General Guidelines and Frequently Asked Questions
- Reducing the size of container images to pre-load will minimize the pool spin up (allocation time). It is recommended to coalesce layers together or use multi-stage builds if using Docker images.
- Please keep in mind that pool allocation speed is dependent upon a lot of factors that Batch Shipyard has no control over.
Frequently Asked Questions
I have an issue or a problem...
- Visit the troubleshooting guide first; your issue may already be cataloged with possible solutions.
- If you checked the troubleshooting guide and your issue is not listed, where do I post an issue about it?
I don't have enough core (or other) quota. How do I increase it?
- Please see this page.
How do new versions and backward compatiblity work?
- Versioning follows
MAJOR-level changes typically will be very large modifications for supporting new features that may fundamentally change how Batch Shipyard works. Expect breaking changes at this level. Prior deprecation paths may be removed.
MINOR-level changes typically will be small to significant changes for supporting new features introduced at the Azure Batch level or integration efforts and features that supplement the current feature set. While breaking changes may happen, we will try to minimize the impact for these changes by providing a deprecation path forward if feasible.
PATCH-level changes are typically hotfixes or non-breaking additions.
- We realize that introducing breaking changes are inconvenient and often burdensome for users. However, we need to achieve a balance of iterating quickly to deliver new solutions and features not only encompassing core Azure Batch and other Azure services but extending to include other technologies at the intersection of cloud Batch processing, HPC, containerization and Big Compute.
- You can always lock in your deployments to a specific version by using a specific release archive, checking out to a specific release tag if using the git repository for consuming releases, or using a specific release tag for the Batch Shipyard CLI Docker image. This will allow you to upgrade on your schedule and at your own convenience.
- At most two versions of a host operating system are supported at any one time.
Feature X is missing. Can you add support for it?
- Please open an issue regarding your request. Pull requests are always welcome!
How do I contribute a recipe?
- Please see this guide.
native under pool
native designates to Batch Shipyard to attempt to create the pool such
that the pool works under native Docker mode where the compute nodes
"natively" understand how to launch and execute Docker containers. Please
understand that only a subset of
platform_image combinations are compatible
native mode. You can refer to the
Batch Shipyard Platform Image support doc
for more information. Compliant
custom images are compatible with
native mode are:
- Batch Shipyard with a provisioned SSH user is no longer necessary to perform actions such as terminating tasks where tasks are still running or deleting jobs with running tasks in cases where normal Batch task termination fails to properly end the Docker container processes.
- Direct execution of container tasks by the Batch node agent itself which lends to cleaner container execution lifecycle management.
- Multi-instance task execution (e.g., MPI job) is cleaner. You can execute
multiple multi-instance tasks per job. If your workload is predominantly
multi-instance, then it is strongly recommended to use
native mode are:
- Singularity containers are not supported.
input_dataof any kind at the task-level is not possible; you must either use
resource_filesor build your own solution.
output_dataoptions are limited and egress to
azure_storageAzure Files (i.e., file shares) is not possible.
- Less aggressive retries of compute node provisioning steps. This can
potentially lead to a greater occurrence of
- Other experimental features may not be supported.
Does Batch Shipyard support Linux custom images?
- Yes, please see the guide.
Does Batch Shipyard support Windows Server Containers?
- Yes, but with some feature, configuration, and CLI limitations. Please see the current limitations doc for more information.
- If you receive OS compatibility mismatches when running your Windows
containers, please ensure you have the correct
--isolationparameter set, if required. You can view the Windows container compatibility matrix here. To learn more about Hyper-V isolation, please see this article.