Microsoft's Azure Big Compute team released version 1.0.0 of its Batch Shipyard toolkit, which enables easy deployment of batch-style Dockerized workloads to Azure Batch compute pools.
The toolkit makes it possible to run parallel jobs in the cloud without having to manage the infrastructure. Applications include parametric sweeps, Deep Learning training with NVIDIA GPUs, and simulations using MPI and InfiniBand, whether the containerized jobs are run on a single machine or hundreds or even thousands of machines.
Some highlighted features:
- Automated Docker Host Engine installation tuned for Azure Batch compute nodes
- Automated deployment of required Docker images to compute nodes
- Accelerated Docker image deployment at scale to compute pools consisting of a large number of VMs via private peer-to-peer distribution of Docker images among the compute nodes
- Automated Docker Private Registry instance creation on compute nodes with Docker images backed to Azure Storage if specified
- Automatic shared data volume support for:
- Azure File Docker Volume Driver installation and share setup for SMB/CIFS backed to Azure Storage if specified
- GlusterFS distributed network file system installation and setup if specified
More...
https://azure.microsoft.com/en-us/blog/deep-learning-simulation-and-hpc-applications-with-docker-and-azure-batch/
