Cloud Bursting Overview
A description of cloud bursting, a typical bursting setup and an overview of steps for setting up cloud bursting.
Cloud bursting is a configuration that is set up between an HPC cluster and a public cloud to deal with peaks in cluster demand. When resource capacity at the HPC reaches a certain point, the demand is directed to a public cloud so there is no interruption of services.
- Microsoft Azure
- Amazon Web Services (AWS)
- Google Cloud Platform (GCP)
- Oracle Cloud Platform
- Orange Cloud Flexible Engine
- Open Telekom Cloud (OTC)
- HUAWEI Cloud
Cloud bursting is managed through PBS Professional by submitting jobs to one or more designated cloud queues. A server periodic hook monitors cloud queues, estimates the demand for cloud nodes, and dynamically adjusts the number of nodes by bursting or unbursting as needed.
Figure 1. Typical Cloud Bursting Setup
Steps for Cloud Bursting
There is some setup required at the cloud provider level before a cloud bursting scenario can be created using Control. There is also PBS Professional configuration that must be done. General steps that are required for bursting:
- Create a boot script that runs after a cloud node is burst.
- Create necessary cloud provider components such as a virtual network and a virtual machine.
- SSH into the cloud provider virtual machine and configure the VM for your site's environment such as installing PBS, mounting file systems, connecting it to the authentication service, installing any applications, etc.
- Create an image of the virtual machine.
- Create a bursting scenario with Control that provides the necessary cloud provider details required for cloud bursting.
- Configure PBS cloud bursting custom resources.
- Create PBS cloud queues.
- Install the PCLM CLI and cloud bursting hook.
- Configure the PBS cloud bursting hook.
- Enable a bursting scenario.