Configure Alibaba Cloud for Cloud Bursting

Create the necessary components in Alibaba Cloud that are required for cloud bursting and then create a Control cloud bursting scenario.

Create an Alibaba Cloud User Account

Create an Alibaba Cloud Resource Access Management (RAM) user account so that you can later create a cloud account in Control.

You must already have signed up for an Alibaba cloud account.

Alibaba Cloud documentation can be found at Overview of RAM users.

Create an Alibaba Cloud RAM user account and grant permissions to the RAM user. Once this task is complete, you should have downloaded a CSV file containing the following:
  • Access Key ID
  • Access Secret Key

This information is necessary to create a cloud account in Control.

  1. Access the logon page of the International site (alibabacloud.com) and click Sign In as RAM User.
  2. Log on to the RAM console by using your Alibaba Cloud account.
  3. In the navigation pane on the left-hand side of the web page, choose Identities > Users
  4. Click Create User. This user will have administrative rights to any machines hosted in the cloud.
  5. Enter the following information to add a user:
    1. For Logon Name, enter a name for the user.
    2. For Display Name, enter a name that will be displayed for the user.
      The name can be anything meaningful to your organization, e.g., pc_clouduser.
    3. For Access type, enable OpenAPI access.
      The user requires this type of access because Control needs to make API calls or use the Alibaba Cloud CLI.
  6. Click OK.
    An AccessKey pair consisting of an Access Key ID and a Access Secret Key are generated for the user.
  7. Click Download CSV File to download the AccessKey pair information for the user to your computer.
  8. To grant administrator permissions to the user, choose Identities > User.
  9. Click Add Permission in Actions column for the user.
  10. Select AdministratorAccess system policy and retain default settings for parameters.
  11. Click OK.
    The system policy is attached to the administrator.

Add an Alibaba Cloud User Account

Add an existing Alibaba cloud user account to Control so that you can create a cloud bursting scenario.

You must have already created an Alibaba cloud user account, given it administrative permissions and downloaded a CSV file containing an Access Key ID and a Access Secret Key. The information in the CSV file is necessary to create a cloud account in Control. If you do not have this information see Create an Alibaba Cloud Account.

While working on Alibaba Cloud it is important to choose a region, the geographic area where data centers are deployed. The region of an Alibaba Cloud resource cannot be changed after the resource is created. A region is further divided into zones, a physical area that has its own power grids and networks. Zones within same region can access each other over an internal network. All regions are provided with a specific Region ID. Refer to the Alibaba cloud documentation to obtain Region IDs for your specific region.

  1. Login to Control.
  2. Click the Cloud tab.
  3. Click Cloud from the Infrastructure menu located on the left-hand side of the web page.
  4. Click
  5. Enter the following to add a cloud account:
    1. For Account name, enter any name for the cloud account.
      The name can be anything meaningful to your organization, e.g., alibaba_cloudaccount.
    2. For Access Key ID, copy and paste the value of Access key ID from the downloaded CSV file.
    3. For Access Secret Key, copy and paste the Access Secret Key value from the downloaded CSV file.
    4. For Region ID enter the Region ID code for your region from the Alibaba Cloud documentation.
  6. Click Create account.

Create a VPC and a vSwitch

Create a Virtual Private Cloud (VPC) so that you can later create a virtual machine for cloud bursting.

Login to your Alibaba Cloud Console and choose a region based on the geographical location of your users. All cloud resources that are created are placed in this region.
  1. Log on to the Alibaba Cloud Management Console.
  2. Move the pointer over the in the upper-left corner. The Products and Services page is displayed.
  3. Search for VPC and click the search result link to open the VPC console.
  4. In the top navigation bar, select the region where you want to create a VPC and a vSwitch.
  5. On the VPCs page, click Create VPC.
  6. Enter the following to create a VPC:
    1. For Region, the region you selected in the top navigation bar is displayed.
    2. For Name, enter any name for the VPC.

      The name can be anything meaningful to your organization, e.g., bursting_vpc.

    3. For IPv4 CIDR block, provide an address range in CIDR notation.
    4. For Description, enter a meaningful description for the VPC.
    5. For Resource Group, select the resource group to which the VPC belongs.
    6. For vSwitch Name, enter a name for the vSwitch.
    7. For Zone, select a zone for the vSwitch from the dropdown.

      vSwitches in different zones can communicate with each other.

    8. For IPv4 CIDR, enter an IPv4 CIDR block for the vSwitch.
      Note: This must be a subset of the CIDR for the VPC. You can add at most 10 vSwitches in each VPC. You cannot change the CIDR block of a vSwitch after it is created.
  7. Click OK.

Create a Virtual Machine

Create a virtual machine in Alibaba Cloud Elastic Compute Service (ECS) console.

Alibaba Cloud documentation can be found at Quick start for Linux instances and Quick start for Windows instances.
  1. Login to the Alibaba cloud console.
  2. Move the pointer over the in the upper-left corner. The Products and Services page is displayed.
  3. Search for Elastic Compute Service and click the search result link to open the ECS console.
  4. Click Instances from the menu located on the left-hand side of the web page.
  5. Click Create Instance.
  6. Click the Custom Launch tab.
  7. In the Basic Configurations section:
    1. For Billing Method, choose the subscription type suitable for your instance.
    2. For Region, select the region you want the insance to be in.
    3. For Zone, select a specific zone or choose random.
      Note: You cannot change the region or zone after the instance is created.
    4. For Instance Type, choose Type-based Selection, x86-Architecture and General Purpose category.
    5. Choose a specific instance type based on your requirements for vCPU, Memory, Clock Speed, Network Bandwidth and pricing.
      Note: Instance types that are available are determined by the region that you select and the resource inventory in the region. You can go to the ECS Instance Types Available for Each Region page to view the instance types available in each region.
    6. For Image, select Public Image, CentOS 7.9 64-bit
    7. For Storage, select System Disk and for Disk specifications and performance choose Enhanced SSD (ESSD). 40GB is minimum recommended.
  8. Click Next.
  9. In the Networking section:
    1. For Network Type, select the required VPC and vSwitch from the list.
    2. For Public IP Address, select Assign Public IPv4 Address.
    3. For Bandwidth Billing, select Pay-By-Traffic.
    4. For Peak Bandwidth, set a speed of 4 Mbps.
    5. For Security Group, choose the default group that is automatically created or Reselect a Security Group that has been previously created.
  10. Click Next.
  11. In the System Configurations (optional) section:
    1. For Logon Credentials, choose a Key Pair.
    2. For Key Pair, select a Key Pair from the list.
    3. Provide an Instance name and a description.
  12. In the Grouping (optional) section:
    1. For Tags, select the tags for the instance.
  13. Click Next.
  14. In the Preview step, confirm the configurations in the Configurations Selected section or click the Edit icon to modify configurations.
  15. Read and select ECS Terms of Service. Click Create Instance.
  16. In the Created message box, click Console to view the instance creation progress on the Instances page. If the instance is created, it is in the Running state.
  17. Copy the public IP address of the instance for use when you want to connect to the ECS instance.

Install the PBS MoM on the VM

Install the PBS MoM on the virtual machine and configure the VM per your site's environment.

The user name for logging into the virtual machine is determined by the SSH key that was provided during the creation of the virtual machine. A startup script is provided during the creation of the bursting scenario. This script is ran after a cloud node is burst and is used to set up the system environment so that nodes burst in the cloud can communicate with the PBS Server. A service specifically designed for cloud instance initialization is cloud-init and is utilized by Control when booting cloud nodes, therefore it must be installed on the VM. For more information about installing cloud-init, refer to the Cloudinit Documentation.
  1. Login to your site's PBS Server as the user account (public SSH key) provided during the creation of the VM.
  2. SSH into the virtual machine using the public IP address of the VM:
    ssh IPV4PublicIP
    where IPV4PublicIP is the public IP address of the virtual machine.
  3. Enter the command:
    sudo -i
  4. Copy the PBS Professional installation package to the VM.
  5. Using the PBS Professional Installation and Upgrade Guide, install and configure the PBS Professional MOM.
  6. Configure the VM for your site's environment such as mounting file systems, connecting it to the authentication service, installing any applications, etc.
  7. Install cloud-init.
  8. Connect the VM to your PBS Professional server and submit a job to ensure it is working as expected.

Create an OS Image

Create an image of the virtual machine configured per your site's requirements.

You must have already create a virtual machine, installed the PBS Professional MoM on the VM and configured the VM per your site's environment.

Alibaba Cloud documentation can be found at Create a custom image from an instance.

  1. Login to the Alibaba cloud console.
  2. Move the pointer over the in the upper-left corner. The Products and Services page is displayed.
  3. Search for Elastic Compute Service and click the search result link to open the ECS console.
  4. Click Instances from the menu located on the left-hand side of the web page.
  5. Click More in the Actions column of the instance from which an image has to be created.
  6. Select Disk and Image > Create Custom Image.
  7. In the Create Custom Image page:
    1. For Custom Image Name, provide a suitable name.
    2. For Custom Image Description, provide a brief description of the image.
    3. You can add Image Family, Resource Groups and tags as per your requirements.
  8. Click Create.
    You can use the Image ID/Name of the custom image in Control to create a cloud bursting scenario.

Create an Alibaba Cloud Bursting Scenario

Create an Alibaba Cloud bursting scenario so that PBS Professional execution nodes can be burst in the cloud.

Open a browser window and login to your Alibaba Cloud Management Console. Open a second browser window and login to Control.

A startup script is provided during the creation of the bursting scenario. This script is ran after a cloud node is burst and is used to set up the system environment so that nodes burst in the cloud can communicate with the PBS Server. On Windows platforms, the startup script must be a PowerShell script. For more information see PowerShell Scripting. On Linux platforms, use a cloud-init script. For more information about writing cloud-init scripts, refer to the Cloudinit Documentation and Cloud Bursting Startup Script.

  1. Navigate to the Control browser window.
  2. Click the Cloud tab.
  3. Click Bursting from the Infrastructure menu located on the left-hand side of the web page.
  4. Click Add Bursting Scenario.
  5. Enter information about the bursting scenario such as name, cloud provider, cloud account, and geographic location.
    1. For Name, enter a name for the cloud bursting scenario.
    2. For Description, enter a description.
    3. For Cloud Account, choose a registered Alibaba Cloud account from the list.
    4. For Region, select a region from the list that matches the region selected in Alibaba Cloud.
  6. Click Next.
  7. For Domain name, provide the domain name of your organization for the cloud bursting network.
    altair.com, burst.altair.com
  8. For Hostname prefix, enter a prefix for the names of the burst nodes.
    If "node" is entered , the cloud bursting nodes will be available to PBS Professional as node1, node2, node3, etc.
  9. For Zone, depending on the region selected for the scenario refer to the Regions and Zones list to identify the suitable Zone ID. e.g., Frankfurt can be eu-central-1b or eu-central-1a.
  10. Navigate to the cloud provider browser window.
  11. Move the pointer over the in the upper-left corner. The Products and Services page is displayed.
  12. Search for VPC and click the search result link to open the VPC console.
  13. Copy the Instance ID/Name of the VPC to be used for the cloud bursting.
  14. Navigate to the Control browser window.
  15. For VPC ID, paste the VPC ID copied from Alibaba Cloud.
  16. Navigate to the cloud provider browser window.
  17. Click vSwitch from the VPC menu located on the left-hand side of the web page.
  18. On the vSwitch page, find the vSwitch that you want to use and copy its ID.
  19. Navigate to the Control browser window.
  20. For V Switch ID, paste the vSwitch ID copied from Alibaba Cloud.
  21. Navigate to the cloud provider browser window.
  22. In the left-side navigation pane, choose Network & Security > Security Groups.
  23. Find the security group that you want to use and copy its ID.
  24. Navigate to the Control browser window.
  25. For Security Group ID, paste the Security Group ID copied from Alibaba Cloud.
  26. Enable Add public IP to VMs if you need to connect to the cloud bursting nodes from the internet.
    It is recommended to leave this disabled.
  27. For Cloud-init script, click Browse, browse for the startup script file and select it.
    This script will run after a cloud node is burst.
  28. Add public SSH keys for any users that you want to have access to the burst nodes:
    1. Login to the PBS Professional Server.
    2. Copy the public SSH key for the user.
      Public key files are usually stored in /.ssh in the user's home directory.
    3. Navigate to the Control browser window.
    4. Click Add to add the public key.
    5. For Public SSH keys, paste the public SSH key.
    6. Repeat steps a through e to add additional public SSH keys.
    The public keys are copied to each burst node for secure connectivity.
  29. For Idle time before unbursting, enter the minimum time that a cloud node can be idle before it is unburst.

    It is expressed as an integer whose units are in seconds.

  30. For Tags, enter a key-value pair to add a label to the cloud node and press ENTER.
    A tag is a label that is assigned to a cloud node. Tags enable cloud nodes to be categorized in different ways, for example, by purpose, owner, or environment.
    The key-value pair must be entered as key:value. The key must be less than 36 characters and the value must be less than 43 characters. The length of both the key + value should be less than 80 characters. Valid values are alphanumeric, dash (-) and an underscore ( _ ).
    Once the tag is added, it will appear within its own bubble.

    Key-Value Tag
    Figure 1. Key-Value Tag
  31. Repeat the previous step to add more tags.

    Additional Tags
    Figure 2. Additional Tags
    Tip: To remove a tag click on the 'x' located within the tag bubble.
  32. Click Next.
  33. Click the Select instance types for bursting link.
  34. Select the instance types to enable for the bursting scenario by clicking the Enabled checkbox located to the far right of the instance type name.
  35. Click Save.
  36. Click the Create an API Key/ Token link.
  37. For Name, enter a name for the token.
  38. For Expiration date, use the calendar widget to specify the expiration date.
  39. Click Add Token.
    An API key is generated and displayed.
  40. Copy and save this API key.
    The API key is only displayed once. It is pasted in the configuration file used to configure the cloud bursting hook.
  41. Click Close.
  42. Enable the bursting scenario by clicking the slider at the top next to the scenario title.
    If an attempt is made to burst nodes with a disabled instance type, the node is not burst and the below message is displayed in the PBS Server logs (if bursting through PBS), the Cloud logs and through the Cloud component user interface.
    TYPE is not enabled

    Where TYPE is the instance type used when attempting to burst the node.

Once the scenario created, it is recommended to Add Quotas and Alerts. Quotas define bursting limits such as the maximum number of cloud nodes to burst.

Alibaba Cloud Regions and Zones

Alibaba Cloud also segregates geography by region and then region by zones. Information related to the regions is available in the Alibaba cloud documentation.

Zone IDs for Specific Regions

Here is a list of zones for each region and their IDs.

{
   "cn-qingdao":[
      "cn-qingdao-c",
      "cn-qingdao-b"
   ],
   "cn-beijing":[
      "cn-beijing-k",
      "cn-beijing-h",
      "cn-beijing-g",
      "cn-beijing-f",
      "cn-beijing-e",
      "cn-beijing-d",
      "cn-beijing-c",
      "cn-beijing-b",
      "cn-beijing-a",
      "cn-beijing-i",
      "cn-beijing-j",
      "cn-beijing-l"
   ],
   "cn-zhangjiakou":[
      "cn-zhangjiakou-a",
      "cn-zhangjiakou-c",
      "cn-zhangjiakou-b"
   ],
   "cn-huhehaote":[
      "cn-huhehaote-a",
      "cn-huhehaote-b"
   ],
   "cn-wulanchabu":[
      "cn-wulanchabu-b",
      "cn-wulanchabu-a",
      "cn-wulanchabu-c"
   ],
   "cn-hangzhou":[
      "cn-hangzhou-k",
      "cn-hangzhou-i",
      "cn-hangzhou-j",
      "cn-hangzhou-h",
      "cn-hangzhou-g",
      "cn-hangzhou-f",
      "cn-hangzhou-b",
      "cn-hangzhou-e",
      "cn-hangzhou-d",
      "cn-hangzhou-c"
   ],
   "cn-shanghai":[
      "cn-shanghai-l",
      "cn-shanghai-b",
      "cn-shanghai-g",
      "cn-shanghai-f",
      "cn-shanghai-e",
      "cn-shanghai-d",
      "cn-shanghai-c",
      "cn-shanghai-a",
      "cn-shanghai-k",
      "cn-shanghai-m",
      "cn-shanghai-n"
   ],
   "cn-shenzhen":[
      "cn-shenzhen-e",
      "cn-shenzhen-f",
      "cn-shenzhen-d",
      "cn-shenzhen-c",
      "cn-shenzhen-a",
      "cn-shenzhen-b"
   ],
   "cn-heyuan":[
      "cn-heyuan-b",
      "cn-heyuan-a"
   ],
   "cn-guangzhou":[
      "cn-guangzhou-a",
      "cn-guangzhou-b"
   ],
   "cn-chengdu":[
      "cn-chengdu-a",
      "cn-chengdu-b"
   ],
   "cn-hongkong":[
      "cn-hongkong-b",
      "cn-hongkong-c",
      "cn-hongkong-d"
   ],
   "ap-northeast-1":[
      "ap-northeast-1b",
      "ap-northeast-1a"
   ],
   "ap-northeast-2":[
      "ap-northeast-2a"
   ],
   "ap-southeast-1":[
      "ap-southeast-1c",
      "ap-southeast-1b",
      "ap-southeast-1a"
   ],
   "ap-southeast-2":[
      "ap-southeast-2b",
      "ap-southeast-2a"
   ],
   "ap-southeast-3":[
      "ap-southeast-3a",
      "ap-southeast-3b"
   ],
   "ap-southeast-6":[
      "ap-southeast-6a"
   ],
   "ap-southeast-5":[
      "ap-southeast-5a",
      "ap-southeast-5b",
      "ap-southeast-5c"
   ],
   "ap-south-1":[
      "ap-south-1b",
      "ap-south-1a"
   ],
   "ap-southeast-7":[
      "ap-southeast-7a"
   ],
   "us-east-1":[
      "us-east-1b",
      "us-east-1a"
   ],
   "us-west-1":[
      "us-west-1b",
      "us-west-1a"
   ],
   "eu-west-1":[
      "eu-west-1b",
      "eu-west-1a"
   ],
   "me-east-1":[
      "me-east-1a"
   ],
   "eu-central-1":[
      "eu-central-1b",
      "eu-central-1a"
   ],
   "cn-nanjing":[
      "cn-nanjing-a"
   ]
}