Configuring Microsoft Azure Cloud Bursting

Register Control with Azure so that you can later create a cloud account in Control.

1. Login in to your Microsoft Azure account.

Register Control to the Azure Active Directory.

2. Using the search box, enter app reg.
   A list of search results is listed.
3. Under Services, click App registrations.
4. Click New registration.
5. Enter the following to register Control to the Azure Active Directory:
   1. For Name, enter the name of the application.
      The name can be anything meaningful to your organization, e.g., pc_cloudapp.
   2. For Supported account types, choose the option that best suits your organization.
      Tip: Click the Help me choose link for additional information about the available options.
   3. For Redirect URI, select Web and enter the URL https://hostname:9500/pc.
      where hostname is the hostname of the machine where the Control Web Interface is installed. This is the URL that is used to login to Control.
6. Click Register.
   Once the application registration is complete, its details are displayed, including an Application ID.

Obtain the application id of the newly registered application.

7. Hover over the Application (client) ID and click when it appears.
8. Store the Application ID to a file.
   The Application ID is used later to create a cloud account in Control.

Create a client secret key for your newly registered application.

9. Click Certificates and secrets from the Manage menu located on the left-hand side of the web page.

   
   
   Figure 1. Certificates and Secrets

10. Under Client secrets, click New client secret.
11. Enter the following to add a client secret:
    1. For DESCRIPTION, enter pc_client_secret.
    2. For EXPIRES, select 24 months.
12. Click Add.
    A client secret key is generated and is displayed under the heading VALUE.
13. Click next to the client secret key.
14. Store the client secret key to a file.
    The client secret key is used later to create a cloud account in Control.

Obtain your Azure subscription ID.

15. Using the search box, enter subscription.
    A list of search results is listed.
16. Under Services, click Subscriptions.
17. Locate and click your subscription.
    Details about the subscription are displayed, including a Subscription ID.
18. Hover over the Subscription ID and click when it appears.
19. Store the Subscription ID value to a file.
    The Subscription ID is used later to create a cloud account in Control.

Assign an access control role to the newly registered application.

20. Click Access control (IAM).

    
    
    Figure 2. Add Access Controls

21. Click Add.
22. Click Add role assignment.
23. In the Add role assignment panel, enter the following to assign a role to the newly registered application:
    1. For Role, select Contributor.
    2. For Assign access to, select user, group, or service principal.
    3. For Select, search for the newly registered application by entering its name, e.g., pc_cloudapp.
    4. Select the application by clicking on it.
24. Click Save.

Obtain your Azure tenant ID:

25. At the top of the web page, click ?.
26. Click Show diagnostics.
    A dialog box is displayed allowing a file called PortalDiagnostics.json to be saved.
27. Open the file using any text editor.
28. Search for tenantId.
29. Store the value of tenantId to a file.
    The Tenant ID is used later to create a cloud account in Control

Add an Azure cloud account to Control so that you can create a cloud bursting scenario or an appliance.

1. Login to Control.
2. Click the Cloud tab.
3. Click Cloud from the Infrastructure menu located on the left-hand side of the web page.
4. Click .
5. Enter the following to add a cloud account:
   1. For Account name, enter any name for the cloud account.
      The name can be anything meaningful to your organization, e.g., azure_cloudaccount.
   2. For Client ID, enter the Application ID generated when Control was registered with the Azure Active Directory.
   3. For Secret Key, enter the client secret key generated for the newly registered Control application.
   4. For AD tenant ID, enter your Azure tenant ID.
   5. For Subscription ID, enter your Azure subscription ID.
6. Click Create account.

Create a resource group that will contain a virtual network and a virtual machine for cloud bursting.

1. Login in to your Microsoft Azure account.
2. Using the search box, enter resource groups.
   A list of search results is listed.
3. Under Services, click Resource Groups.
4. Click Create.

Enter the following to configure the basic settings for the resource group:

5. For Project Details enter the following:
   1. For Subscription, choose the subscription to be billed for the use of the VM.
   2. For Resource group, enter a name for the resource group.
      The name can be anything meaningful to your organization, e.g., bursting_resource_group.
6. For Resource Details enter the following:
   1. For Region, select a location based on the geographical location of users.
7. Click Review + create.
8. Click Create.
   It may take a moment to create the resource group. All resources (networks, virtual machines, etc.) that are created are placed within this resource group. The name of the resource group is required for creating a bursting scenario in Control.

Create a virtual network so that you can later create a virtual machine for cloud bursting.

1. Login in to your Microsoft Azure account.
2. Using the search box, enter virtual networks.
   A list of search results is listed.
3. Under Services, click Virtual networks.
4. Click Create.
5. In the Basics tab, for Subscription, select the same subscription as was selected for the previously created resource group.
6. For Resource group, select the previously created resource group.
7. For Name, enter a name for the virtual network.
   The name can be anything meaningful to your organization, e.g., bursting_virtual_network
8. For Region, select the same geographical location as was selected for the previously created resource group.
9. In the IP Addresses tab, for IPv4 address space, enter an address range for the network using CIDR notation.
10. Click Add subnetand for enter the following:
    1. For Subnet name, enter a name for the virtual machine's subnet.
       The name can be anything meaningful to your organization, e.g., bursting_subnet
    2. For Subnet address range, enter an address range for the subnet in CIDR notation.
    3. Click Add.
11. Click Review + create.
    It may take a moment to create the virtual network. The name of the virtual network is required for creating a bursting scenario in Control.

Create a virtual machine in Azure.

1. Login in to your Microsoft Azure account.
2. Using the search box, enter marketplace.
3. Open the Marketplace by selecting the link from the search results.
4. In the marketplace search box, enter rogue.
   List of images created by Rogue Wave Software is displayed.
5. Click the CentOS-based HPC image name.
   The details of the image are displayed in a page.
6. For Select a plan, choose CentOS-based 7.9 HPC.
7. Click Create.

Enter the following to configure the basic settings for the virtual machine:

8. For Project Details enter the following:
   1. For Subscription, choose the subscription to be billed for the use of the VM.
   2. For Resource group, choose the previously created resource group.
   3. For Virtual machine name, enter a name for the virtual machine.
      The name can be anything meaningful to your organization, e.g., bursting-vm.
   4. For Region, select the same geographical location as was selected for the previously created resource group.
   5. For Availability options, choose No infrastructure redundancy required.
   6. For Size, click the Change size link and select a machine size appropriate for your site's workload based on:
      • the number of cores
      • the amount of memory
      • storage
      • network performance

      Consider the nature of the applications that you plan to deploy on the instance, the number of users that you expect to use the applications, and also how you expect the load to scale in the future. Remember to also factor in the CPU and memory resources that are necessary for the operating system.
9. For Administrator Account, enter a user account :
   This user will have sudo rights and will be able to connect to the VM to install the PBS MoM.
   • On Linux platforms:
     1. For Authentication type, enable SSH public key.
     2. For Username, enter a username of a user account that exists on your site's PBS Server.
     3. For SSH public key, copy the SSH public key (i.e., id_rsa.pub) of the user account and paste it.
   • On Windows platforms:
     1. For Username, enter a username.
     2. For Password, enter a password.
10. For Inbound Port Rules, enter the following:
    1. For Public inbound ports, enable Allow selected ports.
    2. For Select inbound ports:
       • For Linux platforms, enable SSH (22).
       • For Windows platforms, enable RDH (3389).
11. Click Next : Disks >.

Enter the following to configure the storage settings for the virtual machine:

12. For Disk Options, enter the following:
    1. For OS disk size, choose an appropriate disk size based on your site's needs.
    2. For OS disk type, choose one of the following options:
       • Premium SSD
       • Standard SSD
       • Standard HDD
       Choose SSD for I/O-intensive applications, where low latency and high throughput are critical. For testing, consider HDD to keep costs down, as you scale up and down quickly.
13. Click Next : Networking > .

Enter the following to configure the networking settings for the virtual machine:

14. For Network Interface, enter the following:
    1. For Virtual network, choose the virtual network previously created.
    2. For Subnet, choose the subnet you previously created.
15. Click Review + create tab.
16. Review your choices and click Create.
    It may take a few minutes for the VM to be deployed. You will use this virtual machine to create an OS image.

Create an Azure bursting scenario so that PBS execution nodes can be burst in the cloud.

1. Navigate to the Control browser window.
2. Click the Cloud tab.
3. Click Bursting from the Infrastructure menu located on the left-hand side of the web page.
4. Click Add Bursting Scenario.
5. Enter information about the bursting scenario such as name, cloud provider, cloud account, and geographic location.
   1. For Name, enter a name for the cloud bursting scenario.
   2. For Description, enter a description.
   3. For Cloud Account, choose a previously created Azure cloud account from the list.
   4. For Region, select the same geographic location as was selected for the previously created bursting resource group.
6. Click Next.

Enter information about the Azure resources necessary to burst in the cloud.

7. For Domain name, provide the domain name of your organization for the cloud bursting network.
   altair.com, burst.altair.com
8. For Hostname prefix, enter a prefix for the names of the burst nodes.
   If "node" is entered , the cloud bursting nodes will be available to PBS Professional as node1, node2, node3, etc.
9. For Resource group name, place the cursor in the field and choose the resource group created for cloud bursting from the list.
   Tip: Enter the first couple of characters of the resource group name to filter the list.
10. For Network name, place the cursor in the field and choose the virtual network created for cloud bursting from the list.
    Tip: Enter the first couple of characters of the resource name to filter the list.
11. For Subnet name, place the cursor in the field and choose the subnet created for cloud bursting from the list.
    Tip: Enter the first couple of characters of the resource name to filter the list.
12. For Network security group name, place the cursor in the field and choose the network security group created for cloud bursting from the list.
    Tip: Enter the first couple of characters of the resource name to filter the list.
13. Choose one of the following options based on whether you chose to use the Azure managed disk feature when creating the virtual machine:
    • Enable Managed Storage if you enabled the managed disk feature.
    • Disable Managed Storage if you disabled the managed disk feature.
14. If you disabled the managed disk feature:
    1. Navigate to the cloud provider browser window.
    2. Click Resource Groups from the menu located on the left-hand side of the web page.
    3. Select the Resource Group created for cloud bursting.
    4. In the list, locate the image that was created for cloud bursting.
       The Type is Image.
    5. Select the image.
    6. Copy the Linux Source BLOB URI.
15. Navigate to the Control browser window.
16. Choose one of the following options:
    • When the managed disk feature is enabled, place the cursor in the OS image field and choose the OS image created for cloud bursting from the list.
      Tip: Enter the first couple of characters of the resource name to filter the list.
    • When the managed disk feature is disabled, place the cursor in the OS image field and paste the Source BLOB URI copied from Azure.
      Note: If the OS image is contained in a Resource group that is different from the one entered for the bursting scenario, it must be entered as Resource Group Name/OS Image Name or Resource Group Name/URI.
17. Enable Add public IP to VMs if you need to connect to the cloud bursting nodes from the internet.
    It is recommended to leave this disabled.
18. For Maximum number of VMs inside a ScaleSet, specify the limit for your cloud bursting scenario.
    If you use InfiniBand, Azure limits the number of VMs on a scaleset to 100. Without InfiniBand, for a scaleset with managed disk and custom image, you can specify a higher limit.
19. For Cloud-init script, click Browse, browse for the startup script file and select it.
    This script will run after a cloud node is burst.
20. Add public SSH keys for any users that you want to have access to the burst nodes:
    1. Login to the PBS Professional Server.
    2. Copy the public SSH key for the user.
       Public key files are usually stored in /.ssh in the user's home directory.
    3. Navigate to the Control browser window.
    4. Click Add to add the public key.
    5. For Public SSH keys, paste the public SSH key.
    6. Repeat steps a through e to add additional public SSH keys.
    The public keys are copied to each burst node for secure connectivity.
21. For Idle time before unbursting, enter the minimum time that a cloud node can be idle before it is unburst.

    It is expressed as an integer whose units are in seconds.

22. For Tags, enter a key-value pair to add a label to the cloud node and press ENTER.
    A tag is a label that is assigned to a cloud node. Tags enable cloud nodes to be categorized in different ways, for example, by purpose, owner, or environment.
    The key-value pair must be entered as key:value. The key must be less than 36 characters and the value must be less than 43 characters. The length of both the key + value should be less than 80 characters. Valid values are alphanumeric, dash (-) and an underscore ( _ ).
    Once the tag is added, it will appear within its own bubble.

    
    
    Figure 4. Key-Value Tag

23. Repeat the previous step to add more tags.

    
    
    Figure 5. Additional Tags

    Tip: To remove a tag click on the 'x' located within the tag bubble.
24. Click Next.
25. Click the Select instance types for bursting link.
26. Select the instance types to enable for the bursting scenario by clicking the Enabled checkbox located to the far right of the instance type name.
27. Click Save.
28. Click the Create an API Key/ Token link.
29. For Name, enter a name for the token.
30. For Expiration date, use the calendar widget to specify the expiration date.
31. Click Add Token.
    An API key is generated and displayed.
32. Copy and save this API key.
    The API key is only displayed once. It is pasted in the configuration file used to configure the cloud bursting hook.
33. Click Close.
34. Enable the bursting scenario by clicking the slider at the top next to the scenario title.
    If an attempt is made to burst nodes with a disabled instance type, the node is not burst and the below message is displayed in the PBS Server logs (if bursting through PBS), the Cloud logs and through the Cloud component user interface.

    TYPE is not enabled

    Where TYPE is the instance type used when attempting to burst the node.