Altair SmartWorks Analytics

 

Execution Profiles

Execution Profiles are created for specific engines (e.g., Pandas, Apache Spark) on which a workflow is to be executed. The Execution Profiles page is accessed by clicking the Execution Profiles icon in the SmartWorks Analytics menu. The page that displays provides information on all available Execution Profiles as well as their creators. Execution profiles may be edited or removed by clicking on the Actions icon to the right of a profile.  

 

Selecting a specific Execution Profile yields three tabs: Configuration, Sessions, and Internal Connections.

 

Creating an Execution Profile

Steps

  1. In the Execution Profile window, click Add New Execution Profile


  2. Specify the following details for the Execution Profile that you want to create:

  3. Property

    Description

    Engine

    Select an engine from the dropdown list provided.

    Execution Profile Name 

    Specify a name for the Execution Profile.

    Description

    Enter a description for the Execution Profile.

    Session limit

    Use the + or – buttons to select a maximum number of sessions that can be supported by the Execution Profile.

    Select Unlimited (default) if you do not wish to implement a session limit.

    Session limits are only applicable when Single Sign-On (SSO) is not implemented. Imposing session limits can help reduce the chances that concurrent usage of shared execution resources will be pushed beyond their limits.

    You cannot limit the number of sessions of an Execution Profile to a value less than the number of sessions currently in use.

    Jupyter Hub URL

    Specify the Jupyter Hub URL to apply for the Execution Profile. This field is applicable only for Execution Profiles based on the Pandas and Apache Spark engines.

    Guacamole URL

    Specify the Guacamole URL to apply for the Execution Profile. This field is applicable only for Execution Profiles based on the Altair Desktop Suite engine.

    Use Single Sign-On (SSO) Authentication

    Select this setting to enable SSO for the application. Otherwise, when creating an Execution Profile based on the Apache Spark and Pandas engines, provide a username and password to access the Jupyter Hub URL.

    Requirements

    (Optional) Enter a list of newline-separated requirement specifiers (e.g., package name + version number) to install one or more third-party packages.

    If you specify requirements for a new Python package, for example, every session created from this Execution Profile will include this package installed from the Python Packaging Index (PyPI).

    Note that if the package you want to install requires an upgrade to one or more packages already installed on the engine, such as NumPy, this may cause some nodes to work incorrectly.

     

  4. Click Save to save the changes to the Execution Profile.

  5. Optionally click Cancel to discard the changes that you have made.

Modifying or Deleting an Execution Profile

In the Execution Profile window, mouse over the Execution Profile and:

  • Click the Actions icon and then select Edit to modify the selected Execution Profile.

  • Click the Actions icon and then select Remove to delete the selected Execution Profile.

Understanding Execution Profiles, Sessions, and Workflows

You can create multiple Execution Profiles using the same engine.

 

However, by default, SmartWorks Analytics displays only those Execution Profiles created from a single engine in the Execution Profiles tab of the Palette.

 

When multiple Execution Profiles are created from the same engine and you wish to use one of these profiles, you must manually select the profile to use as follows.

 

When an execution profile, especially one using the PySpark engine, is unlinked from

NOTES

  • Execution profiles are automatically linked to a workflow.

  • Each workflow can have only a single session.

  • You can link several Execution Profiles of different engines to the same workflow.

  • You can use the same Execution Profile in different workflows, but a new session must be started for each workflow.

  • When an Execution Profile is unlinked from a node and then reconnected to it, a new session may need to be manually started for the profile and linked to the node if these actions are not completed automatically.  

  • Once a session has been created, you may need to wait ~1 minute for all of the jars to be distributed to the cluster nodes prior to usage in a workflow. Otherwise, the following error may display in the Node Viewer or when you attempt to run the node:

  • {"graphQLErrors":[{"message":"Unexpected error value: { message: \"Did not observe any item or terminal signal within 30000ms in 'map' (and no fallback has been configured)\", locations: [[Object]], path: [\"getRawPreview\"], extensions: { type: \"GENERAL\", code: \"UNEXPECTED_ERROR\", message: \"Did not observe any item or terminal signal within 30000ms in 'map' (and no fallback has been configured)\", traceId: null, requestId: null, details: [Object] }