Altair® Panopticon

 

Livy Spark

Livy is an open source REST interface for interacting with Apache Stark. It supports executing snippets of code or programs such as Scala, Python, Java, and R in a Spark context that runs locally or in Apache Hadoop YARN.

The Livy Spark connector allows you to run these codes and fetch the data in Panopticon Real Time.

Steps:

1.     Select LivySpark from the Data Sources pane. The LivySpark Settings pane and the retrieved LivySpark source are displayed.

 

2.     Enter the Name of the Livy Spark data source, then click .

3.     Enter or select the following properties:

Property

Description

Host

Livy Spark host address.

User Id

User Id that will be used to connect to Livy Spark.

Password

Password that will be used to connect to Livy Spark.

Kind

Currently, the supported kind of connection to be used is pyspark (Interactive Python Spark session).

Request Timeout

Length of time to wait for the server response. Default is 30.

Polling Count

The number of polling done to the Livy Spark server to check if the status of the app is successful. Default limit is 150.

Polling Frequency (in seconds)

Frequency of the polling. Default is 2.

Script

The script to use.

 

4.     Set the row limit of the data set.

5.     Tap the Preview Selected Data Source slider to turn it on.

6.     Click  to display the data preview.