Creating Data Extracts

One of the methods in accessing data is by retrieving only the required results into memory, by querying on demand, pushing aggregation and filtering tasks to underlying big data repositories, or queryable data extracts.

This is commonly known as a ROLAP implementation, where the product is dynamically writing data queries to the underlying data repository and retrieving aggregated and filtered datasets. Given the on-demand nature of this method it is more suitable to exploratory data analysis but requires dynamic query generation.

Data extracts are created in the server administration environment. These server-based data extracts can then be retrieved and used through the Panopticon Data Extract connector.

 

Data Extract Property	Description
Name	Data extract name.
Connector	Data connector used in the extract.
#Rows	Number of rows of the data extract.
#Columns	Number of columns of the data extract.
#Cells	Number of cells of the data extract.
Status	Status of the data extract. Values include IDLE or RUNNING. When saving a new data extract, it is first run and the status changes to RUNNING. Consequently, the Refresh Data icon is disabled. When the data extract is complete, the status changes to IDLE and the Refresh Data icon is enabled.
Last Updated	The Date/Time when the data extract is last completed or updated.

 

 

A sample data extract using the MS Excel connector before refreshing the data:

NOTE: A * symbol appears beside a data extract that is not yet refreshed.

 

After refreshing the data:

 

Steps:

1.     On the Extracts tab, click .

The Extracts tab displays with the following sections:

 

Section/Panel	Description
Extract Name	Name of the data extract. Click the  button to go back to the Data Extracts listing page.
Connector drop-down list	Includes the non-streaming connectors to extract data from.
Save	Save the changes made on the Extracts tab.
Refresh Data	Refresh the data after modifying and saving changes on the Data Extract page. You can also opt to click Cancel Refresh Data .
Details	Display the details of the data extract including the number of rows, columns, cells, status, and the last time it was updated.
Refresh Data Preview	Refresh the data preview.
Expand Data Preview	Expand the Data Preview pane.

 

2.     Enter the Name of the data extract. This should be unique and should only contain letters (a to Z), numbers (0 to 9), and underscores.

3.     Click  or press Enter to apply the name.

4.     Select any of the following non-streaming connectors:

·         Cassandra

·         Elasticsearch 6.x

·         Elasticsearch 7.x

·         InfluxDB

·         JDBC Database

·         JSON

·         Kx kdb+

·         ksqlDB

·         Livy Spark

·         MongoDB

·         MS Excel

·         OneTick

·         OneTick Cloud

·         Python

·         Rserve

·         Splunk

·         Text

·         XML

The tab page changes depending on the selected connector.