Event processing is a method of tracking and analyzing streams of information of an event, and eventually deriving a conclusion from what transpired. CEP is an event processing method which combines data from multiple sources to infer events or patterns that may demonstrate unusual activities or anomalies, consequently requiring immediate action.
The CEP engine provided by Panopticon is named Panopticon Streams and it is built to work with different CEP engines. However, for this version, it will only support Kafka.
Kafka is a distributed streaming platform that lets you publish and subscribe to streams of records. Each record consists of a key, a value, and a timestamp and stores streams of records in categories called topics. Kafka is mainly used for two reasons:
q Building real-time streaming data pipelines that reliably get data between systems or applications
q Building real-time streaming applications that transform or react to the streams of the data
Refer to https://kafka.apache.org/intro.html for more information.
Panopticon Streams enables you to create streaming data pipelines which both transforms and reacts to streaming data. Aside from Kafka, it is also using ZooKeeper and Schema Registry that are provided by Confluent. ZooKeeper is a key component when using Kafka since it allows the configuration and management of clusters in the Kafka servers. The Schema Registry stores a versioned history of all schemas used by Kafka and provides a RESTful interface for storing and retrieving Avro schemas.