Technology
Why the Complex Event Processing emerged:
- Today's information systems produce a flood of data and events (transactions, sensor measurements, logs from technological systems, …)
- There are information hidden in the bulk data that can make processes more effective, increase profits or descrease losses.
How to get the information in time? This is exactly what Complex Event Processing (CEP) is about.
The first CEP systems emerged within applications for stock trading. Instead of complicated analytical and statistical procedures aspiring to find a nugget, CEP system focused on extremely fast collecting fractions of cents driven by relatively simple algorithms. Nowadays, CEP systems are used in wide range of areas as fraud detection, energetics, logistics, financial markets, telecommunication.
The core of Mycroft Mind solution
CEP solution continuously process events from different data sources (system logs, transactions, sensor measurements, etc...). From the event streams detects an interesting patterns and from lots of isolated events makes a picture of actual situation.
Anomalies detection
In many cases it is not clear how to detect an event interesting to capture. For example, we do not know all tricks what cheaters could use. For such cases a method of anomalies detection is available. The system gradually builds and updates profiles of monitored objects and is able to alert the moment when an object behavior becomes distinct from its typical behavior.
Prediction
Thanks to the architecture Mycroft Mind solutions are able to predict events and draw attention to them before they actually occur. This capability is based on the fundamental principles of CEP systems - detected patterns gradually "mature" as the data passes through. So it is possible to recognize when the pattern is almost "ripe" and give notice about the situation in time.
CEP: database inside out
A classic database stores data used by queries running on demand.
A CEP system stores queries (patterns) through which the data flow. As soon as a query (pattern) catches data waiting for, the CEP system reacts immediatelly.
Why the CEP is >Complex< Event Processing
CEP is characteristically multi-tier system. The first tier processes data coming from outside and filters or correlates these events. Output of this tier is again in the form of events, however, more complex ones. These events are processed by similar way and outputs are again events. It is not unususal if CEP system consists of four or more tiers. Thus it is capable to extract an esence from a flow of millions of events and to present just such amount of information that users are able to control and directly use.
Technology
