SSRC talk: Data Stream Systems and Complex Event Processing
Ismail Ari (HP Labs)
All mission-critical applications read or generate raw data streams and require real-time processing of these streams to collect statistics, control flow and detect abnormal patterns. A trend which has gained strong momentum in different industry sectors is the use of Complex Event Processing (CEP) over streams to support critical business processes, thus pushing its use beyond the classical military and financial applications. However, a parallel trend has been the re-architecting of existing business processes with Service Oriented Architecture (SOA) principals to provide integration and interoperability. There is a gap between the new generation of business processes which desperately need CEP and the proposed CEP engines that were not built with SOA in mind.
In this talk, I'll describe the basic concepts in data stream systems and complex event processing. I'll overview the design of our CEP engine called ReCEPtor, which can sense complex events from data streams in real-time. ReCEPtor addresses issues related to CEP system flexibility, interoperability, scalability and performance. I'll show proof-of-concept applications designed for retail and financial industries.