Interesting to see some publicity on Intel many-core CPU research, what they are calling the “single chip cloud computer“. While CEP software companies like TIBCO Software have exploited a distributed approach to support the scalability and reliability needs of extreme event processing - OK, let’s call it XEP as an alternative to XTP - similar techniques are now being put into chips for multi-core handling.

Now folks have talked about re-using powerful graphics chips for event processing for a while now, but these tend to support SIMD (Single Instruction Multiple Data) parallelism. The Intel work seems more on the MIMD (Multiple Instruction Multiple Data) side of parallelism - with the potential to put complete units of event processing (especially event stream processing as sequences of event stream processors) into a single chip - effectively mapping a logical Event Processing Network (per EPTS terminology) directly onto silicon. It could also potentially handle a mapping for “event cloud processing” (more cross-channel and data operations than stream-based) - imagine mapping a Rete pattern matching network to such a CPU - although in the latter case the increase in data handling (i.e. IO) requirements could stymie the approach.

Some of the quotes from the article seem familiar…

Initial multicore chip architectures depended on a set of protocols that assures that each core has the same view of the system’s memory, a technique called cache coherency.

Compare and contrast with eXtreme Event Processing technologies’ use of cache and distributed grid technologies for sharing events and event objects around a set of processes…

The recent work of the design team has centered on developing message-passing techniques for the chip that would scale as more cores are added.

Compare and contrast with eXtreme Event Processing and Event Driven Architecture (EDA) systems’ use of Message Oriented Middleware (or MOM) to share information across multiple processes.

Naturally Intel is a chip vendor rather than a software company, so it looks like their de facto software “solution” is to map to their experimental chip a general purpose operating system i.e. Linux. Now if they were to seriously customise Linux to act as an agent host language for parallel operations, perhaps this could be a future CEP (i.e. EPN) hardware engine!

More details on Intel’s work here…

I had promised Prof David Luckham at the last EPTS meeting I would post some notes from a BCS event I attended a while back, given by the IT department of JET. The topic came up again in a response to Tim Bass’ Linkedin CEP blog where Tim implied the CERN LHC’s event processing was something the EPTS Languages Working Group should have (or should in future be) taken into account - a fair criticism provided you think the LHC and its ilk’s event processing needs are “typical” or repeatable elsewhere.

Back to my notes on JET: they used custom hardware (>21,000 rack mount modules of >400 types) for event-data compression and collection - what could be described as a “hardware tree database” with event channels leading to a “lightweight” software mapping system controlled via APIs. However, the IT systems were mainly concerned with collecting valid events and exceptions for post-experiment analysis - each experiment having something like 22,000 parameters to deal with! Events were not just from the Torus, either - with such expensive experiments they were monitoring things like the data collection hardware. The control systems used a state model and a control loop executing at 500 iterations per second. The experiments were defined as state models that were maintained in a custom content management solution based on RDF. If I recall, their main programming languages were C++ and Fortran.

Interestingly, state management was one of the language aspects covered by the EPTS Languages working group… and state modeling is also a feature of TIBCO Business Events…