This weekend there was an article in a Sunday newspaper for an “intelligent smoke detector” - one that not just sets off an audible alarm, but also sends a message to some registered phones. And last week the BBC reported on a CISCO prediction on the “proliferation of … connected appliances and other smart machines”, and 15Bn internet-connected devices passing 966 exabytes of internet-traffic by 2015.

These predictions are significant: although many of us use multi-connected devices (phone, PC, etc), many of the increase in connectivity will be for machine to machine (M2M) devices, capturing events and passing them on to be aggregated and have business rules and policies applied to context-rich information in real-time (think things like smart grid energy usage, as well as domestic smoke detectors and their ilk).

From the CEP perspective, complex event processing is needed to actually monitor, digest and control all this new data (as events or event streams) and devices. And I’ve heard that one large computer networking company will be presenting on their use of CEP at the TIBCO user group (TUCON) later this year…

I see that last year’s Dagstuhl workshop report has been published, titled “Executive Summary and Manifesto — Event Processing”. This is an excellent 60-page group effort by some of the great-and-the-good in event processing, and recommended reading!

The top-level Table Of Contents is:

Chapter 1: Why event processing?
Chapter 2: What are the characteristics of event processing?
Chapter 3: Synergies and relations to other areas
Chapter 4: Event processing related standards
Chapter 5: Grand challenge: The global event processing fabric and its applications
Chapter 6: Near-term research

Disclaimer/disclosure: TIBCO was a co-author of Chapter 4.

EbizQ is running a series of articles (first here and second here, curiously classified under “Social BPM“) on “dynamic BPM”, quoting luminaries such as Gartner’s Janelle Hill and Jim Sinur, and Forrester’s Clay Richardson.

In the simplest terms, [dynamic BPM] supports rapid “on the fly” process adjustments. “You can respond to emerging conditions and changing business needs, in some cases without any interruption to IT or without having IT get involved,” explains Jim Sinur…

Dynamic BPM is described as having various levels of complexity:

• rule/decision-based selection of processes / process paths - see (good old) decision management / rule technology
• dynamic configuration of parts of processes - usually a variation of the above, with decision rules determining what tasks to do and in which order
• goal-driven processes - selection of processes and process tasks is entirely based on some mechanism for addressing a route to a goal

So lets look at some of the examples mentioned:

• railroads using RFID sensors to monitor passing traincars (a.k.a. “sense and respond”)
• complex algorithms to determine where to deploy troops (normally requiring “situation awareness”)
• government document tracking (a.k.a. “track and trace”)
• pharmaceutical and fragrance design process involving ad hoc partner participation (i.e. “social BPM”)

For sure, CEP and event processing is at least a “major contributor” and in reality probably the solution for the “sense and respond” and “track and trace” application areas. And it helps with “situation awareness” in the real-time domain.

What is fascinating is that the term “dynamic BPM” seems to really mean “any type of process that is not associated with simple, fixed task orchestration” - in other words,  “non-BPM” technologies like CEP.  One perspective is that one of the main drivers for processes to be “dynamic” is the ability to respond to change - i.e. events. Indeed, the need is for many business processes to take account of there being larger quantities and higher rates of events that can be deemed important. Another is to use more sophisticated solutions to solve or optimise more complex problems - think of classical planning and scheduling algorithms and predictive analytics, for example.

Probably the “dynamic BPM complexity levels” described in the article can be mapped to the following, based on increasing levels of event-handling capability:

• pre-defined, late-selected process paths: subprocesses are pre-defined
• no-predetermined-path processes, where rules or more complex algorithms determine what tasks need to take place - sets of tasks are allocated based on conditions and events: tasks are predefined
• goal-oriented and semantic processes, where goal states are reached by the dynamic selection of tasks based on current events and situation: tasks may be constructed on-the-fly, and processes and plans changed according to the situation and events.

There are other viewpoints as well - for example, adaptive case management (for non-automated, changeable processes), currently the subject of an OMG standardisation effort. Related to things like goal-driven fulfillment technologies like TIBCO AF and AC. Perhaps we are returning to - dare I say it - knowledge-based processes ?

I leave the final words to a quote:

…according to Gartner Inc: “By 2013, dynamic BPM will be an imperative for companies seeking process efficiencies in increasingly chaotic environments.”

It’s an interesting observation that the largest suppliers of database technology have also dipped their toes into CEP technology (although some with more enthusiasm than others it seems) - Oracle, Microsoft, IBM, Sybase… and maybe even Teradata. Most of these have naturally gone for event stream processing rather than other variants of CEP, utilising SQL-like query languages. But are they really just hedging their bets against the possible replacement of “big iron” databases with something more 21st Century, like in-memory datastores?

I was reminded of this when reviewing some of the features of TIBCO ActiveSpaces - TIBCO’s “big data” offering. Performance-at-network-speed - check. Persistence-support-for-failover - check. Reliance-on-DBAs-and-Big-Vendors - nope. Everything-done-via-SQL - nope. Transactional-integrity - OK, this is where things get interesting. Why run your transactions through a *file store* (a.k.a. traditional database)? Are not real-time reliable in-memory transactions possible some other way? Of course.

And it seems even DB afficionados are beginning to question the roles databases play in enterprises…

TIBCO has a virtual booth at the MWD Advisors Process Intelligence virtual event, covering TIBCO ActiveMatrix BPM Spotfire (which might also win the award for the longest product name) - which does visual and predictive analytics on your process information - and TIBCO BusinessEvents (i.e. CEP) - which does real-time event analytics, monitoring, and dynamic process control…

The MWDA webinar (in 2 parts) is quite interesting and worth a look: their model of Process Intelligence is layered in 3 parts:

• strategy level: goals, motivations, metrics
• process level: BPMN, CEP patterns, analytic models
• reaction level: decision rules and analytics

From a TIBCO perspective, ActiveMatrix BPM and ActiveMatrix BPM Spotfire cover the metrics, process models and analytics side, with BusinessEvents covering the real-time metrics, dynamic processes, CEP, event-based analytics and decisions. One interesting possibility is to use the BusinessEvents’ state models for strategic goal planning too.

Meanwhile, the Gartner guys are also pushing the “Business Process Improvement” button, releasing a report threatening doom and gloom for organisations who do not control their processes. They talk about:

• undetected but detectable process errors: this being the case for process monitoring and analytics, presumably. Either that or good old business analysis!
• “context awareness” to rejuvenate commodity processes: context being of course state, or what events tell you!

Note that the MWD event sponsors other than TIBCO (a CEP company) are Progress (also mentioning CEP capabilities) and SoftwareAG (again,  also mentioning CEP capabilities). So it could be an interesting year for BPM-CEP convergence?