A Solution Architect posed an interesting question over on the Complex Event Processing LinkedIn discussion board run by Tim Bass, which I reproduce here as it references TIBCO CEP:

In my case I have to think as to monitoring/governing a EDW (enterprise data warehouse) where we have a classic approach (host systems (in different legal entities of the Group) -> staging area -> edw -> dm) that can be thought as a state machine but we have also a lot of Human tasks thinking about data quality processes.
So, my reasoning have to be .. If I have human tasks then I have to build a BPM and using a BAM/CEP in order to control our environment?
In any case could BAM+CEP (re -thinking about Progress or Tibco solutions) without BPM solve my problem?
(just a little detail, my BPM is Websphere Process Server) …

Of course there are multiple terms and technologies in play here:

1. data cleansing and validation - typically rule based (technology: BRE; product: TIBCO BusinessEvents), and often-times requires manual intervention (technology: BPM workflow; product: TIBCO iProcess); note that this could also be considered ETL…
2. data lifecycles - this could be considered a “state” issue (technology: state machine; product: TIBCO BusinessEvents), but is probably orthogonal to “data management” (technology: MDM; product: TIBCO CIM)
3. operational awareness - BAM over IT systems, monitoring their state usually via their state transitions (events) (technology: CEP; product: TIBCO BusinessEvents).

Interestingly, one of TIBCO’s technology tenets is “no rip n’ replace”, meaning customers (like this one) use TIBCO CEP with non-TIBCO BPM. And the need to use a BPM tool will depend on the need to manage the manual interventions (i.e. workflow) and the complexity of the tasks / process models - again a popular TIBCO technology mix is using “TIBCO BPM+” a.k.a. CEP driving BPM.

Of course, an interesting question is also whether the “new operational data” actually represents useful events that should be precessed as such en route to the EDW, for faster decisions…

The Business Rule Forum folk have just published their flier for submissions for their November conference [*1], and I noticed what seems to be a sly misprint (or a slight dig at the technology-focused Complex Event Processing community) in their call for customer case studies:

We also invite your presentation on one or more of these Special Focus Areas …
Business Rules, Decisioning and …
o Complex Event Management (CEP)
…

Most CEP tools claim some level of business ownership of the CEP process (for example, in TIBCO BusinessEvents we provide a decision management application together with a means of specifying decisions as decision tables). However, for the most part the simple events used by the application are read from some event bus or other event source: in other words, they are defined (and maybe managed) externally.  The abstract events being defined and interpreted by the CEP system are “managed” only in that the CEP tool may provide the associated state model, event flow or rules (in the appropriate event processing element) to define the abstract event, and these programming elements may possibly be controlled in a SCM system. Internal events and timer events are also defined inside the CEP application.

Is there a case for business-level or SCM-style management of events? Some end-user organizations believe so: a large credit card organization complained to me once that there was insufficient control over (and re-use in) message bus products, as they allowed developers unconstrained abilities to create overlapping message / event types in an organization requiring costly message/event bridges between applications [*2]. Perhaps the equivalent of MDM for messaging is required [*3] - I prefer “Master Event Management” or MEM over “Complex Event Management”. Would the business like to control the lifecycle and useage of their business messages or events, and maintain them in a Business User Interface? The latter would then tie in with CEP products, as various abstract events could then be published to the business’ Event Management system as potential external events for possible consumption by other systems (such as CEP-based BAM applications working across event-based applications…). It could also tie in with Governance and Business Policy / rule management…

Just a thought…

 Notes:

[1] Disclosure: TIBCO presented at the last year’s conference on Complex Event Processing…

[2] Of course, one could say that unconstrained development of messages is an internal IT control issue…

[3] Disclaimer: TIBCO CIM is an MDM product developed in the same group as the TIBCO BusinessEvents CEP product. Typically master data may be referenced in complex event processing, or master data change events may be useful input events for a CEP application.

With the inauguration of a new US President just around the corner, I’ve been ruminating on the forthcoming changes. It’s not an easy job to begin with, and since there’s so much work to do and he’s got a brand new team, I’m thinking President-elect Obama might be able to use Complex Event Processing and / or Business Activity Monitoring software.

Here’s how… (in the form of a top ten list)

Number 10

A new US Government operations visibility tool for the newly appointed Chief Performance Officer cabinet post.

Number 9

As a new Presidential dashboard with real power.

Number 8

Another reason to stick with his Blackberry - TIBCO CEP product BusinessEvents creates Blackberry alerts.

Number 7

Predict what Venezuelan President Hugo Chavez will say and do next.

Number 6

Choose the new Presidential dog based on historical breed information correlated with new puppy attributes and real-time events.

Number 5

Automatically rebalance his personal 401K before Bernanke announces FED rate changes.

Number 4

New Affinity rewards program for frequent taxpayers.

Number 3

Make sure nobody plays the overnight commodities market with the bailout money float.

Number 2

Keep an eye on Hillary. As an added bonus, Hillary can use it to keep an eye on Bill

And the number 1 reason

oBAMa

That’s what I think, anyway. How about you? Let us know.

Neil Raden blogged on Intelligent Enterprise about Process Intelligence, commenting that that “Operational BI, Pervasive BI, Operational Intelligence, Process Intelligence, BAM, Complex Event Processing, Decision Management and Decision Services” all overlap. This is almost certainly true, but some of these like CEP are technologies, and some like Process Intelligence are more applications of such technologies.

We might draw these together by saying Process Intelligence (insight and advanced behavior on the execution of business processes that are typically under the control of a BPMS)…

• supports Operational Intelligence (insight and advanced behavior on business operations, part of what we at TIBCO call Business Optimization)
• uses
  • operational (real-time / event-based) BI (reporting and dashboards),
  • BAM (monitoring business and process activities) and (other) CEP techniques on process events,
• may use
  • event / decision / change process management for business control of the processes,
  • statistical trending and analytics on histories of process events, for process refinement.

The above is an increasingly common use-case in fact, and not just restricted to individual BPMS implementations. Indeed, businesses can benefit from having a process monitor that is independent of their BPM tooling layer to give them an “independent view”of -all- types of processes and services, regardless of supplier and technology. Quite often such applications include not just event monitoring and pattern detection, but also the manual processes for interventions when required (which can be implemented as BPM). Event sources can be across the fulfillment or production process / value chain, and across department boundaries to give the organization the “big picture”.

Does this mean that “Process Intelligence” -equals- CEP? Of course not. But tools or applications to do this task are likely to embed or include some technology doing “complex event processing”, even if under a different name. And customers are certainly deploying these sorts of applications with TIBCO BusinessEvents.

Notes:

I see there is a conference / workshop on this subject (well of course there would be) - see here for the 2008 details.

Other papers include this one and this one - both dating from 2003.

Mostly of interest to existing Syndera users: TIBCO has released Syndera 3.3.0, which is now available from the TIBCO download site.

This release mostly consists of a “TIBCOization” of the real-time dashboard rather than new functionality per se. Or, if you prefer, Syndera has been “TIBbed”, “TIBCO’d”, or “TIBCOnified”. That means internal, license and packaging changes only.

More later.