I just ran across this interesting conversation and thought I’d chime in. I am not familiar with your specific CEP products. It sounds like they are indeed trying to provide situation awareness to their users. If users of the systems are complaining that they do, it most likely because either the information they really need is not there, or it is being communicated and presented in a manner that is sub-optimal. To overcome these difficulties, product vendors may want to check out “Designing for Situation Awareness” (Endsley, Bolte & Jones, 2003), available on Amazon.com. This book provides a 3 stage process for helping to ensure that your systems are effectively delivering this much needed commodity. It also includes 50 design principles, based on over 20 years of research findings, on how to present the information effectively. It was written for exactly this type of situation, to translate the research base into practical solutions that engineers need.

On the other points, unfortunately expert systems (or automation or AI) does not necessarily enhance SA. In fact, the research shows it can subtly undermine in some cases. The idea of using advanced algorithms (of any type) to help provide comprehension (level 2 SA) and projection (level 3 SA) is definitely a good one. However much care needs to be taken to make sure that many pitfalls are avoided and that the information is really being integrated effectively. This also is covered in the book for those who are interested

cheers,

Mica Endsley

“Keep focusing on solving your customers problems, and don’t be distracted by this pedantic nonsense.”
I’m pretty sure we haven’t lost any focus on our customers! Nontheless, we find that CEP applications are very often used for operational intelligence roles that overlap with situation awareness concepts, making situation awareness a reasonable topic…

Some other interesting links on Situation Awareness:
- Stephen Few on SA Dashboards from his Visual BI blog
- Situation Awareness 101 for management from CIO.com

Cheers

This is all very interesting, but what’s the point? It reminds me of the early days of relational databases when DBMS’ struggled to meet all of Codd’s “rules.” We ended up with 20 years of stagnant, stale lookalike database systems and have only recently broken out of it.

Like the late, great software visionary Mao Zadong said, “I don’t care if a cat has a tail or not, so long as it catches mice.”

Keep focusing on solving your customers problems, and don’t be distracted by this pedantic nonsense.

-NR

peter

“partitioning the rules into smaller manageable sets works in some cases, but not all. Ideally, a large ruleset should be divided into small manageable sets regardless of whether agents are used.”
Yes, normal rule development best practices apply…

“System that compile rules down to pure java code run into JVM PermGen memory issues. Having seen the PermGen issue crop up in rule engines, servlet containers and ejb containers, it’s an important consideration for enterprise applications.”
Note that decisions (that in BE map from its DecisionManager to rulefunctions (aka procedures / sequential mode code) are optimized (for exclusive rulesets) so that you can get say 50K row decision tables into a small amount of code. And of course, distibuting across multiple agents helps here too

“Does BE have the option of running in interpreted mode?”
Not in the sense of a “rule interpreter”, no.

Full documentation is available to signed-up partners at download.tibco.com, of course.

Cheers

System that compile rules down to pure java code run into JVM PermGen memory issues. Having seen the PermGen issue crop up in rule engines, servlet containers and ejb containers, it’s an important consideration for enterprise applications. For example, some rule engines can’t load 20K rules without increasing the PermGen setting to 512Mb. Loading 100K rules would require using 64bit JVM from Sun or IBM.

Does BE have the option of running in interpreted mode?

thanks

peter

On your example, note that typically in BE:
- each message type is typically handled by at most one agent type.
- the agent will consume (filter / initial transforms and processing) the event and share the event information (via the persistence agents / data grid)
- shared event information is then used by any other / multiple rule (and query) agents.

Clearly synchronization (/verification of duplicate rule conditions across agents) needs to be taken care of: but using the data grid as the blackboard seems to work well. In other words, you start with the shared fact model / view when partitioning your rules into separate agents.

In your example: if you add new fact (object) types in your 2 new agents that are not known (or used) by the 4 existing agents, then this is not a big deal (*their* rules will obviously ignore any data they don’t reference). Or if you have added more classification information via subclasses, the rules operating on the superclass will still operate. But you could then selectively update the 4 agents with new / updated rules to use the new information as you need…

FYI the Agent Metamodel and Profile group in OMG under Jim Odell is doing some interesting work on agent organization…

Back on topic: the distributed data grid acts as the application fact base, including history, for “situation representation” and, through rule-based interpretation, “situation awareness”. Conceptually and in software…

Cheers

Say I start up 4 agents with the object model and 4 different rulesets. At T2 some time later, I start up 2 agents with a new model, the communication between the first 4 agents and the second set run into issues. Communication from the first 4 to the second 2 are fine, but communication from the 2 new agents to the 4 initial agents may run into issues if they don’t have those object definitions.

I agree that’s why prolog style systems are used for AI and machine learning.

thanks.

peter

When was the last time a TIBCO customer announced a “real-world problem being solved by a CEP vendor”?
For customer statements on CEP, using CEP software like TIBCO BusinessEvents, see previous blog entries and TUCON announcements. We’ve had a few interesting projects in the last few months and we’ll have to see what and when we can release as case studies. For a Situation Awareness case study from last year consider the Airline Operations example mentioned in this past post on TUCON 2008 …

The “CEP vendors” are doing very little to solve customer problems because “CEP vendors” are selling little more than rule-based engines
Well, you could consider some of the pattern matching mechanisms in, for example, pure ESP tools as “rules”: but I’m not sure all CEP tools qualify as “rule engines” per the standard use of the term. See this past post on the differences between BREs and rule-driven CEP - Part 1 and - Part 2 for a discussion on the differences between CEP tools and rule engines per se. Also, I’m surprised you believe CEP vendors are “selling” while doing “very little to solve customer problems” - it would be interesting to see your data (if any) about any CEP tools being mis-sold, which this statement implies.

…which most customers can solved with PERL scripts, or open source rules engines like Drools.
Notwithstanding the fact that PERL or rule engines are not the same as event processing platforms doing CEP, this statement would seem to be contradicted by the large year-on-year increase in the CEP software tools market. As ultimately all software compiles down to machine code, and these 2 tools can be extended, the statement may be intrinsically correct - but not very useful (you don’t see many people writing databases or BPM tools in PERL either).

Note: Interestingly, we’ve had at least one good BusinessEvents application come from a user “experimenting” with open source rules technology and determining that their requirements need to be solved by BusinessEvents.

Cheers