Events versus Data
Posted by Paul Vincent
A recent question came up regarding the difference between “events” and “data”. After all, to a DBA, an “event” is just another data record but with a timestamp! Of course, that is the “static” view of an event - useful only for archiving and “post (-event) processing” activities like data mining, BI and analytics.
However, a more technical answer is that data is derived from events. Customer data? Derived from customer registrations and associated checks - all events. Amount owed? Derived from purchase events. And so forth. So a better answer for the DBA is that “data” is just “information collected from, and associated with, prior events“.
In Complex Event Processing, of course, we are processing this “event data” as it occurs (within the limitations of our system latency, of course!). Typically this event data is correlated with previous events which can be either “relatively recent” (and considered as stored events or event objects) or “historic” (and considered as traditional data), although these are necessarily imprecise concepts. Access to historic data is typically carried out in CEP tools through good-old-database-services: calling the operational data store associated with legacy systems, for example. Another approach is to “event-ise” the legacy applications, extracting new events from logs and data sources as they occur.
TIBCO examples of the above approaches to data access in event processing are TIBCO BusinessEvents’ database interface (what we call DBConcepts) and TIBCO Adapters (e.g. for ERP and HR systems).
Notes:
The EPTS glossary does not define “data” but does mention that event objects contain data…
Tags: EPTS glossary
Complex Event Processing (CEP) | 12 March 2010 |
10 Comments
Other Links to this Post
RSS feed for comments on this post. TrackBack URI
By Richard Veryard, March 12, 2010 @ 04:26
Hi Paul, I agree that data are derived from events, but clearly that’s not the only way people want to understand data, and I don’t think the EPTS statement that “event objects contain data” is accurate or helpful. I think what underlies both events and data is the notion of difference. See my post on Enterprise as a Network of Events, which was partly triggered by something you posted last year on Events, Rules, Processes and Decisions
http://rvsoapbox.blogspot.com/2009/04/enterprise-as-network-of-events-2.html
By Peter Lin, March 12, 2010 @ 05:18
I think that’s an over simplification. Some data has no relation to real world things, while data about a customer captures information about the account. Clearly the data about an account changes over time, which some people think of as events, but that’s not the whole story. To me, it would be more accurate to see it as temporal versioning. As I understand it, current CEP definitions do not consider temporal versioning a first class concept. In the real world, we have messy things like bi-temporal and quad-temporal versioning where data is back-dated and future-dated.
It feels like the word “event” is being used as a hammer. my bias opinion.
By Paul Vincent, March 12, 2010 @ 07:09
Hi Peter - “temporal versioning”? I like it! FYI TIBCO BE does “temporal versioning” in that any concept can be specified to record history (of updates or changes) and yes, you can go to a particular version in time. We haven’t worked out how to go forward in time yet at the concept level, though, other than through rule-based “predictions”
Hi Richard - well indeed the original question posed was “what is the difference between events and data”, which contrasts with your supposition that events and data are “differences”! But I won’t argue against the notion that “state changes” (my interpretation of “differences”) are what drives systems - and their need for associated data.
Cheers
By Hans, March 12, 2010 @ 07:21
This sounds reasonable, but I’m not sure anyone will benefit from a deep ontological discussion of events vs. data. Are events moving data, or is data the result of static storage of event processing?
Meh - interesting, but a little too abstract to properly debate.
But… no matter how you think of events, the time stamp (and even just the natural ordering of the data) introduces many interesting patterns. There are new ways of partitioning the data (by time). There is the idea of state - conditions that go in and out of realization as we consider different sections of the events/data. There are plenty more patterns that come up in just ten minutes of thinking about it.
So even folks who can’t agree on the ontological discussion, can usually agree on the use cases where events and data differ.
By Neil Ward-Dutton, March 12, 2010 @ 14:07
I’m often caught saying that relational data (and other longterm stored data) are the “shadows cast by events and held for later inspection”. For many decades we (IT people) mainly relied on humans to capture events, work out when they were significant, and then to type information into a computer that then “fossilised” some abstraction/composition of events that had occurred over time.
Now we’re moving from the era of Data Processing (DP) to Event Processing (EP). Yes, longterm data storage still has a massive role to play. But EP technology enables us to complement human event processors with specialised technology versions in many scenarios.
I think “data is just data”. Electronic events are data. RDBMS rows are data. The way those data are processed and stored are different and complementary.
By Opher Etzion, March 13, 2010 @ 02:41
Hi Paul. Some thoughts about the touch points between event and data -
http://epthinking.blogspot.com/2010/03/on-events-versus-data.html
By Peter Lin, March 13, 2010 @ 08:11
Just to be clear, when you say concepts are versioned, does it also apply to event data? What about external data that’s not maintained by BE? I haven’t seen a CEP engine that provides a comprehensive and holistic model for handling temporal versioning. That includes bi-temporal all the way to quad-temporal versioning.
On a different, when will Tibco publish a white paper showing the scalability of BE with respect to rule count and dataset size. I’m thinking from 1-10,000 rules and 100-1,000,000 events/facts.
By Paul Vincent, March 13, 2010 @ 15:32
Hi Peter - Events can contain (and update our) information about entities in the world - and these are usually the things you want to know about history over time. So yes, event data is versioned. But no, versioning does not apply to the (immutable) events themselves.
PS: as to performance white papers… only a million facts? Heh heh…
By Paul Vincent, March 13, 2010 @ 15:36
Opher - I fully concur with your post, and esp “Thus the main issue is not the “versus” issue but the various relationships between the two terms. “.
Hans, Neil - yes, fully agree with your points too.
But… we will continue seeing business IT guys, bought up on a diet of “data processing” as the one-and-only IT pattern, wanting to explore this question.
Cheers
By Peter Lin, March 14, 2010 @ 15:18
A million facts in the working memory should be a breeze for en engine built for processing data streams or event streams. I look forward to a performance white paper from Tibco.