{"id":63,"date":"2008-02-29T15:04:29","date_gmt":"2008-02-29T20:04:29","guid":{"rendered":"http:\/\/haleyai.com\/wordpress\/2008\/02\/29\/cep-crossing-the-chasm-into-bpm-by-way-of-brms\/"},"modified":"2016-06-09T08:21:34","modified_gmt":"2016-06-09T13:21:34","slug":"cep-crossing-the-chasm-into-bpm-by-way-of-brms","status":"publish","type":"post","link":"http:\/\/haleyai.com\/wordpress\/2008\/02\/29\/cep-crossing-the-chasm-into-bpm-by-way-of-brms\/","title":{"rendered":"CEP crossing the chasm into BPM by way of BRMS"},"content":{"rendered":"<p>Complex event processing (CEP) software handles many low-level events to recognize a high-level event that triggers a business process.\u00a0 Since many business processes do not consider low-level data events, BPM may not seem to need event processing.\u00a0 On the other hand, event processing would not be relevant at all if it did not occasionally trigger a business process or decision.\u00a0 In other words, it appears that:<\/p>\n<ul>\n<li>CEP requires BPM but<\/li>\n<li>BPM does not require CEP<\/li>\n<\/ul>\n<p>The first point is market limiting for CEP vendors.\u00a0 Fortunately for CEP vendors, however, most BPM does require event-processing, however complex.\u00a0 In fact, event processing is perhaps the greatest weakness of current BPM systems (BPMS) and business rules management systems (BRMS), as discussed further below.<!--more--><\/p>\n<h3>BPM usually requires event processing<\/h3>\n<p>BPM involving workflow or business to consumer (B2C) interaction is frequently driven by events, however.\u00a0 For example, consider:<\/p>\n<ol>\n<li>a task being completed,<\/li>\n<li>an activity, communication, or response by a consumer,<\/li>\n<li>or, more interestingly, a lack of activity over or after a period of time<\/li>\n<\/ol>\n<p>For example, consider each of the following governing policies:<\/p>\n<ol>\n<li>an application taken by order entry should be forwarded to underwriting<\/li>\n<li>a postal letter received from a customer must be scanned and handled as a fax letter from the customer<\/li>\n<li>a letter warning of imminent collection should be sent to the primary customer on a delinquent account by certified mail if no communication with the customer has occurred within the last thirty days<\/li>\n<\/ol>\n<p>Clearly, managing business processes that involve people requires event processing.<\/p>\n<ul>\n<li>CEP processes low-information events, potentially in high-volume.<\/li>\n<li>BPM processes events that are more directly relevant to the business.<\/li>\n<\/ul>\n<h3>Event-driven inadequacies of BPMS and BRMS<\/h3>\n<p>Note that it is a frequent exaggeration to say that BPM processes events at all.\u00a0 For the most part, BPMS\u00a0 require analysts to define a process to be invoked when events are signaled by program code that has to written and inserted into other systems.\u00a0 BPMS do not typically externalize operational governance and compliance policies as stated above as BRMS do.\u00a0 And, since neither BPMS nor BRMS handle events that arise from a lack of activity over a period of time, the analysis and systems infrastructure to trigger processes arising from a lack of events commonly results in extraordinary development and maintenance costs.<\/p>\n<p>Having pointed this out, let us continue with understanding the relationship between business process management and event processing, however complex.<\/p>\n<h3>Does CEP overemphasize complexity?<\/h3>\n<p>Complexity in event processing can come from an onslaught of simple events, as in telecommunications or computer networks, where individual events typically have little relevance.\u00a0 When combinations or correlations of such events may be relevant, however, event processing complexity obviously increases further.\u00a0 CEP specifically addresses the onslaught of events and the processing necessary to distill raw data into more relevant and typically less frequent information events.<\/p>\n<p>Examples of such CEP include:<\/p>\n<ul>\n<li>alarm management in telecommunications and computer networks<\/li>\n<li>security monitoring in telecommunications and computer networks<\/li>\n<li>performance monitoring in computer systems<\/li>\n<li>algorithmic trading of securities<\/li>\n<\/ul>\n<p>In addition to distilling data into information, complexity can arise in how the event is handled.\u00a0 For example, if a five minute trend indicates that a stock is free-falling, what action should be taken?\u00a0 Or if an authentication for a mailbox has been failing every ten minutes, what should be done?<\/p>\n<h3>CEP or BPM decisions<\/h3>\n<p>This is were the handling of events crosses the bridge from processing, however complex, into a business process that may be as simple as a single decision about how to respond and then invoking the selected process.<\/p>\n<p>That is:<\/p>\n<ol>\n<li>event processing triggers business processing<\/li>\n<li>business processing involves decision making<\/li>\n<li>decision making requires a business rules engine<\/li>\n<\/ol>\n<p>Or course, if the decision making is trivial, an event processing platform may suffice.\u00a0\u00a0 Few business processes have no decision complexity or workflow, however.<\/p>\n<h3>GRC and EDM<\/h3>\n<p>Enterprises necessarily manage their decision making from governance, risk, and compliance (GRC) perspectives and with an eye to improving business performance with experience and over time.\u00a0 To implement such enterprise decision management (EDM), BPMS typically integrate with BRMS which manages externalized GRC and decision making policies that the BRMS automates using a business rules engine (BRE) that is integrated with a BPMS.<\/p>\n<p>To put all this simply, assuming CEP is not overkill for simple events:<\/p>\n<ul>\n<li>CEP triggers BPM that uses a BRMS for EDM<\/li>\n<\/ul>\n<p>Note that decisions are frequently externalized from business processes by embedding BRE in service-oriented architectures (SOA) that are invoked by BPMS.<\/p>\n<h3>CEP as BPMS niche<\/h3>\n<p>So CEP may appear to be a bolt-on that extends BPM to handle events (or the lack of events) more easily and competently.\u00a0 CEP vendors are not happy with this view, since it limits their market to a BPM add-on.\u00a0 As a result, CEP vendors like TIBCO are moving down the stack to incorporate BPM functionality.\u00a0 On the BPMS front, some smaller vendors will move to incorporate CEP to increase their market share in a defined niche.\u00a0 With few exceptions, one of which is discussed below, BPM market leaders are less motivated to incorporate CEP functionality into their BPMS precisely because CEP is a relative small market.<\/p>\n<h3>Where CEP = BPMS + BRMS<\/h3>\n<p>As discussed above, CEP addresses niche markets where the volume or complexity of event processing can be bolted-on to a typical business process.\u00a0 But what if high-volumes of events are directly relevant to a business, such as in algorithmic trading at a hedge fund?\u00a0 BPMS cannot handle the event stream and CEP cannot handle the BPM or the complexities of decision making address by a BRMS.<\/p>\n<p>Every CEP vendor recognizes and targets the securities industry for a substantial share of their revenues. \u00a0More specifically, every CEP vendor targets algorithmic trading.\u00a0 The amount of capital invested algorithmic trading alone is substantial enough that the word niche seems inappropriate. \u00a0\u00a0The market for CEP on Wall Street alone is enough to motivate BPM market leaders to incorporate CEP in their offerings.\u00a0 On the other hand, the need for CEP, not BPM, is so specific and intense in algorithmic trading that some CEP vendors can justify focusing on the application rather than moving down the stack to incorporate BPM functionality.<\/p>\n<ul>\n<li>This contrast seems most apparent between Apama and TIBCO.<\/li>\n<\/ul>\n<p>Whether or not CEP vendors move to incorporate BPM functionality or focus on algorithmic trading, CEP consumers need the technology to support decisions.\u00a0 As CEP becomes more capable in GRC and EDM, it will also have to become more usable by less technically skilled people.\u00a0 <strong>CEP could accomplish all this by following BRMS&#8217; lead.<\/strong><\/p>\n<h3>Mainstream CEP<\/h3>\n<p>Twenty five years ago, rules where written in technical languages.\u00a0 Today, BRMS such as the one we developed at Haley, express polices &#8211; <a href=\"http:\/\/haleyai.com\/wordpress\/2008\/01\/03\/when-rules-meet-requirements\/\">not just rules<\/a> &#8211; in natural language.\u00a0 BRMS would not have <a href=\"http:\/\/haleyai.com\/wordpress\/2007\/12\/13\/business-rules-market-fud\/\">crossed the chasm<\/a> without these advances.\u00a0 CEP needs to do the same.\u00a0 The most likely path for CEP to cross the chasm is to follow BRMS example.<\/p>\n<p>In CEP, there are <a href=\"http:\/\/www.unix.com\/blogimgs\/EVENT.PROCESSING.LANGUAGE.SURVEY.V14.OCT.15.2006.pdf\">two approaches<\/a>.<\/p>\n<ol type=\"1\">\n<li>Extend SQL with syntax related to streams and time windows\n<ol type=\"a\">\n<li><a href=\"http:\/\/www.streambase.com\/products-streamsql.htm\" target=\"_blank\">StreamSQL<\/a> from Streambase<\/li>\n<li><a href=\"http:\/\/www.coral8.com\/blogs\/blog-entry\/cep-and-sql-top-five-myths\">Continuous Computation Language<\/a> from Coral8<\/li>\n<li>BEA <a href=\"http:\/\/dev2dev.bea.com\/blog\/aalves\/archive\/2007\/08\/cep_rules_engin.html\">AquaLogic<\/a> (recently acquired by mainstream BPM vendor IBM)<\/li>\n<li><a href=\"http:\/\/dist.codehaus.org\/esper\/NYJavaSIG_May_30_2007.pdf\">EQL<\/a>, the event-processing language (EPL) of open-source Esper<\/li>\n<\/ol>\n<\/li>\n<li>-Define a Java-like programming language for processing events\n<ol type=\"a\">\n<li>Monitor EPL from Apama<br \/>\n(documentation evasive, but examples <a href=\"http:\/\/rulecore.com\/CEPblog\/?p=298\">here<\/a> and <a href=\"http:\/\/apama.typepad.com\/my_weblog\/2008\/02\/apama-monitorsc.html\">here<\/a>)<\/li>\n<li>TIBCO Business Events<br \/>\n(documentation and examples conspicuously lacking?)<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<p>As with business rules prior to the release of Innovator by Blaze Software (now part of Fair Isaac) and Authority from Haley, the esoteric technical skills required by these CEP languages will <a href=\"http:\/\/www.forrester.com\/Research\/Document\/Excerpt\/0,7211,44418,00.html\" target=\"_blank\">prevent CEP from crossing the chasm<\/a> into more mainstream use, as BRMS has accomplished for business rules and BPMS for processes.<\/p>\n<h3>Making CEP more usable<\/h3>\n<p>I thank and apologize to Coral8, who are exemplary in their product coverage, for using an example of their <a href=\"http:\/\/www.coral8.com\/system\/files\/assets\/pdf\/Coral8TechnologyOverview.pdf\">excellent technical overview<\/a> to demonstrate a point.\u00a0 In section 5.1 they show following stream SQL:<\/p>\n<blockquote><p>Insert Into StreamAvgPrices Select Symbol, Avg(Price) From StreamTrades Keep 5 minutes Where Volume &gt;= 10000 Group By symbol<\/p><\/blockquote>\n<p>Personally, I think this is one of the nicer examples of the stream \/ SQL approach.\u00a0 The code that I have seen from the Java approaches seems more difficult to implement.<\/p>\n<p>Superior CEP expression though it may be, this is not a form of expression that CEP market analysts such as <a href=\"http:\/\/www.forrester.com\/Research\/Document\/Excerpt\/0,7211,44418,00.html\" target=\"_blank\">Forrester<\/a> or BRMS vendors would consider for crossing the chasm.\u00a0 Much more appropriate would be:<\/p>\n<blockquote><p>the average price of a stock for trades over 10,000 shares over the last 5 minutes<\/p><\/blockquote>\n<p>For those who are familiar with my work in BRMS, you can understand how the latter is completely equivalent to the former.\u00a0 I believe, as do BRMS analysts, that such or similarly non-technical expression is a practical market requirement for crossing the chasm.\u00a0 Ilog and other vendors&#8217; approach of authoring pseudo-English statements would clearly be superior to EPL code.<\/p>\n<p>Note that EPLs do not express business processes or address decision making at all.\u00a0 The EPL above corresponds to a noun phrase, not a statement of policy, process, or action.<\/p>\n<h3>Understanding CEP and BPM together<\/h3>\n<p>Although most BRMS do not understand time or what it means to say &#8220;over the last 5 minutes&#8221;, CEP and BPM require a deeper understanding of time, as discussed <a href=\"http:\/\/haleyai.com\/wordpress\/2008\/02\/19\/understanding-events-and-processes-takes-time\/\">here<\/a> in a prior post. \u00a0Given such an understanding and an <a href=\"http:\/\/haleyai.com\/wordpress\/2008\/02\/22\/rules-are-not-enough-knowledge-is-core-to-reuse\/\">ontology<\/a> based on <a href=\"http:\/\/www.fixprotocol.org\/what-is-fix.shtml\">FIX<\/a> or otherwise covering stocks, trades, shares, and other concepts relevant to algorithmic trading, noun phrases like the one above can replace EPL.<\/p>\n<p>Assuming EPL is replaced by noun phrases, the lines between CEP and BPM fall.\u00a0 Such noun phrases will be used within sentences, obviously!\u00a0 Such sentences will constitute the GRC and decision making policies and knowledge involved in EDM and BPM.\u00a0 The CEP will fall out from the same content that BRMS now manage.<\/p>\n<h3>Eventually Eventful BPM<\/h3>\n<p>CEP will cross the chasm but its distinction may also fade as knowledge management becomes formal enough to automate requirements, policies, and rules as they are managed using BRMS today.\u00a0 As BRMS become increasing semantic and logically capable, they will also encompass more and more of BPM complexity (e.g., GRC).<\/p>\n<p>And as BRMS and BPM converge with architectures and ontologies that cover long-running processes that involve workflow and events, EPL will be generated automatically just as Haley Authority now generates technical rules from knowledge managed by stakeholders and subject matter experts who have far less technical skill than CEP currently requires.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Complex event processing (CEP) software handles many low-level events to recognize a high-level event that triggers a business process.\u00a0 Since many business processes do not consider low-level data events, BPM may not seem to need event processing.\u00a0 On the other hand, event processing would not be relevant at all if it did not occasionally trigger &hellip; <\/p>\n<p class=\"link-more\"><a href=\"http:\/\/haleyai.com\/wordpress\/2008\/02\/29\/cep-crossing-the-chasm-into-bpm-by-way-of-brms\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;CEP crossing the chasm into BPM by way of BRMS&#8221;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4,72,7,9,13],"tags":[33,60,62,65,880,894,78,881,86,882,112,123,124,125,150,152,313],"class_list":["post-63","post","type-post","status-publish","format-standard","hentry","category-business-process-management","category-business-rules-management","category-complex-event-processing","category-decision-management","category-knowledge-management","tag-algorithmic-trading","tag-bpm","tag-bpms","tag-brms","tag-business-process-management","tag-business-rules-management","tag-cep","tag-complex-event-processing","tag-compliance","tag-decision-management","tag-edm","tag-event-processing","tag-event-driven-processing","tag-events","tag-governance","tag-grc","tag-workflow"],"_links":{"self":[{"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/posts\/63","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/comments?post=63"}],"version-history":[{"count":1,"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/posts\/63\/revisions"}],"predecessor-version":[{"id":855,"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/posts\/63\/revisions\/855"}],"wp:attachment":[{"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/media?parent=63"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/categories?post=63"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/tags?post=63"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}