Importing documents into a knowledge base

Many users land up wanting to import sentences in the Linguist rather than type or paste them in one at a time.  One way to do this is to right click on a group within the Linguist and import a file of sentences, one per line, into that group.  But if you want to import a large document and retain its outline structure, some application-specific use of the Linguist APIs may be the way to go.

We’ve written up how to import a whole web site  on California Sales & Use Tax:

Affiliate Transactions covered by The Federal Reserve Act (Regulation W)

Benjamin Grosof, co-founder of Coherent Knowledge Systems, is also involved with developing a standard ontology for the financial services industry (i.e., FIBO).  In the course of working on FIBO, he is developing a demonstration of defeasible logic concerning Regulation W of the The Federal Reserve Act.  Regulation W specifies which transactions involving banks and their affiliates are prohibited under Section 23A of the Act.  In the course of doing this, there are various documents which are being captured within the Linguist™ platform.  This is a brief note of how those documents can be imported into the platform for curation into formal semantics and logic (as Benjamin and Coherent are doing). Continue reading “Affiliate Transactions covered by The Federal Reserve Act (Regulation W)”

Knowledge acquisition using lexical and semantic ontology

In developing a compliance application based on the institutional review board policies of John Hopkins’ Dept. of Medicine, we have to clarify the following sentence:

  • Projects involving drugs or medical devices other than the use of an approved drug or medical device in the course of medical practice and projects whose data will be submitted to or held for inspection by the FDA will not be exempt from JHM IRB review UNLESS that use falls within the Emergency Use provisions of 21 CFR 56.102 (d).

As you can see, there are a number of compound words and acronyms, as well as references to the Code of Federal Regulations that need to be defined or recognized to understand this sentence.  Continue reading “Knowledge acquisition using lexical and semantic ontology”

Background for our Semantic Technology 2013 presentation

In the spring of 2012, Vulcan engaged Automata for a knowledge acquisition (KA) experiment.  This article provides background on the context of that experiment and what the results portend for artificial intelligence applications, especially in the areas of education.  Vulcan presented some of the award-winning work referenced here at an AI conference, including a demonstration of the electronic textbook discussed below.  There is a video of that presentation here.  The introductory remarks are interesting but not pertinent to this article.

Background on Vulcan’s Project Halo

Background on Vulcan's Project Halo

From 2002 to 2004, Vulcan developed a Halo Pilot that could correctly answer between 30% and 50% of the questions on advanced placement (AP) tests in chemistry.  The approaches relied on sophisticated approaches to formal knowledge representation and expert knowledge engineering.  Of three teams, Cycorp fared the worst and SRI fared the best in this competition.  SRI’s system performed at the level of scoring a 3 on the AP, which corresponds to earning course credit at many universities.  The consensus view at that time was that achieving a score of 4 on the AP was feasible with limited additional effort.  However, the cost per page for this level of performance was roughly $10,000, which needed to be reduced significantly before Vulcan’s objective of a Digital Aristotle could be considered viable.

Continue reading “Background for our Semantic Technology 2013 presentation”

Semantic Technology & Business Conference (SemTechBiz)

Benjamin Grosof and I will be presenting the following review of recent work at Vulcan towards Digital Aristotle as part of Project Halo at SemTechBiz in San Francisco the first week of June.

Acquiring deep knowledge from text

We show how users can rapidly specify large bodies of deep logical knowledge starting from practically unconstrained natural language text.

English sentences are semi-automatically interpreted into  predicate calculus formulas, and logic programs in SILK, an expressive knowledge representation (KR) and reasoning system which tolerates practically inevitable logical inconsistencies arising in large knowledge bases acquired from and maintained by distributed users possessing varying linguistic and semantic skill sets who collaboratively disambiguate grammar, logical quantification and scope, co-references, and word senses.

The resulting logic is generated as Rulelog, a draft standard under W3C Rule Interchange Format’s Framework for Logical Dialects, and relies on SILK’s support for FOL-like formulas, polynomial-time inference, and exceptions to answer questions such as those found in advanced placement exams.

We present a case study in understanding cell biology based on a first-year college level textbook.

Deep QA

Our efforts at acquiring deep knowledge from a college biology text have enabled us to answer a number of questions that are beyond what has been previously demonstrated.

For example, we’re answering questions like:

  1. Are the passage ways provided by channel proteins hydrophilic or hydrophobic?
  2. Will a blood cell in a hypertonic environment burst?
  3. If a Paramecium swims from a hypotonic environment to an isotonic environment, will its contractile vacuole become more active?

A couple of these are at higher levels on the Bloom scale of cognitive skills than Watson can reach (which is significantly higher than search engines).

As some other posts have shown in images, we can translate completely natural sentences into formal logic.  We actually do the reasoning using Vulcan’s SILK, which has great capabilities, including defeasibility.  We can also output to RIF or SBVR, but the temporal aspects and various things such as modality and the need for defeasibility favor SILK or Cyc for the best reasoning and QA performance.

One thing in particular is worth noting:  this approach does better with causality and temporal logic than is typically considered by most controlled natural language systems, whether they are translating to a business rules engine or a logic formalism, such as first order or description logic.  The approach promises better application development and knowledge management capabilities for more of the business process management and complex event processing markets.

Understanding English promotes better policies and requirements

Capturing some policies from a publication by the Health and Human Services department recently turned up the following….
It’s probably the case that there are more specific lists than just “some list” or “any list”, as suggested below.
This is a good thing about applying deep natural language understanding to policy statements.  It helps you say precisely what you mean, even if you are not using a rule or logic engine, but just trying to articulate your policies or requirements clearly and precisely.

HHS on expedited review

Logic from the English of Science, Government, and Business

Our software is translating even long and complicated sentences from regulations to textbooks into formal logic (i.e,, not necessarily first-order logic, but more general predicate calculus).   As you can see below, we can translate this understanding into various logical formalisms including defeasible first-order logic, which we are applying in Vulcan’s Project Halo.  This includes classical first-order logic and related standards such as RIF or SBVR, as well as building or extending an ontology or description logic (e.g., OWL-DL).

We’re excited about these capabilities in various applications, such as in advancing science and education at Vulcan and formally understanding, analyzing and automating policy and regulations in enterprises.

English translated into predicate calculus

English translated into predicate calculus

a sentence understood by Automata

an unambiguously, formally understood sentence


English translated into SILK and Prolog

English translated into SILK and Prolog

Rules vs. applications of knowledge

I was just asked for some background on business rules and the major players, preferably in the form of videos. The request came in by email, so I didn’t have the opportunity to immediately ask “why”.   Below I give some specific and direct responses, but first a few thoughts about clarifying objectives.

I don’t know of any video that is particularly good from an executive overview standpoint concerning “business rules” or even “decision management” let alone “management of active knowledge”.    My recommendation is to clarify the objective before drilling into “business rules”, which is a technical perspective.  What is it that you are trying to accomplish?  Most abstractly, it could be to manage and improve performance of an activity or an organization.  That kind of answer or focus is the right place to start and then work backwards to the technical approach rather than start with an inadequately conceived technical need.  This is one of the major problems with business rules as an independent market or product line.

Learning from Enterprise Decision Management

While at Fair Isaac, James Taylor saw this clearly.  He articulated the enterprise decision management (EDM) and positioned the business rules capability Fair Isaac acquired with Blaze Software in that space.  That is, more as a strategic objective than as a tool or technology.  This is an example of the proper way to think about business rules.

The decision management perspective was also narrowly focused on point decision making (e.g., using rules) but James and others (e.g., John Lucker of Deloitte) have appropriately expanded the strategy of decision management to include analytics, which produce and inform decision making (i.e., business rules), into a continuous process not of point decision making, but more closed-loop, continuous process improvement.  Over recent years, this has evolved into the broader market of performance management, which also includes performance optimization.

The key thing to consider when considering inquiries about “the applications and market for business rules” is the applications of knowledge.  The “knowledge engineering” community is often too focused on the sources of knowledge.  Focusing on sources rather than opportunities and benefits is a big part of why the business rules market has been subsumed into the business process management market, which is small in comparison to the business intelligence market, the fastest growing segment of which is performance management.

Semantic enterprise performance optimization checklist:

Here’s a checklist to consider when framing your considerations of strategies and tactics that might involve business rules technology:

  1. What knowledge (including policies, regulations, objectives, goals) is involved?
  2. What knowledge is superficial (i.e., derived from or approximations of) versus deeper knowledge?
  3. Will you capture the motivation for a decision rather than how that decision is made using rules?
  4. How will the performance  of your decision management or governance system be evaluated?
  5. Is the knowledge involved in evaluating such performance part of the knowledge that you will capture and management?
  6. How does the manner of evaluation relate to goals and objectives and over what time frames?
  7. Is the knowledge about goals and objectives time frames part of the knowledge to be managed?
  8. Are your decisions rigidly governed in every aspect or do you need the business process to include experimentation and optimization?

Most business rules efforts are focused on contexts so narrow that they are reduced to technical buying criteria without much or any consideration of the above.  That is, most business rule efforts do not even cover point 1 above.  Few reach bullet 2 and only strategic thinkers get to the third.

Specific recommendations for the naive question:

So I went off looking for videos…  You can find some on technical matters involving IBM/Ilog but I didn’t find any good videos from IBM at the business strategy level which concerned knowledge-based process/decision management/governance, which surprised.

A video from the vendors of Visual Rules touches on many of the traditional buying points that IT people typically formulate before evaluating vendors (here).

Although it did not respond to the inquiry, I sent along this video of James’ since it touches on so many of the aspects beyond business rules that are increasingly in vogue, even if it does not go far enough towards things like the business motivation model and the market for performance management, imo.

And for a very thorough background in the form of an analyst presentation that is consistent with all of the above, John Rymer of Forrester is most thorough in the two longer presentations that are here and there.

Please send me any other content that you would recommend!

Paul