In combination, we demonstrate how the following types of reasoning can be easily incorporated into intelligent agents:<\/p>\n
- \n
- textual entailment<\/a><\/li>\n
- resolving metonymy<\/a><\/li>\n
- abductive inference<\/a><\/li>\n<\/ul>\n
Other posts and pages address these topics and the use of natural language knowledge acquisition, representation, and reasoning more formally.\u00a0 This is a casual, introductory post aimed at making the lower end of the intelligent agent spectrum (i.e., chatbots) smarter.<\/p>\n
- \n
- if you are more interested in policy automation, decision management, or compliance, you might be more comfortable starting with this introduction to logical English<\/a> or the 2-part confessions of a production rule vendor<\/a>.<\/li>\n<\/ul>\n
Simple Knowledge Acquisition<\/h3>\n
For simple sentences, natural language processing (NLP) systems can frequently produce knowledge representation that is sufficient for effective question answering beyond chatbots.\u00a0 For example, the following results immediately from parsing a simple sentence.<\/p>\n
![\"\"](\"http:\/\/haleyai.com\/wordpress\/wp-content\/uploads\/food-products-do-not-include-medinces.jpg\" "\\"food")
<\/a><\/p>\nThis knowledge representation is much simpler but not much less practical than fully-specified predicate calculus.\u00a0 The important thing is that it gives us a structure representation that we can work with using the machinery of logic programming and notions such as textual entailment.\u00a0 If you\u2019re interested in the details and how to go further towards \u201caxiomatic knowledge\u201d, please see the notes. [1]<\/sup><\/a>[2]<\/sup><\/a>[3]<\/sup><\/a><\/sup> The Linguist<\/a> <\/span>can make many reliable deductions not shown above, but let\u2019s stick with this most simple use case for purposes of this post.<\/sup>[4]<\/sup><\/a><\/p>\n
The most basic question is what the sentence means.\u00a0 Then we can consider how its meaning is to be used in reasoning (e.g., to answer a question, support a decision, etc.).<\/p>\n
Our common sense tells us that the intended meaning here is that food products and medicines do not intersect (i.e., they are disjoint; having no common members).<\/p>\n
- \n
- So how would we want to reason with this understanding?<\/li>\n<\/ul>\n
We could ask abstract questions about food products or medicines as types of things.\u00a0 Or, we could ask questions about a specific food product or medicine.<\/p>\n
For example, we could ask \u201cIs medicine subject to sales tax?\u201d where we know that food products are not subject to sales tax.<\/sup>[5]<\/sup><\/a><\/p>\n
![\"\"](\"http:\/\/haleyai.com\/wordpress\/wp-content\/uploads\/is-medicine-subject-to-sales-tax.jpg\" "\\"is")
<\/a><\/p>\nNote that the system understands that the above is an interrogative sentence (i.e., a question).<\/p>\n
The following, on the other hand, is a declarative sentence:<\/p>\n
![\"\"](\"http:\/\/haleyai.com\/wordpress\/wp-content\/uploads\/food-products-are-not-subject-to-sales-tax.jpg\" "\\"food")
<\/a><\/p>\nBut, of course, the answer to the question would be \u201cI know nothing that is subject to sales tax.\u201d\u00a0 At least until we say something like:<\/p>\n
![\"\"](\"http:\/\/haleyai.com\/wordpress\/wp-content\/uploads\/every-sale-of-tangible-property-is-subject-to-sales-tax.jpg\" "\\"every")
<\/a><\/p>\nIntelligent Agents\u2019 Inference Engine<\/h3>\n
In order for the logic of the interrogative and declarative sentences above to be used by an intelligent agent, its representation must be made available to some type of inference engine.<\/p>\n
Assuming we have loaded results from the Linguist into Flora-2<\/a>, the following query:<\/p>\n
- \n
- flora2 ?-reading(of(sentence))(?reading,?_)@parse.<\/li>\n<\/ul>\n
Has the following answers (the question is underlined):<\/p>\n
- \n
- ?reading = subject(to)(‘<\/span>\u2286'(medicine),’<\/span>\u2286'(sale(tax)))<\/span><\/li>\n
- ?reading = ‘\u00ac'(include(‘\u2286'(food(product)),’\u2286'(medicine)))<\/li>\n
- ?reading = ‘\u00ac'(subject(to)(‘\u2286'(food(product)),’\u2286'(sale(tax))))<\/li>\n
- ?reading = ‘\u2200\u208a'(\\##7)(‘\u2192'(sale(of)(\\##7,’\u2286'(tangible(property))),subject(to)(\\##7,’\u2286'(sale(tax)))))<\/sup>[6]<\/sup><\/a><\/li>\n<\/ul>\n
Here, \\## introduces what Flora calls a skolem constant.\u00a0 For our purposes here, you can think of these as variables.\u00a0 The fact that they are constants in Flora merely prevents their undergoing unification.<\/sup>[7]<\/sup><\/a><\/p>\n
Identifying Pertinent Knowledge<\/h3>\n
Now we can look at the atomic formulas<\/sup>[8]<\/sup><\/a> of these readings using the following query:<\/p>\n
- \n
- ?- reading(of(sentence))(?_reading),atom(of(reading))(?atom,?_reading).<\/li>\n<\/ul>\n
To which the following answers are received:<\/p>\n
- \n
- ?atom = include(‘\u2286'(food(product)),’\u2286'(medicine))<\/li>\n
- ?atom = sale(of)(\\##7,’\u2286'(tangible(property)))<\/li>\n
- ?atom = subject(to)(\\##7,’\u2286'(sale(tax)))<\/li>\n
- ?atom = subject(to)(‘<\/span>\u2286'(medicine),’<\/span>\u2286'(sale(tax)))<\/span><\/li>\n
- ?atom = subject(to)(‘\u2286'(food(product)),’\u2286'(sale(tax)))<\/li>\n<\/ul>\n
These suggest that the readings that contain atoms about \\##7 or ‘\u2286'(food(product)) being subject to sales tax may be pertinent.<\/p>\n
Matching atoms from the question with atoms from the readings suggests 2 further deliberations:<\/p>\n
- \n
- Could ‘\u2286'(food(product)) include ‘<\/span>\u2286'(medicine)<\/span>?<\/li>\n
- Could ‘<\/span>\u2286'(medicine)<\/span> and \\##7 intersect?<\/li>\n<\/ol>\n
The answer to the first is given by the reading that negates the pertinent atom (i.e., food products do not include medicines).<\/p>\n
- \n
- The answer to the second deliberation involves metonymy<\/a>.<\/li>\n<\/ul>\n
Note that the sentence about sales of tangible property is about the sale rather than the property but the question seems to be about medicine as the property involved in a sale.\u00a0 This is an example of metonymy<\/em>.\u00a0 The question is not whether some medicine in a bathroom cabinet is subject to tax but whether a sale of medicine is subject to tax.<\/p>\n
Metonymy<\/h3>\n
Answering questions requires handling metonymy. \u00a0It\u2019s practically impossible for human beings to avoid using metonymy.\u00a0 It is pervasive.<\/p>\n
- \n
- All we know about sales tax from the above sentences is that sales may be subject to it.<\/li>\n
- Presumably, we would not say that something that is not a sale is subject to (or not) sales tax.<\/li>\n<\/ol>\n
This suggests that we abductively<\/a> infer that the question is about a sale of medicine and that the statement excluding food products from sales tax is about sales of food products.\u00a0 Alternatively, if we do not assume (as in abduction), we can ask.<\/p>\n
One again, we have narrowed the question answering process to the following point:<\/p>\n
- \n
- subject(to)(\\##7,’\u2286'(sale(tax)))<\/li>\n
- \u2192'(sale(of)(\\##7,’\u2286'(tangible(property))),subject(to)(\\##7,’\u2286'(sale(tax)))) where<\/li>\n
- \\##7 matches ‘<\/span>\u2286'(medicine) from our question<\/li>\n<\/ul>\n
The implication would hold if \\##7 was a sale of medicine.<\/p>\n
- \n
- To resolve the metonymy, we take one step away from medicine.<\/li>\n
- In this case, we move to something about medicine.<\/li>\n<\/ul>\n
Let\u2019s make this a little easier… the following answers result from simplifying the prior answers:<\/p>\n
- \n
- ?simplify = sale(of(‘<\/em>\u2286'(tangible(property))))(?X7)<\/em><\/li>\n
- ?simplify = subject(to(‘\u2286'(sale(tax))))(?X7)<\/li>\n
- ?simplify = subject(to(‘<\/span>\u2286'(sale(tax))))(‘<\/span>\u2286'(medicine))<\/span><\/li>\n
- ?simplify = subject(to(‘\u2286'(sale(tax))))(‘\u2286'(food(product)))<\/li>\n<\/ul>\n
Note that the antecedent of the implication is italicized above (the question remains underlined).<\/p>\n
Abductive Inference<\/h3>\n
When we consider metonymy, we simply allow for some function of the phrase used, as in:<\/p>\n
- \n
- subject(to(‘\u2286'(sale(tax))))(?F(‘\u2286'(medicine)))<\/li>\n<\/ul>\n
Unifying this with the 2nd<\/sup> answer above yields:<\/p>\n
- \n
- ?X7=?F(‘\u2286'(medicine)))<\/li>\n<\/ul>\n
Given the implication above, the answer would be yes if ?F resolves the metonymy, as in:<\/p>\n
- \n
- holds(sale(of(‘\u2286'(tangible(property))))(?F(‘\u2286'(medicine)))))<\/li>\n<\/ul>\n
In general, a predicate that holds for something general holds for something more specific, as in:<\/p>\n
- \n
- holds(?predicate(?p(?X))(?predicate(?p(?Y))) :- preposition(?p), subsumes(?x,?Y).<\/li>\n<\/ul>\n
Ontology and Common Sense<\/h3>\n
Our objective in this article is to simply convey how a little knowledge representation and inference is the difference between a chatbot and an intelligent agent.\u00a0 There is more to cover about how an intelligent agent gets domain knowledge, such an understanding of products and services offered by an enterprise.\u00a0 That commonly involves reading pertinent documentation to extract and align domain-specific vocabulary and ontology.\u00a0 Here, we admit, that we gloss over all that.\u00a0 In a future post we will show how we do that, such as we have done for cooking<\/a>.<\/p>\n
If we have loaded an ontology or written a sentence to the effect that tangible property includes medicine, we will deduce the prior statement.<\/p>\n
- \n
- We can deduce that medicine is tangible by reading using common sense, such as:\n
- \n
- anything that contains<\/sup>[9]<\/sup><\/a> something tangible must be tangible<\/li>\n
- knowing (e.g., from WordNet) that medicine is a substance<\/sup>[10]<\/sup><\/a><\/li>\n<\/ul>\n<\/li>\n
- This leads us to a better ontology where we educate the machine about transfers of ownership of property (i.e., sales).\n
- \n
- anything owned is property<\/sup>[11]<\/sup><\/a><\/li>\n
- a sale is a transfer of ownership of property<\/sup>[12]<\/sup><\/a><\/sup>[13]<\/sup><\/a><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n
Obviously, there is much to go into here about the agent’s knowledge, inference capabilities, and reasoning heuristics.\u00a0 We will make the simple claim that a small body of general pragmatics combined with sufficiently precise knowledge from domain documentation can produce a smarter agent than deep learning can obtain with Big Data.\u00a0 The differences are:<\/p>\n
- \n
- A chatbot may respond as if it understands but cannot answer questions if any reasoning is required.\n
- \n
- The chatbot has no explicit, reliable knowledge and does not reason.<\/li>\n
- [Yann] LeCun, head of AI at Facebook: “what’s still missing is reasoning”<\/a><\/li>\n<\/ul>\n<\/li>\n
- An intelligent agent has some knowledge with which it does reason.<\/li>\n<\/ul>\n
Learning or Clarifying by Asking<\/h2>\n
Unlike a chatbot, an intelligent agent engaged in dialogue may wish to clarify the intent of a question or even ask for help in figuring out the answer to a question.<\/p>\n
If we have neither loaded an ontology nor read enough to deduce that medicine is tangible, the agent could ask…\u00a0 depending on the user, their goal, and the context of the interaction.<\/p>\n
In XSB and Flora, we have the well-founded semantics<\/sup>[14]<\/sup><\/a>, which allows us to express the following:<\/p>\n
- \n
- subsumes(?x,?y) :- \\neg subsumes(?x,?y), ask(subsumes(?x,?y)).<\/li>\n<\/ul>\n
Which invokes the action to ask only if it is not already known or provable (positively or negatively) that one subsumes the other.\u00a0 (This is different from Prolog\u2019s strictly procedural behavior).<\/p>\n
Putting It Together<\/h2>\n
In either case, once we have that the metonymy around medicine can be resolved by:<\/p>\n
- \n
- ?F=sale(of(?_))<\/li>\n<\/ul>\n
We answer the question by deducing the following:<\/p>\n
- \n
- subject(to(‘\u2286'(sale(tax))))(sale(of(‘\u2286'(medicine))))<\/li>\n<\/ul>\n
Typically, we resort to metonymy when no answer is found without it.\u00a0 So we ask the question without metonymy and, if no answers are obtained, we pursue the question again, this time allowing metonymy.<\/p>\n
Of course, in an actual dialog, we might ask for confirmation of the metonymy and, in either case, render the confirmations and answers above as English sentences.<\/p>\n
For example, the agent might say any of the following:<\/p>\n
- \n
- Yes.<\/li>\n
- Yes, a sale of medicine is subject to sales tax.<\/li>\n
- Yes, if you were asking about sales of medicine.<\/li>\n<\/ul>\n
As simple as this example may be, it demonstrates general techniques, just a few of which allow an agent to be more intelligent than a chatbot.\u00a0 All it takes is a little thinking (i.e., knowledge and reasoning).<\/p>\n
<\/sup>[1]<\/sup><\/a> Bunt, Harry. “Semantic under-specification: Which technique for what purpose?.”<\/a> Computing meaning<\/em>. Springer, Dordrecht, 2008. 55-85.<\/p>\n<\/sup>[2]<\/sup><\/a> We ignore the quantifier for \u201cfood\u201d here but could establish its variable as equal to the variable for products, as is almost universally the case for compound nouns.<\/p>\n
<\/sup>[3]<\/sup><\/a> We ignore the quantifier for \u201cnot\u201d here, which refers to the situation in which the negation holds.\u00a0 In effect, we interpret the negation \u201cuniversally\u201d.<\/p>\n
<\/sup>[4]<\/sup><\/a> For example, the Linguist realizes that:<\/p>\n
- \n
- the focal referent here is the noun phrase headed by \u201cproducts\u201d (i.e., ?x6)\n
- \n
- e.g., it\u2019s scope is outermost and includes the quantification of ?e16<\/li>\n
- e.g., that its quantifier is universal given its referent occurs as plural<\/li>\n
- the quantification for a subject of an event typically has wider scope\n
- \n
- i.e., the quantification of ?x6 here has scope over the quantification of ?e16<\/li>\n
- that \u201cfood products\u201d is a compound noun having a single referent\n
- \n
- e.g., the quantification for ?x9 has the same scope as ?x6<\/li>\n
- e.g., the variables ?x9 may co-reference or be unified with ?x9<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n
<\/sup>[5]<\/sup><\/a> We can be as precise as necessary here, but for simplicity we will omit the distinction of various taxes and their individuality per jurisdiction.<\/p>\n
<\/sup>[6]<\/sup><\/a> The Linguist also generates the following Flora axioms for the last reading:<\/p>\n
- \n
- subject(to)(?x7, ‘\u2286'(sale(tax))) :- sale(of)(\\##7,’\u2286'(tangible(property))).<\/li>\n
- \\neg sale(of)(\\##7,’\u2286'(tangible(property))) :- \\neg subject(to)(?x7, ‘\u2286'(sale(tax))).<\/li>\n<\/ul>\n
The second rule reflects that (by default) something that is not subject to sales tax cannot be a sale of tangible property.<\/p>\n
- \n
- Most business rule systems would not handle this logic.Flora provides defeasible logic to handle the exceptions.<\/li>\n<\/ol>\n
<\/sup>[7]<\/sup><\/a> See this background information<\/a> or this page at Wikipedia<\/a> for further information.<\/p>\n
<\/sup>[8]<\/sup><\/a> definitions for terms of logic can be found here<\/a><\/p>\n
<\/sup>[9]<\/sup><\/a> in this WordNet sense<\/a><\/p>\n
<\/sup>[10]<\/sup><\/a> see this WordNet sense<\/a> and its definition and hypernyms<\/p>\n
<\/sup>[11]<\/sup><\/a> see this WordNet sense<\/a><\/p>\n
<\/sup>[12]<\/sup><\/a> see this WordNet sense<\/a> of sale or this in FrameNet<\/a><\/p>\n
<\/sup>[13]<\/sup><\/a>
![\"\"](\"http:\/\/haleyai.com\/wordpress\/wp-content\/uploads\/CycSale.jpg\" "\\"CycSale\\"")
<\/a><\/p>\n<\/sup>[14]<\/sup><\/a> see this background<\/a> for more information<\/p>\n","protected":false},"excerpt":{"rendered":"
Deep learning can produce some impressive chatbots, but they are hardly intelligent.\u00a0 In fact, they are precisely ignorant in that they do not think or know anything. More intelligent dialog with an artificially intelligent agent involves both knowledge and thinking.\u00a0 In this article, we educate an intelligent agent that reasons to answer questions.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6,16,17,243],"tags":[687,41,821,865,97,707,866,185,419,867,211,214,888,347,238,868,896,269,548,495,573],"class_list":["post-1223","post","type-post","status-publish","format-standard","hentry","category-cognitive-agents","category-natural-language","category-ontology","category-question-answering","tag-ai","tag-artificial-intelligence","tag-automated-reasoning","tag-chatbot","tag-cyc","tag-framenet","tag-intelligent-agent","tag-knowledge-acquisition","tag-logic-programming","tag-metonymy","tag-natural-language-processing","tag-nlp","tag-ontology","tag-opencyc","tag-prolog","tag-qa","tag-question-answering","tag-semantics","tag-textual-entailment","tag-under-specified-representation","tag-wordnet"],"_links":{"self":[{"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/posts\/1223","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/comments?post=1223"}],"version-history":[{"count":5,"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/posts\/1223\/revisions"}],"predecessor-version":[{"id":1231,"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/posts\/1223\/revisions\/1231"}],"wp:attachment":[{"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/media?parent=1223"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/categories?post=1223"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/tags?post=1223"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
- Most business rule systems would not handle this logic.Flora provides defeasible logic to handle the exceptions.<\/li>\n<\/ol>\n
- the focal referent here is the noun phrase headed by \u201cproducts\u201d (i.e., ?x6)\n
- subject(to(‘\u2286'(sale(tax))))(sale(of(‘\u2286'(medicine))))<\/li>\n<\/ul>\n
- ?F=sale(of(?_))<\/li>\n<\/ul>\n
- An intelligent agent has some knowledge with which it does reason.<\/li>\n<\/ul>\n
- a sale is a transfer of ownership of property<\/sup>[12]<\/sup><\/a><\/sup>[13]<\/sup><\/a><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n
- knowing (e.g., from WordNet) that medicine is a substance<\/sup>[10]<\/sup><\/a><\/li>\n<\/ul>\n<\/li>\n
- anything that contains<\/sup>[9]<\/sup><\/a> something tangible must be tangible<\/li>\n
- We can deduce that medicine is tangible by reading using common sense, such as:\n
- holds(?predicate(?p(?X))(?predicate(?p(?Y))) :- preposition(?p), subsumes(?x,?Y).<\/li>\n<\/ul>\n
- holds(sale(of(‘\u2286'(tangible(property))))(?F(‘\u2286'(medicine)))))<\/li>\n<\/ul>\n
- ?X7=?F(‘\u2286'(medicine)))<\/li>\n<\/ul>\n
- ?simplify = sale(of(‘<\/em>\u2286'(tangible(property))))(?X7)<\/em><\/li>\n
- Could ‘<\/span>\u2286'(medicine)<\/span> and \\##7 intersect?<\/li>\n<\/ol>\n
- ?atom = subject(to)(‘\u2286'(food(product)),’\u2286'(sale(tax)))<\/li>\n<\/ul>\n
- ?reading = subject(to)(‘<\/span>\u2286'(medicine),’<\/span>\u2286'(sale(tax)))<\/span><\/li>\n
- flora2 ?-reading(of(sentence))(?reading,?_)@parse.<\/li>\n<\/ul>\n
- So how would we want to reason with this understanding?<\/li>\n<\/ul>\n
- resolving metonymy<\/a><\/li>\n