{"id":504,"date":"2013-06-15T10:52:45","date_gmt":"2013-06-15T15:52:45","guid":{"rendered":"http:\/\/haleyai.com\/wordpress\/?p=504"},"modified":"2018-07-14T17:19:46","modified_gmt":"2018-07-14T22:19:46","slug":"deep-question-answering-watson-vs-aristotle","status":"publish","type":"post","link":"http:\/\/haleyai.com\/wordpress\/2013\/06\/15\/deep-question-answering-watson-vs-aristotle\/","title":{"rendered":"Deep question answering: Watson vs. Aristotle"},"content":{"rendered":"<p>At the SemTech conference last week, a few companies asked me how to respond to IBM&#8217;s Watson given my involvement with <a href=\"http:\/\/haleyai.com\/wordpress\/2013\/06\/06\/acquring-rich-logical-knowledge-from-text-semantic-technology-2013\/\">rapid knowledge acquisition<\/a> for <a href=\"http:\/\/haleyai.com\/wordpress\/2013\/05\/28\/background-for-our-semantic-technology-2013-presentation\/\">deep question answering at Vulcan<\/a>.\u00a0 My answer varies with whether there is any subject matter focus, but essentially involves extending their approach with deeper knowledge and more emphasis on logical in additional to textual entailment.<\/p>\n<p>Today, in a discussion on the LinkedIn NLP group, there was some interest in finding more technical details about Watson.\u00a0 A year ago, IBM published <a href=\"http:\/\/ieeexplore.ieee.org\/xpl\/tocresult.jsp?reload=true&amp;isnumber=6177717\">the most technical details to date about Watson<\/a> in the IBM Journal of Research and Development.\u00a0 Most of those journal articles are available for free on the web.\u00a0 For convenience, here are my bookmarks to them.<\/p>\n<ul>\n<li><a href=\"http:\/\/researcher.watson.ibm.com\/researcher\/files\/us-heq\/W%284%29%20QUESTION%20ANALYSIS%2006177727.pdf\">Question analysis: How Watson reads a clue<\/a><\/li>\n<li><a href=\"http:\/\/researcher.watson.ibm.com\/researcher\/files\/us-heq\/W%285%29%20DEEP%20PARSING%2006177729.pdf\">Deep parsing in Watson<\/a><br \/>\nGood technical details on the two parsing approaches taken.\u00a0 Using deep parsing, such as we have (e.g., using the ERG in <a href=\"http:\/\/haleyai.com\/wordpress\/2012\/12\/19\/project-sherlock\/\">Project Sherlock<\/a>) and disambiguation is a viable approach for the background knowledge that IBM dismisses too quickly, however (see below).\u00a0 Note that in order to train NLP systems in new domains, you have to go through the same process for thousands of sentences, so disambiguation technology as in <a href=\"http:\/\/haleyai.com\/wordpress\/2013\/05\/29\/translating-english-into-logic-using-the-linguist\/\">the Linguist<\/a> is appropriate even if proofs are based more on textual entailment than logical deduction.<\/li>\n<li><a href=\"http:\/\/researcher.watson.ibm.com\/researcher\/files\/us-heq\/W%286%29%20TEXT%20RESOURCE%20ACQUISITION%2006177719.pdf\">Textual resource acquisition and engineering<\/a><\/li>\n<li><a href=\"http:\/\/www.andrew.cmu.edu\/user\/ooo\/watson\/05%20automatic%20knowledge%20extration.pdf\">Automatic knowledge extraction from documents<\/a><\/li>\n<li><a href=\"http:\/\/researcher.watson.ibm.com\/researcher\/files\/us-heq\/W%288%29%20NEEDLES%20IN%20HAYSTACK%2006177720.pdf\">Finding needles in the haystack: Search and candidate generation<\/a><\/li>\n<li><a href=\"http:\/\/researcher.watson.ibm.com\/researcher\/files\/us-heq\/W%289%29%20TYPE%20COERCION%2006177730.pdf\">Typing candidate answers using type coercion<\/a><\/li>\n<li><a href=\"http:\/\/researcher.watson.ibm.com\/researcher\/files\/us-heq\/W%2810%29%20TEXTUAL%20EVIDENCE%2006177731.pdf\">Textual evidence gathering and analysis<\/a><\/li>\n<li><a href=\"http:\/\/researcher.watson.ibm.com\/researcher\/files\/us-heq\/W%2811%29%20RELATION%20EXTRACTION%2006177734.pdf\">Relation extraction and scoring in DeepQA<\/a><\/li>\n<li><a href=\"http:\/\/researcher.watson.ibm.com\/researcher\/files\/us-heq\/W%2812%29%20STRUCTURED%20DATA%2006177725.pdf\">Structured data and inference in DeepQA<\/a><br \/>\nIBM takes a hard line against deep knowledge here.\u00a0 They have the upper hand in the argument due to their impressive results, but more precise knowledge would only improve their performance.\u00a0\u00a0 You can find more on this debate <a href=\"https:\/\/www.research.ibm.com\/deepqa\/faq.shtml#20\">in the Deep QA FAQ<\/a> and <a href=\"http:\/\/haleyai.com\/documents\/AnybodyKnowBuddy.pptx\">this presentation <\/a>which includes the the followings slide:<br \/>\n<a href=\"http:\/\/haleyai.com\/documents\/AnybodyKnowBuddy.pptx\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-510\" title=\"Why not QA using logic and NLP\" src=\"http:\/\/haleyai.com\/wordpress\/wp-content\/uploads\/2013\/06\/Why-not-QA-using-logic-and-NLP-300x174.png\" alt=\"\" width=\"300\" height=\"174\" srcset=\"http:\/\/haleyai.com\/wordpress\/wp-content\/uploads\/2013\/06\/Why-not-QA-using-logic-and-NLP-300x174.png 300w, http:\/\/haleyai.com\/wordpress\/wp-content\/uploads\/2013\/06\/Why-not-QA-using-logic-and-NLP-1024x594.png 1024w, http:\/\/haleyai.com\/wordpress\/wp-content\/uploads\/2013\/06\/Why-not-QA-using-logic-and-NLP.png 1404w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><!--more--><\/li>\n<li><a href=\"http:\/\/researcher.watson.ibm.com\/researcher\/files\/us-heq\/W%2813%29%20SPECIAL%20QUESTIONS%2006177732.pdf\">Special Questions and techniques<\/a><\/li>\n<li><a href=\"http:\/\/researcher.watson.ibm.com\/researcher\/files\/us-heq\/W%2814%29%20IMPLICIT%20RELATIONSHIPS%2006177721.pdf\">Identifying implicit relationships<\/a><br \/>\nWatson uses semantic networks and algorithms such as spreading activation in various ways, including lexically, syntactically and, to some degree, semantically using Wikipedia, for example.\u00a0 (The examples here demonstrate this combined with the capabilities described in the next paper.)<\/li>\n<li><a href=\"http:\/\/www.andrew.cmu.edu\/user\/ooo\/watson\/13%20fact-based%20question%20decomposition.pdf\">Fact-based question decomposition in DeepQA<\/a><br \/>\nExplains how Watson handles questions with multiple constraints which exposes much of Watson&#8217;s sophistication.<br \/>\n<a href=\"http:\/\/haleyai.com\/wordpress\/wp-content\/uploads\/2013\/06\/Mallory-and-Hillary.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-509\" title=\"Mallory and Hillary\" src=\"http:\/\/haleyai.com\/wordpress\/wp-content\/uploads\/2013\/06\/Mallory-and-Hillary-300x197.png\" alt=\"\" width=\"300\" height=\"197\" srcset=\"http:\/\/haleyai.com\/wordpress\/wp-content\/uploads\/2013\/06\/Mallory-and-Hillary-300x197.png 300w, http:\/\/haleyai.com\/wordpress\/wp-content\/uploads\/2013\/06\/Mallory-and-Hillary-1024x674.png 1024w, http:\/\/haleyai.com\/wordpress\/wp-content\/uploads\/2013\/06\/Mallory-and-Hillary.png 1528w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/li>\n<li><a href=\"http:\/\/www.andrew.cmu.edu\/user\/ooo\/watson\/14%20a%20framework%20for%20merging%20and%20ranking%20answers.pdf\">A framework for merging and ranking of answers in DeepQA<\/a><br \/>\nThe essence of the evidential framework for assessing confidence in candidate answers which allows Watson to combine multiple candidate generation and answer scoring components via machine learning.\u00a0 Although the approach is described comprehensively, many technical critical details, such as the nature of the features considered by the answering scoring and learning components, are omitted.<\/li>\n<li><a href=\"http:\/\/www.andrew.cmu.edu\/user\/ooo\/watson\/15%20making%20watson%20fast.pdf\">Making Watson fast <\/a><br \/>\nLots of technical details about parallelism and low-level optimizations  (e.g., RAM-residence using 32-bit instead of 64-bit Java, int[], and  UTF-8).\u00a0 Also provides a variety of insights into the use of RDF and  information retrieval (e.g., Sesame and Indri).<br \/>\n<a href=\"http:\/\/haleyai.com\/wordpress\/wp-content\/uploads\/2013\/06\/Deep-QA-UIMA-architecture.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-508\" title=\"Deep QA UIMA architecture\" src=\"http:\/\/haleyai.com\/wordpress\/wp-content\/uploads\/2013\/06\/Deep-QA-UIMA-architecture-300x223.png\" alt=\"\" width=\"300\" height=\"223\" srcset=\"http:\/\/haleyai.com\/wordpress\/wp-content\/uploads\/2013\/06\/Deep-QA-UIMA-architecture-300x223.png 300w, http:\/\/haleyai.com\/wordpress\/wp-content\/uploads\/2013\/06\/Deep-QA-UIMA-architecture.png 782w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/li>\n<li><a href=\"http:\/\/www.andrew.cmu.edu\/user\/ooo\/watson\/16%20simulation,%20learning,%20optimization%20in%20game%20strategy.pdf\">Simulation, learning, and optimization techniques in Watson&#8217;s game strategies<\/a><\/li>\n<li><a href=\"http:\/\/researcher.watson.ibm.com\/researcher\/files\/us-heq\/W%2819%29%20JEOPARDY%20INTERFACE%2006177728.pdf\">In the game: The interface between Watson and Jeopardy! <\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>At the SemTech conference last week, a few companies asked me how to respond to IBM&#8217;s Watson given my involvement with rapid knowledge acquisition for deep question answering at Vulcan.\u00a0 My answer varies with whether there is any subject matter focus, but essentially involves extending their approach with deeper knowledge and more emphasis on logical &hellip; <\/p>\n<p class=\"link-more\"><a href=\"http:\/\/haleyai.com\/wordpress\/2013\/06\/15\/deep-question-answering-watson-vs-aristotle\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Deep question answering: Watson vs. Aristotle&#8221;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[10,16,243],"tags":[516,518,550,547,185,211,237,896,548,549,306],"class_list":["post-504","post","type-post","status-publish","format-standard","hentry","category-formal-logic","category-natural-language","category-question-answering","tag-deep-qa","tag-digital-aristotle","tag-disambiguation","tag-ibm-watson","tag-knowledge-acquisition","tag-natural-language-processing","tag-project-halo","tag-question-answering","tag-textual-entailment","tag-treebanking","tag-vulcan"],"_links":{"self":[{"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/posts\/504","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/comments?post=504"}],"version-history":[{"count":17,"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/posts\/504\/revisions"}],"predecessor-version":[{"id":1517,"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/posts\/504\/revisions\/1517"}],"wp:attachment":[{"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/media?parent=504"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/categories?post=504"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/haleyai.com\/wordpress\/wp-json\/wp\/v2\/tags?post=504"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}