Deep natural language understanding (NLU) is different than deep learning, as is deep reasoning. \u00a0Deep learning facilities deep NLP and will facilitate deeper reasoning, but it\u2019s deep NLP for knowledge acquisition and question answering that seems most critical for general AI.\u00a0 If that\u2019s the case, we might call such general AI, \u201cnatural intelligence\u201d.<\/p>\n
Deep learning on its own delivers only the most shallow reasoning and embarrasses itself due to its lack of \u201ccommon sense\u201d (or any knowledge at all, for that matter!).\u00a0 DARPA, the Allen Institute, and deep learning experts have come to their senses about the limits of deep learning with regard to general AI.<\/p>\n
General artificial intelligence requires all of it: deep natural language understanding<\/sup>[1]<\/sup><\/a>, deep learning, and deep reasoning.\u00a0 The deep aspects are critical but no more so than knowledge (including \u201ccommon sense\u201d).<\/sup>[2]<\/sup><\/a><\/p>\n To think there must be some reasoning using some knowledge.<\/p>\n There are several problems to realizing intelligence within an agent or system, the most immediate of which is how it acquires knowledge.\u00a0 Here, we suggest that knowledge is most easily acquired by understanding language (e.g., English).\u00a0 There seems to be little controversy on this point since the most intelligent agents obtain most of their knowledge by \u201creading\u201d (e.g., Wikipedia).<\/p>\n Thanks to improving natural language understanding capabilities, knowledge acquisition is less of a practical problem today than it has been for decades.\u00a0 Today, we can educate machines at scale.<\/p>\n Millions of sentences will be translated from English into axiomatic knowledge and machines will reason with great power with their thinking guided by deep learning.\u00a0 With 10 years, more knowledge will be encoded for machine reasoning than a human being can read \u2013 let alone comprehend – in a lifetime.<\/p>\n Of course, this begs questions, such as:<\/p>\n Staying in the present, where is reasoning technology that shows the way or what to expect?<\/p>\n Personally, I am not aware of too much, which is disappointing.\u00a0 There are important islands, but the field of knowledge-based artificial intelligence has been steamrolled since the eighties expert and frame-based systems.\u00a0 Most recently, real breakthroughs in neural networks and deep learning have taken the air out of the room for knowledge-based AI.\u00a0 The pendulum is now swinging back, however, such as with regard to \u201cexplainable\u201d AI and \u201ccommon sense\u201d, the lack of which limit the usefulness and intelligence that learning without reasoning can deliver.<\/p>\n As an entrepreneur, I was impressed with Siri and Watson, but I have never found them impressive from an AI perspective.\u00a0 From an engineering perspective, Watson was tremendous, however.\u00a0 And the deep NLP component behind Watson was on the right track.<\/p>\n The \u201cintelligent agent\u201d tools from Microsoft (LUIS), Amazon, IBM, and others, especially all the chatbot froth, seem nothing more than scripting tools with simple notions of state and limited types of queries rather than AI platforms. \u00a0I recall Viv, from the creators of Siri, being a visual programming tool prior to its acquisition by Samsung.\u00a0 Such platforms are not going to deliver more general or deeper AI.<\/p>\n Advances in speech recognition and synthesis in Siri and Alexa (mostly due to deep learning) have been impressive, but they are more like search engines than intelligent agents.\u00a0 Searching the web for Siri or Alexa and \u201cinference engine\u201d finds nothing of interest.\u00a0 (I expect this to change in the near future.)<\/p>\n All the interesting stuff is hidden in the academic literature, research projects, and very few companies.<\/p>\n Years ago, Carnegie Mellon and Stanford were home to the most significant work in cognitive and knowledge-based AI.\u00a0 Today, probably due to diversion of research funding, there are fewer and smaller pockets of work.<\/p>\n Corporate research projects are, of course, hard to spot.\u00a0 I know of several at Google, Intel, IBM, Apple, and elsewhere, but even they are isolated within their companies. \u00a0The emphasis is on big data, deep learning, and more mundane commercial applications, such as personalization of advertising.<\/p>\n One area at the intersection of academic and corporate research worth touching on is theorem proving, including applications of deep learning such as in Google\/DeepMind\u2019s \u201cneural Turing machines\u201d.<\/sup>[4]<\/sup><\/a> This is an area we are working in.\u00a0 We believe so-called embeddings and approaches such as demonstrated by Alpha Zero\u2019s success at Go will have a significant impact.\u00a0 We believe this needs to be coupled with concepts like \u201cbounded rationality\u201d and defeasibility.\u00a0 You can find more about the latter on this site and you can gain insight on the intersection of deep reasoning with deep learning from Google\u2019s Deep Math<\/sup>[5]<\/sup><\/a> research.\u00a0 In general, though, the field of theorem proving has been stable this decade.<\/sup>[6]<\/sup><\/a><\/p>\n The few companies specializing in knowledge-based AI are an interesting lot.\u00a0 I have to include Wolfram Alpha<\/sup>[7]<\/sup><\/a>, of course.\u00a0 Cycorp<\/sup>[8]<\/sup><\/a> is more notable, in my opinion, while Alpha is more usable \u201coff the shelf\u201d.\u00a0 Also keep an eye on little known Soar Technology.<\/sup>[9]<\/sup><\/a><\/p>\n Aside from these we get into small vendors of logic programming technology, largely based on Prolog.\u00a0 We go into this family of logic programming technologies elsewhere<\/a>.\u00a0 To sum up our thinking, a bounded-rationality, defeasible logic theorem proving system that incorporates the well-founded semantics, distributed representations, and deep-learning search heuristics is to be expected.<\/p>\n <\/sup>[2]<\/sup><\/a> Doug Lenat of Cyc and our advisor at Inference has been shining light on this since the early eighties<\/p>\n <\/sup>[3]<\/sup><\/a> Expect them to pass medical school admissions and post-graduate exams within a few years.<\/p>\n <\/sup>[4]<\/sup><\/a> https:\/\/en.wikipedia.org\/wiki\/Neural_Turing_machine<\/a><\/p>\n <\/sup>[5]<\/sup><\/a> https:\/\/github.com\/tensorflow\/deepmath<\/a><\/p>\n <\/sup>[6]<\/sup><\/a> See the \u201cThousands of problems for theorem provers\u201d (TPTP<\/a>)<\/p>\n <\/sup>[7]<\/sup><\/a> https:\/\/www.wolframalpha.com<\/a><\/p>\n <\/sup>[8]<\/sup><\/a> http:\/\/www.cyc.com\/<\/a><\/p>\n <\/sup>[9]<\/sup><\/a> https:\/\/soartech.com\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":" Deep natural language understanding (NLU) is different than deep learning, as is deep reasoning. \u00a0Deep learning facilities deep NLP and will facilitate deeper reasoning, but it\u2019s deep NLP for knowledge acquisition and question answering that seems most critical for general AI.\u00a0 If that\u2019s the case, we might call such general AI, \u201cnatural intelligence\u201d. Deep learning … <\/p>\n\n
\n
\n
<\/sup>[1]<\/sup><\/a> deep NLP is a topic throughout Automata, Inc’s web site<\/a><\/p>\n