![\"\"](\"http:\/\/haleyai.com\/wordpress\/wp-content\/uploads\/NLP_pipeline_example.png\" "\\"NLP_pipeline_example\\"")
<\/a><\/p>\nI was reading a paper <\/a>today about NLP pipelines for question answering that used machine learning to find what tools are good at what tasks and to configure a pipeline by selecting the best tool for a given task from each of the types of components in the pipeline.\u00a0 The paper has a long list of various components, so I checked a few out.\u00a0 Most of those of interest were available on the web so that they could be easily composed into pipelines without a lot of software setup.\u00a0 Looking at these I quickly tired in disappointment.\u00a0 Here are some of the reasons.<\/p>\n I am not surprised by these results.\u00a0 NLU is hard.\u00a0 But they are not particularly strong results either.\u00a0 I’m surprised that people find such results useful (if they do).<\/p>\n I know that Cal. App. is an organization (i.e., California Court of Appeal), but I’m not surprised this system missed it, but perhaps it should have identified California?<\/p>\n I understand that the People can be confusing, especially at the beginning of a sentence, but a knowledgeable or learned system would have recognized People as referring to persons (the public) or an organization (e.g., a government).<\/p>\n Similarly, a knowledge or learned system would recognize this as a reference to a legal case and perhaps classified it with regard to organization, place, and event.<\/p>\n I guess I can understand why this system doesn’t bother identifying single words as concepts.\u00a0 But I don’t really understand how “oil well” is a concept but not “oil well driller” or “rock bit” or even “effective life”.\u00a0 It’s important to understand that “transfer of title” is a concept or event here, too. (Perhaps it should even classify ‘(a) of this section’ as referring to a work.)<\/p>\n It’s pretty clear that a company is an organization, not just a concept, so I’m a little surprised by these errors.\u00a0 It missed section 6352 as a work (as above), so this system would not be very good for legal or contract purposes.<\/p>\n The mistake on “private” is interesting, especially that it did not identify “private individuals” as either a concept or as persons.<\/p>\n In real life, text is not always pretty.\u00a0 This is real life. The prior examples are verbatim from the California Sales and Use Tax Law & Regulations.<\/p>\n The following text occurs in the body of a medical test prep exam from one of our clients.\u00a0 Note that it misses lots of people here.\u00a0 It’s also almost completely arbitrary about what it classifies as a concept.\u00a0 It’s particularly odd that it classifies an adjective as a concept when the concept should be “clinical features”.\u00a0 It misses “Pancreas” (twice) and “Blumgart’s Surgery” as referring to a person or organization and being a concept specializing “surgery”.\u00a0 It also misses that various parts f this are references to works.\u00a0 And it misses that 2010 and 2012 are references to years (events).<\/p>\n The following may be fine, but how do you use it? How do those modifiers of 23 and year-old relate to each other? Isn’t she the subject of the telling??\u00a0 That’s enough for me.<\/p>\n The following is a little difficult to “parse”, especially the first row.\u00a0 I don’t get the ‘that’ as the subject nor ‘her be’ as the object of several of these. (And I wonder, what is ‘it’ as a predicate?)<\/p>\n I expect activities to be the subject of ‘require’ and the objective of ‘participates’.\u00a0 I expect her to be the subject of ‘participates’ and (part of) the object of ‘requires’.<\/p>\n In the “you get what you pay for” category, here are some results from Amazon’s Comprehend service.\u00a0 I like that it got the section as an entity and I don’t fault it for perceiving some negative sentiment here (except as follows).<\/p>\n To the extent such things (like Amazon) do well with the smallest noun phrases, they could also do some prepositional phrases, which would be nice, but they are far from understanding what phrases complement (i.e., other phrases or clauses).<\/p>\n I would be more impressed if it noted the more interesting phrases here, such as ‘collect or pay’ or ‘collect or pay a use tax’ or ‘duty to collect or pay a use tax’.\u00a0 On the other hand, it takes a knowledge or very learned system to do that.<\/p>\n And even more impressed if it recognized (with useful confidence) that its the company being relieved from a duty that is negated (without negative sentiment).<\/p>\n<\/a><\/p>\n<\/a><\/p>\n<\/a><\/p>\n<\/a><\/p>\n<\/a><\/p>\n<\/a><\/p>\n