{"id":1244,"date":"2018-05-09T12:53:13","date_gmt":"2018-05-09T17:53:13","guid":{"rendered":"http:\/\/haleyai.com\/wordpress\/?p=1244"},"modified":"2018-07-13T18:27:28","modified_gmt":"2018-07-13T23:27:28","slug":"importing-documents-into-a-knowledge-base","status":"publish","type":"post","link":"http:\/\/haleyai.com\/wordpress\/2018\/05\/09\/importing-documents-into-a-knowledge-base\/","title":{"rendered":"Importing documents into a knowledge base"},"content":{"rendered":"

Many users land up wanting to import sentences in the Linguist rather than type or paste them in one at a time.\u00a0 One way to do this is to right click on a group within the Linguist and import a file of sentences, one per line, into that group.\u00a0 But if you want to import a large document and retain its outline structure, some application-specific use of the Linguist APIs may be the way to go.<\/p>\n

We’ve written up how to import<\/a> a whole web site\u00a0 on California Sales & Use Tax<\/a>:<\/div>\n
\"\"<\/a><\/div>\n
There are quite a few ways to do this, but we took the step by step approach of:<\/div>\n
\n
    \n
  1. walk the web site and extract the content of the law into a number of files<\/li>\n
  2. process the files into a the content of individual sections within an outline structure<\/li>\n
  3. break the content down into individual sentences<\/li>\n
  4. create the outline structure and populate it with the sentences<\/li>\n<\/ol>\n
    The resulting knowledge base is ready for text analysis, automated parsing, and disambiguation, as shown here:<\/div>\n
    \"\"<\/a><\/div>\n
    \n

    Our favorite case is importing pages from web sites, such as Wikipedia or regulatory text that appears on-line.\u00a0 For the most part, the text is available as HTML.\u00a0 In some cases, it’s available as Word documents.\u00a0 And sometimes it’s only available as PDF.<\/p>\n