[PDF] Top 20 Analysing Errors of Open Information Extraction Systems

... a particular system. According to Fader et al. (2011), missing extractions are often caused by argument-finding heuristics choosing the wrong arguments, or failing to extract all possible ar- guments. One example is the ... See full document

... Information extraction (IE) systems aim to identify structured relations, like tuples, from unstructured sources such as documents or web ...IE systems learn an extractor for each target ... See full document

... supervised open infor- mation extraction (Open IE) framework that leverages an ensemble of unsupervised Open IE systems and a small amount of labeled data to improve system ... See full document

... two systems, with absolute gains in F-score between 4 and ...respective systems on the test set, except that we used our relation candidate extraction method for the tree kernel of (Xu et ... See full document

... Complex sentences: Because T EXT R UNNER only uses shallow syntactic features it has a harder time on sentences with complex structure. SRL-IE, because of its deeper processing, can better handle complex syntax and ... See full document

... Systematic errors in the clause splitter are the likely cause of these er- ...the errors made in the clause splitter manifest across a range of sentences more often than the fine-grained patterns of Ollie ... See full document

... Evaluating Open IE systems is a difficult prob- lem. Mesquita et al. evaluated SONEX by auto- matically matching a sample of the entity pairs their system identified from the Spinn3r corpus against a ... See full document

... current Open IE systems, which concentrate on identifying explicit relations, ...by open class ...for Open IE: first confirming whether there is a rela- tion between two entities, and then ... See full document

... available systems will cor- rectly ...various systems in Table 1, extractions ...available systems (often implicitly) focus on either compactness ... See full document

... In order to experiment with the 7 systems used in this paper, we bundled them as a web service. A client application need only submit a sentence and a list of OIE system names to perform extraction. All ... See full document

... A prototype was developed to assess the proposed approach. We used as input the same test corpus used by ReVerb 5 (Etzione et al., 2011), composed of 500 sentences from the web. The OpenNLP6 tagger was used to label the ... See full document

... Information Extraction (IE) is the task of turning the unstructured information expressed in natural lan- guage (NL) text into a structured representation in the form of relational tuples consisting ... See full document

... (QA) systems have been a long-standing quest of AI ...structed open vocabulary (subject; predicate; ob- ject) style tuples have broader coverage, but have only been used for simple questions where a single ... See full document

... OIE systems extracted two binary facts that are not false, but in- complete, as the interaction between all three entities in this sentence can only be adequately modeled us- ing an ternary ... See full document

... that better confidence scores would result in better error modeling performance. Hence, we investigated methods for improving the confidence estimates, focusing specifi- cally on introducing new features that might ... See full document

... of Open IE evaluation by providing a large gold benchmark ...prominent Open IE systems, resulting in a clearer formulation of the Open IE ...high-quality Open IE corpus of more than 10K ... See full document

... terms of dependency parse features that would cap- ture the non-contiguous relation phrases in Table 3. Previous work has shown that dependency paths do indeed boost the recall of relation extraction ... See full document

... IE systems for extracting tuples from plain ...rule-based systems require exten- sive engineering when the patterns become di- verse and ...IE systems based on end-to-end frameworks, such as sequence ... See full document

... Matching function Similar to other cases in NLP, we would like to allow some variability in the predicted tuples. For example, for the sen- tence The sheriff standing against the wall spoke in a very soft voice we would ... See full document

... Conventional Open Information Extrac- tion (Open IE) systems are usually built on hand-crafted patterns from other NLP tools such as syntactic parsing, yet they face problems of error ... See full document