IMoJIE: Iterative Memory-Based Joint Open Information Extraction
Table 3: Comparison of various OpenIE systems - non-neural, neural and proposed models. (*) Cannot compute AUC as Sense-OIE, MinIE do not emit confidence values for extractions and released code for Span-OIE does not provision calculation of confidence values. In these cases, we report the Last F1 as the Opt. F1
['[BOLD] System', '[BOLD] Metric Opt. F1', '[BOLD] Metric AUC', '[BOLD] Metric Last F1']
Among the systems under consideration, the best non-neural system reached Last F1 of 51.5, whereas the best existing neural model could only reach 49.2. Deeper analysis reveals that CopyAttention produces redundant extractions conveying nearly the same information, which CaRB effectively penalizes. RnnOIE performs much better, however suffers due to its lack of generating auxilliary verbs and implied prepositions. Example, it can only generate (Trump; President; US) instead of (Trump; is President of; US) from the sentence “US President Trump…”. Moreover, it is trained only on limited number of pseudo-gold extractions, generated by Michael et al.