Simple Large-scale Relation Extraction from Unstructured Text
Table 1: Mean and standard deviation of the F-score values at 0.5 threshold across three trials for the Wikidata and Alexa KBs, using the MaxEnt, fastText, and HypeNET (our Keras implementation with word embeddings training and bi-directional LSTMs). The MaxEnt model did not have any variance across the trials.
['[EMPTY]', 'Relation', '[BOLD] HypeNET  [ITALIC] μ(F-score)', '[BOLD] HypeNET  [ITALIC] σ', '[BOLD] fastText  [ITALIC] μ(F-score)', '[BOLD] fastText  [ITALIC] σ', '[BOLD] MaxEnt F-score']
The main takeaway from these results is that the more advanced architecture of HypeNET does not offer a significant advantage over that of fastText when used with (almost) the same input features. As an added benefit, the fastText classifier is dramatically faster than the HypeNET model, with a reduction of training time from around 75 minutes to less than a minute. However as the results of the MaxEnt model show, the features alone are not enough. It is fastText’s (and HypeNET’s) ability to create higher-dimensional representations of these discrete features that provide the best results.