ESBM: An Entity Summarization BenchMark
Table 4: Average F1 over all the entities in a dataset. For the nine existing entity summarizers, significant improvements and losses over each other are indicated by ▲ and ▼ (p<0.05), respectively. Insignificant differences are indicated by ∘.
['[EMPTY]', 'DBpedia  [ITALIC] k=5', 'DBpedia  [ITALIC] k=10', 'LinkedMDB  [ITALIC] k=5', 'LinkedMDB  [ITALIC] k=10']
Overall Results of Existing Entity Summarizers. We compare nine existing summarizers using one-way ANOVA post-hoc LSD and we show whether the difference between each pair of them is statistical significant at the 0.05 level. Among existing summarizers, BAFREC achieves the highest F1 under k=5. It significantly outperforms six existing summarizers on DBpedia and outperforms all the eight ones on LinkedMDB. It is also among the best under k=10. MPSUM follows BAFREC under k=5 but performs slightly better under k=10. Other top-tier results belong to KAFCA on DBpedia and FACES-E on LinkedMDB. Results of Supervised Learning. Four methods (MultilayerPerceptron, AdditiveRegression, REPTree, and RandomForest) outperform all the existing entity summarizers on both datasets under both size constraints, and two methods (SMOreg and LinearRegression) only fail to outperform in one setting. The results demonstrate the powerfulness of supervised learning for entity summarization. Further, recall that these methods only use standard models and rely on features that are used by existing entity summarizers. It would be reasonable to predict that better results can be achieved with specialized models and more advanced features. However, creating a large number of ground-truth summaries for training is expensive, and the generalizability of supervised methods for entity summarization still needs further exploration.