DivGraphPointer
Table 2. The performance of keyphrase extraction on five benchmarks. The results of the former six methods are taken from (Meng et al., 2016) and the results of CNN and CopyCNN are taken from (Zhang et al., 2017). The significance levels (** 0.01, * 0.1) between different methods (GraphPointer v.s SeqPointer, DivGraphPointer v.s GraphPointer) are also provided.
['[EMPTY]', '[BOLD] Inspec  [BOLD] F1@5', '[BOLD] Inspec  [BOLD] F1@10', '[BOLD] NUS  [BOLD] F1@5', '[BOLD] NUS  [BOLD] F1@10', '[BOLD] SemEval  [BOLD] F1@5', '[BOLD] SemEval  [BOLD] F1@10', '[BOLD] Krapivin  [BOLD] F1@5', '[BOLD] Krapivin  [BOLD] F1@10', '[BOLD] Kp20k  [BOLD] F1@5', '[BOLD] Kp20k  [BOLD] F1@10']
For each method, the table presents the performance of generating 5 and 10 keyphrases with respect to F1 measure. We also include the truncated NDCG measure of generating 10 keyphrases in the table. The best results are highlighted in bold. We can see that for most cases (except RNN and CNN), the supervised models outperform all the unsupervised algorithms. This is not surprising since the supervised models are trained end-to-end with supervised data. The RNN model and CNN do not perform well on nearly all datasets (e.g., Inspec, NUS, SemEval, and Krapivin). The reason is that they cannot generate words that are not in the vocabulary (OOV words) during the decoding process, while in this paper we only allow to generate words appeared in the given documents. This problem is addressed by the SeqPointer model, which utilizes the pointer network to copy words from the source text, and hence the performance is significantly improved. We can also see such phenomenon on CopyCNN.