Learning to Generate Wikipedia Summaries for Underserved Languages from Wikidata
Table 4: Automatic evaluation of our model against all other baselines using BLEU 1-4, ROUGE and METEOR for both Arabic and Esperanto Validation and Test set.
['[EMPTY]', '[BOLD] Model', '[BOLD] BLEU 1 Valid.', '[BOLD] BLEU 1 Test', '[BOLD] BLEU 2 Valid.', '[BOLD] BLEU 2 Test', '[BOLD] BLEU 3 Valid.', '[BOLD] BLEU 3 Test', '[BOLD] BLEU 4 Valid.', '[BOLD] BLEU 4 Test', '[BOLD] ROUGEL Valid', '[BOLD] ROUGEL Test', '[BOLD] METEOR Valid.', '[BOLD] METEOR Test']
We evaluate the generated summaries from our model and each of the baselines against their original counterparts from Wikipedia. > token is generated) are excluded from the evaluation. We use a set of evaluation metrics for text generation: BLEU Papineni et al. We achieve at least an enhancement of at least 5.25 and 1.31 BLEU 4 score in Arabic and Esperanto respectively over the IRext, the strongest baseline. The introduction of the copy actions to our encoder-decoder architecture enhances our performance further by 0.61−1.10 BLEU (using BLEU 4). In general, our copy actions mechanism benefits the performance of all the competitive systems.