NCLS: Neural Cross-Lingual Summarization
Table 7: Human evaluation results. IF, CC and FL denote informative, concise, and fluent respectively.
['Model', 'En2Zh IF', 'En2Zh CC', 'En2Zh FL', 'Zh2En IF', 'Zh2En CC', 'Zh2En FL']
Human Evaluation. We conduct the human evaluation on 25 random samples from each of the En2ZhSum and Zh2EnSum test set. We compare the summaries generated by our methods (including TNCLS, CLS+MS, and CLS+MT) with the summaries generated by GLTran. Three graduate students are asked to compare the generated summaries with human-corrected references, and assess each summary from three independent perspectives: (1) How informative the summary is? (2) How concise the summary is? (3) How fluent, grammatical the summary is? Each property is assessed with a score from 1 (worst) to 5 (best). The conciseness score and fluency score of TNCLS are comparable to those of GLTran. This is because both GLTtrans and TNCLS employ a single encoder-decoder model, which easily leads to under-generation and repetition. Our CLS+MS and CLS+MT can significantly improve the conciseness and fluency of generated summaries, which shows that these methods can generate shorter summaries and reduce grammatical errors. In conclusion, TNCLS can generate more informative summaries, but it is difficult to improve the conciseness and fluency. However, with the help of MT and MS tasks, conciseness and fluency scores can be significantly improved.