INS: An Interactive Chinese News Synthesis System
Table 1: Manual evaluation results
['Method', 'Read.', 'Struc.', 'Topic.', 'Redun.', 'Overall']
The summarization methods we choose include Lead, Coverage, Centroid, and TextRank. Centroid and TextRank can work on either the sentence unit (i.e., Centroid-sen, TextRank-sen) or the text block unit (i.e., Centroid-blk, TextRank-blk). Thus we have six baselines. The baselines are compared with INS that does not involve user interaction. Each baseline and INS generate a synthesis article with 1000 words for each news event and 10 news topics will be manually evaluated. We employ 16 Chinese college students as judges. Each student evaluates one or two news topics. We make sure each student judge all the 7 articles for each news topic and each news topic is evaluated by 3 students. The judges are asked to give a rating score between 1 and 6 from the following aspects: readability, structure, topic diversity, redundancy removal, and overall responsiveness. From the table, we can see that INS achieves the best performance in every aspect, which proves the effectiveness of our INS system. We also perform pairwise t-tests when comparing our system with baselines, and find that INS significantly outperforms sentence-based methods in every aspect and block-based methods in almost all aspects (p-value < 0.05).