End-to-End Abstractive Summarization for Meetings
Table 3: Ablation results of HMNet on AMI’s test set. “+role text” means the role vector is not used, but the role name is prepended to each turn’s transcript.
['Model', 'ROUGE-1', 'R-2', 'R-SU4']
Ablation Study. We conduct an ablation study of HMNet to verify the effectiveness of its various components. When the role vector is removed, the ROUGE-1 score drops 4.3 points. The “+role text” setting removes the role vector. Instead, it prepends the role name to each turn’s transcript. Its performance is higher than that of “-role vector” setting in ROUGE-2 and ROUGE-SU4, but still falls behind HMNet. This indicates the effectiveness of the vectorized representation of speaker roles. When HMNet is without the hierarchy structure, i.e. the turn-level transformer is removed and role vectors are appended to word-level embeddings, the ROUGE-1 score drops as much as 7.0 points. Thus, all these components we propose both play an important role in the summarization capability of HMNet.