Exploring Domain Shift in Extractive Text Summarization
Table 3: Rouge-1 performance of our four learning strategies on the MULTI-SUM dataset. ΔR=|Rouge(\textscin−domain)−Rouge(\textscout−of−domain)| . A smaller ΔR indicates corresponding model has a better generalization ability. Bold numbers are the best results, and red ones indicate the minimum performance gap between source and target domains. The grey rows show the models’ average performance under three evaluation settings.
['[BOLD] Domains', 'Model [ITALIC] IBasic', 'Model [ITALIC] IIBERT', 'Model [ITALIC] IIITag', 'Model [ITALIC] IVMeta']
Note that we select two sentences for MULTI-SUM domains and three sentences for CNN/Daily Mail due to the different average lengths of reference summaries. , we observe that the domain-aware model outperforms the monolithic model under both in-domain and out-of-domain settings. The significant improvement of in-domain demonstrates domain information is effective for summarization models trained on multiple domains. Meanwhile, the superior performance on out-of-domain further illustrates that, the awareness of domain difference also benefits under the zero-shot setting. This might suggest that the domain-aware model could capture domain-specific features by domain tags and have learned domain-invariant features at the same time, which can be transferred to unseen domains.