Unity in Diversity: Learning Distributed Heterogeneous Sentence Representation for Extractive Summarization
Table 2: Comparison Results (%) on DUC Datasets
['2001  [BOLD] System', '2001  [BOLD] ROUGE-1', '2001  [BOLD] ROUGE-2', '2002  [BOLD] System', '2002  [BOLD] ROUGE-1', '2002  [BOLD] ROUGE-2', '2004  [BOLD] System', '2004  [BOLD] ROUGE-1', '2004  [BOLD] ROUGE-2']
This is encouraging because despite having not so complex deep network architecture the HNet-B system is able to learn efficient document-dependent semantic features. It also outperforms the Reg_Manual baseline which uses human-compiled features for obtaining the document-independent features and the graph-based summarization system LexRank. This is due to the fact that the additional BLSTM network used in HNet-B helps in learning temporal (sequential) dependencies among variable length phrases exploiting past as well as future context. Finally, our proposed model (HNet/HNet (T)) significantly outperforms our basic systems HNet-B/HNet-B (T) which supports the fact that HNet is equipped with suitable deep network architecture for procuring latent semantic features (document-independent features) from a sentence. In the following, we analyze different aspects of the proposed system. Contribution of Document Independent Features To explore the contribution of the learned document independent features towards the saliency estimation of a sentence prior to the summary, we follow a simple approach. For each sentence, we ignore document dependent features by setting the ϕe vector to 0, and then applying the regression transform to calculate the saliency score. We refer to this model as HNet−. This setting helps us in analyzing the intuitive features latent in our heterogeneous representation of the sentence without consideration of the contextual features. The drop in points has resulted due to the absence of document dependent features. Therefore, we can conclude that document independent features have a major contribution towards saliency estimation of a sentence. Contribution of Transfer Learning Method The fact that increase in training data results in better performance as the system becomes more robust motivated us to pre-train the HNet-B and HNet systems on Daily Mail dataset first and then fine-tune the systems to multi-document summarization setting. We refer to these systems as HNet-B(T) and HNet(T). HNet(T) is the best performing system amongst the HNet variants.