A Hybrid Word-Character Model for Abstractive Summarization
Table 1: Vocabulary sizes available from the LCSTS 1.0 dataset in different representation units and the used by existing methods. The low portion of the used vocabulary in a word-based presentation tested by existing methods is caused by the limitation of GPU memory. The words are segmented from documents using the jeiba utilities.
['Vocabulary size Word-based', 'Encoder 961,468', 'Usage', 'Decoder 428,051', 'Usage']
Such a design is motivated by the observation: In the Chinese language, words are more precise to provide information than characters because Chinese characters are highly polysemous, but characters are shorter and more flexible than words. On the one hand for input data, it is less ambiguous to represent articles using words than characters. In the LCSTS 1.0 dataset we use for experimental validation, there are 960k words made up by mere 10k characters. Using word embedding units expands the limit of a vocabulary bank in terms of size to train an effective encoder to capture the meaning contained in the word relationship. On the other hand for output targets, since summarized sentences are highly concentrated and it is common in Chinese to shorten long phrases for convenience, e.g. 奥运委员会 (the International Olympic Committee) is abbreviated by 奥委会 where the term 奥运 (Olympic Game) is reduced to 奥 and 委员会 (Committee) to 委会, characters are more flexible than words to represent the output sentences because of their definitely shorter length.