Neural Word Segmentation with Rich Pretraining
Table 5: Influence of word contexts.
['[BOLD] Word', '[BOLD] P', '[BOLD] R', '[BOLD] F']
Word Context. Without using word information, our segmentor gives an F-score of 95.66% on the development data. Using a context of only w−1 (1-word window), the F-measure increases to 95.78%. This shows that word contexts are far less important in our model compared to character contexts, and also compared to word contexts in previous word-based segmentors Zhang et al. This is likely due to the difference in our neural network structures, and that we fine-tune both character and character bigram embeddings, which significantly enlarges the adjustable parameter space as compared with Zhang et al. The fact that word contexts can contribute relatively less than characters in a word is also not surprising in the sense that word-based neural segmentors do not outperform the best character-based models by large margins. Given that character context is what we pretrain, our model relies more heavily on them.