Exploring Segment Representations for Neural Segmentation Models
Table 3: Effect of fine-tuning (FT) segment embedding on development data. For CoNLL03 data, a named entity is “out-of-vocabulary” when it is not included in the training data as a named entity.
['[ITALIC] model', 'CoNLL03', 'CTB6', 'PKU', 'MSR']
We study the effect of fine-tuning the segment embeddings by imposing SEmb-Homo into our model. We find that our models benefit from fixing the segment embeddings on CoNLL03. While on MSR, fine-tuning the embeddings helps. Further study on the out-of-vocabulary rate shows that the OOV rate of MSR is very low, thus fine-tuning on segment embeddings help to learn a better task-specified segment representation. However, on CoNLL03 data whose OOV rate is high, fine-tuning the segment embedding harms the generalization power of pre-trained segment embeddings.