Exploring Segment Representations for Neural Segmentation Models
Table 2: The NER and CWS results of the baseline models and our neural semi-CRF models with different input composition functions. spd represents the inference speed and is evaluated by the number of tokens processed per millisecond.
['[EMPTY]', '[EMPTY]', 'NER CoNLL03', 'NER CoNLL03', 'CWS CTB6', 'CWS CTB6', 'CWS PKU', 'CWS PKU', 'CWS MSR', 'CWS MSR', '[EMPTY]']
We first consider the problem of representing segments via composing input units and compare different input composition functions. From this table, the SRNN and SCONCATE achieve comparable results and perform better than the SCNN. Although CNN can model input sequence at any length, its invariance to the exact position can be a flaw in representing segments. The experimental results confirm that and show the importance of properly handling the input position. Considering SCNN’s relatively poor performance, we only study SRNN and SCONCATE in the following experiments. Comparing with NN-Labeler, structure prediction models (NN-CRF and neural semi-CRF) generally achieve better performance. But the difference between the neural structure prediction models is not significant. NN-CRF performs better than the best neural semi-CRF model (SCONCATE) on NER while the both SRNN and SCONCATE outperform NN-CRF on three CWS datasets. We address this to the fact either the NN-CRF or the neural semi-CRF merely takes input-level information and not sufficiently adopts segment-level information into the models.