Deep Stacking Networks for Low-Resource Chinese Word Segmentation with Transfer Learning
Table I: Results of Bi-LSTM on high-resource and low-resource datasets. For low-resource datasets, there is a clear result decline compared with high-resource datasets, MSR and CTB8.
['Datasets', 'Train', 'Test', 'F-score']
All of the models above achieve very promising results, however, their success relies on large-scale annotated datasets. Neural networks with conventional architectures fail in low-resource datasets due to the lack of labelled data. In this paper, we choose a widely used neural network, Bi-directional Long Short Term Memory Network (Bi-LSTM), to show this failure. We conduct experiments on two high-resource datasets and several low-resource datasets. The datasets will be introduced in the experiment section. It proves that insufficient annotated data poses obstacles for training a powerful neural word segmentation model. Fortunately, there are a number of multi-domain corpora which can be used to improve the performance. Thus, in this paper, we propose to incorporate multi-domain knowledge to deal with the lack of training data. However, simply borrowing knowledge from other domains probably results in “Dataset Bias”. First, a model loses accuracy when the target data distribution is different with the source data distribution. “Dataset Bias” leads to model conflicts which make it difficult to directly integrate models.