Complex Structure Leads to Overfitting:A Structure Regularization Decoding Method for Natural Language Processing
Table 1: Comparing SR decoding with the baseline BLSTM models. As we can see, when the order of the model increases, the performance of BLSTM deteriorates most of the time. With SR decoding, the structural complexity is controlled, so that the performance of the higher order model is substantially improved. It is also interesting that the improvement is larger when the order is higher.
['Test score', 'Chunking', 'English-NER', 'Dutch-NER']
First, we apply the proposed scalable multi-order decoding method on BLSTM (BLSTM-SR). As we can see, when the order of the model is increased, the baseline model worsens. The exception is the result of the Dutch-NER task. When the order of the model is increased, the model is slightly improved. It demonstrates that, in practice, although complex structure models have lower empirical risks, the structural risks are more dominant.