An Exploration of Inference Network Architecturesfor Sequence Labeling
Table 1: Test results for all tasks. Inference networks, gradient descent, and Viterbi are all optimizing the BLSTM-CRF energy. Best result per task is in bold.
['[EMPTY]', 'Twitter POS Tagging CNN', 'Twitter POS Tagging BLSTM', 'Twitter POS Tagging seq2seq', 'NER CNN', 'NER BLSTM', 'NER seq2seq', 'CCG Supertagging CNN', 'CCG Supertagging BLSTM', 'CCG Supertagging seq2seq']
The inference networks use the same architectures as the corresponding local baselines, but their parameters are trained with both the local loss and the BLSTM-CRF energy, leading to consistent improvements. CNN inference networks work well for POS, but struggle on NER and CCG compared to other architectures. BLSTMs work well, but are outperformed slightly by seq2seq models across all three tasks. Using the Viterbi algorithm for exact inference yields the best performance for NER but is not best for the other two tasks.