Hierarchically-Refined Label Attention Network for Sequence Labeling
Table 3: Effect of attention layer. † denotes the model without attention sublayers except for the last layer.
['[BOLD] Model', '[BOLD] Accuracy (%)']
The effectiveness of Model Structure. In BiLSTM-LAN w/o attention, we remove the attention inference sublayers from BiLSTM-LAN except for the last BiLSTM-LAN layer. This model is reminiscent to BiLSTM-softmax except that the output is based on label embeddings. It gives an accuracy slightly higher than that of LSTM-softmax, which demonstrates the advantage of label embeddings. On the other hand, it significantly underperforms BiLSTM-LAN (p-value<0.01), which shows the advantage of hierarchically-refined label distribution sequence encoding.