Hierarchically-Refined Label Attention Network for Sequence Labeling
Table 2: WSJ development set. E: label embedding size, H: hidden size, L: number of layers.
['[BOLD] Model', '[BOLD] # E/# H', '[BOLD] # L', '[BOLD] Acc', '[BOLD] # Param']
Label embedding size. Notable improvement can be achieved when the label embedding size increases from 200 to 400 in our model, but the accuracy does not further increase when the size increases beyond 400. We fix the label embedding size to 400 for our model. Number of Layers. Yang et al. As can be seen, a multi-layer model with larger hidden sizes does not give significantly better results compared to a 1-layer model with a hidden size of 400. We thus chose the latter for the final model. For BiLSTM-LAN, each layer learns a more abstract representation of word and label distribution sequences. More layers do not empirically improve the performance. We thus set the final number of layers to 3. Model Size vs CRF. We observe that: (1) As the model size increases, both BiLSTM-CRF and BiLSTM-LAN see a peak point beyond which further increase of model size does not bring better results, which is consistent with observations from prior work, demonstrating that the number of parameters is not the decisive factor to model accuracy; and (2) the best-performing BiLSTM-LAN model size is comparable to that of the BiLSTM-CRF model size, which indicates that the model structure is more important for the better accuracy of BiLSTM-LAN.