Porous Lattice Transformer Encoder for Chinese NER
Table 6: An ablation study of our proposed model. For model without lattice-aware self-attention (-LASA), we take character sequence as input, and one character just computes multi-head self-attention weights with its adjacent characters and the shared pivot node. For model without porous mechanism (-PM), we directly utilize multi-head LASA to aggregate the weighted information of each word with fully-connected attention connections. For PLTE-LASA-PMHA, we apply multi-head self-attention to each pair of elements from the input character sequence.
['[BOLD] Models', '[BOLD] OntoNotes', '[BOLD] MSRA', '[BOLD] Weibo', '[BOLD] Resume']
, we conduct an ablation study on four datasets. (2) By introducing the porous mechanism (PM), we can enhance the ability of capturing useful local context which is beneficial to NER, while maintaining the strength of capturing long-term dependencies. (3) PLTE-PM performs worse than PLTE-LASA-PM, which confirms that the standard LASA is not suitable for NER because it takes into account all the signals and disperses the distribution of attention, while NER may be benefited more from localness modeling. (4) PLTE-LASA outperforms PLTE-LASA-PM on most datasets, which shows that the porous mechanism can also benefit self-attention when only taking characters as input.