Glyce: Glyph-vectors for Chinese Character Representations
Table 8: Impact of different training strategies.
['Strategy', 'Precision', 'Recall', 'F1', 'Accuracy']
As can be seen, the BERT-glyce-joint outperforms the rest two strategies. Our explanation for the inferior performance of the joint strategy is as follows: the BERT layer is pretrained but the glyph layer is randomly initialized. Given the relatively small amount of training signals, the BERT layer could be mislead by the randomly initialized glyph layer at the early stage of training, leading to inferior final performances. As can be seen, transformers not only outperform BiLSTMs and CNNs, but also the BiMPM structure, which is specifically built for the sentence pair classification task. We conjecture that this is because of the consistency between transformers and the BERT structure.