SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check
Table 4: The performance of BERT and SpellGCN evaluated by official tools on SIGHAN 2014 and SIGHAN 2015. FPR denotes the false positive rate and A denotes the accuracy. D-A and C-A denote detection accuracy and correction accuracy.
['SIGHAN 2014', 'FPR', 'D-A', 'C-A', 'D-P', 'D-R', 'D-F', 'C-P', 'C-R', 'C-F']
FASpell used their own metrics, which are different from the sentence-level false postive and false negtivate counting strategy of the official evaluation toolkit. We used the scripts by PGNet and FASpell to compute their metrics for fair comparison. Actually, SpellGCN consistently improves the performance when evaluated by the PGNet/FASpell scripts and the official evaluation toolkit. We will add the FPR results in our revision. The FPR scores are 14.1% (SpellGCN) v.s. 15.3% (BERT) on SIGHAN 14, and 13.2% (SpellGCN ) v.s. 13.6% (BERT) on SIGHAN 15. FPR on SIGHAN 13 is statistically meaningless since almost all the tested sentences have the spelling errors.