ChID: A Large-scale Chinese IDiom Dataset for Cloze Test
Table 10: Performance comparison using different idiom representations.
['[EMPTY]', '[BOLD] Dev', '[BOLD] Test', '[BOLD] Ran', '[BOLD] Sim', '[BOLD] Out']
We can observe remarkable drops from idiom embedding to average character embedding + MLP and to average character embedding for all the models, where all the differences are significant (sign test, p-value <0.01). The results indicate that the other two idiom representation methods are worse than treating an idiom as an independent semantic unit. This study also implies that idiom representation is a key factor for the success of Chinese reading comprehension with idiom. In other words, a good cloze test model should have not only a proper model structure, but also a good method to represent idioms.