Automated Chess Commentator Powered by Neural Chess Engine
Table 2: Human evaluation results. Models marked with * are evaluated only for the Description, Quality, and Comparison categories. The underlined results are significantly worse than those of SCC-mult(*) in a two-tail T-test (p<0.01).
['[BOLD] Models', '[BOLD] Fluency', '[BOLD] Accuracy', '[BOLD] Insights', '[BOLD] Overall']
The human annotators are required to be good at playing chess. That is to say, they are the true audiences of the commentator researches and applications. By introducing human evaluations, we further reveal the performances in the perspective of the audiences. We further demonstrate the efficacy of our models with significantly better overall performances than the retrieval-based model and previous state-of-the-art ones. It is worth noting that the evaluations about Accuracy and Insights show that our models can produce more precise and thorough analysis owing to the internal chess engine. SCC-mult and SCC-strong perform better than SCC-weak in Accuracy and Overall scores. It also supports the points that the our commentary model can be improved with better internal engine.