Dolphin: A Spoken Language Proficiency Assessment System for Elementary Education
Table 3. Experimental results on semantic relevance detection.
['[EMPTY]', '[BOLD] LR', '[BOLD] GBDT', '[BOLD] TextCNN', '[BOLD] Transformer', '[BOLD] Our']
We conduct pair wise t-tests between our approach and every baseline mentioned above and the results show that the prediction outcomes are statistically significant at 0.05 level. As we can see, our Transformer based multiway attention network shows superior performance among all baselines. It not only aggregates sentence information within Transformer encoder layer, but matches words in both video clips and VFT questions from multiway attention layer. We aggregate all available semantic information by an inside aggregation layer. Due to the lack of any contextual information, LR have the worst classification performance. TextCNN and Transformer improve the accuracy by considering the sentence level sequential information. However, it ignores the semantic relationship between video clips and VFT questions.