Course Concept Expansion in MOOCs with External Knowledge and Interactive Game
Table 4: Online evaluation results.
['[EMPTY]', '[ITALIC] Cr@10', '[ITALIC] Cr@50', '[ITALIC] Cr@100']
By collecting deletion data of the expanded concepts, we can peep whether our expansion results are really helpful to the MOOC users. In order to quantitatively observe the user’s feedback on the model, we propose Average Correction Rate(Cr) as a game-to-model evaluation metric. This metric is the ratio of the number of times the user deletes the top n concepts to the total deletions, and is formally denoted as Cr=∑ni=1del(ei)/∑|Ec|j=1del(ej). Higher Cr indicates less users think the expansion results are helpful. From this perspective, our method is the most helpful to them. For Cr@10, PUL performs a slight advantage at 0.002. However, once the range of observation is broadened to 50 or 100, our method shows an obvious ascendancy. Besides, after adding classifier and game, user satisfaction further increases, reducing the Cr by 0.004 and 0.049 at Cr@50 and Cr@100, reiterating the improvement of these components.