Annotating Student Talk in Text-based Classroom Discussions
Table 2: Inter-rater reliability for pairs P1 and P2.
['[BOLD] Moves', '[BOLD] Argumen- tation (kappa)', '[BOLD] Specificity (qwkappa)', '[BOLD] Domain (kappa)']
Kappa values for specificity and knowledge domain are in the 0.81−1 range which generally indicates almost perfect agreement These results show that our proposed annotation scheme can be used to produce reliable annotations of classroom discussion with respect to argumentation, specificity, and knowledge domain.