A Report on the Complex Word Identification Shared Task 2018
Table 2: The number of annotators for different languages
['Language', 'Native', 'Non-native', 'Total']
For the German annotation task, we have fewer annotators than the other languages. Unlike the English annotation task, non-native annotators have a higher inter-annotator agreement (70.66%) than the native annotators (58.5%). There were slightly more non-native participants than native ones, but the number of native annotators who completed the same number of HITs was considerably larger. This means that although there were more non-native participants, they did not participate equally in all HITs.