DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehension
Table 8: Performance in accuracy (%) on the DREAM dataset. Performance marked by ⋆ is reported based on 25% annotated questions from the development and test sets.
['[BOLD] Method', '[BOLD] Dev', '[BOLD] Test']
We report the averaged accuracy of two annotators as the human performance. The proportion of valid questions (i.e., an unambiguous question with a unique correct answer option provided) that are manually checked by annotators on the annotated test and development sets is regarded as the human ceiling performance.