Leveraging Schema Labels to Enhance Dataset Search
Table 1: For each task, the number of pairs assigned to each relevance label.
['Task #', 'off topic', 'poor', 'good', 'excellent']
Every single task-dataset pair was judged by three annotators and we take the majority vote as the relevance label. If no majority agreement is achieved, we take the average of the scores as the final label.