Computing Lexical Contrast
Table 11: Percentage of contrast questions correctly answered by the automatic method, where different questions sets correspond to target–answer pairs of different kinds. The automatic method did not use WordNet seeds for this task. The results shown for ‘ALL’ are micro-averages, that is, they are the results for the master set of 1269 contrast questions.
['[EMPTY]', '# instances', 'P', 'R', 'F']
The column ‘# instances’ shows the number of questions in each of the datasets. The performance of our method on the complete dataset is shown in the last row ALL. Observe that the F-score of 0.85 is markedly higher than the score obtained on the GRE-preparatory questions. This is expected because the GRE questions involved vocabulary from a higher reading level, and included carefully chosen distractors to confuse the examinee. The automatic method obtains highest F-score on the datasets of gradable adjectives (0.90), antipodals (0.89), and complementaries (0.89). The precisions and recalls for these opposites are significantly higher than those of disjoint opposites. The recall for reversives is also significantly lower than that the gradable adjectives, antipodals, and complementaries, but precision on reversives is quite good (0.93).