Representing Verbs as Argument Concepts
Table 3: Accuracy of argument identification
['[EMPTY]', 'k', 'Probase AC', 'Probase BL', 'Probase SP', 'WordNet AC', 'WordNet BL', 'WordNet SP', '[dir=SE,height=2em,trim=rl]RVSRL']
We set k=5,10,15 for AC, BL and SP. Due to its limited scale, ReVerb cannot recognize many argument instances in the test data, and thus often labels true arguments as negative. SRL, on the opposite side, tends to label everything as positive because the SRL classifier is trained based on features extracted from the context, which remains the same even though we exchange the arguments. Thus, SRL still labels the argument as positive. Comparing with BL and SP, AC considers the coverage of verb arguments, the parsing errors and overlap of concepts to give an optimal solution with different values of k. Consequently, our algorithm generates a set of concepts which cover more precise and diversed verb argument instances. The accuracy decreases when we use WordNet as the taxonomy because WordNet covers 84.82% arguments in the test data while Probase covers 91.69%. Since arguments that are not covered by the taxonomy will be labeled as incorrect by both methods, the advantage of our algorithm is reduced.