A Dependency-Based Neural Network for Relation Classification
Table 7: Results of evaluation on the SemEval-2010 dataset.
['Model', 'Additional Features (AF)', 'F1 with AF', 'F1 without AF']
Since many of our baselines are neural network models, it is convenient for them to use some features extracted with external resources or tools to enhance performance. We call these features “additional features” (AF) and list them in the second column. The F1-measures on SemEval-2010 dataset with/out these additional features are shown in the last two columns. SVM achieves a comparable result, though the quality of feature engineering highly relies on human experience and external NLP resources. MV-RNN models the constituent parse trees with a recursive procedure and its F1-measures with/out AF are about 1.7 percent and 4.6 percent lower than those of DepNN. This to some extent indicates that our proposed ADP structure is more suitable for relation classification task. Meanwhile, MV-RNN is very slow to train, since each word is associated with a matrix. Both CNN and FCM use features from the whole sentence and achieve similar performance. DT-RNN is the worst of all baselines, though it also considers the information from shortest dependency paths and attached subtrees. As we analyze, shortest dependency paths and subtrees play different roles in relation classification. But, we can see that DT-RNN does not distinguish the modeling processes of shortest paths and subtrees, and deems the representation of each node as a linear combination of its children.