DTCA: Decision Tree-based Co-Attention Networks for Explainable Claim Verification
Table 2: The performance comparison of DTCA against the baselines.
['Dataset', 'Measure', 'SVM', 'CNN', 'TE', 'DeClarE', 'TRNN', 'MTL-LSTM', 'Bayesian-DL', 'Sifted-MTL', 'Ours']
We observe that: [leftmargin=*] SVM integrating semantics from claim content and comments outperforms traditional neural networks only capturing semantics from claim content, like CNN and TE, with at least 4.75% and 6.96% boost in accuracy on the two datasets respectively, which indicates that semantics of comments are helpful for claim verification. On the whole, most neural network models with semantic interaction between comments and claims, such as TRNN and Bayesian-DL, achieve from 4.77% to 9.53% improvements in accuracy on the two datasets than SVM without any interaction, which reveals the effectiveness of the interaction between comments and claims. TRNN, Bayesian-DL, and DTCA enable claims and comments to interact, but the first two models get the worse performance than DTCA (at least 1.06% and 1.19% degradation in accuracy respectively). That is because they integrate all comments indiscriminately and might introduce some noise into their models, while DTCA picks more valuable comments by DTE. Multi-task learning models, e.g. MTL-LSTM and Sifted-MTL leveraging stance features show at most 3.29% and 1.86% boosts in recall than DTCA on the two datasets respectively, but they also bring out noise, which achieve from 1.06% to 21.11% reduction than DTCA in the other three metrics. Besides, DTCA achieves 3.11% and 2.41% boosts than the latest baseline (sifted-MTL) in F1-score on the two datasets respectively. These elaborate the effectiveness of DTCA.