Contradiction Detection for Rumorous Claims
Table 1: Threads (left) and iPosts (right) RTE datasets compiled from 4 crisis events: amount of pairs per entailment type (ENT, CON, UNK), amount of unique rumorous claims (#uniq clms) used for creating the pairs, amount of unique tweets discussing these claims (#uniq tws).
['[BOLD] event', '[BOLD] ENT', '[BOLD] CON', '[BOLD] UNK', '[BOLD] #uniq  [BOLD] clms', '[BOLD] #uniq  [BOLD] tws', '[BOLD] ENT', '[BOLD] CON', '[BOLD] UNK', '[BOLD] #uniq  [BOLD] clms', '[BOLD] #uniq  [BOLD] tws']
This collection is holds 5.4k RTE pairs generated from about 500 English tweets using semi-automatic 3-way RTE labeling, based on semantic or numeric mismatches between the rumorous claims annotated in the data. The proportion of contradictory pairs (CON) amounts to 25%. iPosts dataset examples are given below. We proposed to model two types of contradictions: in the first both tweets encode the claim target (iPosts), in the second typically only one of them (Threads). The Nearest Centroid algorithm performs poorly on the CON class in Threads where textual overlap is typically small especially for the CON and UNK classes, in part due to the absence of the claim target in replies. However, the Random Forest algorithm’s performance is not affected by this factor. The advantage of RF on the Threads data can be explained by its property of training several weak classifiers on parts of the feature vectors only. By this boosting strategy a usually undesirable combination of relatively long feature vectors but few training observations can be tackled, holding for the Threads data that due to its extreme skewedness (cf. Results indicate the benefit of RF classifiers in such sparse data cases.