An Empirical Evaluation of various Deep Learning Architectures for Bi-Sequence Classification Tasks
Table 1: Argument Mining Datasets.
['[BOLD] Task', '[BOLD] Train', '[BOLD] Dev', '[BOLD] Test', '[BOLD] Problem', '[BOLD] Class']
We have carried out extensive evaluation of the above architecture variants over a wide range of datasets related to argumentation mining as well as datasets appealing to the larger natural language community like textual entailment and question answering data. We consider data with class imbalance problem as well as balanced and we do not restrict ourselves to binary classification by working with multiclass dataset as well. Claim Sentence This is the current benchmark dataset for the Claim Detection task. There are a total 47183 canditate claims distributed among 33 motions. There are 56985 labelled candidates for 57 different motion topics. STUDY Evidence : For evidence type STUDY, the dataset consists of 33534 labelled candidates for 49 motion topics. Interesting point to be noted here is that all the datasets above have very low number of positives and the architectures we are evaluating need to be resilient to the class imbalance problem for these datasets. This dataset consist of around 500K instances evenly distributed across all three classes. So, here we have a multiclass problem in a balanced setting. : There are around 29K labelled question/answer pairs at our disposal.