DiscSense: Automated Semantic Analysis of Discourse Markers
Table 1: Discourse marker prediction accuracy percentages on Wiki20 and Discovery datasets. Human Raters and Decomposable Attention are from [15]. Bi-LSTM is from [22] and the last two are ours.
['[EMPTY]', 'Wiki20', 'Discovery']
The accuracy of BERT on the Discovery test data is quite high given the large number of classes (174, perfectly balanced) and sometimes their low semantic distinguishability. This accuracy is significantly higher than the score of the Bi-LSTM model in the setup of \newcitesileo2019discovery. The BERT model finetuned on Discovery outperforms human performance reported on Wiki20 with no other adaptation than discarding markers not in Wiki20 during inference. These results suggest that the BERT+Discovery model captures a significant part of the use of discourse markers; in the following section, we will apply it to the prediction of discourse markers for individual categories.