BUT-FIT at SemEval-2020 Task 5: Automatic detection of counterfactual statements with deep pre-trained language representation models
Table 3: Different batch sizes and learning rates used to train RoBERTa-large models, results of the best checkpoint on the validation part of the data.
['[BOLD] batch size', '[BOLD] learning rate', '[BOLD] best acc', '[BOLD] best F1']
For Subtask 1, we adapted the baseline provided by the task organizer to asses how more classical machine learning approaches perform on the dataset. After seeing the subpar performance, we turned our attention to pre-trained LRMs, namely RoBERTa and ALBERT.