Big Bird: Transformers for Longer Sequences
Table 10: Comparison.
['Model', 'F1']
The corresponding machine learning task is to classify a given DNA fragment as promoter or non-promoter sequence. We use the dataset compiled by Oubounyt et al. Dreos et al., We finetuned the pretrained BigBird model from above, using the training data and report F1 on test dataset. We see that BigBird achieve nearly perfect accuracy with a 5% jump from the previous best reported accuracy.