Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing
Table 10: Text classification performance comparison under the large-scale pretraining.
['[BOLD] Model', '[BOLD] IMDB', '[BOLD] AG', '[BOLD] DBpedia', '[BOLD] Yelp-2', '[BOLD] Yelp-5', '[BOLD] Amazon-2', '[BOLD] Amazon-5']
Similar to the GLUE benchmark results, compared with the previous result based on Transformer, with fewer FLOPs, the proposed F-TFM achieves comparable results.