Multi-Scale Self-Attention for Text Classification
Table 3: Test Accuracy over MTL-16 datasets. “SLSTM” refer to the sentence-state LSTM [32].
['[BOLD] Dataset', 'Acc (%) MS-Trans', 'Acc (%) Transformer', 'Acc (%) BiLSTM', 'Acc (%) SLSTM']
Multi-Scale Transformer achieves 1.5 and 3.56 points against Transformer on these two datasets, respectively. Meanwhile, Multi-Scale Transformer also beats many existing models including CNNs and RNNs.