Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Modeling
Table 2: Time cost and memory consumption of the different methods on SNLI. Time(s)/epoch: average training time (second) per epoch. Memory(MB): Training GPU memory consumption (Megabyte). Inference Time(s): average inference time (second) for all dev data on SNLI with test batch size of 100.
['[BOLD] Model', '[BOLD] Time(s)/epoch', '[BOLD] Memory(MB)', '[BOLD] Inference Time(s)', '[BOLD] Test Accuracy']
Compared to DiSAN with the same test accuracy, Bi-BloSAN is much faster and more memory efficient. In terms of training and inference time, Bi-BloSAN is 3∼4× faster than the RNN models (Bi-LSTM, Bi-GRU, etc.) . It is as fast as CNNs and multi-head attention but substantially outperforms them in test accuracy. In terms of training memory, Bi-BloSAN requires similar GPU memory to the RNN-based models and multi-head attention, which is much less than that needed by DiSAN.