QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
Table 3: Speed comparison between our model and RNN-based models on SQuAD dataset, all with batch size 32. RNN-x-y indicates an RNN with x layers each containing y hidden units. Here, we use bidirectional LSTM as the RNN. The speed is measured by batches/second, so higher is faster.
['Training', 'QANet  [BOLD] 3.2', 'RNN-1-128 1.1', 'Speedup  [BOLD] 2.9x', 'RNN-2-128 0.34', 'Speedup  [BOLD] 9.4x', 'RNN-3-128 0.24', 'Speedup  [BOLD] 13.3x']
To measure the speedup of our model against the RNN models, we also test the corresponding model architecture with each encoder block replaced with a stack of bidirectional LSTMs as is used in most existing models. Specifically, each (embedding and model) encoder block is replaced with a 1, 2, or 3 layer Bidirectional LSTMs respectively, as such layer numbers fall into the usual range of the reading comprehension models All of these LSTMs have hidden size 128. We can easily see that our model is significantly faster than all the RNN based models and the speedups range from 3 to 13 times in training and 4 to 9 times in inference.