Fast Reading Comprehension with ConvNets
Table 3: Comparing variants with different number of layers. EM stands for exact match score. The scores are multiplied by 100. DrQA uses a much larger pre-trained word embedding resulting in more parameters.
['Model', '# of params', 'Dev EM', 'Dev F1']
We can see that dilation helps improve answer quality a bit. The choice of non-linear unit has a huge impact on answer quality with GLU performs much better than ReLU. Without residual learning (removing the shortcut connection), the model can never converge to fair point.