DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering
Table 3: Performance, Inference Speed and Memory for different models on SQuAD.
['[EMPTY]', 'Performance (Squad-F1)', 'Speed (GFLOPs)', 'Memory (MB)']
Small Distilled or Large Decomposed? DeFormer-BERT-large is 1.6 times faster than the smaller BERT-base model. Decomposing the larger model turns out to be also more effective than using the smaller base model (+2.3 points) This shows that with decomposition, a large Transformer can run faster than a smaller one which is half its size, while also being more accurate.