GECToR – Grammatical Error Correction: Tag, Not Rewrite
Table 8: Inference time for NVIDIA Tesla V100 on CoNLL-2014 (test), single model, batch size=128.
['[BOLD] GEC system', '[BOLD] Time (sec)']
Speed comparison. We measured the model’s average inference time on NVIDIA Tesla V100 on batch size 128. For sequence tagging we don’t need to predict corrections one-by-one as in autoregressive transformer decoders, so inference is naturally parallelizable and therefore runs many times faster. Our sequence tagger’s inference speed is up to 10 times as fast as the state-of-the-art Transformer from Zhao et al.