Fully Quantized Transformer for Machine Translation
Table 6: Evaluation of our quantization method on the WikiText-2 and WikiText-103 language modeling tasks.
['Precision', 'Size (Mb)', 'Compression', 'WikiText-2 Loss', 'WikiText-2 PPL', 'WikiText-103 Loss', 'WikiText-103 PPL']
To evaluate if our quantization scheme generalizes well to other tasks, we evaluate it on two language modeling datasets: WikiText-2 and WikiText-103. The task consists of predicting the sequence {xt+1,⋯,xt+n+1} from the input sequence {xt,⋯,xt+n}. We trained four Transformer models, each with different precision. All models consist of two Transformer encoder layers, with the embedding and hidden size set to 200. Multi-Head Attention has two heads with key and value size 64. The final word projection layer’s weights are shared with the embedding layer. Models were trained for 10 epochs with a batch size of 20 and sequence length of 35. Learning rate is set to 5, dropout to 0.2 and gradient clipping to 0.25. Loss is computed on every element of the output sequence. Validation was computed every epoch to determine the best candidate. Loss and perplexity are computed on the test set and averaged over 10 trials for WikiText-2 and 3 trials for WikiText-3.