Neural Machine Translation with Byte-Level Subwords
Table 2: Transformer models used in the experiments (using the notations in Vaswani et al. 2017).
['Model', '[ITALIC] N', '[ITALIC] dmodel', '[ITALIC] dff', '[ITALIC] h', '[ITALIC] Pdrop', 'Params']
We set attention and ReLU dropout to 0.1, except Si-En for which we use 0.2. We use 0.2 residual dropout for Tbase models in X-En. We use a kernel size of 5 and a padding of 2 on both sides for all convolutional layers.