Learning to Encode Position for Transformer with Continuous Dynamical Model
Table 2: Experimental results of various position encoders on the machine translation task.
['[EMPTY]', '[ITALIC] Transformer-Base En-De', '[ITALIC] Transformer-Base En-Fr', '[ITALIC] Transformer-Large En-De', '[ITALIC] Transformer-Large En-Fr']
Among all the data/model combinations, our proposed FLOATER at all blocks outperforms two other position encoders.