Depth-Adaptive Transformer
(a) IWSLT De-En - Valid
['[EMPTY]', 'Uniform', '[ITALIC] n=1', '[ITALIC] n=2', '[ITALIC] n=3', '[ITALIC] n=4', '[ITALIC] n=5', '[ITALIC] n=6', 'Average']
5 Conclusion We extended anytime prediction to the structured prediction setting and introduced simple but effective methods to equip sequence models to make predictions at different points in the network. We compared a number of different mechanisms to predict the required network depth and find that a simple correctness based geometric-like classifier obtains the best trade-off between speed and accuracy. Results show that the number of decoder layers can be reduced by more than three quarters at no loss in accuracy compared to a well tuned Transformer baseline.