BP-Transformer: Modelling Long-Range Context via Binary Partitioning
Table 6: BLEU score vs context length on different models
['Context length', '0', '32', '64', '128']
Similar to tu2018learning and miculicich-etal-2018-document , we found a small context length is enough for achieving good performance on IWSLT for Document-Level Translation. However, as we increases context size, the performance of BPT does not get worse as these models and Transformers, suggesting the inductive bias encoded by BPT makes the model less likely to overfit.