Fast and Simple Mixture of Softmaxeswith BPE and Hybrid-LightRNN for Language Generation
Table 8: Ablation study on the importance of translating OOV words. The BLEU score are evaluated on the validation set of IWSLT.
['Model', 'OOV Translation', 'BLEU']
The comparison is shown in Tab. Removing the generated OOV words do lead to a performance decrease of 0.05 BLEU score. However, when both Hybrid-LightRNN and BPE are disabled from translating OOV, BPE is still better than Hybrid-LightRNN by a gap of 0.07 BLEU score. This result indicates that the encoding function learned by BPE better captures the data statistics than the encoding learned by Hybrid-LightRNN, showing that Hybrid-LightRNN has a lot of potentials for improvements.