Multi-Task Neural Models for Translating Between StylesWithin and Across Languages
Table 4: BLEU scores of various FSMT models. “+Tag” indicates using formality tags for bilingual data while “Random” indicates using randomly selected bilingual data.
['Model', '+Tag?', 'Random?', 'FR→Formal-EN', 'FR→Informal-EN']
Automatic Evaluation. We compute BLEU scores on the held out test set for all models as a sanity check on translation quality. Because there is only one reference translation of unknown style for each input sentence, these BLEU scores conflate translation errors and stylistic mismatch, and are therefore not sufficient to evaluate FSMT performance. We include them for completeness here, as indicators of general translation quality, and will rely on human evaluation as primary evaluation method. Based on BLEU scores, we select MultiTask-random as the representative of multi-task FSMT and compare it with NMT-constraint and PBMT-random during our human evaluation.