Tied Multitask Learning for Neural Speech Translation
Table 2: The multitask models outperform the baseline single-task model and the pivot approach (auto/text) on all language pairs tested. The triangle model also outperforms the simple multitask models on both tasks in almost all cases. The best results for each dataset and task are highlighted.
['[EMPTY]', 'Model ASR', 'Model MT', 'Search ASR', 'Search MT', 'Mboshi CER', 'French BLEU', 'Ainu CER', 'English BLEU', 'Spanish CER', 'English BLEU']
We compare our proposed models against three baselines and one “skyline.” The first baseline is a traditional pivot approach (line 1), where the ASR output, a sequence of characters, is the input to a character-based NMT system (trained on gold transcriptions). The “skyline” model (line 2) is the same NMT system, but tested on gold transcriptions instead of ASR output. The second baseline is translation directly from source speech to target text (line 3). The last baseline is the standard multitask model (line 4), which is similar to the model of Weiss et al.