Fluent Translations from Disfluent Speech in End-to-End Speech Translation
Table 3: End-to-end model performance evaluated with new fluent references. Comparing average single reference scores (1Ref) vs multi-reference scores using both generated references (2Ref).
['[BOLD] Model', '[BOLD] Metric', '[BOLD] dev  [BOLD] 1Ref', '[BOLD] dev  [BOLD] 2Ref', '[BOLD] test  [BOLD] 1Ref', '[BOLD] test  [BOLD] 2Ref']
Comparing the disfluent and fluent models, we see that METEOR scores are almost the same while BLEU scores are lower with the disfluent model. This is as we would hope: with our fluent model, we want to generate translations that are semantically the same but with disfluencies removed. Therefore similar METEOR scores with similar recall (52) on the fluent references are encouraging. For BLEU, however, the disfluencies generated by the disfluent model break up n-grams in the fluent references, thereby lowering scores. A filtering or tagging system may not capture all false starts or corrections, leading to lower fluency, and requires labeled spans. Treating the post-processing step as a monolingual translation task (MonoMT) rather than a filtering task allows for reordering and insertions, which we saw boost fluency. We trained a 4-layer BiLSTM encoder-decoder model to translate between the disfluent and fluent English references and applied this to the output of the end-to-end disfluent model.