A language score based output selection methodfor multilingual speech recognition
Table 3: WER results
['[EMPTY]', '1st decoding with LM0', 'Rescoring with LM2', 'Rescoring with LM1', 'Proposed method']
One pass decoding with the proposed method To experiment that our method can automatically select the best result without having to identify the input language, we did two experiments. The first experiment was to simulate the case of the input language is known in advance. We firstly decoded all 6 testing sets to obtain lattices using AM-2 and LM0. Afterward rescored English sets by LM1 and Vietnamese sets by LM2. In the second experiment, we combined all six testing sets into one set to simulate that the decoder will not know in advance the input language. It could be English or Vietnamese at random. Lattices of this combined set were also obtained by AM-2 and LM0 at the first decoding step. But at the rescoring step, both LM1 and LM2 were used to produce the best paths and applied the proposed method to select final outputs. Finally, we separated the final outputs based on their utterance ID back to the six corresponding testing sets to individually measure WER for each one. Comparing the results of these two experiments shown that our method produced a similar performance without a language identifier for input signals. There was a bit worse on Conversation-test and YouTube-test sets. We found that these sets contain some very short utterances with only one or two words. For these cases the language scores given by LM1 and LM2 could be similar since the output sentences chosen probably have meaning and are made up of homonyms between the two languages.