Speech-Based Visual Question Answering
Table 2. Word Error Rate from Kaldi speech recognition
['Noise (%)', 'WER (%)']
Word error rate (WER) is used to measure the accuracy of speech to text transcriptions. WER is defined as follows: WER=(S+D+I)/N Where S is the number of substitutions, D is the number of deletions, and I is the number of insertions. N is the total number of words in the sentence being translated. WER is not expected to be a perfect measure of transcription accuracy, since some words are more essential to the meaning of a sentence than other words. For example, missing the word dog in the sentence what is the dog eating is more detrimental than missing the word the, but we nevertheless employ it to convey a general notion of how many words are understood by the ASR. Naturally the more noise there is, the higher the word error rate becomes. Due to transcription errors, there are resulting questions that contain words not seen in the original datasets. These words, as mentioned above, are indexed as 0 and are masked when fed into TextMod.