Visually grounded cross-lingual keyword spotting in speech
Table 3: Analysis of errors by a human annotator of the top ten retrievals on development data. Percentages (%) indicate the absolute drop in P@10 due to that error type.
['Error type', 'XVisionSpeech Count', 'XVisionSpeech %', 'XBoWCNN Count', 'XBoWCNN %']
Transcriptions of the top English utterances retrieved using XVisionSpeechCNN for a selection of German keywords. Errors from both XVisionSpeechCNN and XBoWCNN were presented to the annotator in shuffled order. For both models, around 10% of the retrievals marked as errors are actually correct. The bulk of errors from XVisionSpeechCNN is due to semantically related retrievals. These retrievals are marked as errors, but could actually be useful depending on the type of retrieval application. If type (1) and type (2) errors are not counted as incorrect, XVisionSpeechCNN and XBoWCNN would achieve a P@10 of 91% and 95%, respectively (but, again, this will depend on the use-case). We leave a larger analysis, which will also measure recall (not only top retrievals), for future work.