Segmental Audio Word2Vec: Representing Utterances as Sequences of Vectors with Applications in Spoken Term Detection
Table 3: The spoken term detection performance in Mean Average Precision (MAP) for the proposed SSAE compared to the audio Word2Vec embeddings trained with spoken words segmented with other methods for different languages. The random baseline (Ran.) simply assigned a random score to each pair of query and document. Standard frame-based DTW is the primary baseline, while the oracle segmentation is the upper bound.
['Lang.', 'Ran.', 'DTW', 'Embeddings (Different Seg.) GAS', 'Embeddings (Different Seg.) HAC', 'Embeddings (Different Seg.) SSAE', 'Embeddings (Different Seg.) Oracle']
The performance for embeddings trained with ground truth word boundaries (oracle) in the last column serves as the upper bound. The random baseline in the first column simply assigned a random score to each pair of query and document. From the table, it is clear that the oracle achieved the best and significantly better performance than all other methods on all languages. SSAE outperformed the DTW baseline by a wide gap. This is probably because DTW may not be able to identify the spoken words if the speaker or gender characteristics are very different, but such different signal characteristics may be better absorbed in the audio Word2Vec training. These experimental results verified that the embeddings obtained with SSAE did carry the sequential phonetic structure information in the utterances, leading to the better performance in STD here. The performance of embeddings trained with GAS and HAC are not too far from random in most cases. It seems the performance of the spoken word segmentation has to be above some minimum level, otherwise the audio Word2Vec couldn’t be reasonably trained, or spoken word segmentation boundaries had the major impact on the STD performance.