Unsupervised neural and Bayesian models for zero-resource speech processing
Table 4.3: Performance of the Bayesian segmental model on TIDigits1 and TIDigits2, using random initialization. Results from both the constrained (K=15) and unconstrained (K=100) segmental Bayesian models are shown.
['Model', 'TIDigits1 (%) WER', 'TIDigits1 (%) Cluster purity', 'TIDigits1 (%) Boundary  [ITALIC] F-score', 'TIDigits2 (%) WER', 'TIDigits2 (%) Cluster purity', 'TIDigits2 (%) Boundary  [ITALIC] F-score']
TIDigits2 was kept as unseen data up to this point. Using the setup developed on TIDigits1, we repeated exemplar extraction and segmentation separately on TIDigits2. Three iterations of exemplar refinement were used. Across all metrics, performance is better on TIDigits2 than on TIDigits1: WERs drop by 6.2% and 3.9% absolute for the constrained and unconstrained models, respectively; cluster purity improves by around 3% absolute; and boundary F-score is higher by 6% absolute. The figure shows that every cluster is dominated by data from a single ground truth digit. Furthermore, all digits apart from ‘eight’ are found in a single cluster. Although these digits are consistently decoded as the same sequence of clusters, WER counts the extra clusters as insertion errors. These small differences in the discovered word types results in a non-negligible difference in WER between TIDigits1 and TIDigits2.