AutoMOS: Learning a non-intrusive assessor of naturalness-of-speech
Table 2: RMSE and correlation results (reflecting median fold, except as indicated)
['Metric / Model', 'Baselines Bias-only', 'Baselines NNet(utt. length)', 'AutoMOS Raw', 'AutoMOS Quantized', 'Ground-truth Sample human rating']
How well can we rank synthesizers relative to one another? To perform this evaluation, we use the above five-fold cross validation to predict a MOS for each utterance using the AutoMOS instance from which it was held-out. We then average at the synthesizer-level, giving us a total of 36 Esynth(MOSpredicted),Esynth(MOStrue) pairs upon which we evaluate.