MaSS: A Large and Clean Multilingual Corpus of Sentence-aligned Spoken Utterances Extracted from the Bible
Table 3: Result of the manual inspection of the speech alignment quality performed on 8 language pairs (100 sentences). Scale is from 1 to 5 (higher is better). Last column refers to the number of evaluators for a given language pair.
['[EMPTY]', 'x̄', '[ITALIC] σ', '[BOLD] med', '[BOLD] min', '[BOLD] max', '[BOLD] # Eval.']
Evaluation scores are good, with a mean value of 4.41. Moreover, for every language pair evaluated (except for FR-ES), the median score is the maximum score, hence confirming the quality of the alignment. However, when trying to quantify rater’s agreement, we obtained mixed results. Percentage of agreement with tolerance 1 (meaning raters differing by one-scale degree are interpreted as agreeing) varies from 59.6% (EN-RO) to 95.96% (EN-HU).