MaSS: A Large and Clean Multilingual Corpus of Sentence-aligned Spoken Utterances Extracted from the Bible
Table 5: Correlation between median rank and 1) alignment quality (from manual evaluation) 2) syntactic distance between languages (measured with lang2vec).
['[BOLD] Languages', '˜ [ITALIC] r', '[BOLD] Quality (x̄)', '[BOLD] Syntactic dist.']
Following this experiment, we investigated the correlation between the median rank and two variables: the quality of the alignment (human evaluation) and the syntactic distance between the language pairs (using the lang2vec library [littell2017uriel]). While there is no correlation between the rank and the syntactic distance, there is a strong negative correlation with respect to the human evaluation (significant for p<0.1). One possible explanation for this result may be that higher quality alignments (measured by the human evaluation ˜x) lead to a slightly easier corpus for the speech-speech retrieval task (difficulty being measured by the rank ˜r). If confirmed, this result would suggest that speech-to-speech retrieval scores are a good proxy for rating alignment corpus quality, as performed for text by \newciteDBLP:journals/corr/abs-1907-05791 through the use of NMT.