An Evaluation of Neural Machine Translation Models on Historical Spelling Normalization
Table 8: The average edit distance of the changed spellings in test set and the average edit distance of the incorrectly normalized changed spellings.
['[EMPTY]', '[BOLD] English', '[BOLD] German', '[BOLD] Hungarian', '[BOLD] Icelandic', '[BOLD] Swedish']
For the incorrectly normalized changed spellings, the average edit distance is smaller than 2. In other words, we just need less than two edits to translate an incorrectly normalized spelling into the correct one. In the incorrect normalizations, Swedish has the shortest average edit distance 1.54, and English has the longest average edit distance 1.81. Intuitively, if a spelling has smaller edit distance, it is easier for the model to normalize this spelling correctly. That is to say, the average edit distance of incorrectly normalized spellings will be larger compared to the average edit distance before normalization. For example, the edit distance between “mōdanac” and “mondák” is 6, yet the model can normalize it correctly. Although the model normalized “mėgbètèǵeitnc̣” into “megbetegíteniúk”, which is not identical to the modern spelling “megbetegítenék”, the edit distance nevertheless decreased from 9 to 2. We hypothesize that this could be due to the fact that Hungarian belongs to a different language family than the other four languages.