A Multitask Learning Approach for Diacritic Restoration
Table 2: Performance of the joint diacritic restoration model when different related tasks are considered. Bold numbers represent the highest score per column. Almost all scores are higher than the base model BASE (char). * denotes statistically significant improvements compared to the baselines. Lex refers to the percentage of words that have incorrect lexical diacritics only, excluding syntactic diacritics.
['[BOLD] Task', '[BOLD] WER', '[BOLD] DER', '[BOLD] LER/Lex', '[BOLD] OOV WER']
When we consider WordToChar as input to the diacritic restoration model, we observe statistically significant improvements for all evaluation metrics. This is justified by the ability of word embeddings to capture syntactic and semantic information at the sentence level. The same character is disambiguated in terms of the surrounding context as well as the word it appears in (e.g. the character t in the word cat would be represented slightly different than t in a related word cats or even a different word table). We consider both character based model as well as WordToChar based model as our baselines (BASE).