Adapting Sequence to Sequence models for Text Normalization in Social Media
Table 8: OOV words that our secondary character model has normalized correctly (blue) or incorrectly (red)
['[BOLD] Source', '[BOLD] Prediction', '[BOLD] Confidence (%)']
Futhermore, we perform error analysis on the secondary character-level model that is trained on synthetic adversarial examples of word pairs. In total, our model correctly normalizes 16.22% of unseen source words. Most frequent errors that are correctly normalized are elongating the last characters, deleting last character [g] in gerund, swapping or replacing characters and typos. There are also several typos that the model was not able to correct, such as missing whitespaces or editing words unnecessarily.