Word-level Lexical Normalisation using Context-Dependent Embeddings
Table 6: The F1-Scores of our word-level LN model when initialised with each embedding technique. The datasets do not contain the   label to denote words that do not require normalisation.
['Model architecture Embedding weights', 'Bi-GRU  [BOLD] P. dist.', 'Bi-GRU  [BOLD] Co-oc.', 'Bi-GRU  [BOLD] Word2Vec', 'Bi-GRU  [BOLD] FastText', 'Bi-GRU  [BOLD] ELMo', 'Feed-forward  [BOLD] BERT']
6.1.4 Effectiveness of the   label To determine the effectiveness of augmenting the dataset with   labels (whereby a token does not require normalisation) , we retrain and evaluate our models on the original datasets that do not contain these labels. The performance dropped significantly from 0.80 to 0.40 for the smaller datasets (Twitter and Australian Accidents).