Word-level Lexical Normalisation using Context-Dependent Embeddings
Table 3: The F1 scores of our word-level GRU on each of the three datasets when the embedding layer is initialised with one of three probability distributions.
['[EMPTY]', '[BOLD] Uniform', '[BOLD] Normal', '[BOLD] Cauchy', '[BOLD] Best']
There is little variance between distributions, although the model trained on the US Accidents dataset benefited relatively significantly from the cauchy distribution. Freezing the weights appears to make no difference to the F1-score. Overall we find that the most appropriate probability distribution to initialise the weights of the embedding layer depends entirely on the dataset.