Learning to Compute Word Embeddings On the Fly
Table 1: Exact match (EM) ratio for different models on SQuAD development and tests set. “dict” stands for dictionary, “MP” stands for mean pooling.
['model', 'EM dev', 'EM test']
We report the exact match ratio as computed by the evaluation tools provided with the dataset, which is basically the accuracy of selecting the right answer. We report the average over multiple runs on the development set. Looking at the results one can see that adding any external information results in a significant improvement over the baseline model (B) (3.7 - 10.5 points). When the dictionary alone is used, mean pooling (D3) performs similarly to LSTM (D4). For the model with mean pooling, we verified that the matrix W in computing ec(w) is necessary for best results (see model D2), and back-propagating through the process of reading definition is helpful (see model D1). We thereby establish that our method is preferable to the one by Long et al.