NE-Table: A Neural key-value table for Named Entities
Table 2: Results on structured-QA task
['Model', 'Retrieval accuracy (%)']
While the test accuracy for With-NE-Table is 100%, it drops to 81% for W/O-NE-Table model. Further analysis shows that out of the 19% drop, 11% is due to OOV NEs encountered at test time. These OOV NEs are in the DB, and hence are part of the vocabulary for the W/O-NE-Table model, but have random embeddings which did not change during the training time (as they were never encountered during the training). The rest 8% drop can be attributed to the model’s inability to learn good embeddings for unique NEs that were rarely seen during training. However, these issues do not pose a problem for our With-NE-Table model, since we generate embedding for a NE on the fly for each question based on the context. This solves both problems: a) whether an NE occurred rarely or b) it was not present in training data at all. The With-NE-Table model should also easily scale to large datasets with any number of NEs without drop in performance.