Identifying Domain Adjacent Instances for Semantic Parsers
Table 3: AUC for domain-adjacent instance identification, using ablated versions of Surprise with kNN.
['[EMPTY]', 'Basketball', 'Blocks', 'Calendar', 'Housing', 'Publications', 'Recipes', 'Restaurants', 'Social']
We can see that Frequency performs slightly worse than CBOW and PreTrained performs even worse than that. We can conclude that the combination of the weighting schema and the tuned vectors is what makes Suprise effective.