Identifying Domain Adjacent Instances for Semantic Parsers
Table 2: AUC for domain-adjacent instance identification, using kNN as the domain-adjacent model.
['[EMPTY]', 'Basketball', 'Blocks', 'Calendar', 'Housing', 'Publications', 'Recipes', 'Restaurants', 'Social']
We first directly evaluate the identification of domain-adjacent instances: Surprise generally performs the best on this evaluation; and, in general, the simpler models tend to perform better, suggesting that more complex approaches tune too much to the training data.