Zero-shot Text-to-SQL Learning with Auxiliary Task
Table 2: Statisitics of WikiSQL test set. W-full is original WikiSQL test set and W-0, W-1,⋯, W-6 are subsets split by the number of shots (number of a table occurrences in the training data).
['[BOLD] dataset', '[BOLD] number of shots', '[BOLD] #questions']
X-axis represents different subsets of WikiSQL test set, split by how many times a table occurs in training data. We split the test set based on the number of shots (the number of a table occurrences in training data). We report experiments on the original full WikiSQL test set as well as different subset based on the number of shots, especially on the zero-shot testing case. Full-model is consistently better than single Gen-model in WHERE clause prediction. The biggest gap between Full-model and Gen-model in WHERE clause accuracy is on test subset W-0. This shows that Full-model generalizes better than Gen-model for the unseen test tables. We also found that Full-model accuracy on W-4 is slightly lower than that on W-3. We believe this is due to the fact that table itself is the other fact affecting models’ performance, in addition to the amount of training tables.