Robust Text-to-SQL Generation with Execution-Guided Decoding
Table 1: Test and Dev accuracy (%) of the models on WikiSQL data, where Accsyn refers to syntactical accuracy and Accex refers to execution accuracy. “+ EG (k)” indicates that model outputs are generated using the execution-guided strategy with beam size k.
['Model', 'Dev Accsyn', 'Dev Accex', 'Test Accsyn', 'Test Accex']
We report both the syntactical accuracy Accsyn corresponding to the ratio of predictions that are exactly the ground truth SQL query, as well as the execution accuracy Accex corresponding to the ratio of predictions that return the same result as the ground truth when executed. Note that the execution accuracy is higher than syntactical accuracy as syntactically different programs can generate the same results (e.g., programs differing only in predicate order). In execution-guided decoding, we report two model variants, one using a beam size of 3 and the other a beam size of 5.