CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases
Table 7: Accuracy of user dialog act prediction on the development and test sets.
['Model', 'Dev', 'Test']
The result of Majority indicates that about 40% of user questions cannot be directly converted into SQL queries. This confirms the necessity of considering a larger set of dialogue actions for building a practical NLIDB system. Even though TBCNN can predict around 85% of user intents correctly, most of the correct predictions are for simple classes such as INFORM_SQL, THANK_YOU, and GOODBYE etc. The F-scores for more interesting and important dialog acts such as INFER_SQL and AMBIGUOUS are around 10%. This indicates that improving the accuracy on user DAT prediction is still important.