SParC: Cross-Domain Semantic Parsing in Context
Table 7: Performance stratified by question turns on the development set. The performance of the two models decrease as the interaction continues.
['Turn #', 'CD-Seq2Seq', 'SyntaxSQL-con']
We note that SyntaxSQL-con scores higher in question match but lower in interaction match compared to CD-Seq2Seq. A possible reason for this is that SyntaxSQL-con adopts a stronger context-agnostic text-to-SQL module (SyntaxSQLNet vs. Seq2Seq adopted by CD-Seq2Seq). The higher performance of CD-Seq2Seq on interaction match can be attributed to better incorporation of information flow between questions by using turn-level encoders Suhr et al. Overall, the lower performance of the two extended context-dependent models shows the difficulty of SParC and that there is ample room for improvement.