StaQC: A Systematically Mined Question-Code Dataset from Stack Overflow
Table 2. Comparison of BiV-HNN and baseline methods.
['Model', 'Python Testing Set Precision', 'Python Testing Set Recall', 'Python Testing Set  [ITALIC] F1', 'Python Testing Set Accuracy', 'SQL Testing Set Precision', 'SQL Testing Set Recall', 'SQL Testing Set  [ITALIC] F1', 'SQL Testing Set Accuracy']
On both datasets, BiV-HNN substantially outperforms heuristic baselines Select-First and Select -All by more than 15% in F1 and accuracy. This demonstrates that our model can collect QC pairs with much higher quality than heuristic methods used in existing research. In addition, when compared with LR and SVM, BiV-HNN achieves 7%∼9% higher F1 and accuracy on Python dataset, and 3%∼5% better F1 and accuracy on SQL dataset. The gain on SQL data is relatively smaller, probably because interpreting SQL programs is a relatively easier task, implied by the observation that both simple classifiers and BiV-HNN can have around 85% F1.