Graph-based, Self-Supervised Program Repair from Diagnostic Feedback
Table 6: Effect of different program corruption procedures. We train repair models using different program corruption methods (our DrPerturb, Gupta et al. (2017)’s, and random token dropout), and evaluate the trained models on the DeepFix raw test set (full repair rate %). Bold score indicates the best corruption algorithm.
['[BOLD] Repair Model', '[BOLD] Corruption Procedure', '[BOLD] Corruption Procedure', '[BOLD] Corruption Procedure']
Different program corruption procedures. Gupta+17, the original DeepFix work that corrupts delimiters or drops variable declarations only (so a subset of our Syntax and ID-typo module), and Random, a baseline that randomly drops tokens. We apply DrPerturb, Gupta+17, Random on the DeepFix data to create corresponding training sets (containing 156, 113, 170 types of compiler errors respectively). We then evaluate repair models trained on each of those training sets on the DeepFix raw test set We find that models trained by DrPerturb significantly outperform Gupta+17 (+10% repair rate), suggesting that the diverse set of errors covered by DrPerturb is useful. Additionally, while Random produces more distinct types of compiler errors than DrPerturb in terms of the number (170 vs 156), models trained by DrPerturb outperform Random by more than 10%, suggesting that DrPerturb generates a more useful distribution of errors than the Random baseline.