Could you have gotten it to automatically fix the errors by taking the output of the tests and some code for context, passing errors to chatgpt, asking for a diff perhaps, automatically apply, retest repeat? Might work in some cases anyway.