Figure from new paper coming soon. Biggest change in latest GPT models (e.g., ChatGPT/davinci-003) is both improvement in accuracy and less sensitivity to prompt engineering. Tricks like saying "I am an expert at programming" or copyright notices do not gain as much accuracy.