ChatGPT has the ability to do in-context learning (ICL), where the model is fed few examples and the task simultaneously and predict the output. It produces meta-gradients through forward computation (not back propagation), then apply to the original model to create an ICL model.