GPT3 is 96 layers and will meta-learn some amount of chain-of-reasoning from layer to layer, but it's limited by model depth. More data/training allow it to make bigger leaps per layer, though, as ChatGPT shows.