But as seen with chatGPT, a pretty small change in architecture/training can have a pretty large effect in terms of usability. I expect a more fundamental breakthrough sometime soonish, where we'll go beyond just scaling transformers further (a