That depends upon the size of the model. Rote memorization is for specific items (like memorizing a book). chatGPT relates to generalized concepts. You are right that there are things missing, but the analogy is not quite right.