ChatGPT can tell me the state capitals. Is that because it's memorized the set of 50 or because it's learned to model how humans choose state capitals? If we masked a state capital from the training set, could it predict it?