I think it is amply clear to anyone who tinkers with ChatGPT that these transformers have 2 big weaknesses: 1lack of memory 2inability to do symbolic reasoning. Would love to hear how people are engineering solutions around these!