1. there's nothing mystical about model architecture all neural networks with a hidden layer are arbitrary function approximators 2. it's trained to estimate P(next words | previous words), the chatgpt trick effectively inc