You train a decoding language model with 300 billion parameters and then you got a ChatGPT If you think this is cool, wait for the next year's GPT-4. It supposedly has 1.2 trillion (or 2 trillion, not sure) parameters