What users don't know about ChatGPT. ChatGPT is a transformer-based language model, which means it uses attention mechanisms to process input. ChatGPT is pre-trained on a massive amount of text data, but it can be fine-tuned on specific tasks or domains.