However, it's important to note that ChatGPT is made up of a few main components, such as: (i) a pre-training document generator, which is based on the 2017 "Attention is All You Need" paper by Google, (ii) fine-tuning to match desired human behavior (RLHF),