1/6 GPT-3 model that lies in the heart of ChatGPT is gigantic. It has 175B weights and requires 800 Gb for storage. Its training cost is estimated to be This makes it impossible to train the model from scratch for all but a few potential users.