A lot of what ChatGPT has seen (during pre-training, which I assume you are counting in the calculation) was garbage though. Far better and smaller models could be trained if they could "read" what domain experts read, but unfortunately, most