if we wanted to match the compression ratio of GPT3, (again assuming 100t params) we would need 1,142 TB of text. i'm sure chatGPT's free usage is definitely in part linked to gathering data, but its a small drop compared to what's needed, and is likely more for RL tuning