Its exciting because it gives a model access to the long tail of code. We made The Pile by downloading the top N repos by star count. Thats great, but I suspect lots of the skill of ChatGPT is from all the rest of the repos. 2.7TB would