It suggests GPT2 was not considered that big a deal at the time. IIRC they were really into RL at that point. I read a Brockman tweet this week painting the decision to release ChatGPT was a bit spur of the moment; maybe it was like that.