At a guess: they found they needed a different tokenizer to deal with code text (chatgpt descends from a code-trained variant). Itd split unrecognized stuff down to single characters instead of bigrams.