In a way, RLHF alone cannot possibly inject this level of (seemingly additional) world knowledge demonstrated in ChatGPT (in comparison to GPT-3). This level of inherent capabilities can only come from scale.