In reinforcement learning (RL), the reward function is typically hardcoded, such as the game score in Atari games. ChatGPTs data-driven reward model is a powerful idea. Another example is our recent MineDojo work that learns reward from tons of Minecraft YouTube videos: 9/