This is my overall takeaway from ChatGPT in an alignment context fwiw. RLHF is ontologically incoherent because GPT-N is not an agent, turning it into an agent is a bad idea because any subagent it samples becomes the agent. If you insist then make "browsing: enabled" out of band