Someone implemented a framework to allow you to train PaLM (Googles 540B parameter LM) using the same reinforcement learning strategy as ChatGPT! Open source strikes again. To start, heres the part where you initially pre-train the model: