We need more research on how the human feedback being used to train models like ChatGPT (RLHF) is/should be collected. Who are the raters? How are they weighing accuracy, politeness, interestingness, etc? What proxies of wellbeing can be used to validate and improve the process?