The TL;DR is that human feedback is the "special sauce" that puts ChatGPT apart from its predecessors, but it 1) introduces a lot of places for humans to bias the algorithms 2) can depend on absolutely terrible human labor conditions