Lets say LLM models like chatGPT are using human preference RL at scale. Then failure improve performance significantly as many predict could come down to multi-agent RL problems. Preferences change, credit assignment is convoluted.