One concern of alignment research is the cost of aligning ML models to a goal (an alignment tax). If that tax is too steep, some will ignore alignment methods so they can lap competitors. Looks like the tax is real: chatGPT loses in-context learning!