The possibility is there alright, but ChatGPT for instance had a watermark embedded in the outputs thatll help the model detect when future training data is from ChatGPT. If therell be such a pollution, itll probably be from text that other LLMs have generated.