There are over 200K language modeling datasets on Hugging Face, I bet a large portion of them were generated with LLMs, and all LLMs to date have been trained on copyrighted data. So they are all tainted.

But philosophically, I wonder if it's allright to block that, it techincally follows the definition of copyright. It does not carry the expression, but borrows abstractions and facts. That's exactly what is allowed.

If we move to block synthetic data, then anyone can be accused of infringement when they reuse abstractions learned somewhere else. Creativity would not be possible.

On the other hand models trained on synthetic data will never regurgitate the originals because they never saw them.