I don’t think the problem is that GPT is sourcing from an unreliable corpus, but that it’s taking fragments and combining them in grammatically-correct but semantically-incorrect ways?

yeah good luck with that, it's going to be a very tall order to integrate PageRank with neural networks. It's not just something you can do in a year or two.

Why? As a starting point you can importance-weight training samples with the PageRank output.