Judging racist insinuation from tone of voice, emphasis, and social context can definitely be valid. But I think it's really not a great idea to jump to negative conclusions about what a speaker said based primarily on the racial identity of the speaker. And making judgments primarily based on identity is what Paul is criticizing.