The exception, to me, is if I'm questioning whether the comment was in good faith or not, where the trackrecord of the user on a given topic could go some way to untangle that. It happens rarely here, compared to e.g. Reddit, but sometimes it's mildly useful.

I recognize twenty or so of the most frequent and/or annoying posters.

The leaderboard https://news.ycombinator.com/leaders absolutely doesn't correlate with posting frequency. Which is probably a good thing. You can't bang out good posts non-stop on every subject.

Same, which is why it would be cool to see. Perhaps there are people I both upvote and downvote?

Either you got the direction wrong or you'd support someone who is wrong just because you like them.

You're wrong in both cases :)

Maybe try rereading my comment?

You're right. But I still disagree with you. Both ways are wrong if you want to maintain a constructive discussion.

Maybe you don't like my opinions on cogwheel shaving but you will agree with me on quantum frobnicators. But if you first come across about my comments on cogwheel shaving and note the user name, you may not even read the comments on quantum frobnicators later.