This seems not to be the only reason:

There seems to be a consistent tendency that some people downvote comments not because the comments are wrong but because accepting the comment would undermine their position/opinion. See anything that touches into google/apple/feminism