I wouldn't consider downvotes to mean something is prohibited. I did in the past because that's what I limited myself to downvoting, but people just downvote what they don't like for whatever reason. All you can do is to not let it get to you.