The problem is they get treated as if they're a member of the opposite side, not a third viewpoint. I can't count the number of times I've been called a Nazi and a communist in the same day.
FWIW, coming from the far left here, I don't really consider either of these positions "opposite" me but they are both clear opponents to my vision of meaningful change.
There's no neutrality or inherent virtue in a centrist position: it's a side like any other and you don't get credit just for inhabiting it. That doesn't make anyone calling you a nazi or communist justified in doing so or correct in their assessment. But it does indicate that neither considers your stance a compromise or allied position in relation to their own.
There's also no value in being an extremist either, you're not a significant member of any party that would lead to change in this country, nor will any of your goals come to fruition, you are outnumbered heavily and it looks like thankfully that will be the case for the remainder of my lifetime.
I find that I must act in alignment with my moral values to the best of my understanding and ability. It's not up to me to discover or calculate the "value" of those results, and that's not any part of my motivation.
I don't know what it would be like to select political goals based on their likelihood in the face of such incredible violence and injustice as we create. I'm grateful I can't lathe my soul in such a way. Seems chill though.
Not too long ago, advocating for gay marriage was considered an extremist position, but that eventually passed. On the other side, advocating for the illegality of abortion even for a child rape victim would be considered an extreme position in most of the US even though it is the current law in several states.
The Overton window can move surprisingly quickly, and policies which you consider extremist now won't be in a decade.
What fraction of people have to agree with something before it isn't extreme? Support for marriage equality among Americans was as high as 25% even back in 1990, if XKCD is to be believed.
