That's like saying I shouldn't say smoking is bad because it's not the action of smoking that is bad for you, it's what is in the smoke that is. Dairy is bad because dairy is a package consisting of too much saturated fat, and cholesterol and hormones which you don't need to consume at all. It's also increasingly consisting of antibiotics and other drugs which are also bad for you.
Not all dairy products contain fat. And antibiotics aren't a problem of dairy alone, but of industrial farming in general.
You make it sound like eating dairy is always unhealthy, similar to smoking. But I doubt that's the case and I don't know of any evidence supporting that.
Alright, some dairy products have less fat because producers spend money on making the product less unhealthy. But it still has cholesterol, and hormones and other bad stuff, and what are the benefits to consuming dairy over all the alternatives there is that has none of the bad stuff? There is none. Why take something like dairy and spend money and energy on producing those and making them more healthy than they really are, when you can get all the other benefits from other foods that also has even more benefits and none of the bad stuff, and also contributes to climate change much less, and doesn't need to include the exploitation, the suffering, and the murder of animals, when we don't need dairy and actually have to go out of our way to make it so it doesn't kill those consuming it?
And yes, antibiotics and other drugs is also a problem for meat products. I'm also against producing and consuming those, if you were wondering.
> But it still has cholesterol, and hormones and other bad stuff, and what are the benefits to consuming dairy over all the alternatives there is that has none of the bad stuff?
As you said yourself: Not all alternatives are better. Some of them are even worse. You'll have to take a closer look to what's inside, that's why I don't like the generalization "Dairy is bad!".