I think the issue with vegetarianism is that it doesn't accept the premise that animals should not be exploited. Vegetarianism supports much of the exact same inhumane and cruel acts that the meat industry commits, and they're inextricably tied. It's really odd to say "Killing and exploiting animals is fine as long as flesh isn't actually on my plate." In all honesty, a vegetarian's hypothetical inverse, i.e. someone who eats meat but not dairy, eggs, and honey, could be more "ethical" than them in the grand scheme of things. Due to the fact that vegetarianism tends to manifest symbolically ("as long as flesh isn't actually on my plate..."), there will be less of a desire for people to adhere to it with serious ethical principles in mind. It's hard to say "Killing animals is wrong, and that's why I'm a vegetarian" when vegetarianism still more-or-less funds industries that are, once again, inextricably tied to the blood that the meat industry has on its hands.

And just to clear up this misconception while I'm at it, veganism isn't "perfect" either, but I don't see it as such. I see it as a moral baseline actually. Especially under capitalism, no one can avoid the cruel acts that happen to animals in its perfect entirety, but veganism seems to be a practical direction to start (for many at least, I will concede that not literally every single person on the planet can go vegan), but it's very difficult for carnists to accept the premise of ethical veganism. That's why excuses that aren't really good are seen as adequate justifications for carnists to not be vegan. Bottom line is that, since I already see veganism as imperfect, I don't really see vegetarianism as a reasonable thing to enable or promote.