Even on the bad faith "you're anti war? Well what about the nazis?" level, there were pacifists who weren't fascists during WWII. Bertrand Russell comes to mind.

Edit: My mistake, Bertrand Russell was a lifelong pacifist, including during WWI, but supported war against Hitler. Which, IMO, makes the point even stronger. One can be against war even when it is an existential threat, and still not against war when the threat is a serious as fascism.