And I don't want any cop-out answers like "Oh these guys are pacifists cause they make others do the fighting for them", I mean a genuine faction of people with power, who reject war, even when it's convenient for them to do war, and even when going to war would be considered the right thing to do by the masses, out of a genuine principle of pacifism. How do you make those people seem like a convincing faction?
So I feel like you have 3 routes,
most obvious and probably requires the least weird stuff is they limit access to themselves. Either they hide from possible aggressors, they make themselves mobile so as to flee from possible aggressors (à la ComradeSharkfucker's comment) or they prevent access to their home by possible aggressors (flying city, force fields, mountain village, undersea towns).
They make harming them less valuable than leaving them alone. Say they have super advanced technology or magic and a desire to help others but refuse to help anyone who harms them and their tech or magic doesn't work for other species. Or maybe when they're killed the person who killed them is mentally merged with them permanently, which has led to an aversion to violence in their society, where taking life is normally done to preserve someone's consciousness beyond death and only by a good friend who is willing to hear their thoughts for the rest of their life. Other cultures also don't want all their soldiers to get mind melded with a bunch of pacifists and leave them alone.
Their conception of violence is different from others and they're willing to do something in self defence that others don't want to experience but which they don't think of as harmful (teleported to another continent, or dimension [a safe one], "infected" with super empathy, time-warped back into a baby [you're now going to live another 37 years how can you say I've harmed you?]).