Stop caring. There is nobody you could be that will please everybody, and if there was...there'd be nothing of you left as you morphed to fit the situation and company around you. Just be yourself and ignore their passive aggressive BS or avoid if possible.

(yes, it is that simple but still harder than it sounds. Ask yourself "Do I need to give a shit about this?" and the answer will usually be no.)