I’ve always gotten along better with women than men, especially as I started reaching maturity and realized just how much I didn’t fit in and didn’t like so much toxic masculinity nonsense and how I expressed how I felt

This was me as well. I used to be concerned that I was just around women because of hope for sex but 1. It turns out I wasn't and 2. Non-men just gave off so much better and more accepting vibes, for lack of a better word.

My earliest memories where social groups were forming I was always outcast from the boys, and I still don't know exactly why, but either because I didn't give off toxicity or because I wasn't welcome and learned not to be toxic from those communities, I've just always felt much more comfortable outside most male spaces. They really are awful, I've been able to join in only when being on or rooting for the same team, where the sides are predetermined. Otherwise I'm at a loss at how to act and enjoy myself in those spaces.

And now as an adult, yeah, most old male friends I have have gone one of the routes you described and even when given the time of day are difficult to talk down from their terrible preconceived masculine notions.

I think I also get along well with non-men because I otherwise present like a normal man, but I'm not a complete predatory dick and strive to be caring, considerate, and listen an learn from them. I'm fine being a cis-man and hope to be and to demonstrate how one can be that and also not toxically masculine, as society defines it, which is probably appealing to those around me who only witness the norm. Go figure being a Pinko slots into that nicely.