The idea that Nobunaga "Preferred men" isn't really well known. He maybe had a gay relationship with a person who held a similar post to Yasuke (Mori Ranmaru), but he also had multiple children with a concubine.
Now its entirely possible that he preferred having sex with his retainer to his concubine, who the heck knows. I certainly don't. But I don't think there's much support for the idea.
But let's accept that. Let's even assume Nobunaga was gay as hell and any sexual encounter with a woman he had were out of duty or something. That still doesn't mean we can just assume that Yasuke was a sex partner to Nobunaga. We don't have any evidence for any kind of relationship like that between Oda Nobunaga and Yasuke as far as I'm aware, and I think we would if it was well known.
The relationship between a personal attendant like Yasuke or Mori Ranmaru with their lord would likely take the form of Shudo, a sort of formalized typically pederastic homsexual coupling. Shudo was expected to be at least somewhat exclusive, and Nobunaga was already supposedly with Mori Ranmaru.