Personally I get too dang hot when trying to fall asleep with my SO. And as someone who tosses and turns back and forth about a dozen times before finally falling asleep I feel bad doing that while having them right next to me, so I try not to, which can sometimes just make it all the harder for me to fall asleep. So I guess ymmv