Sleeping in caves is the enemy of natural selection. It lets people with too little body hair survive the cold.

In long-long-long terms it will screw people over when the next big ice-age comes.