We don't.

I'd figure even going full stereotypical gender roles you'd have some on a berry hunt since what if you later encounter edible edible plant or fungal life on a standard hunt or fishing trip? You going to let those calories just sit in the sun? No, hunting is a slow act, there's enough time to gather berries or shrooms or whatever so you don't go home without food if the prey evades you. I'd expect hunts to be a lot more varied than expected since there's so much work to be done. Even our proto human ancestors knew more is better when it comes to fulfillment of work capacity. I also suspect gender roles as we think of them is a relatively modern thing.