The solution is as follows: the entire party rolls against a DC (typically passive perception though it could just be opposed), and so long as half of the party (round up) succeeds, you are sufficiently stealthy. Realistic? Not necessarily, but it lets a stealthy player do something without having to go off on their own and get killed.