I have the best success using a towel snap against flies.

Sometimes I think I don't even hit them but it stuns them so they fall to the floor.