Helium doesn't decay, but it does rise to the top of the atmosphere, and then slowly escapes into space.

It mostly comes from radioactive decay in the earth. An alpha particle is a helium nucleus, so anything that alpha decays is slowly making more helium. I think we get a lot of our supply as a side product of natural gas extraction.