Couldn't we have a lead box lined with these radiation to electricity converters with a small amount of radioactive material in the center, and have an energy generating device that would last for thousands or even millions of years? Imagine putting the sun in a box lined with solar cells, but on a much smaller scale.
Is there a reason this wouldn't work?
No one has directly answered your question.
The answer is yes, you can create photovoltaic cells better optimise to pick up high energy light such as that from nuclear decay (gamma radiation). However, the power generated by photovoltaics is limited more by intensity of the light, and not the energy per photon (wavelength). For physical reasons is hard to capture the energy of high energy light, so gamma photovoltaics are low power concepts.
There is an idea going around to grow diamond with c14 and also harvest that c14 decay with a diamond based photovoltaic. Making everlasting batteries, albeit radioactive and microwatt. (Specifics are probably wrong, working from memory.)