Oh.

So I looked this up out of morbid curiosity, and found what everyone already kind of knew after reading that:

Dahl initially wrote them as African pygmies who were brought as slaves to work in Willy Wonka's factory.