As much as I think Musk is dumb and shitty, von Braun was personally aware that concentration camp slave labor was being used to make his rockets and continued anyway. According to some prisoners, he also ordered corporal punishment of slave workers and had a hand in selecting who was fit to work and who would be sent to an extermination camp. However, we can't say for sure because of lack of documentation.

Even if we assume that the prisoner allegations are mistaken identity or whatever, I think directly being in contact with and benefiting from the Nazi death machine is quite a lot worse than Elon.