Any insults coming from their enemies are negated in their moral connotation because they come from people they perceive as absolutely evil. If you're described as a Nazi by someone you think is evil, it's like you're actually a Nazi that's morally good, but with all the other attributes of a Nazi. Same for dictator, threat to democracy, etc. But being called weird just makes them, well, weird. There isn't really a moral connotation to invert. It's just an accurate statement that, even when flipped around, still stings.