Conventionally it is used as an insult so that is why it is one. The contents of the world are not the important part here

In school there were several insults people used against each other (in Hindi) that we did not exactly know the meaning of. But like knowing they were bad is good enough