Isn't the whole point of Goku that his family were low class nobody's and that's why he was sent to a backwater planet like Earth?

Yes, but then he started his own bloodline and now everyone important in the DBZ universe is his kid, his grandkid, or an immediate relation.

Like the whole reason Vegeta is always so pissed is that a low class Saiyan like Goku keeps being stronger than Vegeta, a Prince.

I think one of the most recent movies finally has Vegeta win a fight against Goku and claim the title of Most Powerful Saiyan. But the implication is that this is just a continuation of their endless rivalry that will echo through the ages.