It is pretty rare for someone born and raised in a country to have a foreign accent from one of their parents if they weren't also surrounded by more people with the same accent. I'm not sure what Barron's schooling/social life as a kid was like, but if it was more isolated than your average child because of his wealthy circumstances, that definitely contributed.