I tried different font settings in the font settings and it didn't improve much (font hinting, anti aliasing, custom DPI settings, different font size)

The font is the default one which is Ubuntu Regular with font size set to 10

Sub pixel order is set properly to RGB Linux Mint xfce

Even when running windows in a virtual machine, the font rendering in it is miles ahead of what I got on my Linux setup!!!