It might be different from country to country. In my country the liberal consensus is that if we're having a "free and honest debated on foreigners" (ie. politicians competing about who can be most racist) then people will not turn to "the far right" (i.e. wear swastikas, if you don't wear a swastika then you're just ordinary "right", not "far").

Our dear succdem minister of justice is even on the record setting that he believes lax hatespeech laws acts as a release valve, preventing "real" racism.