I think some would call them that, though it depends a little bit on the individual group. I would lump most trots in with ultras personally, as they tend to do all the usual ultra things like spending more time denouncing AES than working on overthrowing capital, and trots especially seem to more frequently abandon their ideals as they get older, becoming "how I left the left" sorts of conservatives.