They get arrested, and for community service, they have to do the quest or get hanged..hung.
Punish murderhoboing severely. Party gets wiped and it won't be fun.
A god has decided to smite your murderhobo. You have been struck by Holy lightning for 100d20 damage. You have disadvantage.
I was a player in a campaign where two of the other players just full-on sprinted past every sign post the (Pathfinder) GM threw up saying, "This character is here to give quests. He is much higher level than you. Don't attack him. He will slaughter you."
They pulled out their weapons and made to attack the guy and my CN character was like, "You have fun with that. I don't want to die today." The quest giver and our other party member tried to talk them out of it for a couple rounds AFTER they started attacking (and doing no damage, naturally). The NPC then started using non-lethal spells to try to defuse the situation. A couple rounds of that and the GM finally gave up went hostile. Two or three AoE attacks later and both characters were gone.
They rolled new characters, but I didn't last much longer in that campaign.
One of the things I like about the CofD games like Mage is it has a mechanical tracker for doing vile stuff. "Sure, you can curse him to vomit forever but that's going to call for a Wisdom check." Losing wisdom means you're worse at containing spell miscasts. Uncontained spell paradoxes will ruin your day. And hitting wisdom 0 means the character is unplayable.
I've stopped a lot of nonsense with "that'll be a wisdom check, if you're sure you want to do that"
Yeah, I remember playing D&D in high school. If you don't have a lot of maturity, it's easy to fall into violent power fantasies that the freedom of a TRPG offers versus a CRPG.
This is why I'm really glad for modern session 0 agendas and safety tools to set expectations and keep people on the same page.
I actually kind of embrace the players doing whatever they want, so long as it's not some way to harass other players or the DM. If they're doing something to deliberately ruin the game, that's not acceptable. If they're doing something the DM didn't expect or account for, that's just the game baybee. It be how it is