Maybe if the admins could make visible votes an option you could turn off, would that be a better solution? Anyone who doesnt want to see them could simply choose to do so.

And as to the algorithm you can always choose to sort by new or old. Sorting by Top, Hot, and Active are already optional, I dont see why removing them would make anything better.