I think it's very much a process of experimentation. There are certain elements of gamification that I have seen online that has been motivating that I've never seen anyone have a negative experience with.

For example, I don't think forum titles based on total post count ever caused any negative outcomes other than perhaps raising the number of junk posts a few users made to pad post count as they chased titles. This is largely a non-issue on a site like this in my opinion and I'd go as far as suggesting that more fluffy engagement is not a bad thing, encouraging people to interact in ANY way at all seems good to me and runs in contradiction with reddit philosophy which has always been "don't interact unless you've got something genuinely worthwhile to add" at least in the old days.

I also don't think the trophy system on reddit is bad. They're essentially like military medals given for distinguishing yourself in service. This can definitely be done to encourage (and reward) extremely high effort posting. The parameters for it can be kept a secret as well so that admins can hand them out personally for certain things on a case by case basis (without ever admitting to doing so).

There's definitely elements of gamification I would not want to go near and care in the specific things that are explored is needed so as not to upset the ecosystem. With that said though the removal of downvotes was a DRASTIC change in direction for the site and I think very few people would argue it was a bad choice now. I think the community has a lot of trust in the team and is willing to let it experiment a bit and decide if certain things are good or bad after testing.