To me a problem with karma systems that are based on a specific relatively well-defined ruleset is that you still end up with a score that doesn't really convey much. For instance, the karma I have is made up of meaningful discussions, which involve telling others about relevant facts, as well as one liner jokes and a few quotes from random dead folks that are smarter than I am.
I always wonder how useful it is to really try to get down to defining what it is you want out of the conversations, and then giving the ability to narrow based on that. Obviously, the narrowing would have to be quick, because it's hard enough to get people to click an up/down arrow once, but if it were to include even two or three more specific categories, or flags, could you put enough information into the system to automate some of the problem folks.
For instance, having a way for users to flag a post as vitriolic, or not containing real information, and then collating the data and either stopping that user from posting, or putting them into a state where their posts are pending.
I also wanted to mention that it seems as though he does address not having illusions in the last sentence: "I hope this will improve the quality of HN comments significantly, but we'll need your help to make it work, and your forbearance if, as usually happens, some things go wrong initially."
I think all of that spitballing is predicated on having a well-defined course you'd like conversations to follow with well-defined rules that don't block the flow of useful discourse, or stifles the community.
This isn't a topic I've done any research on. Just throwing some thoughts out there. I appreciate the parent post, though, because it raised a lot of notions I hadn't thought of immediately when I read the pending idea. I think I tend to lean toward giving people with higher karma the ability to flag people into a status that makes there posts go into pending, according to a set of defined criteria (like were posted) rather than starting folks in that state. I think it would be less impactful to the rate of conversation, but obviously then you have people who make accounts to get around it, which would be solved by the solution to be implemented.
Also everything I'm suggesting is complicated and possibly overkill/overengineered. I wonder if there's an elegant way to get more useful classification information into the voting system to give it more focussed goals.
It's been a decade since I've posted there with any regularity, but this more or less lays out why I feel like I still haven't seen a more elegant solution to moderating comments than Slashdot's metamoderation system.
Comments not only receive a +/- minus score, but they receive a reason for that score. Additionally, meta-moderators are selected randomly from eligible moderators who go behind and moderate whether the scores given to a particular comment were justified. Get metamoderated down often enough and you lose your ability to moderate.
It baffles me that every site on the web keeps using simple up/down systems and then complains about how hard it is to generate quality discussion when this is hardly a new problem set.
I haven't visited Slashdot on a regular basis in the better part of ten years, but it tended to 15 years ago. Back then, it was the kind of place where you'd find people like John Carmack randomly chiming in on a thread.^1
And that's also entirely beside the point.
Does HN have the same level of comment quality as reddit? They both use simple up/down systems.
The point is that Slashdot's metamoderation system better leverages the community as a means of directing discussion. The initial upvote has a reason attached to it, and the meta level moderation allows the community to filter out people that are moderating inappropriately. What constitutes appropriate is going to depend on the standards in place for that community.
I wouldn't expect to see an option to moderate a comment +1, Funny on HN, for example. The system itself, however, is designed specifically to bring quality content to the top.
But I think you're making the mistake of conflating the design of a system that is meant to address one problem with the state of a site that declined for other reasons years ago.
> It baffles me that every site on the web keeps using simple up/down systems and then complains about how hard it is to generate quality discussion when this is hardly a new problem set.
It sounded like you were saying "if those sites just used a better moderation system, it would be easy to generate quality discussion". But the moderation system you propose does not generate quality discussion on slashdot.
If that's not what you're saying, then I'm not sure what particular benefit you think slashdot's sytem does have. What does it mean for a system to be good at directing discussion, but not at generating high-quality discussion? Why is it useful to be able to revoke mod privileges, if that doesn't generate high-quality discussion?
Or are you saying something like, it would be easy to generate quality discussion with slashdot's system given certain other factors (where simple +/- does not make this easy), but slashdot lacks those factors?
You bring up another interesting point for me. How do you even know when the problem is or isn't solved? There are two common themes I see in forums like this (and of course, many others of varying nuance). One group has been around for a long time and says it's just not like the old days, and a second group says that it is, and that the former are looking at the past through rose tinted glasses.
I can't objectively say that slashdot was better or worse here. Firstly, because I didn't go to slashdot nearly as often as I do here, and secondly, because I haven't done anything to thoroughly define and measure what I would consider a quality community.
If my problem is that the community is bad, I think I need a way to show to myself that it really is. I need a set of guidelines which I want my community to interact according to, or standards for the goodness I'm trying to push, and a way to see how much goodness there is, then a plan to drive that goodness.
And I'm not even saying that you need to be rigorous about it. Maybe that would help, but I think you can get some ideas with some general notions like tiny surveys, or something like that. I guess you could start getting more interesting and use sentiment analysis after auto-classifying posts, looking for meta content talking about the board.
As another aside, I think more specialized, smaller fragmented communities tend to be better for me, which is why I also enjoy subreddits like r/types and such, but again, I think that by no objective standard
>you still end up with a score that doesn't really convey much
Exactly. Mostly I try to make constructive comments, but the most upvotes I have received were on comments which were hateful to one of the tech companies - they might still not be the best comments in pg's dictionary.
I always wonder how useful it is to really try to get down to defining what it is you want out of the conversations, and then giving the ability to narrow based on that. Obviously, the narrowing would have to be quick, because it's hard enough to get people to click an up/down arrow once, but if it were to include even two or three more specific categories, or flags, could you put enough information into the system to automate some of the problem folks.
For instance, having a way for users to flag a post as vitriolic, or not containing real information, and then collating the data and either stopping that user from posting, or putting them into a state where their posts are pending.
I also wanted to mention that it seems as though he does address not having illusions in the last sentence: "I hope this will improve the quality of HN comments significantly, but we'll need your help to make it work, and your forbearance if, as usually happens, some things go wrong initially."
I think all of that spitballing is predicated on having a well-defined course you'd like conversations to follow with well-defined rules that don't block the flow of useful discourse, or stifles the community.
This isn't a topic I've done any research on. Just throwing some thoughts out there. I appreciate the parent post, though, because it raised a lot of notions I hadn't thought of immediately when I read the pending idea. I think I tend to lean toward giving people with higher karma the ability to flag people into a status that makes there posts go into pending, according to a set of defined criteria (like were posted) rather than starting folks in that state. I think it would be less impactful to the rate of conversation, but obviously then you have people who make accounts to get around it, which would be solved by the solution to be implemented.
Also everything I'm suggesting is complicated and possibly overkill/overengineered. I wonder if there's an elegant way to get more useful classification information into the voting system to give it more focussed goals.
Well. Enough rambling on my part.