My may concern with this system: Sledgehammer meets tack.
The comments on HN aren't perfect, but they're far from bad when compared to other sites of this nature. There has been a downwards trend most probably due to the increasing popularity of HN. A response is warranted. However, this system has the potential to silence a lot of high quality comments on any threads that aren't on the front-page for an extended period of time. Thus, you get a feedback loop. Good posts require quality discussion to stay on top, but must stay on top to get quality discussion going with this added approval lag.
I think you should ease these changes in as conservatively and gradually as possible. For example, apply it only to the top page at first, and reduce the number of endorsements required for display to 1. You might also consider merely greying out comments that have not yet been endorsed, as currently happens to down-voted comments. Another option would be to apply the endorsement system only after threads have reached a certain age so as to jump-start discussions. Additionally, I would recommend that authors of a parent post should be able to see all child posts regardless of their karma. Below, Babuskov raised the point that the endorsement system will obstruct useful back-and-forth discussions between sub-kilokarma users in buried threads that often takes the place of a private messaging system on HN. This would fix that more effectively than merely reducing the endorsement requirement.
You should not entertain any illusions that you can flip the switch and watch this system work perfectly, and that you will therefore be able to avoid confusing people with many changes over a lengthy period of time. Tweaking will almost certainly be required.
I'm not sure that I qualify to contribute a reply (and that may be a problem here) - I've read HN for 4+ years and have only started actively contributing very recently on this account - I have neither karma nor history of posting in my favour.
One of the most powerful things about HN is that it is currently not like reddit - we don't end up with 400+ comments that sit at the bottom of the page gathering dust and downvotes. I feel that a system of pending comments plays better in a more moderated community - especially so as sometimes the most seemingly innocuous comments here can be important or generate discussion that would not otherwise occur.
If the system were to be brought in immediately I'd like to see something along the lines of a 24 hour window; all comments are published after 24 hours with karma directly correlating to the reduction of that window - i.e. Those users with karma of 1440+ can comment with relative impunity on the assumption that their past contributions merit their voice being heard instantly, whilst users with lower karma must either wait for their comment to be 'approved' by a higher user or of course wait out the period of 1440-karma. A relatively simple fix, I would hope, given that you are willing to let 1000+ karma users basically moderate the comments.
I absolutely agree with beloch that these changes should be eased in as conservatively as possible: the worst thing that can happen to HN, and the tech industry as a whole, is presenting itself as a closed community in which you must gain favour to advance - we all quite literally live on the new blood that so often disrupts the status quo.
This really worries me as well. I'm in the somewhat the same boat as you, a good relatively recent anecdote is one of my early threads, an Ask HN thread which while it never got more than a few votes, got enough productive answers and discussion that it cleared up some issues on my part. (also, the entire discussion took place within the first 24 hours by primarily low karma users, after which the thread was dead.) If that discussion wouldn't be feasible without "approval" that entire exchange and the benefit I drew from it would have been unobtainable.
I must admit I do not feel compelled by this change.
I think this is a great idea, it's simple (one karma = one minute) and it addresses many of the problems I foresee with this new rules, in particular getting stuck in "pending limbo" indefinitely (given that you can't post if your previous comment is still pending).
It still makes it take extremely long for a new user to obtain 1000 karma, though. Much harder than previously, I imagine. Although maybe not, since there will be less posts visible, those that are will get relatively more karma points?
I agree with this entirely. As it currently stands, implementing this policy in such a abrupt way will hurt discussion and negatively impact the quality of the community.
And although there are ways to improve the actual implementation of the system, I still think it is fundamentally flawed for a few reasons.
First, any system in which a comment is assumed to be spam or trash until proven otherwise will produce less meaningful discussion. Maybe most of the good comments will be filtered through; but some will invariably be missed, and people with valid ideas will not have their comments seen by anyone. Moreover, from my understanding, the HN user base is primarily in North America. (I could be wrong.) If this is true, won't users in other countries be disproportionately affected by this kind of oversight because there are simply fewer high-karma users in the rest of the world?
This is fundamentally a debate over whether HN should have 50% fewer good comments and 100% fewer bad ones, or whether we should just get everything. I believe that latter.
Second, this program places excess responsibility in the hands of high-karma users. HN is now moving from a passive system (eg. one where everything is visible and the best stuff gets voted to the top), to one where every comment must be scrutinized and evaluated just to be visible. Logistically, I don't see how this works. Users with over 1000 karma are rare. From what I've seen they are less than 5% of the total HN user base. This minority is now going to be responsible for evaluating every single comment?! I truly do not see the small number of high-karma users on HN being able to sort through every comment. Many, many comments will be missed or ignored.
Overall, I believe that any effort to stop spam on HN should be given consideration, but this system poses substantial logistical problems and will, in all likelihood, hurt the community.
I do particularly enjoy the mostly civil posts here. I've always held out HN as an example of an open community commenting system that works. I personally disagree with the new policy.
Since I don't comment often, this probably doesn't mean much, but this is the last comment I'll post on HN until the policy is reversed.
I didn't even know HN had a commenting problem until HN told me. As is often the case, the comments are pretty informative on this topic too.
Well, so long everyone. Logging out for now. Someone please let me know on Twitter if this gets reversed.
"Users with over 1000 karma are rare. From what I've seen they are less than 5% of the total HN user base of."
But a much higher percentage of people who actually post. The top 10 posts on this page (for me, right now) are about 50% from such users, and of the top 10 newcomments right now there's only 3 from users with 1000+ Karma, but several more from >900. If you're participating on the site for a while, it doesn't take that long.
You might also consider merely greying out comments that
have not yet been endorsed, as currently happens to down-
voted comments.
I agree with this, except that pending comments should probably be visually differentiated from downvoted ones (maybe a shade of green like new usernames). Otherwise, you risk subtly prejudicing a reader that has grown accustomed to the original meaning of the greying-out.
Either way, I'd advocate for a "showpending" toggle for users with under 1000 karma.
I think it's a good idea, as the quality of comments have been degrading.
Perhaps after a post has reached a certain age (few hours) comments would slowly open up to users with less and less karma. Just a thought, but I agree a "show pending" toggle would be nice too.
It is important that we encourage new users to comment on HN. The reality is that early posts from new users may not make the grade set by 1000+ Karma users. Let them be visible as otherwise you may discourage such new users from re engaging/commenting further.
Color coding comments to flag they as 'pending' is fine. But complete hiding them sounds too strong
Its great to have the conversation on how to improve HN comments
It's possible for HN to be increasing in popularity while decreasing in # of submissions. For instance, if more users are coming to the site, the site is growing in popularity. If the X percent of users who normally submit the most frequently are decreasing their submission rates, submissions will go down despite the user growth.
Of course, this scenario assumes that # of submissions follows a power law distribution: a small percentage of posters make the majority of submissions. That pattern emerges on many content sites, especially sites with active forms of communal self-moderation. I have no idea if it's the case here, but it seems feasible.
IMO, the quality has degraded due to more people viewing the site, not the submissions. If I get my post on the frontpage of HN I can expect roughly 1,000 hits per hour. This wasn't the case 4 years ago when I first starting lurking here.
There are two problems with this:
1. More people (of poor quality) upvoting stories that attract the most attention (link bait, stories that aren't relevant, etc)
2. More people (of poor quality) upvoting comments that are not relevant to the discussion.
No graph will be able to chart this - it's purely subjective.
Does that graph include comments under "submissions", or only stories? I bet the latter. There are only so many conceivably-appropriate-for-HN stories out there, and many of the rest get taken out as spam.
If the number of comments, on the other hand, hadn't gone up since 2012, I'd be shocked.
To me a problem with karma systems that are based on a specific relatively well-defined ruleset is that you still end up with a score that doesn't really convey much. For instance, the karma I have is made up of meaningful discussions, which involve telling others about relevant facts, as well as one liner jokes and a few quotes from random dead folks that are smarter than I am.
I always wonder how useful it is to really try to get down to defining what it is you want out of the conversations, and then giving the ability to narrow based on that. Obviously, the narrowing would have to be quick, because it's hard enough to get people to click an up/down arrow once, but if it were to include even two or three more specific categories, or flags, could you put enough information into the system to automate some of the problem folks.
For instance, having a way for users to flag a post as vitriolic, or not containing real information, and then collating the data and either stopping that user from posting, or putting them into a state where their posts are pending.
I also wanted to mention that it seems as though he does address not having illusions in the last sentence: "I hope this will improve the quality of HN comments significantly, but we'll need your help to make it work, and your forbearance if, as usually happens, some things go wrong initially."
I think all of that spitballing is predicated on having a well-defined course you'd like conversations to follow with well-defined rules that don't block the flow of useful discourse, or stifles the community.
This isn't a topic I've done any research on. Just throwing some thoughts out there. I appreciate the parent post, though, because it raised a lot of notions I hadn't thought of immediately when I read the pending idea. I think I tend to lean toward giving people with higher karma the ability to flag people into a status that makes there posts go into pending, according to a set of defined criteria (like were posted) rather than starting folks in that state. I think it would be less impactful to the rate of conversation, but obviously then you have people who make accounts to get around it, which would be solved by the solution to be implemented.
Also everything I'm suggesting is complicated and possibly overkill/overengineered. I wonder if there's an elegant way to get more useful classification information into the voting system to give it more focussed goals.
It's been a decade since I've posted there with any regularity, but this more or less lays out why I feel like I still haven't seen a more elegant solution to moderating comments than Slashdot's metamoderation system.
Comments not only receive a +/- minus score, but they receive a reason for that score. Additionally, meta-moderators are selected randomly from eligible moderators who go behind and moderate whether the scores given to a particular comment were justified. Get metamoderated down often enough and you lose your ability to moderate.
It baffles me that every site on the web keeps using simple up/down systems and then complains about how hard it is to generate quality discussion when this is hardly a new problem set.
I haven't visited Slashdot on a regular basis in the better part of ten years, but it tended to 15 years ago. Back then, it was the kind of place where you'd find people like John Carmack randomly chiming in on a thread.^1
And that's also entirely beside the point.
Does HN have the same level of comment quality as reddit? They both use simple up/down systems.
The point is that Slashdot's metamoderation system better leverages the community as a means of directing discussion. The initial upvote has a reason attached to it, and the meta level moderation allows the community to filter out people that are moderating inappropriately. What constitutes appropriate is going to depend on the standards in place for that community.
I wouldn't expect to see an option to moderate a comment +1, Funny on HN, for example. The system itself, however, is designed specifically to bring quality content to the top.
But I think you're making the mistake of conflating the design of a system that is meant to address one problem with the state of a site that declined for other reasons years ago.
> It baffles me that every site on the web keeps using simple up/down systems and then complains about how hard it is to generate quality discussion when this is hardly a new problem set.
It sounded like you were saying "if those sites just used a better moderation system, it would be easy to generate quality discussion". But the moderation system you propose does not generate quality discussion on slashdot.
If that's not what you're saying, then I'm not sure what particular benefit you think slashdot's sytem does have. What does it mean for a system to be good at directing discussion, but not at generating high-quality discussion? Why is it useful to be able to revoke mod privileges, if that doesn't generate high-quality discussion?
Or are you saying something like, it would be easy to generate quality discussion with slashdot's system given certain other factors (where simple +/- does not make this easy), but slashdot lacks those factors?
You bring up another interesting point for me. How do you even know when the problem is or isn't solved? There are two common themes I see in forums like this (and of course, many others of varying nuance). One group has been around for a long time and says it's just not like the old days, and a second group says that it is, and that the former are looking at the past through rose tinted glasses.
I can't objectively say that slashdot was better or worse here. Firstly, because I didn't go to slashdot nearly as often as I do here, and secondly, because I haven't done anything to thoroughly define and measure what I would consider a quality community.
If my problem is that the community is bad, I think I need a way to show to myself that it really is. I need a set of guidelines which I want my community to interact according to, or standards for the goodness I'm trying to push, and a way to see how much goodness there is, then a plan to drive that goodness.
And I'm not even saying that you need to be rigorous about it. Maybe that would help, but I think you can get some ideas with some general notions like tiny surveys, or something like that. I guess you could start getting more interesting and use sentiment analysis after auto-classifying posts, looking for meta content talking about the board.
As another aside, I think more specialized, smaller fragmented communities tend to be better for me, which is why I also enjoy subreddits like r/types and such, but again, I think that by no objective standard
>you still end up with a score that doesn't really convey much
Exactly. Mostly I try to make constructive comments, but the most upvotes I have received were on comments which were hateful to one of the tech companies - they might still not be the best comments in pg's dictionary.
They can roll it out tonight and immediately roll it back if it is stifling conversation in a way that causes more harm than it resolves. They can also iterate on it if any part of it works in a way that isn't optimal.
Essentially, if you trust HN/YC to not leave something horribly broken, there is nothing to worry about. The code is not set in stone, and you can bet they'll be watching closely for anything not working well about the new system.
Social systems can be broken in a way that won't manifest until a certain type of discussion comes up. So while I trust HN/YC to do their best to protect and promote discourse, I don't know that the types of conversation that come up during the evaluation period will be representative of all future conversation on HN
That's true. The same argument can be made for any system that is put in place, though. Even the current one likely has types of discussions that "break it" in some way. There's even evidence of such breakage if you include arguments/bickering as something that the system should prevent.
We can summarize this change as moving from a system where (at least some of) the system's weaknesses are known, to a system where the people running HN believe it should be a better system, but it still probably has points where it breaks in some way. Doing changes like this and reacting to breakage is the only way that progress can be made.
Like you said, this change may lead to the case that "high quality comments on any threads that aren't on the front-page" will not be shown. The lack of those comments and subsequently the lack of a vibrant discussion might discourage other users from participating, leading to the article never making it to the front page.
I foresee this causing an even worse problem than low quality comments: The front page will be flooded with stories that make it their not because of the quality of the accompanied discussion, but because of the "clickability" of the article title.
However, it seems like there is a relatively simple solution. The number of "endorsements" required to activate a comment could be inversely proportional to the articles points. Or as some other users have suggest, it could be inversely proportional to the quantity of other comments on the same article.
I'd also like to point out that plenty of us with plenty of karma also have some degree of RSI (repetitive strain injury, sometimes loosely called carpal tunnel syndrome (that's just one of many forms of RSI)), in part acquired by what allowed us to get so much karma.
No way can I afford to spend a lot of mousing and clicking on a zillion "pending" comments to make them visible to the hoi polli. Heck, ignoring the greater good, why would I personally care, I've probably got enough karma to see everything.
Good to see someone point out a flaw on basing the success of the site on the dependability of the 1k+ group. They need not all be benevolent dictators (or even interested) in spirit.
I share your concerns. I think that this system would be best applied in combination with some sort of flamewar detection. In other words, only enable it in threads that have high comment:upvote ratios, or in threads where there is high velocity back-and-forth or downvoting.
The comments on HN aren't perfect, but they're far from bad when compared to other sites of this nature. There has been a downwards trend most probably due to the increasing popularity of HN. A response is warranted. However, this system has the potential to silence a lot of high quality comments on any threads that aren't on the front-page for an extended period of time. Thus, you get a feedback loop. Good posts require quality discussion to stay on top, but must stay on top to get quality discussion going with this added approval lag.
I think you should ease these changes in as conservatively and gradually as possible. For example, apply it only to the top page at first, and reduce the number of endorsements required for display to 1. You might also consider merely greying out comments that have not yet been endorsed, as currently happens to down-voted comments. Another option would be to apply the endorsement system only after threads have reached a certain age so as to jump-start discussions. Additionally, I would recommend that authors of a parent post should be able to see all child posts regardless of their karma. Below, Babuskov raised the point that the endorsement system will obstruct useful back-and-forth discussions between sub-kilokarma users in buried threads that often takes the place of a private messaging system on HN. This would fix that more effectively than merely reducing the endorsement requirement.
You should not entertain any illusions that you can flip the switch and watch this system work perfectly, and that you will therefore be able to avoid confusing people with many changes over a lengthy period of time. Tweaking will almost certainly be required.