Also, so much for 1-on-1 comment threads that are deeply buried and are not intended to be prominently displayed to anyone else. I've had lots of interesting conversations like that.
There's something like a "scrolling cost" -- people are only willing to skim so much of a comments thread, without seeing something interesting to them, before closing it.
This is why HN dislikes humorous fluff-posts: they both easily rise to the top, and encourage humorous fluff-replies, which means the first few screenfuls of comments will be guaranteed to induce the kind of "scroll-pain" that makes people close the tab.
Collapsing threads do the job pretty well. And you're always going to get comments that aren't relevant to you, because you're not the only person doing the endorsing. If a comment is useless fluff to you, but gets endorsed by someone else, then you still have a scrolling problem. Not so with collapsable commeents.
They use a very simple system, collapsing posts after a certain depth, and hiding more comments after a certain number (10, I believe) at the level below the top comment (sorted by score obviously).
It works extremely well, and thanks to the fast JS collapsing, it's not at all a hassle to read a subthread that happened to get collapsed if it piques your interest.
It's so simple it may easily be overlooked in its obviousness, but really you don't need a very complex system that is strictly a lot better than no collapsing at all.
What's wrong with (old) slashdot (I mean, technically) ?
While I'm usually a kind of hard core html-first, ajax/js/webapps later kind of guy -- I'd love for the comments to be served up in a json-blob, with a couple of user-settable preferences ("Show only comments rated higher than N, hide threads with lower (median/mean) rating than N, show all direct replies to my comments -- and similar).
You have to read a thread enough to know you want to collapse it, which incurs exactly the same "scroll-pain" as actually scrolling. It's not a physical tax from the action required to scroll, but rather a mental stress from making a decision to skip something. Enough of that stress building up at once, and you decide that the comments page itself is probably skippable.
If uninteresting comment threads could start collapsed, that'd be great. But they won't, because humor-fluff and other such things are superstimuli for upvotes, so you can't use anything about the vote tally to determine collapsed-ness. (If there was a secondary voting system--like, say, if enough people collapsing a thread would make it start collapsed for others--this might work. But then you'd have to take into account the people who collapse everything as they read it, to mark their place...)
The problem is, your definitions of what is uninteresting, useless and fluff are subjective, and no more relevant than anyone else's, which leads to a conflict of interests within the community about the bounds of what Hacker News content should and shouldn't be.
I think the models which would satisfy the most people are opt-in, in this case, choosing to collapse threads and ignore users rather than expecting the hivemind to do it for you.
Perhaps if users had a custom set of filters which automatically collapsed threads for them based on their own criteria, that would solve part of the problem. But I don't think it's too much to ask of people to actually take the minor effort to form an opinion on what they read, or curate their own account, if they expect content they don't like to be hidden from themselves and potentially from others.
Disagree on your first point, the top comment is often obviously spawning a huge argument that goes nowhere because it touched some hot-button political issue that was a minor part of the story.
Of course this is the most obvious solution, least intrusive that changes the least things and as far as I can see has only positive side-effects.
But for some reason or other pg won't do that.
I know there's some extensions/bookmarklets and I use them, but they can't do the auto-collapsing based on score+amount, the very simple yet elegant algorithm Reddit employs, pretty much the one thing that is absolutely necessary to have a well-usable threaded commenting system.
Instead, pg "just wrote the simplest thing first", or something.
I think there's a difference between a constructive conversation and a flame war.
If the worry is that comment threads are too long, HN could implement something like reddit, where you click to read additional comments in a long thread.
Two commenters X,Y with sub-threshold karma (<1,000) could never have a dialogue (two-way, real-time) as a third party Z would need to endorse their each and every comment.
Two commenters X,Y (Karma 1,000+ each) in substantially different timezones with a-synchronous dialogue (eg, overnight replies) would need a third party z to endorse each and every comment (at least until the other wakes up).
Two commenters X,Y (Karma 1,000+ each) with opposing views, could never have a real-time dialogue without a third party z to endorse each and every comment (unless #)
Hopefully these are at least helpful to dilineate.
I think for what HN aims to be, keeping everything public is a good thing. I seem to find quite a few "one-on-one" threads that get the occasional input from a third,fourth,fifth participant -- and also a few I find interesting even if I'm not participating.
When everyone knows that what's being discussed is public, it tends to keep the tone more conversational and clear -- I think. The "feel free to contact me, email in profile"-response seems to work well enough for those that do want a private (albeit not anonymous) conversation?
