Better yet (IMHO), like HN. In my field, I know where to find papers, etc. I'm interested in other fields, and not every paper; just rank them by a HN-like algorithm.
I said a few days ago, in my dream HN would be mostly papers. That's where the richest, most accurate, most intellectually curious information is - far beyond most things on HN.
But what are you reading? I'm very interested in finding more and better!
I find research papers to be an order of magnitude richer, more accurate, more thoughtful than almost anything else. Everything else is too slow and doesn't answer most of the questions I have. Research papers (and books) not only answer the questions, they take me much further than what I knew existed - what I'd hope for from experts.
Still need some solutions: How do you find the absolute gems?
That they exist - of course - is no more helpful than pointing someone to a library of 100 million books and telling them there are some gems in there.
I think people are happy to scroll/browse around. Our time is very limited; if we read the 0.001% 'best' (however defined) material, we'd never come close to finishing. I don't want to spend much time on anything remotely average.
What is "remotely average" to you may be completely exceptional to someone else.
I just finished a book about Andrée de Jongh, a 23 year old Belgium woman who lead the Comet Line, one of the largest escape networks in Europe during WW2 helping return some of the men left behind at Dunkirk and other downed pilots and aircrew.
She personally took ~24 round trips from Brussles, Belgium to Gibralter, Spain thorugh Nazi occupied France. A one-way distance of over 1300 miles, crossing the Pyrenees.
I found the actual book not the best, but the story was amazing. I've since done a deep dive on the Comet Line and more about de Jongh.
Would you have read something like this? What % best would this be to you? I don't think I would be able to find anything similar in a research paper or anything in academia. Its the most interesting thing I've consumed in the last few months.
To answer your main question though:
> How do you find the absolute gems?
I consume a lot, and of those, some are excellent. Some are bad. I wouldn't know though unless I checked myself.
I understand your approach, and I think everyone does that to some extent, and most people do it to a great extent. I try to impose more discipline on my content choices, not always succeessfully. For me, it's paid off very well.
> What is "remotely average" to you may be completely exceptional to someone else.
I think that's taking relativism to a point of paralyzation. While judgments will differ between people, that clearly doesn't make them useless.
> I found the actual book not the best, but the story was amazing. I've since done a deep dive on the Comet Line and more about de Jongh.
IMHO, that curiosity and exploration is the most important thing.
> Would you have read something like this? What % best would this be to you? I don't think I would be able to find anything similar in a research paper or anything in academia. Its the most interesting thing I've consumed in the last few months.
Honestly, I'm tempted by the story, but because you said it wasn't the best, probably not. Also, I work hard to limit my history and biography to serious, scholarly sources: I want to understand and learn as much of the reality of things as possible; we never actually perceive reality, of course, even in front of our noses (or especially then), but I find a lot of popular histories/etc are sensationalized or more biased.
You'd be amazed what you can find in scholarly sources. It's incredibly rich, fertile, beautiful, exciting stuff - if you're the curious type, far more than the popular sources. People just used the tools their habituated to, and those lead to the 'popular' stuff - that was my situation too. Fortunately, I knew I just needed new habits and it would be just as easy.
So here's some tips if you are interested or if anyone is (written assuming no familiarity):
* For browsing books, look at what university presses publish - which includes the pinnicales of the most brilliant people's life works, and which covers all sorts of fantastic ground you hardly know about. You can usually find reviews.
* Also, to learn about something in particular, use Google Scholar to look up research. Start with literature reviews - the expert reviews all that is relevant and presents it to you. From the reader's point of view, it's incredible - they do your homework for you, and they are experts in the field. There are entire review articles (Google Scholar has a filter for them), and the beginning of any scholarly paper has a literature review - just pick a recent one. Then you will know the landscape and can proceed from there.
* To skim a book, etc.; join the Internet Archive's lending library (free, quick signup), Hathi Trust, and Libby (via your local library) - all offer immediate, free checkout of electronic versions of books.
I think limiting yourself to research papers alone will deprieve you of ideas which are just developing, which may not have an official body of research or even any published papers about. Ideas we as a society or culture might only beginning to tackle.
I find the discussion around a thing to be sometimes more interesting than the thing itself - a research paper might spark way more interesting discussions than the paper itself.
Take the "Attention Is All You Need" paper. How many have actually read it vs. how much has been written about it, or about the things it led to?
There is a lot packed into those 10 pages, but I've found the thousands of discussions around transformers and GPT to be way more interesting.
I'm not sure where the absolute statements came from - only research papers, etc.
> find the discussion around a thing to be sometimes more interesting than the thing itself
Here we differ. The discussions are mostly BS, mostly misinformation, ignorance, jokes, etc. It takes a lot of reading to find a few gems. What an expert writes, in their own domain, about something they've specifically studied in detail, is far more valuable IMHO, filled with beautiful things.
I don't rule out all discussion (obviously!). But think of the papers as comments in forums - or as blog posts - but instead of a misinformed hot take, the commenter did a bunch of research in the existing literature, carefully constructed an experiment, and tried out their idea themself - and furthermore, the commenter is an expert themself, like some people who post to HN.
It would still be great to learn where you find good material, of any kind.
I said a few days ago, in my dream HN would be mostly papers. That's where the richest, most accurate, most intellectually curious information is - far beyond most things on HN.