I think it's little appreciated that planets solve many of the practical problems of feed crawling.
If you wanted to follow 2000 blogs yourself you'd find it is really a hassle. You can follow one planet and its easy.
For that matter, if 2000 people want to follow your blog (and many other blogs) they are going to generate 2000 requests per polling period. It is not wonder why people like [1] get so exasperated. There are three kinds of polling periods: (1) too fast, (2) too slow, (3) both at the same time. Instead of having 2000 people poll your blog too often, one planet can poll your blog. It improves the scalability and economics of the system dramatically.
(e.g. the difficulty of finding a good polling regime is one of 10 or 20 or so unappreciated reasons why RSS has remained nerdcore)
Data point 3: does not work for me. Also using NetNewsWire. “Throttled or blocked” changes nothing, if it’s down for half of visitors then it’s down for half of visitors. I highly doubt NetNewsWire is breaching spec or poorly behaved. I bet rachelbythebay is just throttling by IP address and really has a 24 hour limit of exactly 1, so only one person per IP per day can subscribe.
I'm on the Feedly train boat too since the sinking of Google Reader. Besides the ai bs it doesn't really let you search something in your feeds without a subscription, so once I tried Feedbin.
But I went back because third world issues (and not a fan of microtransactions either) I could not find a way (if any) to change the layout on desktop. I really like the minimal list view at Feedly (and even on mobile too).
big fan of feedmail [1] for a "dumb" reader. It just delivers it to your inbox. I have a folder set up called "feed" and I can do all the email things to the items landing in that folder. Every email costs a credit, you can buy 10 000 credits for $10, which works out to $0.001 per article.
The only thing it misses is any sort of highlight saving deal, but for those I just save the article to zotero and annotate it there.
I love RSS but how can we get more of it? Walled-ish gardens seem to dominate. Many good producers are on platforms that just don't syndicate. What kind of pressure can we, consumers, put on?
I'm sorry to be the pessimist here, but I doubt it. HN users are the most likely to use RSS in the first place. I sincerely doubt that RSS is going to make a comeback this year with all my non-IT friends using it.
Realism shared. I work with journalists, exactly the type of normies who would most benefit from this technology. Years of evangelism has had no effect whatsoever. Nobody they know is using it, they don't see the logo anywhere, or a big friendly "Get started" button. It's all so unfamiliar and technical-sounding. Even the name itself was a disaster, IMO: a hopelessly geeky and opaque acronym. It should have been called Webfeeds! It's all such a wasted opportunity.
In this case having more people asking for it or expecting it can contribute to expanding the availability of RSS feeds to more websites.
While the vast majority of technical stuff I follow uses RSS, the same cannot be said for some other resources I like to read.
While some are kind enough to enable them when asked, I don't expect them to want to support them for just a handful of people in their target audience.
Honestly that might be for the best in some ways. I see rss as "allowed to exist" because not enough people use it. There's plenty of ways to subscribe to things that I enjoy right now that could be nuked if suddenly enough people were using it that they weren't seeing the conversion rates they wanted.
Try Rss-Bridge [1] when the website does not have any feed, it might have an integration already. It also supports custom CSS-selectors to create feeds, or even use SEO-Sitemaps for your advantage to generate a feed from it.
I wish it was easier to find out what my friends have been up to without getting them to sign up for some platform they’ve never heard of, then post in multiple places in perpetuity, and move on again when that platform also goes to shit.
A hard problem but surely not unsolvable. It belongs in a pg “please solve these big problems” essay.
That's an interesting problem. I'm thinking about creating a local-first RSS Reader that syncs using github. It seems doable to create a personal feed based on my feed and publish it also to github.
Not that this is novel in any way, but I just started a repo call Subcurrent yesterday for the Astoria Tech Meetup in NYC at our Saturday hack session. Subcurrent aims to provide a feed aggregator page made of our community members' feeds.
https://github.com/astoria-tech/subcurrent
I did not know that Meetup.com exposes RSS feeds at all, so I will be adding that to our Subcurrent instance since our group keeps events on Meetup.com.
I had never heard of Kill the Newsletter, but I'm a fan sight-unseen. Substack at least has feeds. You can append `/feed` to the newsletter's URL.
For me, that advanced reader is Thunderbird. It has amazing customizability with userchrome.css, web extensions, about:config and developer tools from Firefox.
1. Convergent desktop/mobile app that syncs between devices (open source and local)
2. Performant for large numbers of feeds
3. Integrated browser and automagical discovery and organization of feeds while you browse
4. Multiple taxonomies and viewing layout options, chronological, by subject etc
5. Advanced filtering by keywords
6. Transparent and pluggable locally runing algorithms to track usage and inform the user of patterns and if desired adjust presentation
In a sense an advanced RSS 'reader' is what the web "browser" should have evolved to. There is really no real boundary between these two clients.
An advanced RSS reader is essentially a more dynamic browser that queries the internet in more ways that the user-initiated visit of a bookmarked url or typing something into a search form.
I'm using a combination of FreshRSS (self-hosted) and Readrops (for Android).
I feel like #1 is handled by this combination. But, I wonder what convergent means here for you?
I don't notice any performance issues. But, this feels like a simple task for a bunch of RSS feeds. Did you notice performance issues with other readers?
There is something very interesting about what you call "automagical discovery." To me this is the biggest hole in my RSS experience. I want something that gives me magical discovery within the feeds I have based on my reading experiences. I subscribe to a lot of feeds because of one article, and then don't care about 90% of the other articles, but there are 10% that I do really want to read, but don't have the focus to find them. Is that what you mean by automagical?
Why do you want #4?
Filtering by keywords seems interesting.
FreshRSS has a bunch of analytics that comes with the server. I never really use it, but it is interesting. I would be happy to share what I can see based on my limited usage over a few months if this would serve your purposes. I would be interested in understanding what you mean by that.
The one thing I wish I had in my current setup is a way to take notes in a centralized way. When I'm on Android, I suppose I can copy and use a share intent. And, on a browser, I could install an extension to do that. But, it feels like that is an interesting opportunity for someone. I wish FreshRSS could layer a JavaScript app on posts, for example, and then I could build whatever I want. It has an extension API, but strangely documented.
"Convergence" was a term used by Ubuntu in their brief foray into mobile Linux. Alas it never led anywhere, but it still embodies a sort-of holy grail of cross-platform apps with uniform native feel.
On performance, yes, in my experience various readers (have not tried them all!) will choke if the feed list grows beyond a hundred or so. If you make consistent use of RSS you don't even need to be a power user to reach this number. E.g. think of the typical total of browser bookmarks of youtube subscriptions people have.
On reader magic, yes, the empowerment of users in discovering relevant content was quite central in early visions of the web. The idea was that with metadata (linked data) and intelligent clients you will have a sort of decentralized version of the "you might also like" functionality that is now such a mainstay of centralized platforms.
Taxonomies and flexible layouts can really transform your experience. It gives you the option to switch between a reddit-like, subject-oriented view of feeds, or a mixed chronological order.
I am actually thinking of possibly trying out FreshRSS. Akregator is what I am currently using and ticks a few requirements [2] but does not seem to be having much further ambition (and being a native Qt application it requires serious commitment to contribute anything).
Yes your point about extensions is quite important. People have different ideas what makes "an advanced reader" and one way to reach broader community adoption is to make it easy to add functionality.
I've got FreshRSS installed. Email me at the address in my profile if you want to try it. Or, just create an account at reader.rss.surf on your own, I think I've enabled sign ups.
FreshRSS is interesting because you can customize it to use postgresql instead of SQLite (which I did). Normally each user gets their own unique SQLite database, but in my case it is split across postgresql and SQLite now. The way they structure their tables and databases is a bit strange across users.
Atom and RSS are functionally equivalent despite their technical differences. Effectively no one has ever implemented Atom push capabilities so Atom is primarily only ever used for syndication. The term "RSS" is just a generic term for "XML syndication feed".
I'm working on a feed reader, called Lighthouse (https://lighthouseapp.io/). It combines RSS feeds with read-it-later, by putting new content into Inbox, where you can either archive or bookmark. Bookmarked content shows up in Library.
I'm trying to build a new corner of the old web with my social link sharing site https://lynkmi.com, and every tag automatically has an RSS feed so you don't need to know anything about them to set one up, or even need an account to follow one.
The main idea is instead of following everything a person posts you can just follow a subset of their interests. So if I post about Irish Dairy Innovations [0] and also about Advice [1] you can follow whichever combination of those you like.
If you'd like to sign up, my email is in my bio. And if you don't want to sign up, my email is still in my bio.
I asked because I didn't know how to get exactly that, so I'm building a site that finds RSS feeds from sites on the front page of HN. Then, at the end of the day it emails me with a summary if I miss one.
I use Feedly, which as a RSS reader is ok but probably one of the better ones available on iOS/iPadOS platform. I loved Google Reader back in the day, but Google loves to kill every product of theirs that I actually like and used.
I do the same. I do it because I subscribe to sites other than HN, and by subscribing to the HN feed I can get the articles from HN as well as the other sites I subscribe to all in the same place, ie my feed reader (also using Feedly). Nothing overly original here as I assume personal content aggregation was one of the original intentions of RSS.
I prefer to consume HN via my feed reader. I feel like I receive most or all of the original topics/posta without regard to algorithm ranking. So I will see flagged and dead posts and can make my own determination on the topic without a bunch of HN folks determining what I will see. Of course by then commenting is disabled, but that’s fine.
I need to find some way to see a chronological feed of what people I am friends with on facebook post. I literally get randomly suggested posts from unrelated accounts, and scrolling through them all will start to repeat them, never showing me content from some people who are actual friend connections.
i built this site (https://fanzine.world) to help people discover micro-creators and share their top 5 faves. not sure what to do with it. maybe someone here has a suggestion?
Chronological feeds are awful. You'll never see anything from the people who post occasionally because they get drowned out by the people who are posting all the time.
There may be some algorithms that deliberately magnify hate because that's a way to increase engagement, but if you want to create one of those algorithms you can make a training set based on chronological feed + boosting/retweets/reposts.
I'm amazed at how people keep making failing RSS readers that keep failing with the same failing user interfaces that have been failing since 1999; everybody knows RSS has been failing but they never ask why or if we have a choice.
We still see the readers that make you mark things as read, that take their cues from email and newsreaders, that, when you subscribe to N feeds show you N boxes with a list of items, etc.
My RSS reader works like TikTok because I'm not afraid of algorithms.
It's one thing to sort a list of things by one criteria, say "a". If you have two criteria, say "a" and "b" you can sort by "a" and break ties by "b" or you can sort by a+b or a-b or something like that... But that's not the same as optimizing both things. Maybe you can say "there are these 5 people who are special to me who I never want to miss their posts" but I think people would struggle to maintain rulesets and might not really be happy with the results they get.
In search rankings for instance you night have a document score (like Pagerank) that tries to identify the quality of a document, and you might have a query-document score that identifies the relevance of a document to a query. It's not trivial at all to find a way to blend those that gives you queries that are both relevant and quality as opposed to just one or the other.
The greatest weakness of my current RSS reader is that it's slow, depending on how much I am using it, articles could be delayed anywhere from a day to a week. For certain kinds of articles [1] recency doesn't matter, but other articles [2] have a definite shelf-life and if you repost them too late you look like a total dope.
I'd like to let articles about sports "cut the line" in front of higher quality articles about other topics but it's really hard to find a balance that's right because I don't want to get flooded with lower quality sports articles. It's one thing to say "let people make up their mind about how to balance these things" but when you really try it you find it's pretty hard. Not only do I have to change the whole way my pipeline works (can't be a batch job anymore) but it's not clear to me how to tune up the selection criteria.
If I understand your challenge correctly it could be helped with a reader that gives a taxonomy to classify feeds, as you then get implicitly multiple timelines - one per feed category. Linux desktop readers like akregator and the now abandoned quiterss provide that and it works fine.
I don't think people are objecting to suggestive feeds in general; they're objecting to suggestive feeds whose primary objective is to keep you scrolling for as long as humanly possible to maximize company revenue.
You can read x articles a day; your system ingests y articles a day.
x=y is perfect but requires close-to-perfect balance (if x=0.9y to 1.1y maybe you can adjust your reading habit to your your feed)
if x>y then your system isn't showing you enough, if y<x you are going to miss things you subscribe to based on some arbitrary or random characteristic.
With an algorithmic feed of some kind you choose to read x items a day, your system shows you the best x items a day out of y based on some set of criteria and constraints.
These things are common sense but seemingly nonsensical to a lot of people. For instance our impoverished rights-based discourse (see [1]) about "free speech" presupposes that 100% of people can read 100% of what everybody else posts, realistically platforms can only show people some fraction of what gets posted so one thing is going to get more visibility and other things get less and that's a choice -- it could be random but it's still a choice. (As Rush would put it, "if you choose not to decide, you still have made a choice")
I think the discussion is so impoverished that we never hear that an algorithm could choose to do anything other than maximize profits for a platform, when in fact that is just one thing an algorithm could do out of countless options.
> With an algorithmic feed of some kind you choose to read x items a day, your system shows you the best x items a day out of y based on some set of criteria and constraints.
And the whole point of the people who want RSS is that they disagree with the idea that "the algorithm" shows the best x items.
The whole point is that they don't want "the algorithm". And your answer is "Have you heard of the algorithm? I think it's a solution to your problem".
For most people y > x. You're choosing what posts to see even if the feed is chronological. For a chronological feed, x is the most recent posts you happen to see when checking your feed reader. And x will skew towards people who post at the same time of day you tend check your feed reader. People who post more will also take a larger distribution of x.
Any algorithm would seem to be an improvement on that, right? For example, an algorithm that sampled evenly from all sources. That would at least mean x was distributed across all your sources, so people who post more didn't crowd others out.
> if x>y then your system isn't showing you enough
We've grown used to having a practically infinite amount of things presented to us all the time, but I've come to believe recently that limiting the amount of posts you see can be a fine (and maybe healthier) choice. There's value in being able to say "I'm caught up, I'm done", it can be a natural stopping point.
I'm not as good about limiting myself like this as I would like to be, but it is a goal of mine.
Some arbitrary process has to reduce the 1,000,000,000 pieces of content produced today to maybe 10 or 100 you can handle.
The very concept of "news" ("fake news" is bad because it is news not because it is fake) is an act of violence against the fabric of reality because a huge number of things happened today, but CNN reported the same 10 of them 100 times. They could maybe cover 1000 different news stories (still a fraction of maybe 100,000 things that happened) but that's a non-starter because someone who watched the network at 3pm would have a totally different impression than someone who watched it at 5pm.
> Some arbitrary process has to reduce the 1,000,000,000 pieces of content produced today to maybe 10 or 100 you can handle.
In my case, I'm the process that does the reduction by subscribing to specific feeds. (Not sure if that's "arbitrary", but I could see it argued either way.)
I admit I miss some content that I would probably like to see. :) But that's just life how it's always been.
As my wife put it, "You don't have enough years in your life to read all the good books, so stop wasting time with bad ones." That's how I think of it. A handful of high-quality, low-volume feeds is great. (But I also follow HN and my curated Lemmy feed, so I get some of the firehose.)
The only "news" I have in my feed is local news. Any interesting national news hits HN or Lemmy.
Again, that's just me. I respect your different usage pattern.
> You'll never see anything from the people who post occasionally because they get drowned out by the people who are posting all the time.
I use inoreader and track basically everything I want in a huge list of feeds.
It's pretty trivial to mark stuff as "always flag this", and then leave the rest of the pile as "scan through and manually tag anything else"
Anything low volume goes on the "always" flag list as it takes one review a day of the new content to decide if it's something I care about.
Ultimately what I want, and what most people want, is the ability to just hook up to various data streams and apply rules to it. From there filtering as desired comes pretty easily.
> Chronological feeds are awful. You'll never see anything from the people who post occasionally because they get drowned out by the people who are posting all the time.
My preference would be chronological feeds, but following few enough accounts that I can see every single post in it. Then that's not a problem.
An algorithm doesn't have to reward engagement or time spent on the app. It can reward or punish anything at all -- and that's what it is about algorithms.
My feed reader (that I wrote myself) rewards things that I thumbs up and punishes things that I thumbs down.
Is this mysterious feed reader one that you made yourself?
I agree about the need to have some means of filtering. This is a major weakness of RSS, alongside onboarding and discovery. Algorithms serve the purpose of filtering well. The problem is their opacity.
What people are calling an "algorithm" is really a heuristic.
In the public discourse over "algorithms", the definition seems to be "something that maximizes your (time on site|outrage|clicking)" and not the real definition
In mathematics and computer science, an algorithm (/ˈælɡərɪðəm/ ⓘ) is a
finite sequence of mathematically rigorous instructions, typically used
to solve a class of specific problems or to perform a computation. [1]
By that definition any well-defined process that creates your feed (sorting chronologically, alphabetically) is an "algorithm"; in the machine learning age it is easy to make an algorithm that selects for anything at all, except for the capital-T Truth.
Let it go Paul. It's like the word "hacker". We ain't getting it back.
It's sad the popular perception of a fundamental computer science
concept, the great progeny of Al-Khwarizmi Musa, the life work of
heroes like Donald Knuth, is sullied and stained with dire B-movie
comic-book Bond villainy. Thank the hooligan broligarchs and their
manchild ambitions for trashing that bit of computing culture.
Even my mum spits in the dust like a cowboy at the sound of
"Algorithm".
A good way to find interesting blogs is to subscribe to a few planets.
These are essentially aggregations of blog related to some project/topic.
https://planet.gnome.org/ https://planet.kde.org/ https://planet.mozilla.org/ https://planet.documentfoundation.org/
PS. If you know any good planets worth skimming, please add to below :)
That said, I don't really have a good RSS reader that syncs across devices. I currently use Feedly, but it tries to be too smart.