Knowledge is power

Many of you may know me as a huge Warhammer 40,000 nerd. I like the absurd and over-the-top nature of it, the grimdark setting and stagnant vision of the future where, what is left of humanity, slowly rots from within. Many small quotes encapsulate the universe very well, but the following one from Dawn of War is one of my favorites:

“Knowledge is power, hide it well.”

Astartes Librarian

Similar to a Black Mirror episode, this quote perfectly reflects our current strange reality of hiding most of the Internet’s articles worth reading behind walls of ads, paywalls or questionable design ideas. Knowledge has not become entirely inaccessible, but similar to what happened in the academic community there is a cost to accessing articles: Your privacy, your sanity, or both.

Saving articles for later-reading has turned into a small ecosystem: Safari has a built-in method to store pages. Mozilla, until recently, ran Pocket and then shut it down. Instapaper remains stable and clean commercial solution I really like, and on the open-source front Wallabag is a viable alternative. There are great offline apps such as Later. Saving articles can also take an archival character, which ArchiveBox supports nicely.

I already have my feed-reader where I read everything else. All of these services are extra layers and apps that are too heavy for someone who just wants to read articles, offline, in a clean layout, without ads, without intrusive layouts or pop-up videos or spyware. This particular problem has already been solved many years ago, so rather than re-inventing the wheel I can instead leverage existing, well-established web-standards to let me read arbitrary articles from the web in the comfy environment of my feed-reader, next to my subscriptions.

If you subscribe via RSS/Atom to blogs, podcasts or Youtube channels, then RSS-Librarian is one more way for you to tunnel the Internet through your feed-reader.

Version One

RSS-Librarian is a read-it-later service for RSS purists. You can store articles from the web in your own personal RSS/Atom feed and use your favorite feed-reader software to read your stored articles later. RSS-Librarian uses no database and works without accounts.

I am happy to announce that RSS-Librarian has now arrived at Version One and is ready for production. Since my first article a lot has happened.

The original RSS-Librarian was kept extremely simple at just 211 lines of code in a single file. RSS-Librarian is written in PHP to make it as easy as possible to drop it on a VPS (virtually zero hosters come without PHP support these days). The script receives a URL as input, runs it through a readability service to extract the raw content, and then attaches it to a randomly generated feed file, no database required. The random feed can be revisited to add more articles to it. Once subscribed to it via a feed-reader, the reader software does the rest. I personally am using readers like FeedMe which download the full content of a feed, images included, to my phone, making all stored articles available offline for later-reading during for instance flights.

Since then, I have addressed several things:

1. Bugs: The code structure was mixed up with HTML quite badly. After some iterations it got better and easier to maintain. Still not perfect, but much easier to read. Separating code from structure also got rid of a lot of state-tracking bugs.
2. External Configuration: RSS-Librarian now reads its configuration from a JSON file rather than needing source modifications. The Instance Info section at the footer of the webpage provides an overview of the current configuration.
3. Feed Validation: The original RSS 2.0 output created invalid feeds. This has been fixed.
4. Local Readability: Rather than relying on a centralized external readability service, one can now download an extra library and run the extraction of article content locally.
5. Disabling Content Extraction: Content extraction can now be turned off. Whilst this seems strange, some feed-readers like FeedFlow do not read RSS item content anymore and instead rely on a built-in reader-view similar to the ones in browsers to pull content from the URL directly. To save redundant space in these cases this option will skip adding any content to the feed.
6. Atom Support: The Atom feed format is slighlty more convenient when it comes to storing an author without an email address, and some other small things. I decided to leave the format RSS-Librarian uses up to the user. One can easily switch between both and RSS-Librarian will automatically convert between formats for existing feeds once a user adds a new article.
7. User Onboarding: On the original test-instance I noticed many feeds were repeatedly created with just one article in them, suggesting users forgot to bookmark their randomly created feed ID. Now users are onboarded with a warning before adding their first article.
8. Mobile CSS: RSS-Librarian now has a beautiful and easy to tap-on user interface for mobile.
9. iOS Share integration: With iOS Shortcuts I was able to create a share-integration that allows any app to share a URL with your bookmarked RSS-Librarian feed.
10. Stylization: Both custom CSS for RSS-Librarian and XSLT for the feeds can now be configured. When opening the feed’s XML file one now gets a full preview of all stored articles in a nicely formatted webpage rather than just raw XML. If no XSLT is present the code will default to feedreader.xyz for a quick preview of the feed. Both the logo and the FavIcon can be configured as well.
11. Administration: All users of an RSS-Librarian instance are pseudonymous, meaning they have an ID but no other identifier. To inform users of the instance about downtimes, changes or any other admin related notices, a special ID is pre-generated on first start up which lets an admin attach a message to all existing user feeds. Furthermore, the admin interface allows cleaning up abandoned feed files. The configuration also contains a field to add contact information for the instance administrator, which defaults to the Github issues of RSS-Librarian. The administrator ID has its own feed that keeps track of any administrator operations and URLs added by any feed managed on the instance.

There are also two contributions I want to mention:

1. Alex Faxå cleaned up the code and added more robust cURL parameters to make RSS-Librarian appear more like a regular browser.
2. Abe Ramseyer added the ability to see the list of stored items and optionally remove them again.

To summarize, the code is now much easier to deploy, much easier to modify, has less bugs and creates valid RSS and Atom feeds. The user interface is visually better and easier to use on a phone.

RSS-Librarian running via Fennec on a LineageOS phone

A full documentation on how to use RSS-Librarian, how to self-host and configure it and an FAQ is now availble on the Github Wiki of the project.

Store it well

Most importantly though, I want to announce the official RSS-Librarian instance hosted at

https://www.rsslibrarian.ch/librarian.php

The old test instance hosted on hstn.me had several issues that caused subscription errors, particularily with online readers like FreshRSS, since it’s HTTPS certificate did not work correctly and regularily rerouted linked XML files to error pages. Additionally hstn.me would often attach extra URL parameters that needed to be filtered out from that particular instance. W3C feed validation for the same reasons would regularily fail. For Version One therefore I wanted to unveil a stable main instance everyone can use for free. If you have any bugs to report, feature requests or PRs, do not hesitate to open a Github issue.

Join the great crusade against the enshitification of the web. We do not need to burn it all down and re-create it from the ashes, the foundational standards for a better web are already here.