"Knowledge is power, store it well."

What is RSS-Librarian?

RSS-Librarian is a read-it-later service for RSS purists. You can store articles from the web in your own personal RSS/Atom feed and use your favorite feed-reader software to read your stored articles later. RSS-Librarian uses no database and works without accounts.

The project was born out of a personal frustration of mine: My workflow for reading anything I am interested in is by adding a star in my feed-reader to an article, which necessitates that anything I want to read is somehow a RSS/Atom subscription, and that isn't true for single articles I get sent by someone.

RSS-Librarian solves this issue with a self-hostable PHP file by extracting content from articles using a readability service and directly writing them as new entries into a personal feed, without requiring special libraries, a database or user accounts.

Consider RSS-Librarian if you want to:

• Store a collection of articles in a feed-reader application
• Avoid third-party read-it-later services such as Wallabag, Instapaper, Pocket etc.
• Minimize the amount of necessary apps for reading articles
• Get rid of accounts and not sign up to anything
• Read articles (offline) in a readable format, but not categorize or store them indefinitely
• Synchronize stored articles to multiple devices
• Optionally be able to self-host the whole architecture

How to use

Go to a librarian instance and add a first URL of an article you would like to read later.

You will be assigned a 64 character-long random ID that is unique and yours. It is practically impossible to guess this ID, so unless you give it to someone no-one else can access your feed.

On the next page you will see two important things:

1. Your personal URL: Store this URL in your bookmarks! This allows you to add more articles to your personal feed.
2. Your personal feed: This is your feed that you can subscribe to with your RSS/Atom reader application. It is unique and can only be managed with the personal URL above.

Confirm you have bookmarked your personal URL and your personal feed.

After you press Confirm a fresh feed will be created, and with your personal URL you may later return to add more articles to your personal feed.

To subscribe to your personal feed, select a feed-reader software for your device. Here are some suggestions:

• Capy Reader (Android)
• NetNewsWire (iOS/MacOS)
• FeedFlow (Windows/Linux)
• FeedBro (Firefox/Chrome/Brave)
• More…

Install a feed reader and subscribe to your personal feed.

IMPORTANT: After adding your first link you need to bookmark your personal URL in your browser so you can keep adding articles to your feed! If you loose your your personal URL you will have to create a new feed.

Tools

Feed Preview

The Feed Preview is a quick way to read your stored articles in a browser. If you are on a secondary device and have not set up a feed-reader yet, it's an easy way to preview your last stored articles.

Validate Feed

In the rare instance your reader software is not displaying articles correctly, the Validate feed link can check if the feed validates correctly and has no errors. Warnings can lead to varying behavior in different feed-readers but should not cause any other issues. Please open a Github issue for any errors you see.

Feed Bookmarklet

The Feed Bookmarklet can be added to your Browser bookmarks. If you click it, the current open page will be added to your feed.

Apple Shortcut

On iOS and macOS you can use a Apple Shortcut to be able to quickly share URLs with your RSS-Librarian feed. To do this, simply install this shortcut.

Once installed, you can click the share button in any application and share a URL to RSS-Librarian and it will be added automatically. This saves time, as you no longer need to copy and paste the URL by hand.

IMPORTANT: This shortcut only works for one bookmarked RSS-Librarian personal URL (i.e. the page you use to manage your feed). It assumes that in your bookmarks there is a URL ending on librarian.php?id= and then appends the shared URL as a parameter.

How it works

IMPORTANT: This description assumes you are using the default configuration.

RSS-Librarian has two parameters: librarian.php?id=HASH&url=SOMEPAGE.

• id is a random ID for a personal feed. If this parameter is not supplied, RSS-Librarian will generate a new one and add a feed file corresponding to id into the subfolder feeds/.
• url is a URL you submit to RSS-Librarian, whose content will be extracted and added to the feed corresponding to id.

For each url posted to RSS-Librarian, the extracted content will be added to a feed file derived from id - if it exists in the feeds/ folder. If it does not exist, it will be generated and written. The feed file will store a maximum of 25 entries before removing the oldest one and adding the new one.

Example

Suppose your id is set to 123, and url is https://ohshitgit.com, or simply put you call librarian.php?id=123&url=https://ohshitgit.com. In this case the content of https://ohshitgit.com is extracted and put into feeds/123.xml. You can subscribe to feeds/123.xml to retreive the article a feed-reader of your choice.

How to self-host

Quick & basic setup

In the most basic use-case RSS-Librarian requires you to host a single file librarian.php, which you can drop onto any host that supports PHP with no other requirements. Simply save this file and upload it to your host via your preferred SFTP application.

IMPORTANT: RSS-Librarian creates a directory and feed files inside it, as well as a configuration file rsslibrarian-config.config in the same directory the PHP file has been uploaded to. This requires that this directory is writable for the local group (default setting in most cases).

Adding local readability extraction

RSS-Librarian uses the RSS extraction of FiveFilters by default. If you can spare some extra space on your host (~822 KB), you can easily add the ability to do the extraction of content locally without having to rely on FiveFilters.

To do this, first install composer by following the instructions until you have composer.phar next to librarian.php in the directory of RSS-Librarian and execute:

$ php composer.phar require fivefilters/readability.php:4.1.0

This will create a directory called vendor inside your checkout, which you simply upload together with librarian.php to your host. If the RSS-Librarian sees that the vendor directory exists it will switch to use the local readability functionality.

If your Readability checkout lives in a different path, you can use mod_readability in the configuration file to point to that path.

Adding customized extraction

Not every webpage can be extracted through Readability. As an example, webpages that do not contain content formatted as an article such as video content or pages that primarily rely on API metadata either do not nicely or not extract at all through Readability.

For this purpose, or for customization, you can provide a custom PHP script to configure extraction manually. The file, by default called custom_extractor.php, must contain the following stub to be recognized by RSS-Librarian:

function custom_extractor($url)
{
    return [];
}

A successfully extracted page needs to, at minimum, return the following array:

return [
    'author' => '...',
    'content' => '...',
    'title' => '...'
]

Content is supposed to follow the RSS 2.0 and Atom guidelines on minimal HTML, or be pure text.

An optional enclosure field can contain an array of a URL to a file, its mime-type and it's length in bytes (can be just set to 0):

    'enclosure' => ['URL', 'mime/type', 0]

If your custom extractor script lives in a different path, you can use mod_custom_extractor in the configuration file to point to that path.

Access to walled content

RSS-Librarian by default only has access to pages that are publicly accessible, meaning they are not locked behind accounts, paywalls or potentially DDos protection such as Anubis.

If you wish to let RSS-Librarian have access to pages that need accounts, you can export your cookies from a browser to a text file and setup RSS-Librarian to use them in the configuration by pointing cookie_file to the exported cookies.

IMPORTANT: Remember to protect the cookies file on your host so as not leak login credentials!

Configuration

On first launch, RSS-Librarian will generate a file on your host called rsslibrarian-config.json. It's a JSON file that you can edit locally and modify the settings RSS-Librarian runs on.

Instance administration

IMPORTANT: Never reveal the admin_id to anyone!

Protect sensitive information

When deploying an instance, it is crucial to prevent the http server from listing files in the configured feeds directory (default: feeds/) to protect user IDs.

The configuration file contains the administrator's admin_id which can be used to send messages to all users and therefore must also not be readable from the web.

Finally, if you want to serve feed XML files directly using your stylesheet information, ensure that your host is configured for the right mimetype.

The easies solution is to add the following lines to a .htaccess file in the same directory as librarian.php:

# Protect configuration
<Files "/rsslibrarian-config.json">
    require all denied
</Files>

# Protect feeds directory
<Directory "/feeds">
    require all denied
</Directory>

# Set correct mimetype for RSS feed
AddType application/rss+xml xml

Admin Interface

On first startup, RSS-Librarian generates a random ID which will be used for the administration interface. It is stored as admin_id in the configuration file. Suppose the admin_id is set to abc. You can open librarian.php?id=abc and will be presented with an alternate interface which displays the following simple administrator functions.

Send a message to all feeds

Users of your instance are pseudonymous, meaning they have an ID, but cannot otherwise be identified or notified.

With this field you can attach a simple messsage entry to all feeds on the current instance, for instance to notify users of potential downtime, URL changes etc.

Users of your instance will see the message in their feed-readers once they pull the latest stored articles.

Clean up stale/abandonded feeds

Feeds will become stale over time. One can consider two types of abandoned feeds:

1. Feeds that have not been updated in a long time. This could be because the user has lost their personal feed ID, or because they simply no longer use the feed.
2. Accidentally created feeds. These are feeds that have been created on first try, and their user has abandonded it or forgot to bookmark their personal URL.

By pressing Clean up RSS-Librarian will remove all feeds matching the above two categories. You can configure the minimum time threshold at which point a feed is considered abandoned either way to free up space.

IMPORTANT: Removing feeds does not invalidate a users ID. If they return to the instance after a feed is deleted, RSS-Librarian will simply re-create the feed file once they add a new article.

Subscribe to logging

Every URL added by and ID on the instance is logged into a feed generated for the admin_id. This feed can be subscribed to with a feed reader just like any other. Simply visit the admin interface and click Subscribe.

Public instances

FAQ

Why is it called RSS-Librarian instead of Feed-Librarian?

The RSS in the name RSS-Librarian is used because for most people it is easier to recognize for reading feeds. RSS-Librarian supports both Atom and RSS 2.0 as containers, and may even support JSON-Feeds in the future. Whilst the technology targets feed-readers, the name is simply chosen for easier recognition.

Can I host my own RSS-Librarian?

Yes. Look up the section How to self-host. A basic instance is simple to set up. If you plan to host a public instance, please open a open a Github issue and let everyone know!

Is there a list of public instances?

You can look up all instances in this table.

Why is RSS-Librarian only storing a finite number of articles?

RSS/Atom feeds are used to communicate new articles to a reader software such as NetNewsWire or a web-based feed-reader such as FreshRSS. The feed-reader software you use will retain articles depending on its capabilities and configuration: Typically feed-readers store starred articles indefinitely, and all others for 3 months. RSS-Librarian is the simplest possible solution to store articles for later-reading based on your reader, not the server it is hosted on.

If you are searching for a read-it later archive and management solution, I can highly recommend Wallabag, which you can also self-host and which allows you to store extracted articles on a server with a database-backend for tagging, and of course RSS support!

How can I subscribe to all articles of a webpage instead of individual ones?

In this case you need to ask the webpage administrator to add a RSS/Atom feed that you can subscribe to with you reader. Many webpages today sadly do not provide this obvious functionality. In such cases you perhaps want to get in touch with these three projects, which provide feeds for webpages that do not have it:

• RSS-Bridge
• RSSHub
• RSSBox