feed2exec manual page

Synopsis

feed2exec {add,ls,rm,fetch,import,export}

Description

This command will take a configured set of feeds and fire specific plugin for every new item found in the feed.

Options

--version	Show the version and exit.
--loglevel	choose specific log level [default: WARNING]
-v, --verbose	show what is happening (loglevel: VERBOSE)
-d, --debug	show debugging information (loglevel: DEBUG)
--syslog LEVEL	send LEVEL logs to syslog
--config TEXT	configuration directory
-h, --help	Show this message and exit.

Examples

Saving feed items to a Maildir folder:

feed2exec add "NASA breaking news" https://www.nasa.gov/rss/dyn/breaking_news.rss --folder nasa
feed2exec fetch

This creates the equivalent of this configuration file in ~/.config/feed2exec/feed2exec.ini:

[DEFAULT]
output = feed2exec.plugins.maildir
mailbox = '~/Maildir'

[NASA breaking news]
folder = nasa
url = https://www.nasa.gov/rss/dyn/breaking_news.rss

Send new feed items to Transmission:

feed2exec add "Example torrent list" http://example.com/torrents/feed --output feed2exec.plugins.exec --args 'transmission-remote marcos.anarc.at -a '{item.link}' -w /srv/incoming'

Send new feed items to Mastodon, using the toot commandline client:

feed2exec add "My torrent" http://example.com/blog/feed --output feed2exec.plugins.exec --args 'toot post "{item.title} {item.link}"'

Send new feed items to Twitter, using the tweet commandline client from python-twitter:

feed2exec add "My torrent" http://example.com/blog/feed --output feed2exec.plugins.exec --args 'tweet "{item.title:40s} {item.link:100s}"'

Show feed contents:

feed2exec add "NASA breaking news" https://www.nasa.gov/rss/dyn/breaking_news.rss --output feed2exec.plugins.echo --args "{item.title} {item.link}"
feed2exec fetch

Commands

• fetch:

  fetch [--parallel | -p | --jobs N | -j N] [--force | -f] [pattern]
  

  The fetch command iterates through all the configured feeds or those matching the pattern substring if provided.

  --force	skip reading and writing the cache and will consider all entries as new
  --catchup	do not run output plugins, equivalent of setting the output plugin to feed2exec.plugins.null
  --parallel	run parsing in the background to improve performance
  --jobs N	run N tasks in parallel maximum. implies --parallel which defaults to the number of CPUs detected on the machine

• add:

  add [--output PLUGIN [--args ARG [ARG [...]]] [--filter PLUGIN] NAME URL
  

  The add command adds the given feed NAME that will be fetched from the provided URL.

  --output PLUGIN
  	use PLUGIN as an output module. defaults to feed2exec.plugins.maildir to store in a mailbox. use feed2exec.plugins.null to just fetch the feed without fetching anything.
  --args ARGS	pass arguments ARGS to the output module. supports interpolation of feed parameters using, for example {title}
  --filter PLUGIN
  	filter feed items through the PLUGIN filter plugin
  --mailbox PATH	folder to store email into, defaults to ~/Maildir.
  --folder PATH	subfolder to store the email into

  Those parameters are documented more extensively in their equivalent settings in the configuration file, see below.

• ls:

  The ls command lists all configured feeds as JSON packets.

• rm:

  rm NAME
  

  Remove the feed named NAME from the configuration.

• import:

  import PATH
  

  Import feeds from the file named PATH. The file is expected to have outline elements and only the title and xmlUrl elements are imported, as NAME and URL parameters, respectively.

• export:

  export PATH
  

  Export feeds into the file named PATH. The file will use the feed NAME elements as title and the URL as xmlUrl.

Files

Configuration file

Any files used by feed2exec is stored in the config directory, in ~/.config/feed2exec/ or $XDG_CONFIG_HOME/feed2exec. It can also be specified with the --config commandline parameter. The main configuration file is in called feed2exec.ini. The above commandline will yield the following configuration:

[NASA breaking news]
url = https://www.nasa.gov/rss/dyn/breaking_news.rss
output = feed2exec.plugins.echo
args = {title} {link}

Naturally, those settings can be changed directly in the config file. Note that there is a [DEFAULT] section that can be used to apply settings to all feeds. For example, this will make all feeds store new items in a maildir subfolder:

[DEFAULT]
output = feed2exec.plugins.maildir
folder = feeds

This way individual feeds do not need to be indivudually configured.

The following configuration parameters are supported:

name

Human readable name for the feed. Equivalent to the NAME argument in the add command.

url

Address to fetch the feed from. Can be HTTP or HTTPS, but also file:// resources for test purposes.

output

Output plugin to use. Equivalent to the --output option in the add command.

args

Arguments to pass to the output plugin. Equivalent to the --args option in the add command.

filter

Filter plugin to use. Equivalent to the --filter option in the add command.

mailbox

Store emails in that mailbox prefix. Defaults to ~/Maildir.

folder

Subfolder to use when writing to a mailbox. By default, a slugified version of the feed name (where spaces and special character are replaced by -) is used. For example, the feed named “NASA breaking news” would be stored in ~/Maildir/nasa-breaking-news/.

catchup

Disable output plugin execution. In this mode, the feed is still read and parsed, but new entries are not added to the database.

pause

Completely skip feed during fetch. Similar to catchup, but doesn’t fetch the feed at all and doesn’t touch the cache.

Here is a more complete example configuration with all the settings used:

# this section will apply to all feeds
[DEFAULT]
# special folder location for maildir. I use this when I have multiple
# accounts synchronized with Offlineimap
mailbox = ~/Maildir/Remote/

# a feed to store NASA breaking news entry in a "nasa" subfolder
[NASA breaking news]
url = https://www.nasa.gov/rss/dyn/breaking_news.rss
folder = nasa

# some maildir storage require dots to get subfolders. for example,
# this will store messages in INBOX/feeds/images/ on Dovecot
[NASA image of the day]
url = https://www.nasa.gov/rss/dyn/lg_image_of_the_day.rss
folder = .feeds.images

# retweet hurricane news
[NASA Hurricane breaking news]
url = https://www.nasa.gov/rss/dyn/hurricaneupdate.rss
output = feed2exec.plugins.exec
args = tweet "{item.title:40s} {item.link:100s}"

# same, but on the mastodon network
#
# we can have multiple entries with the same URL without problems, as
# long as the feed name is different. it does mean that the feed will
# be fetched and parsed multiple times, unfortunately.
#
# this could be improved to include the '{item.summary}' and extra markup,
# for example.
[NASA Hurricane breaking news - Mastodon]
url = https://www.nasa.gov/rss/dyn/hurricaneupdate.rss
output = feed2exec.plugins.exec
args = toot "{item.title} {item.link}"
# output is disabled here. feed will be fetched and parsed, but no
# toot will be sent
catchup = True

# same, but on the Pump.io network
[NASA Hurricane breaking news - Pump]
url = https://www.nasa.gov/rss/dyn/hurricaneupdate.rss
output = feed2exec.plugins.exec
args = p post note "{item.title} {item.link}"

# crude podcast client
[NASA Whats up?]
url = https://www.nasa.gov/rss/dyn/whats_up.rss
output = feed2exec.plugins.exec
# XXX: this doesn't handle errors properly: if there is a feed without
# enclosures, the whole thing will crash.
args = wget -P /srv/podcasts/nasa/ "{item.enclosures[0].href}"
# feed is paused here. feed will not be fetched and parsed at all and
# no post will be sent.
pause = True

Cache database

The feeds cache is stored in a feed2exec.sqlite file. It is a normal SQLite database and can be inspected using the normal sqlite tools. It is used to keep track of which feed items have been processed. To clear the cache, you can simply remove the file, which will make the program process all feeds items from scratch again. In this case, you may want to use the null output plugin to avoid doing any sort of processing to catchup with the feeds.

See also

feed2imap(1), rss2email(1)