API documentation
This is the API documentation of the program. It should explain how to create new plugins and navigate the code.
Controller module
This is the core modules that processes all feeds and talks to the storage. It’s where most of the logic lies, although the parsing is still currently done inside the model. It dispatches the plugin logic to the plugin module.
Model
The “model” keeps track of feeds and their items. It handles configuration and cache storage.
Main entry point
The main entry point of the program is in the
feed2exec.__main__
module. This is to make it possible to call
the program directly from the source code through the Python
interpreter with:
python -m feed2exec
All this code is here rather than in __init__.py
to avoid
requiring too many dependencies in the base module, which contains
useful metadata for setup.py
.
This uses the click
module to define the base command and
options.
Plugins
Plugin interface
In this context, a “plugin” is simply a Python module with a defined interface.
- feed2exec.plugins.output(feed, item, lock=None, session=None)[source]
load and run the given plugin with the given arguments
an “output plugin” is a simple Python module with an
output
callable defined which will process arguments and should output them somewhere, for example by email or through another command. the plugin is called (fromfeed2exec.feeds.parse()
) when a new item is found, unless cache is flushed or ignored.The “callable” can be a class, in which case only the constructor is called or a function. The
*args
and**kwargs
parameter SHOULD be used in the function definition for forward-compatibility (ie. to make sure new parameters added do not cause a regression).Plugins should also expect to be called in parallel and should use the provided
lock
(a multiprocessor.Lock object) to acquire and release locks around contentious resources.Finally, the FeedManager will pass along his own
session
that should be reused by plugins to do requests. This allows plugins to be unit-tested and leverages the built-in cache as well.The following keywords are usually replaced in the arguments:
{item.link}
{item.title}
{item.description}
{item.published}
{item.updated}
{item.guid}
The full list of such parameters is determined by the :module:feedparser module.
Similarly, feed parameters from the configuration file are accessible.
Caution
None of those parameters are sanitized in any way other than what feedparser does, so plugins writing files, executing code or talking to the network should be careful to sanitize the input appropriately.
The feed and items are also passed to the plugin as keyword arguments. Plugins should especially respect the
catchup
argument that, when set, forbids plugins to do any permanent activity. For example, plugins MUST NOT run commands, write files, or make network requests. In general, “catchup mode” should be fast: it allows users to quickly catchup with new feeds without firing plugins, but it should also allow users to test configurations so plugins SHOULD give information to the user about what would have been done by the plugin withoutcatchup
.- Parameters:
- Return object:
the loaded plugin
Note
more information about plugin design is in the Matchtitleregex document.
- feed2exec.plugins.filter(feed, item, lock=None, session=None)[source]
call filter plugins.
very similar to the output plugin, but just calls the
filter
module member instead ofoutput
Todo
common code with output() should be factored out, but output() takes arguments…
- feed2exec.plugins.resolve(plugin)[source]
resolve a short plugin name to a loadable plugin path
Some parts of feed2exec allow shorter plugin names. For example, on the commandline, users can pass maildir instead of feed2exec.plugins.maildir.
Plugin resolution works like this:
search for the module in the feed2exec.plugins namespace
if that fails, consider the module to be an absolute path
Note
actual plugins are documented in the Plugins document.
Utilities
Those are various utilities reused in multiple modules that did not fit anywhere else.