API documentation¶

This is the API documentation of the program. It should explain how to create new plugins and navigate the code.

Feeds module¶

This is the core modules that processes all feeds and takes care of the storage. It’s where most of the logic lies.

Main entry point¶

The main entry point of the program is in the feed2exec.__main__ module. This is to make it possible to call the program directly from the source code through the Python interpreter with:

python -m feed2exec

All this code is here rather than in __init__.py to avoid requiring too many dependencies in the base module, which contains useful metadata for setup.py.

This uses the click module to define the base command and options.

Plugins¶

Plugin interface¶

In this context, a “plugin” is simply a Python module with a defined interface.

feed2exec.plugins.output(feed, item, lock=None)[source]¶

load and run the given plugin with the given arguments

an “output plugin” is a simple Python module with an output callable defined which will process arguments and should output them somewhere, for example by email or through another command. the plugin is called (from feed2exec.feeds.parse()) when a new item is found, unless cache is flushed or ignored.

The “callable” can be a class, in which case only the constructor is called or a function. The *args and **kwargs parameter SHOULD be used in the function definition for forward-compatibility (ie. to make sure new parameters added do not cause a regression).

Plugins should also expect to be called in parallel and should use the provided lock (a multiprocessor.Lock object) to acquire and release locks around contentious resources.

The following keywords are usually replaced in the arguments:

{item.link}
{item.title}
{item.description}
{item.published}
{item.updated}
{item.guid}

The full list of such parameters is determined by the :module:feedparser module.

Similarly, feed parameters from the configuration file are accessible.

Caution

None of those parameters are sanitized in any way other than what feedparser does, so plugins writing files, executing code or talking to the network should be careful to sanitize the input appropriately.

The feed and items are also passed to the plugin as keyword arguments. Plugins should especially respect the catchup argument that, when set, forbids plugins to do any permanent activity. For example, plugins MUST NOT run commands, write files, or make network requests. In general, “catchup mode” should be fast: it allows users to quickly catchup with new feeds without firing plugins, but it should also allow users to test configurations so plugins SHOULD give information to the user about what would have been done by the plugin without catchup.

Parameters:	feed (dict) – the feed metadata item (dict) – the updated item
Return object:	the loaded plugin

Note

more information about plugin design is in the Writing new plugins document.

feed2exec.plugins.filter(feed, item, lock=None)[source]¶

call filter plugins.

very similar to the output plugin, but just calls the filter module member instead of output

Todo

common code with output() should be factored out, but output() takes arguments…

feed2exec.plugins.resolve(plugin)[source]¶

resolve a short plugin name to a loadable plugin path

Some parts of feed2exec allow shorter plugin names. For example, on the commandline, users can pass maildir instead of feed2exec.plugins.maildir.

Plugin resolution works like this:

search for the module in the feed2exec.plugins namespace

if that fails, consider the module to be an absolute path

Note

actual plugins are documented in the Plugins document.

Utilities¶

Those are various utilities reused in multiple modules that did not fit anywhere else.

various reusable utilities

feed2exec.utils.slug(text)[source]¶

Make a URL-safe, human-readable version of the given text

This will do the following:

decode unicode characters into ASCII
shift everything to lowercase
strip whitespace
replace other non-word characters with dashes
strip extra dashes

This somewhat duplicates the Google.slugify() function but slugify is not as generic as this one, which can be reused elsewhere.

>>> slug('test')
'test'
>>> slug('Mørdag')
'mordag'
>>> slug("l'été c'est fait pour jouer")
'l-ete-c-est-fait-pour-jouer'
>>> slug(u"çafe au lait (boisson)")
'cafe-au-lait-boisson'
>>> slug(u"Multiple  spaces -- and symbols! -- merged")
'multiple-spaces-and-symbols-merged'

This is a simpler, one-liner version of the slugify module.

taken from ecdysis

feed2exec.utils.make_dirs_helper(path)[source]¶

Create the directory if it does not exist

Return True if the directory was created, false if it was already present, throw an OSError exception if it cannot be created

>>> import tempfile
>>> import os
>>> import os.path as p
>>> d = tempfile.mkdtemp()
>>> make_dirs_helper(p.join(d, 'foo'))
True
>>> make_dirs_helper(p.join(d, 'foo'))
False
>>> make_dirs_helper('')
False
>>> make_dirs_helper(p.join('/dev/null', 'foo')) # doctest: +ELLIPSIS
Traceback (most recent call last):
    ...
NotADirectoryError: [Errno 20] Not a directory: ...
>>> os.rmdir(p.join(d, 'foo'))
>>> os.rmdir(d)
>>>

feed2exec.utils.find_test_file(name='.')[source]¶: need to be updated from ecdysis

feed2exec.utils.find_parent_module()[source]¶

find the name of a the first module calling this module

if we cannot find it, we return the current module’s name (__name__) instead.

taken from ecdysis