Subscribe to RSS Feeds via OpenWhisk

Share: Share on FacebookTweet about this on TwitterShare on Google+Share on LinkedInShare on RedditEmail this to someonePrint this page

OpenWhisk is a “serverless” execution environment that follows an event driven model, where user code is automatically executed in response to an external event. An example of an external event can be a database update, pushing code to Github or even a simple HTTP request from a web or mobile application.

Rather than manually listening for events and calling OpenWhisk when an event occurs, OpenWhisk feeds allow for connecting external event producers with OpenWhisk actions.

RSS

RSS is a web syndication format that enables delivery of frequently changing web content. It is widely used by news publishers, blogs or even podcasts to announce newly available episode(s). As changing content could be treated as a new event, RSS feeds could be considered as event providers. This seems to fit well with the OpenWhisk feed paradigm. Therefore, in this article we will explore how to connect RSS sources to OpenWhisk through an OpenWhisk feed.

As Figure 1 illustrates, feeds emit events that invoke triggers when an external event occur. Triggers can be associated with actions through rules, thus allowing additional code to be executed in the form of an action.

Screen Shot 2016-08-01 at 11.35.55 AM

Figure 1: OpenWhisk Architecture

 

High Level Architecture:

Subscribing to a RSS feed from OpenWhisk is controlled by two main components:

  • RSS Package – holds RSS feed
  • Event Provider Service – long running service that polls RSS sources periodically

rssarchitecture

Figure 2: RSS Package and RSS Event Provider Service High Level Design

OpenWhisk RSS Package:

OpenWhisk users may subscribe to RSS Sources using the OpenWhisk RSS Package. This package holds a feed that could be provided as a parameter when creating a trigger:

The code above will create a trigger that delivers the content of newly published feed items related to the 2016 Presidential Election.

Creating an RSS Feed

Feed is an abstraction for a stream of events, all of which belong to a trigger. A feed is controlled by a feed action that provides lifecycle operations such as create, delete, pause, and resume, via a whisk action interface.

The RSS feed action is a normal OpenWhisk action but accepts the following parameters:

  • lifecycleEvent: ‘CREATE’ or ‘DELETE’
  • triggerName: the fully-qualified name of the trigger which contains events produced from this feed
  • authKey: the Basic auth credentials of the OpenWhisk user who owns the trigger
  • url: RSS feed endpoint
  • pollingInterval: interval at which polling is performed
  • filter: comma separated list of keywords to filter on

The lifecycleEvent signifies whether to register(CREATE) or unregister(DELETE) the trigger with the event provider:

OpenWhisk distinguishes normal actions from feed actions by an annotation. In order to turn a normal action into a feed action, annotate it with the keyword “feed” and set its value to true:

RSS Event Provider Service

The RSS event provider service uses a “polling” strategy to obtain content from RSS feed sources and transmit it to OpenWhisk. The service polls an RSS endpoint periodically, checks if feed items are newly published and fires the trigger that was registered with the RSS source.

The service is implemented using Node.js and Cloudant. The service listens for incoming HTTP requests from the RSS feed action. Cloudant persists the trigger information(triggerName, pollingInterval, url and filter) so this can be leveraged during the polling process. Feed items have to meet certain criteria in order to be propagated to OpenWhisk:

  • published within the user specified period from the current time
  • (optional) user specified list of keywords need to be present in the feed title or description

Feed items are piped to OpenWhisk through the OpenWhisk API.

The service could be hosted on the user’s local machine or on Bluemix as a Cloud Foundry application.

Try it Out!

Subscribing to your favorite feeds via OpenWhisk is now easy! Download the OpenWhisk RSS Package project on Github and install the RSS package along the supporting event provider service.  Instructions are provided to install the RSS package locally or on the Bluemix platform.

Share: Share on FacebookTweet about this on TwitterShare on Google+Share on LinkedInShare on RedditEmail this to someonePrint this page

Leave a Reply

Your email address will not be published. Required fields are marked *