Sporkmonger

Sitting here at Canada on Rails… Just released two new gems, FeedTools 0.2.24 and FeedUpdater 0.1.0.

I rewrote the HTTP retrieval code for FeedTools, and it now has full support for HTTP proxies as well as basic auth. Even better, I pulled all the HTTP stuff out of the load_remote_feed! method and put it into its own helper module so that the advanced HTTP stuff is available throughout FeedTools. Additionally, in the midst of the rewrite, the strange timeouts that would sometimes occur when caching was enabled, those went away by magic. Yay.

There is also a nice handy new tool I wrote for correctly using FeedTools in a Rails app. It doesn’t have to be used within a Rails app, but certainly, that’s where it will work best. It probably won’t work on Windows though, due to a lack of fork().

To use, sudo gem install feedupdater, then cd to your Rails app and feed_updater install. This will install the feed_updater script into your Rails app’s scripts directory, add a new config file for controlling it, and it unpacks the gems for FeedTools and FeedUpdater into your vendor directory. You will need to also write a new updater script for FeedUpdater. You’ll probably want to put this in your lib folder, and point to it with the config/feed_updater.yml config file.

Note: The install command will overwrite the currently existing feedtools or feedupdater directory in the vendor directory.

Here’s an example file (included with FeedUpdater, in the example directory):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

class CustomUpdater < FeedTools::FeedUpdater
  on_update do |feed, seconds|
    logger.info(
      "Loaded '#{feed.href}'.")
    logger.info(
      "=> Updated (#{feed.title}) in #{seconds} seconds.")
  end
  
  on_error do |href, error|
    logger.info("Error updating '#{href}':")
    logger.info(error)
  end

  on_complete do |updated_feed_hrefs|
  end
end

You would use the on_update method to copy data from the parsed feed into whatever database tables or other locations where it’s required.

Then once you’ve got your custom updater script, cd to your Rails app and run script/feed_updater start.

I will likely update FeedUpdater again fairly soon.

Update: Now, with multithreading!