Older blog entries for abram (starting at number 23)

Hep News

A couple of exciting things happened yesterday:

• Wari Wahab sent in the first ever donation towards the development of Hep, in the amount of 5 dollars.  Thanks, Wari!
• Tim O'Reilly wrote that he's looking for a tool to send e-mail to his blog.

So, feeling motivated, I'm staying up late, finishing the last few items on my to-do list, and getting ready to release Hep 0.3.  And this time I mean it!

Another Test Post

I'm getting ready to release Hep 0.3, which will support posting to an Advogato diary.  Just posting this to make sure it works.

test post

Last night I added preliminary support for filters to Hep. You can copy
and/or redirect incoming messages to other places besides your Inbox,
based on criteria you specify.

I was hoping to finish the configuration UI for filters and protocols,
and get a new alpha release of Hep out today. But I ran into some
issues. First, I learned that the default Jabber server configuration
doesn't like it when a connection sends too many messages in a short
period of time (it gives you a bad 'karma' rating and stops accepting
messages for a few seconds), which was a problem since I had set up a
filter to copy all incoming messages to a jabber address. Then I found
that Jabber (with the default server configuration) also doesn't like it
when you have more that 5 connections open at a time from a single IP
address, which breaks my one-connection-per-destination setup. So I'm
going to have to rethink the way the Jabber bot will work.

Also I'm occasionally getting some kind of race condition where Hep uses
100% CPU, which definitely wasn't happening before.

I've been adding a lot of new code to Hep over the past few days, and I
think soon it will be time to step back and think about the design a bit
before I get sloppy in my rush to add cool new features. I'll release a
new alpha version with all my latest changes early next week. Then I
might pause for a week or two, let people do some testing, collect
feedback and bug reports and suggestions, get outside and enjoy the fall
weather instead of spending so much time coding.

I was home sick from work on Tuesday, so I took the opportunity to hack on Hep a bit.  And since I wasn't feeling well I didn't feel bad about ignoring the important-but-not-fun items on my TODO list, and playing with some interesting things instead.

So Jabber support is now working in Hep.  You give Hep a jabber account and password, and it logs in as username@server/Hep (with priority 2), and once for each of your destinations (username@server/DestinationName, priority 1).  Messages send to username@server/Hep (or just username@server since it Hep has the highest priority) go to your Hep Inbox.  Messages sent to the other resources get posted to the appropriate destinations (you configure a list of jabber accounts to accept messages from, to prevent strangers from sending messages to your weblog).

I also added the concept of "protocol handlers" to Hep, so that it can figure out how to send a message to a mailto: or jabber: address.  A protocol works much like a destination, except that every time you send a message you also specify an address.  That way you don't have to set up destinations for every e-mail or jabber address you might want to send a message to - you just set up your e-mail configuration once, and then specify the address when you send the message.  (I haven't figured out all the details of how this will work, but in the short term it should let Hep handle filters like "Copy all messages from Slashdot to this e-mail address").

Hep's architecture is making me happy.  Adding Jabber support was only 100 lines of code (thanks to the nice jabber.py library).  The rest of the work is all generic support for protocol handlers, which can be added in the future by dropping modules in the messaging/protocols directory.

Also I made a diagram of how Hep works, which may or may not help other people understand it.

I'll try to get another snapshot release ready this week, so that other people can start playing with this stuff.  And I really should write up some developer documentation...

I've put a new version of Hep up for testing (see the bottom of the page).  This is an alpha release, meant for developers and curious people.  It has some nice new features:

• Uses Mark Pilgrim's rssfinder module to find the RSS feed for a web page, searching syndic8's database if the page itself doesn't contain a link to the feed.  This means that Hep should be able to find the RSS feed for almost any site that has one.
• Also uses Mark's rssparser module to parse RSS feeds, which works around some buggy feeds and eliminates the problems some people were having with Python's XML libraries.
• Lets you set up message destinations, and has an SMTP server that accepts mail sent to those destinations.   Now you can send e-mail to your weblog (through the Blogger or MetaWeblog APIs), or your advogato diary, or both at once.
• Lets you view your Inbox on a web page, so you can read and delete messages without using your mail client if you want.

A complete changelog is here.

But before you download, let me remind you that there are reasons that this is an alpha release:

• The SMTP server has NO SECURITY, and only works for the first Hep user you create.  Don't run it on a shared or public system unless you want the whole world to be able to post to your weblog.
• The SMTP server hasn't been thoroughly tested and might occasionally lose an outgoing mail message.  A suggestion:  put yourself on the bcc list for any important messages that you send.  That way you can verify that your message got through.
• The first time you run the new version you'll probably get a lot of messages you've already seen (the way it generates message IDs is a little different, so all RSS messages look new).

These things will all be fixed soon enough, so if you're just using Hep as an RSS aggregator and you're happy with the way it's working I recommend you wait to upgrade, at least until this release gets into the beta stage.

If you're an existing user and you want to upgrade, here's what you need to know to get started fast.   Edit the hep.ini file that comes with the distribution (don't replace it with your old one - there's some new stuff in there related to the SMTP server, and it won't work without it).  Set the SMTP upstream_server=your.smtp.server.   Set your mail client to use localhost, port 5337 for outgoing mail.   Send mail to [destination_name]@hep.   And when you set up your destinations don't put spaces or special characters in your destination name.

If you're interested in seeing an almost complete list of things that need to be done before 0.3 is ready, check out the TODO file in the distribution.