Last night I added preliminary support for filters to Hep. You can copy
and/or redirect incoming messages to other places besides your Inbox,
based on criteria you specify.
I was hoping to finish the configuration UI for filters and protocols,
and get a new alpha release of Hep out today. But I ran into some
issues. First, I learned that the default Jabber server configuration
doesn't like it when a connection sends too many messages in a short
period of time (it gives you a bad 'karma' rating and stops accepting
messages for a few seconds), which was a problem since I had set up a
filter to copy all incoming messages to a jabber address. Then I found
that Jabber (with the default server configuration) also doesn't like it
when you have more that 5 connections open at a time from a single IP
address, which breaks my one-connection-per-destination setup. So I'm
going to have to rethink the way the Jabber bot will work.
Also I'm occasionally getting some kind of race condition where Hep uses
100% CPU, which definitely wasn't happening before.
I've been adding a lot of new code to Hep over the past few days, and I
think soon it will be time to step back and think about the design a bit
before I get sloppy in my rush to add cool new features. I'll release a
new alpha version with all my latest changes early next week. Then I
might pause for a week or two, let people do some testing, collect
feedback and bug reports and suggestions, get outside and enjoy the fall
weather instead of spending so much time coding.
I was home sick from work on Tuesday, so I took the opportunity to hack on Hep a bit. And since I wasn't feeling well I didn't feel bad about ignoring the important-but-not-fun items on my TODO list, and playing with some interesting things instead.
So Jabber support is now working in Hep. You give Hep a jabber account and password, and it logs in as username@server/Hep (with priority 2), and once for each of your destinations (username@server/DestinationName, priority 1). Messages send to username@server/Hep (or just username@server since it Hep has the highest priority) go to your Hep Inbox. Messages sent to the other resources get posted to the appropriate destinations (you configure a list of jabber accounts to accept messages from, to prevent strangers from sending messages to your weblog).
I also added the concept of "protocol handlers" to Hep, so that it can figure out how to send a message to a mailto: or jabber: address. A protocol works much like a destination, except that every time you send a message you also specify an address. That way you don't have to set up destinations for every e-mail or jabber address you might want to send a message to - you just set up your e-mail configuration once, and then specify the address when you send the message. (I haven't figured out all the details of how this will work, but in the short term it should let Hep handle filters like "Copy all messages from Slashdot to this e-mail address").
Hep's architecture is making me happy. Adding Jabber support was only 100 lines of code (thanks to the nice jabber.py library). The rest of the work is all generic support for protocol handlers, which can be added in the future by dropping modules in the messaging/protocols directory.
Also I made a diagram of how Hep works, which may or may not help other people understand it.
I'll try to get another snapshot release ready this week, so that other people can start playing with this stuff. And I really should write up some developer documentation...
I've put a new version of Hep up for testing (see the bottom of the page). This is an alpha release, meant for developers and curious people. It has some nice new features:
Sitting on a bench outside the coffee shop this afternoon, I think I finally figured out how Jabber instant messaging can be connected to Hep. I'm putting my thoughts up here for comments. This is a first draft. Let me know what you think.
I've been playing with my new wireless card (as you may have guessed
from my last post). I may write up something about what I've learned -
as a wireless newbie I was disappointed by the lack of good real-world
documentation on what to expect from wireless - things like what kind of
signal strength you need to get a connection, how to tell if the network
you're on is using encryption, etc. Maybe I just haven't looked in the
right places, but I've spent the time on Google.
I'm still working on Hep 0.3, but I think I'm going to have to push the
release back to next week, and even then it will probably be an
"0.3-alpha" release. Hep 0.3 has a built in SMTP proxy server that
watches messages going to your ISP's SMTP server. If it sees a message
addressed to <address>@hep, it delivers it to a destination you've
configured. I'm sending this message to weblog@hep (my weblog), and
cc'ing advogato@hep (my advogato diary). It seems to be working so far,
but the thought of people sending all their e-mail through Hep makes me
nervous: suddenly Hep will be handling information that's actually
important. So I want to do some good testing, and let the brave and
curious use it for a while before I do a "real" release.
I added a new feature to Hep. Here's how it works. You give Hep the URL of a web page you'd like to subscribe to. Hep grabs the page from the URL, and passes it (along with the original URL and HTTP headers) to each of its message source drivers. Each driver gets a chance to analyze the page information and try to get messages from that page. If it can, it returns a working configuration. If not, Hep goes on to the next driver.
Currently I've added support for community sites using Slash, Virgule, or PHP-Nuke, weblogs that use either a <link> tag or an xml.gif image to point to their RSS feed, and individual diaries on Virgule-powered sites. And I wrote a simplified "Subscribe" bookmarklet that just passes Hep the URL of the web page you're currently looking at. This makes the process of adding a message source almost completely painless:
All this and more in Hep 0.2.3, coming soon!
There's a new version of Hep (now officially called Hep Messaging Server) available for download on my website. The GUI from version 0.1 is gone, replaced by web and POP3 servers that let you configure Hep through your browser, and get messages through your e-mail client.
Sending messages to weblogs still isn't supported, but it will be in the next couple of weeks. So Hep 0.2 doesn't really do anything more than 0.1 did - it works as a news aggregator, and that's about it. But internally it has a lot of new features - built in POP3 and web servers, support for multiple users, and a cleaned up API, which gives me a good platform to build on.
Now that it doesn't have its own client interface, Hep should run on any operating system that supports Python and the Python standard libraries. I've been developing it on Linux, but this morning I tested it on Windows and it ran, without errors, the first try. Hooray for Python!
I've released a new hep tarball that fixes a bugs in the 0.1 version, where messages with long Subject lines and/or IDs would get corrupted. The new also version stores it's messages in maildir directories, which is faster and much less prone to corruption. Also, it uses a slightly different format for the ID numbers it generates for RSS newsfeeds.
This means that, if you're currently using Hep 0.1, you'll lose whatever messages are currently in your Inbox, and the first time you run this version it will re-download messages that you've already read and deleted. All your accounts will be intact, though.
Thanks to everyone who's given me feedback on Hep over the past two days!
New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.
Keep up with the latest Advogato features by reading the Advogato status blog.
If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!