3 Feb 2014 dan   » (Master)

MuDDLe, a faster simpler maildir downloader

One of the services running on my old Bytemark VM was the Dovecot IMAP server. When I started thinking about configuring it on the replacement box I realised that I don’t actually need IMAP these days, so, er, why bother? So I didn’t. But what I do need is a way of getting the Maildir on that machine onto other machines, and everything I looked at to do this job (other than rsync) was fearsomely complicated because it also catered for a zillion other file formats and/or transports. Or because it wanted to sync in both directions, which I don’t really care about that much.

So, muddle

In essence, what it does is this: connect to the remote host, find list of files in cur/ and new/, compare with similar list on local host, create tar stream of differences, transfer it, unpack each transferred file into tmp/ and atomically rename into cur/ when done.

This adds up to one transfer of file names and a second transfer of all the file contents. Each of these is one-way and distinctly non-chatty, so it should have reasonably good network performance characteristics and you can use ssh compression if your network bête noire is bandwidth itself and not just latency.

It might not be as simple as possible, but at the same time it might also be simpler. For example, and as alluded to above, the download is one-way only, so it won’t e.g. update the server to mark messages as read. If you care about that stuff, this is not for you.

(Why not rsync? It can’t detect that a rename from new/foo to cur/foo:2, is a rename, so treats the latter as a new file. Which is a teensy bit suboptimal)

Syndicated 2014-02-02 23:27:24 from diary at Telent Netowrks

Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!