6 Jun 2000

Tue, 6 Jun 2000 20:58:09 GMT

Oops, Jerzy Balamut, the solid pop3 author pointed out that the pop3 protocol dosn't count bytes stuffed line beginings in the size returned by the LIST and STAT commands. I checked the RFC's, and he's right.

It's not that bad actually, cos i've been thinking about writing a script that takes a mail on stdin and writes it into a maildir with the right line endings etc.

It's needed for MTA's that arn't as flexable as exim, and i can now encode the file size without byte stuffed line beginings in the filename (a bit of a hack, but not a problem).

26 May 2000

Fri, 26 May 2000 22:55:16 GMT

got TOP working, and made exim 'byte-stuff' lines with a single '.' (which is how the pop server tells the client that it's finished sending the mail).

It all seems to work! (and, more importantly, is RFC complient).

Now it's supper time ;-)

26 May 2000

Fri, 26 May 2000 18:49:22 GMT

I'm setting up a high capacity pop server to replace one thats thrashing it's self to death at the moment, so I've been trying to find the most efficent pop server for what I want to do.

At the moment we are running qpopper, but if a mailbox gets above 50Mb or so the server really slows down as qpopper copies the entire mailbox before and after a user transfers their mail.

We need a system that works well with:

Large (50Mb+) mailboxes
Many users accessing large mailboxes at the same time

To get around the problem of large mailboxes I want a pop server that uses maildir, and for vairous reasons I want one that runs standalone, i.e. not through inetd.

After some searching through vairous mailing list archives and freshmeat I decided upon Solid POP3.

The problem now is that the wire-format for POP3 has all line endings as \r\n, where as the normal unix line endings are just \n. This means you have to scan through each message to get a count of the lines so you can account for transmitting all the line endings as \r\n - you need to know the size of each message so you can give the right responce to the LIST command.

With the test mailbox I was using (an excessive 1Gb in size and 329 messages) this takes 2 minutes!, which is WAY to long as far as user interactivity goes!

Now, since the only processes that are going to be touching these mailboxes are the MTA i'm using (exim), and the pop server I could do whatever I wanted with the mailbox format.

I set up exim to deliver all the mails with \r\n line endings, and then created a new mailbox type in the solid pop3 source, crlf_maildir.

Since we don't need to munge the line endings any more all we have to do for tha vairous pop commands is:

Getting the data for LIST is just a question of looping through readdir and stat'ing each file for it's size.
RETR is just a mmap and write.
TOP is just mmap, loop through counting line endings, and then write. (fetchmail uses TOP <msg> 99999999, and so might other pop clients, so if we check to see if no. of lines >= size of message we can just chuck out the message as a whole...)
... and everything else is as before

I've implemented most of this, and it now takes less than a second to authenticate a client, rather than 2 minute, which is a nice imporvement ;-)

Expect to see a new version of Solid POP3 soon!

Advogato blog for jasper

6 Jun 2000

26 May 2000

26 May 2000