My home computers use Postfix/Fetchmail for sending and getting email from Comcast and Gmail (Reference: Gmail on Linux Box).
I recently added IMAP to this configuration. This greatly improves things, since I can easily preserve my mail on the server, when reading/storing it from a laptop. One thing: I hate using the Linux account passwords for email. It can be a problem if ssl isn't setup correctly on the client. Passwords, Linux account passwords, would go across the Internet unencrypted. So, IMAP is configure to use sasldb for authentication. Not PAM. That way if my kids screw the client up, Linux passwords are separate and safe.
Mutt works well with IMAP(S) by storing certificates. If you have multiple servers, you can easily configure your .muttrc file to authenticate and grab mail automatically.
I've informally documented the setup in "Linux Tips" (TIP 188 and TIP 190), which includes necessary entries in "/etc/imapd.conf", creating users with saslpasswd2, changes to "/etc/postfix/main.cf" and "/etc/postfix/master.cf", and needed RPMS to install with Fedora Core.
I hope this helps.