IMAP server performance
If your IMAP server is started from [x]inetd, you may find you can get a moderate performance boost by switching to a standalone mode of operation. (Ie. IMAPd is running all the time, waiting for connections instead of being started on demand when a connection attempt is made.) Almost all popular IMAP servers, including Cyrus and UW-imapd, may be set up this way.
Some IMAP servers (typically UW IMAP) are set up by default to scan the user's entire home directory for mail. If they have a lot of files, this can take a pretty long time (comparatively), plus clutter up the subscribe list. Configure your IMAP server to look for mail below a specific directory, a good choice might be ~/Mail/ or ~/mail/ If you have to make this directory, be certain you get the permissions & ownership correct, and also make sure you tell your incoming delivery agent where to put the new mail!
Dovecot IMAP server
Use of the Dovecot IMAP server (free and fast, and supposedly the next default IMAP server for Red Hat Enterprise 4) speeds up SquirrelMail for even very large folders without migrating to maildir format. You can switch from UW-IMAP using mbox format directly to Dovecot and turn an almost unusable SquirrelMail into a fast system. Dovecot uses indexing which significantly speeds up header listing, etc. I had used an IMAP proxy before to help. After installing Dovecot, I just disabled the proxy because Dovecot alone is much faster than UW-IMAP with the proxy.
I'll second that suggestion! Forget about all the other optimizations and switch to Dovecot. I'm running multi-GB maildirs and SquirrelMail was almost unusable with Courier IMAP. Switching to Dovecot changed the response time from minutes to less than a second. -- email@example.com
Cyrus IMAP server
Cyrus is a very rich IMAP server with support for many IMAP extensions. Mail is located in the Cyrus mailstore which looks a bit like Courier maildir. Combined with a database, where Cyrus stores information like flags, certain headers and bodystructures, it makes Cyrus very fast.
Cyrus is widely used in large scale installations thanks to its performance.
Cyrus IMAP is bundled with SIEVE support, a server side filter tool. See also the [SIEVE Mail Filters] plugin.
More tips for specific servers