To solve performance problems I've now started to switch my SMTP servers from using the "forkserver" version of qpsmtpd to using the "prefork" version.
Under testing qpsmtpd-prefork performed significantly better than the qpsmtpd-forkserver for handling incoming SMTP connections.
The loadavg of one machine has dropped from a constant 2.xx to 0.4x!
I'd love to see what the asynchronous server would behave like, but that would require re-writing all my plugins to work in an asynchronous manner which would be a significant undertaking.
(It would be nice if the qpsmtpd package available to Debian would allow you to choose between the two version of the server - I will file a wishlist bug.)