fetchmail is a network gateway program. Its main purpose is to translate between POP3 or IMAP remote-mail protocols and the Internet's native SMTP protocol for email exchange. It is in extremely widespread use on Unix machines that use intermittent SLIP or PPP connections to Internet service providers, and as such probably touches an appreciable fraction of the Internet's mail traffic.
fetchmail has no fewer than 60 command-line options (which, as we'll establish later in this book, is probably too many), and a number of other options that are settable from the run-control file but not from the command line. Of all these, the most important — by far — is -v, the verbose option.
When -v is on, fetchmail dumps each one of its POP, IMAP, and SMTP transactions to standard output as they happen. A developer can actually see the code doing protocol with remote mailservers and the mail transport program it forwards to, in real time. Users can send session transcripts with their bug reports. Example6.1 shows a representative session transcript.
Example6.1.An example fetchmail -v transcript.
fetchmail: 6.1.0 querying hurkle.thyrsus.com (protocol IMAP) at Mon, 09 Dec 2002 08:41:37 -0500 (EST): poll started fetchmail: running ssh %h /usr/sbin/imapd (host hurkle.thyrsus.com service imap) fetchmail: IMAP< * PREAUTH [42.42.1.0] IMAP4rev1 v12.264 server ready fetchmail: IMAP> A0001 CAPABILITY fetchmail: IMAP< * CAPABILITY IMAP4 IMAP4REV1 NAMESPACE IDLE SCAN SORT MAILBOX-REFERRALS LOGIN-REFERRALS AUTH=LOGIN THREAD=ORDEREDSUBJECT fetchmail: IMAP< A0001 OK CAPABILITY completed fetchmail: IMAP> A0002 SELECT "INBOX" fetchmail: IMAP< * 2 EXISTS fetchmail: IMAP< * 1 RECENT fetchmail: IMAP< * OK [UIDVALIDITY 1039260713] UID validity status fetchmail: IMAP< * OK [UIDNEXT 23982] Predicted next UID fetchmail: IMAP< * FLAGS (\Answered \Flagged \Deleted \Draft \Seen) fetchmail: IMAP< * OK [PERMANENTFLAGS (\* \Answered \Flagged \Deleted \Draft \Seen)] Permanent flags fetchmail: IMAP< * OK [UNSEEN 2] first unseen in /var/spool/mail/esr fetchmail: IMAP< A0002 OK [READ-WRITE] SELECT completed fetchmail: IMAP> A0003 EXPUNGE fetchmail: IMAP< A0003 OK Mailbox checkpointed, no messages expunged fetchmail: IMAP> A0004 SEARCH UNSEEN fetchmail: IMAP< * SEARCH 2 fetchmail: IMAP< A0004 OK SEARCH completed 2 messages (1 seen) for esr at hurkle.thyrsus.com. fetchmail: IMAP> A0005 FETCH 1:2 RFC822.SIZE fetchmail: IMAP< * 1 FETCH (RFC822.SIZE 2545) fetchmail: IMAP< * 2 FETCH (RFC822.SIZE 8328) fetchmail: IMAP< A0005 OK FETCH completed skipping message [email protected]:1 (2545 octets) not flushed fetchmail: IMAP> A0006 FETCH 2 RFC822.HEADER fetchmail: IMAP< * 2 FETCH (RFC822.HEADER {1586} reading message [email protected]:2 of 2 (1586 header octets) fetchmail: SMTP< 220 snark.thyrsus.com ESMTP Sendmail 8.12.5/8.12.5; Mon, 9 Dec 2002 08:41:41 -0500
fetchmail: SMTP> EHLO localhost fetchmail: SMTP< 250-snark.thyrsus.com Hello localhost [127.0.0.1], pleased to meet you fetchmail: SMTP< 250-ENHANCEDSTATUSCODES fetchmail: SMTP< 250-8BITMIME fetchmail: SMTP< 250-SIZE fetchmail: SMTP> MAIL FROM:<[email protected]> SIZE=8328 fetchmail: SMTP< 250 2.1.0 <[email protected]>... Sender ok fetchmail: SMTP> RCPT TO:<esr@localhost> fetchmail: SMTP< 250 2.1.5 <esr@localhost>... Recipient ok fetchmail: SMTP> DATA fetchmail: SMTP< 354 Enter mail, end with "." on a line by itself # fetchmail: IMAP< ) fetchmail: IMAP< A0006 OK FETCH completed fetchmail: IMAP> A0007 FETCH 2 BODY.PEEK[TEXT] fetchmail: IMAP< * 2 FETCH (BODY[TEXT] {6742} (6742 body octets) *********************.**************************. ********************************.************************.*********** **********.***********************.*************** fetchmail: IMAP< ) fetchmail: IMAP< A0007 OK FETCH completed fetchmail: SMTP>. (EOM) fetchmail: SMTP< 250 2.0.0 gB9ffWo08245 Message accepted for delivery flushed fetchmail: IMAP> A0008 STORE 2 +FLAGS (\Seen \Deleted) fetchmail: IMAP< * 2 FETCH (FLAGS (\Recent \Seen \Deleted)) fetchmail: IMAP< A0008 OK STORE completed fetchmail: IMAP> A0009 EXPUNGE fetchmail: IMAP< * 2 EXPUNGE fetchmail: IMAP< * 1 EXISTS fetchmail: IMAP< * 0 RECENT fetchmail: IMAP< A0009 OK Expunged 1 messages fetchmail: IMAP> A0010 LOGOUT fetchmail: IMAP< * BYE hurkle IMAP4rev1 server terminating connection fetchmail: IMAP< A0010 OK LOGOUT completed fetchmail: 6.1.0 querying hurkle.thyrsus.com (protocol IMAP) at Mon, 09 Dec 2002 08:41:42 -0500: poll completed fetchmail: SMTP> QUIT fetchmail: SMTP< 221 2.0.0 snark.thyrsus.com closing connection fetchmail: normal termination, status 0
The -v option makes what fetchmail is doing discoverable (by letting you see the protocol exchanges). This is immensely useful. I considered it so important that I wrote special code to mask account passwords out of -v transaction dumps so that they could be passed around and posted without anyone having to remember to edit sensitive information out of them.
This turned out to be a good call. At least eight out of ten problems reported get diagnosed within seconds of a knowledgeable person's eyes seeing a session transcript. There are several knowledgeable people on the fetchmail mailing list — in fact, because most bugs are easy to diagnose, I seldom have to handle them myself.
Over the years, fetchmail has acquired a reputation as a rather bulletproof program. It can be misconfigured, but it very seldom outright breaks. Betting that this has nothing to do with the fact that the exact circumstances of eight out of ten bugs are rapidly discoverable would not be smart.
We can learn from this example. The lesson is this: Don't let your debugging tools be mere afterthoughts or treat them as throwaways. They are your windows into the code; don't just knock crude holes in the walls, finish and glaze them. If you plan to keep the code maintained, you're always going to need to let light into it.
