Exim is designed to work efficiently on systems that are permanently connected to the Internet and are handling a general mix of mail. In such circumstances, most messages can be delivered immediately. Consequently, Exim does not maintain independent queues of messages for specific domains or hosts, though it does try to send several messages in a single SMTP connection after a host has been down, and it also maintains per-host retry information.

Policy controls are now an important feature of MTAs that are connected to the Internet. Perhaps their most important job is to stop MTAs being abused as “open relays” by misguided individuals who send out vast amounts of unsolicited junk, and want to disguise its source. Exim provides flexible facilities for specifying policy controls on incoming mail:

In a conventional Exim configuration, users are able to run private filters by setting up appropriate .forward files in their home directories. See chapter 22 (about the redirect router) for the configuration needed to support this, and the separate document entitled Exim’s interfaces to mail filtering for user details. Two different kinds of filtering are available:

User filters are run as part of the routing process, described below.

Every message handled by Exim is given a message id which is sixteen characters long. It is divided into three parts, separated by hyphens, for example 16VDhn-0001bo-D3. Each part is a sequence of letters and digits, normally encoding numbers in base 62. However, in the Darwin operating system (Mac OS X) and when Exim is compiled to run under Cygwin, base 36 (avoiding the use of lower case letters) is used instead, because the message id is used to construct file names, and the names of files in those systems are not always case-sensitive.

The detail of the contents of the message id have changed as Exim has evolved. Earlier versions relied on the operating system not re-using a process id (pid) within one second. On modern operating systems, this assumption can no longer be made, so the algorithm had to be changed. To retain backward compatibility, the format of the message id was retained, which is why the following rules are somewhat eccentric:

After a message has been received, Exim waits for the clock to tick at the appropriate resolution before proceeding, so that if another message is received by the same process, or by another process with the same (re-used) pid, it is guaranteed that the time will be different. In most cases, the clock will already have ticked while the message was being received.

The only way Exim can receive mail from another host is using SMTP over TCP/IP, in which case the sender and recipient addresses are transferred using SMTP commands. However, from a locally running process (such as a user’s MUA), there are several possibilities:

In the three cases that do not involve TCP/IP, the sender address is constructed from the login name of the user that called Exim and a default qualification domain (which can be set by the qualify_domain configuration option). For local or batch SMTP, a sender address that is passed using the SMTP MAIL command is ignored. However, the system administrator may allow certain users (“trusted users”) to specify a different sender address unconditionally, or all users to specify certain forms of different sender address. The -f option or the SMTP MAIL command is used to specify these different addresses. See section 5.2 for details of trusted users, and the untrusted_set_sender option for a way of allowing untrusted users to change sender addresses.

Messages received by either of the non-interactive mechanisms are subject to checking by the non-SMTP ACL, if one is defined. Messages received using SMTP (either over TCP/IP, or interacting with a local process) can be checked by a number of ACLs that operate at different times during the SMTP session. Either individual recipients, or the entire message, can be rejected if local policy requirements are not met. The local_scan() function (see chapter 41) is run for all incoming messages.

Exim can be configured not to start a delivery process when a message is received; this can be unconditional, or depend on the number of incoming SMTP connections or the system load. In these situations, new messages wait on the queue until a queue runner process picks them up. However, in standard configurations under normal conditions, delivery is started as soon as a message is received.

When Exim accepts a message, it writes two files in its spool directory. The first contains the envelope information, the current status of the message, and the header lines, and the second contains the body of the message. The names of the two spool files consist of the message id, followed by -H for the file containing the envelope and header, and -D for the data file.

By default all these message files are held in a single directory called input inside the general Exim spool directory. Some operating systems do not perform very well if the number of files in a directory gets very large; to improve performance in such cases, the split_spool_directory option can be used. This causes Exim to split up the input files into 62 sub-directories whose names are single letters or digits.

The envelope information consists of the address of the message’s sender and the addresses of the recipients. This information is entirely separate from any addresses contained in the header lines. The status of the message includes a list of recipients who have already received the message. The format of the first spool file is described in chapter 52.

Address rewriting that is specified in the rewrite section of the configuration (see chapter 31) is done once and for all on incoming addresses, both in the header lines and the envelope, at the time the message is accepted. If during the course of delivery additional addresses are generated (for example, via aliasing), these new addresses are rewritten as soon as they are generated. At the time a message is actually delivered (transported) further rewriting can take place; because this is a transport option, it can be different for different forms of delivery. It is also possible to specify the addition or removal of certain header lines at the time the message is delivered (see chapters 15 and 24).

A message remains in the spool directory until it is completely delivered to its recipients or to an error address, or until it is deleted by an administrator or by the user who originally created it. In cases when delivery cannot proceed – for example, when a message can neither be delivered to its recipients nor returned to its sender, the message is marked “frozen” on the spool, and no more deliveries are attempted.

An administrator can “thaw” such messages when the problem has been corrected, and can also freeze individual messages by hand if necessary. In addition, an administrator can force a delivery error, causing a bounce message to be sent.

While Exim is working on a message, it writes information about each delivery attempt to its main log file. This includes successful, unsuccessful, and delayed deliveries for each recipient (see chapter 48). The log lines are also written to a separate message log file for each message. These logs are solely for the benefit of the administrator, and are normally deleted along with the spool files when processing of a message is complete. The use of individual message logs can be disabled by setting no_message_logs; this might give an improvement in performance on very busy systems.

All the information Exim itself needs to set up a delivery is kept in the first spool file, along with the header lines. When a successful delivery occurs, the address is immediately written at the end of a journal file, whose name is the message id followed by -J. At the end of a delivery run, if there are some addresses left to be tried again later, the first spool file (the -H file) is updated to indicate which these are, and the journal file is then deleted. Updating the spool file is done by writing a new file and renaming it, to minimize the possibility of data loss.

Should the system or the program crash after a successful delivery but before the spool file has been updated, the journal is left lying around. The next time Exim attempts to deliver the message, it reads the journal file and updates the spool file before proceeding. This minimizes the chances of double deliveries caused by crashes.

The main delivery processing elements of Exim are called routers and transports, and collectively these are known as drivers. Code for a number of them is provided in the source distribution, and compile-time options specify which ones are included in the binary. Run time options specify which ones are actually used for delivering messages.

Each driver that is specified in the run time configuration is an instance of that particular driver type. Multiple instances are allowed; for example, you can set up several different smtp transports, each with different option values that might specify different ports or different timeouts. Each instance has its own identifying name. In what follows we will normally use the instance name when discussing one particular instance (that is, one specific configuration of the driver), and the generic driver name when discussing the driver’s features in general.

A router is a driver that operates on an address, either determining how its delivery should happen, by assigning it to a specific transport, or converting the address into one or more new addresses (for example, via an alias file). A router may also explicitly choose to fail an address, causing it to be bounced.

A transport is a driver that transmits a copy of the message from Exim’s spool to some destination. There are two kinds of transport: for a local transport, the destination is a file or a pipe on the local host, whereas for a remote transport the destination is some other host. A message is passed to a specific transport as a result of successful routing. If a message has several recipients, it may be passed to a number of different transports.

An address is processed by passing it to each configured router instance in turn, subject to certain preconditions, until a router accepts the address or specifies that it should be bounced. We will describe this process in more detail shortly. First, as a simple example, we consider how each recipient address in a message is processed in a small configuration of three routers.

To make this a more concrete example, it is described in terms of some actual routers, but remember, this is only an example. You can configure Exim’s routers in many different ways, and there may be any number of routers in a configuration.

The first router that is specified in a configuration is often one that handles addresses in domains that are not recognized specially by the local host. These are typically addresses for arbitrary domains on the Internet. A precondition is set up which looks for the special domains known to the host (for example, its own domain name), and the router is run for addresses that do not match. Typically, this is a router that looks up domains in the DNS in order to find the hosts to which this address routes. If it succeeds, the address is assigned to a suitable SMTP transport; if it does not succeed, the router is configured to fail the address.

The second router is reached only when the domain is recognized as one that “belongs” to the local host. This router does redirection – also known as aliasing and forwarding. When it generates one or more new addresses from the original, each of them is routed independently from the start. Otherwise, the router may cause an address to fail, or it may simply decline to handle the address, in which case the address is passed to the next router.

The final router in many configurations is one that checks to see if the address belongs to a local mailbox. The precondition may involve a check to see if the local part is the name of a login account, or it may look up the local part in a file or a database. If its preconditions are not met, or if the router declines, we have reached the end of the routers. When this happens, the address is bounced.

As well as being used to decide how to deliver to an address, Exim’s routers are also used for address verification. Verification can be requested as one of the checks to be performed in an ACL for incoming messages, on both sender and recipient addresses, and it can be tested using the -bv and -bvs command line options.

When an address is being verified, the routers are run in “verify mode”. This does not affect the way the routers work, but it is a state that can be detected. By this means, a router can be skipped or made to behave differently when verifying. A common example is a configuration in which the first router sends all messages to a message-scanning program, unless they have been previously scanned. Thus, the first router accepts all addresses without any checking, making it useless for verifying. Normally, the no_verify option would be set for such a router, causing it to be skipped in verify mode.

As explained in the example above, a number of preconditions are checked before running a router. If any are not met, the router is skipped, and the address is passed to the next router. When all the preconditions on a router are met, the router is run. What happens next depends on the outcome, which is one of the following:

If an address reaches the end of the routers without having been accepted by any of them, it is bounced as unrouteable. The default error message in this situation is “unrouteable address”, but you can set your own message by making use of the cannot_route_message option. This can be set for any router; the value from the last router that “saw” the address is used.

Sometimes while routing you want to fail a delivery when some conditions are met but others are not, instead of passing the address on for further routing. You can do this by having a second router that explicitly fails the delivery when the relevant conditions are met. The redirect router has a “fail” facility for this purpose.

The preconditions that are tested for each router are listed below, in the order in which they are tested. The individual configuration options are described in more detail in chapter 15.

Note that require_files comes near the end of the list, so you cannot use it to check for the existence of a file in which to lookup up a domain, local part, or sender. However, as these options are all expanded, you can use the exists expansion condition to make such tests within each condition. The require_files option is intended for checking files that the router may be going to use internally, or which are needed by a specific transport (for example, .procmailrc).

When a message is to be delivered, the sequence of events is as follows:

Exim’s mechanism for retrying messages that fail to get delivered at the first attempt is the queue runner process. You must either run an Exim daemon that uses the -q option with a time interval to start queue runners at regular intervals, or use some other means (such as cron) to start them. If you do not arrange for queue runners to be run, messages that fail temporarily at the first attempt will remain on your queue for ever. A queue runner process works its way through the queue, one message at a time, trying each delivery that has passed its retry time. You can run several queue runners at once.

Exim uses a set of configured rules to determine when next to retry the failing address (see chapter 32). These rules also specify when Exim should give up trying to deliver to the address, at which point it generates a bounce message. If no retry rules are set for a particular host, address, and error combination, no retries are attempted, and temporary errors are treated as permanent.

There are many reasons why a message may not be immediately deliverable to a particular address. Failure to connect to a remote machine (because it, or the connection to it, is down) is one of the most common. Temporary failures may be detected during routing as well as during the transport stage of delivery. Local deliveries may be delayed if NFS files are unavailable, or if a mailbox is on a file system where the user is over quota. Exim can be configured to impose its own quotas on local mailboxes; where system quotas are set they will also apply.

If a host is unreachable for a period of time, a number of messages may be waiting for it by the time it recovers, and sending them in a single SMTP connection is clearly beneficial. Whenever a delivery to a remote host is deferred,

Exim makes a note in its hints database, and whenever a successful SMTP delivery has happened, it looks to see if any other messages are waiting for the same host. If any are found, they are sent over the same SMTP connection, subject to a configuration limit as to the maximum number in any one connection.

When a message cannot be delivered to some or all of its intended recipients, a bounce message is generated. Temporary delivery failures turn into permanent errors when their timeout expires. All the addresses that fail in a given delivery attempt are listed in a single message. If the original message has many recipients, it is possible for some addresses to fail in one delivery attempt and others to fail subsequently, giving rise to more than one bounce message. The wording of bounce messages can be customized by the administrator. See chapter 45 for details.

Bounce messages contain an X-Failed-Recipients: header line that lists the failed addresses, for the benefit of programs that try to analyse such messages automatically.

A bounce message is normally sent to the sender of the original message, as obtained from the message’s envelope. For incoming SMTP messages, this is the address given in the MAIL command. However, when an address is expanded via a forward or alias file, an alternative address can be specified for delivery failures of the generated addresses. For a mailing list expansion (see section 46.2) it is common to direct bounce messages to the manager of the list.

If a bounce message (either locally generated or received from a remote host) itself suffers a permanent delivery failure, the message is left on the queue, but it is frozen, awaiting the attention of an administrator. There are options that can be used to make Exim discard such failed messages, or to keep them for only a short time (see timeout_frozen_after and ignore_bounce_errors_after).