Sender Authentication Best Practices
Self
Virginia Beach
VA
23464
US
dougfoster.emailstandards@gmail.com
General
DMARC
Mailing Lists
Sender Authentication contributes to the goal of detecting and blocking maliciously impersonated email identifiers.
Sender Policy Framework (SPF) (RFC 7208) validates the RFC5321.MailFrom address, and
Domain-based Message Authentication, Reporting, and Conformance (DMARC) (RFC 7489) validates
the RFC5322.From header address. Both techniques may produce a result other than PASS on a message that the
recipient considers acceptable and wanted. This document describes best practices for integrating SPF and DMARC into
an email filtering strategy.
Sender Authentication contributes to the goal of detecting and blocking maliciously impersonated email
identifiers.
Sender Policy Framework (SPF)
validates the RFC5321.MailFrom address, and
Domain-based Message Authentication, Reporting, and Conformance (DMARC)
validates the RFC5322.From header address. For both techniques, a result of PASS indicates that the message is judged to be free of impersonation, and therefore free of malicious impersonation risk. Any message that does not produce PASS can therefore be considered at risk of possible impersonation. From a risk standpoint, the evaluator has little reason to distinguish between results of NOPOLICY, PERMERROR, SOFTFAIL, or FAIL. However, malicious impersonation is only one of many reasons that a message does not produce PASS, and therefore malicious impersonation can only be concluded after all other possibilities have been ruled out.
A typical email filtering solution has two primary components, sender filtering and content filtering. Sender filtering evaluates the identifiers in a message to assign a reputation to the sender of the message. Upon completion of sender filtering, the message has three possible dispositons:
- Privileged: The message is from a highly trusted sender and is exempted from some or all content filtering checks. Sender Authenticaton
PASS is required for Privileged mode to be granted safely. Great harm could result if a malicious impersonation is granted privileged mode,
allowing it to bypass both sender filters and content filters.
- Normal: The message sender is not rejected, but the messages must pass all content filtering checks to be allowed.
- Unwanted: The message is blocked based on sender identity and reputation alone. For purposes of this document,
no distinction is made whether the message is blocked with notification via an SMTP reject result, blocked with notification
via a Non-Delivery Report, or silentlty discarded without notification.
A basic assumption is that unwanted and malicious email originates from unwanted and malicious email senders.
While content filtering and sender authentication will flag or block single messages that are suspicious,
their greatest benefit accrues when they are used to identify malicious message sending entities,
so that all messages from that sender can be blocked. Consider the situation where a message is identified as malicious and blocked,
but no follow-up action is taken. When the attacker changes techniques, and a subsequent attack succeeds, the
system administrator will appear negligent. This document assumes a continuous improvement process with these characteristics:
- Content filtering or sender authentication identify a set of one or more suspicious messages with common characteristics.
- The message set is reviewed in depth to assess whether the messages are wanted or unwanted.
- If the message set is determined to be unwanted, the message sender is identified and a local policy is implemented to block messages from that sender.
- If the message set is determined to be acceptable, but was flagged as suspicious by content filtering,
then a local policy is implemented to give the message sender Privileged mode.
- If the message set is determined to be acceptable, but was flagged as suspicious because of sender authentication failure,
then a local policy is implemented to provide alternate authentication for that message set. Note that alternate authentication is different from
exemption. Exemption from sender authentication provides exemption for both intented and impersonation messages. Alternate authentication
maintains a distinction between the intended messages and their impersonators.
- When all other explanations have been exhausted, Sender Authentication failure alone may provide evidence of malicious intent. The difficulty
is ruling out all other possible explanations.
Because any message may require Priviledged mode processing, any message must be verifiable, using either SPF and DMARC or a local policy
which provides equivalent results. Local policy extends SPF to define acceptable relationships between a verfied server
identifier and the RFC5321.MailFrom domain, and extends DMARC to define acceptable relationships between
a verified RFC5321.MailFrom
and the RFC5322.From domain. The
supplemental authentication techniques for this purpose are explained later in this document.
Some messages will be categorized as unwanted because of previously configured rules or trusted reference sources. These messages are blocked, but
content may be preserved for subsequent review or other purposes.
Messages which fail Content Filtering are blocked. A subsequent review is used to determine if content filtering needs to be updated, if a sender block rule
is needed for specific senders, or if a Privilege Mode exemption is needed for trusted senders.
When a message set is determined to be unwanted, an analysis is necessary to identify the identifiers which correctly represent
the source of the attack. In most cases, one or more identifiers can be found which represent the source of the attack, and a single-attribute block
rule is created for each such identifier. In less common situations, a system manager may determine that an identifier was fraudulent,
but the underlying identifiers are not wholly malicious. If this occurs, a multiple-attribute block rule is needed to block the specific
combination of identifiers. In either configuration, verification of the identifiers is not necessary to the purposes of a block rule.
Messages which produce an authentication FAIL result will carry the highest presumption of malicious impersonation,
but exceptions will occur. Content Filtering will provide additional insight into the nature of the message,
so disposition should be delayed until this data has been collected. For messages that pass Content Filtering, a
sufficient defense is to quarantine these messages and prioritize
them for review and categorization. Confirmed malice will justify a local policy to block
all messages from the malicious source. Acceptable messages will be granted a local policy to provide alternate authentication.
Messages which pass both Sender Authentication and Content Filtering have no detected risk and are consequently released to the user. User complaints
could cause the sender to be reviewed and content filtering rules to be enhanced.
Some messages will produce neither PASS nor FAIL, because of missing, misconfigured, or non-enforcing domain owner policies. These messages
will be forwarded to content filtering, and may be blocked on that basis. The remainder will be messages that have no detected risk,
but may have undetected impersonation. The evaluator must decide whether to release such messages to the user or quarantine them for prior review.
The sheer volume of such messages will often cause an organization to release such messages to the final recipient, while retaining
the option of after-the-fact review. The risk of doing so is proportional to the effectiveness of the content filtering system. As
after-the-fact review identifies additional message sources to be blocked and additional messages sources to be granted local authentication policies,
the volume of ambiguous results will steadily decline. When the ambiguous volume is sufficiently controlled, the evaluator can switch from
after-the-fact review to quarantine with prior review.
SPF and DMARC only provide results if the domain owner has published a policy. The policy is generally considered actionable if the policy
returns a result of DMARC FAIL, or SPF FAIL without DMARC PASS. Even then, the FAIL result can be misleading if the domain owner's
implementation is not consistent with its policy, or if the message transit has caused the message to lose authentication.
For additional discussion of these issues, refer to Interoperability Issues between Domain-based Message Authentication,
Reporting, and Conformance (DMARC) and Indirect Email Flows.
RFC 7208 and RFC 7489 provide
no guidance for
evaluators to cope with incorrect, ambiguous, or No Policy results. Without exception management, Sender Authentication dies as soon as an exception
is necessary. A poorly designed exception process may enable the very impersonations that Sender Authentication is intended to prevent.
To handle exceptions appropriately, and to ensure that any acceptable message can be configured as authenticated, additional authentication
techniques are needed:
- DMARC using a local alignment policy: When a DMARC policy is not provided, DMARC-equivalent verification can be applied using an alignment rule configured by local policy
to produce an equivalent PASS or FAIL result.
- DKIM with alternate allignment: When a verified DKIM signature is available, and the From domain is not DMARC-aligned, but
the From address is determined to have an acceptable relationship to the DKIM domain, the message is considered equivalent to DMARC
PASS. SPF PASS is optional when the From address is authenticated by a DKIM signature.
- SPF with alternate alignment: When the RFC5321.MailFrom domain is verified with SPF PASS, and the From domain is not DMARC-aligned,
but is determined to have an acceptable relationship to the MailFrom domain, the message is considered equivalent to DMARC PASS.
- SPF with alternate server: When a message does not produce SPF PASS, but the RFC5321.MailFrom domain is determined to have an acceptable
relationship to the server organization, equivalence to SPF PASS can be established using the Source IP,
a forward-confirmed HELO name that represents the server organization, or a forward-confirmed Reverse DNS name which represents the server
organization. The SPF-equivalent PASS result can then be used to test the RFC5321.From domain for DMARC-equivalent PASS.
DMARC combines eight different authentiction types into a single result, but the eight types do not provide equal
confidence. Evaluators should be aware of the possibility of false PASS, and consider that risk when developing local
policies, especially local alignment policies.
DKIM and SPF are the two underlying authentication techniques for DMARC, and within each technique there are four possible
alignment methods: Self authenticates self, parent authenticates child, child authenticates parent, and sibling
authenticates sibling,
When a message is sent from a multi-tenant server, SPF assumes that the server owner has
administrative controls which prevent clients from impersonating each other, or that all clients are well behaved. DKIM, by contrast, has
the much simpler expectation that the server owner is able to ensure that each client's private key data is kept private to
themselves, out of reach from other clients in the same environment. Consequently, DKIM PASS always conveys a higher level
of confidence than SPF PASS. Of course, a DKIM signature also ensures that authentication is preserved during
forwarding (when done without content changes), so DKIM also provides confidence to a broader range of evaluators. In short, domain
owners who wish to promote maximum trust by evaluators should ensure that all of their messages have verifiable DKIM signatures.
Similarly, the different types of alignment provide different levels of confidence. When the authenticating
domain exactly matches the authenticated domain, trust in the result is maximized. The three varieties of relaxed
alignment depend on correctly identifying the organizational domain. The Public Suffix List from publicsuffix.org is the
reference used by most organizations, but it is not an authoritative source, is modified on a continuing basis, is developed
for cross-site-scripting defenses rather than email defensies, and has detectable errors and omisssions. While the frequency
of errors may be low, the impact of an error may be significant. If the PSL lands too low, the evaluator may not see a DMARC
policy at all, or may see the wrong policy. If the PSL lands too high, the evaluator may authenticate sibling organizations
as if they were sibling domains within a single organization. The impact of a PSL error varies with the type of authentication:
- Parent-authenticates-child is the most trustworthy form of relaxed alignment. DNS is managed hierarchically, so a parent domain has
unlimited control over a child domain if they choose to seize such control. Most organizations also operate on hierarchical control, so
the DNS structure is consistent with the real world that it is modelling. The risk of a registry domain impersonating a registered client
is assumed to be inherently low, but is within the rights of a parent domain. Consequently, a parent-authenticates-child relationship
carries a high level of confidence, nearly equivalent to strict alignment.
- Child-authenticates-parent is a little less intuitive, because it is not consistent with the heirarchical way that
organizations operate. Curiously, it is the most commonly observed form of relaxed alignment. An evaluator can assume
that if a child domain is improperly used to impersonate a parent domain, the response from the parent domain will be
swift and harsh. Consequently, a child-authenticates-parent relationship retains a significant degree of confidence.
-
Sibling-authenticates-sibling does not have an intuitive relationship to the control mechanisms of hierarchical organizations.
More importantly, sibling authentication is vulnerable to PSL errors used for malicious purposes. Consequently, it is the
least trustworthy from of relaxed authentication.
Domain owners who seek to maximize evaluator trust should utilize those authentication types which provide the highest confidence to evaluators.
When configuring local authentication policies, evaluators have the option of applying any of four confidence levels, to obtain
their desired balance between maximizing policy coverage and maximizing confidence in a PASS result.
Forwarding creates significant challengers for the evaluator seeking to eliminate impersonation. The evaluator needs to assess the reputation of both the originator and the forwarder, but in most cases, the greater concern applies to the originator. The process of forwarding:
- obscures the originating organization behind the forwarding organization,
- will either invalidate SPF PASS on the original RFC5321.MailFrom domain, or replace it completely to obtain SPF PASS on the forwarder domain,
- may involve content changes that invalidate DKIM signatures, and
- may alter the RFC5322.From header to evade DMARC problems on altered messages.
Consequently, an ideal evaluation process needs to include:
- identifying that one or more forwarding operations occurred,
- verifying through administrative measures that the recipient wants the forwarded stream,
- deciding how much filtering trust can be delegated to the forwarder,
- inferring, as much as possible, the identity of the originating server organization, RFC5321.MailFrom address, and RFC5322.From address,
- applying a reputation based on all of the above, and
- applying a disposition based on the reputation and the results of content filtering.
Forwarding may be easier to detect in aggregate than on single messages, because a forwarding configuration will produce a
stream of messages from a single server organization, on behalf of a wide variety of RFC5322.From domains. Within a single
message, these data sources may be available to help detect forwarding:
- An ARC Chain can be parsed to extract prior identifiers and their prior verification state.
- Received headers containing a "for user@domain" clause may reveal the original destination address. An MX lookup on that domain
may permit determination of the Received entry which indicates arrival at the forwarding organization's MX server.
- The sequence of organization changes and private knowlehdge can be used to identify organization roles: submitting agent domain, mailstore
domain, outbound gateway domain, or forwarding domain. Organizational changes are inferred when the Received header chain transits
between public and private IP addresses, and when two adjacent public IP addresses are identified as belonging to different server domains.
- An RFC5322.To address list which does not contain the RFC5321.RcptTo address can indicate many things, one of which is a forwarding operation.
- An RFC5321.MailFrom address that is encoded using the Sender Rewriting Scheme (SRS) can be decoded to reveal the prior RFC5321.MailFrom address or addresses. However, SRS encoding is also used by some servers upon origination, to detect invalid bounce messages.
- Forwarding without RFC5321.MailFrom rewrite is one of many reasons that a message will fail to produce SPF PASS.
While these are potentially useful clues, this document does not attempt to provide an algorithm for optimal interpretation of these clues.
Evaluators must consider that Received headers may be forged and ARC Sets may mislead. Consequently, header parsing is most
reliable when it is used to identify messages that are unwanted because of detected identifiers with an unacceptable reputation.
Evaluators who wish to choose to accept forwarded mail, but cannot perform detailed header parsing, will need to authenticate
based on the forwarder identity alone. For messages forwarded without RFC5321.MailFrom rewrite, this requires a local policy to
provide alternate authentication for the RFC5321.MailFrom address. The policy would need to specify that any MailFrom domain is
acceptable as long as the forwarding server identity matches the expected Source IP or the server host domain matches the expected
domain name and is forward confirmed. For messages forwarded with RFC5321.MailFrom rewrite to provide SPF PASS based on
the forwarder's domain, the local policy would need to specify that all RFC5322.From domains are acceptable as long
as the RFC5321.MailFrom domain has the expected value and is confirmed by SPF PASS or a local policy equivalent. In either case,
the evaluator is implicitly assuming that either the forwarder will intercept and block malicious impersonation, or that the
evaluator's content filtering will be strong enough to intercept and block any malicious impersonation in the forwarded stream.
Forwarders should consider that many evaluators will not attempt to navigate the complexity of originator detection. Instead, any unacceptable
messages will be assigned to the reputation of the forwarder, which could lead to obstruction of messages unrelated to the forwarded stream. Consequently,
forwarders should apply the same rigorous filtering to forwarded messages
as they use to protect their own domain.
Additionally, forwarders should be aware of the impact that content-altering spam filters will have on forwarded mail. An increasing
number of spam filters add disclaimers to distinguish external mail from internal mail, or rewrite embedded links to provide
click-time protection. Any such change will invalidate the originator's DKIM signatures, causing an authentication
problem if the message is forwarded in its altered state. Consequently, organizations that use content-altering spam
filters should not forward mail, and organizations that routinely forward mail should not use content-altering featues of their spam filter.
This memo includes no request to IANA.
The goal of sender authentication is to separate well-identified messages from potentially-impersonated messages. Successful
sender authentication is a necessary pre-requisite to privileged-mode handling of a document, but it is not a reason to
grant that privilege. Sender authentication simply ensures that any reputation information, obtianed by other
methods, can be applied correctly.
Evaluators must remember that both sender infrastructure and sender reputation can change over time and therefore
they must remain vigilant. Previously trusted sources may become compromised or even retired and
reassigned to other, less trustworthy, uses. Similarly, a previously untrusted source may have earned that designation
because of an infection which has been subsequntly corrected. Coping with these sources of variability is outside the scope of this document.
Becuase of the additional risk associated with sibling relationships, evaluators may wish to treat authentication
based on sibling alignment more carefully than other forms of relaxed alignment.