MTA-STS Enforces Authenticated TLS for SMTP Delivery

SMTP STARTTLS can encrypt mail transport, but ordinary opportunistic TLS permits delivery to continue when encryption is unavailable. That compatibility behavior leaves room for an active intermediary to suppress STARTTLS or redirect delivery toward an unintended server.

SMTP MTA Strict Transport Security, defined by RFC 8461, gives a recipient domain a policy channel for conforming sending MTAs. The policy states which MX hosts are acceptable and whether delivery must use TLS with a valid PKIX certificate. In enforce mode, a sender does not silently downgrade when those checks fail.

Policy discovery uses DNS and HTTPS for separate roles

An MTA-STS deployment has a DNS indicator and an HTTPS policy document. For example.com, the DNS record is published at:

_mta-sts.example.com. IN TXT "v=STSv1; id=20260923a;"

The id identifies a policy revision. A sender that sees a different value can fetch the current policy from the fixed HTTPS location:

https://mta-sts.example.com/.well-known/mta-sts.txt

The policy host must present a certificate valid for its mta-sts DNS name and chaining to a CA trusted by the sending MTA. DNS announces the presence and revision of the policy; HTTPS carries the policy body and authenticates the policy host through PKIX.

This split is significant because the TXT record is not the complete transport policy. Its revision identifier is a cache signal, not an ordered version number and not a substitute for the HTTPS document.

The policy constrains acceptable SMTP destinations

A policy contains a version, mode, one or more mx entries, and a max_age value. A compact policy can look like this:

version: STSv1
mode: enforce
mx: mail.example.com
mx: *.mail.example.net
max_age: 604800

The mx entries define host patterns that may receive mail under the policy. A wildcard is restricted to the complete left-most label. Thus *.mail.example.net can match a.mail.example.net, but it does not match mail.example.net or a.b.mail.example.net.

For a candidate MX to pass policy validation, its host name must match an allowed pattern, the server must support STARTTLS, and the certificate presented during the TLS handshake must be valid for that MX host under the PKIX checks required by the specification.

A certificate valid only for the policy host does not authorize an SMTP MX. Policy retrieval and SMTP delivery are distinct TLS connections with distinct host identities.

Enforce mode turns transport failure into deferred delivery

The policy mode controls sender behavior after validation failure. In enforce mode, a conforming sender must not deliver to an MX that fails the policy’s host, STARTTLS, or certificate checks. The sender can try another candidate MX that satisfies the policy.

If delivery still cannot proceed, the failure is treated as transient rather than immediately converting a transport-security problem into plaintext delivery. Before permanently failing the message, the sender checks DNS for a policy revision so that a corrected policy can take effect.

testing mode is different. It permits delivery as though the MTA-STS validation failure did not block the message, while implementations that also support TLS Reporting can report policy failures. none indicates that the domain has no active MTA-STS policy and is also part of the defined policy-removal procedure.

These modes make deployment state explicit. Moving directly to enforce without verifying every production MX, certificate chain, and routing path can turn a policy mistake into delayed mail.

Cached policy carries security state across DNS interference

A sender can cache a successfully retrieved policy for its max_age. If live discovery later fails but a valid, non-expired cached policy exists, RFC 8461 requires the sender to keep applying that cached policy.

That cache is part of the downgrade resistance. An attacker able to block an unauthenticated DNS response may suppress the TXT indicator during initial discovery. Once a sender has a valid cached policy, suppressing a later TXT response does not immediately erase the policy state.

The protection has a boundary: first contact can still be exposed to interference before a policy has been acquired. MTA-STS does not authenticate DNS with DNSSEC and does not claim to remove that initial-discovery risk. The specification recommends refreshing cached policies before expiry so a short interruption at the expiry boundary is less likely to discard established policy state.

Operationally, max_age therefore affects both change latency and persistence. A long value keeps an established policy active through longer discovery failures, while policy changes require careful handling because senders can legitimately retain older cached state until its lifetime ends.

Policy removal must account for existing caches

Deleting the TXT record and HTTPS endpoint is not enough to immediately remove an MTA-STS policy from senders that already cached it. Those senders can continue enforcing the cached document until its max_age expires.

RFC 8461 defines mode: none for controlled withdrawal. A domain can publish a new none policy with a short lifetime, update the TXT id so senders fetch it, then wait for previously served policies to age out before removing the discovery record and policy endpoint.

This sequence avoids treating cache persistence as an implementation accident. Cached policy is expected protocol state, so deployment and rollback procedures need to include it.

MTA-STS and DANE use different authentication foundations

MTA-STS and DANE for SMTP address related downgrade and destination-authentication risks, but their trust models differ. DANE uses DNSSEC-authenticated TLSA records. MTA-STS uses an HTTPS policy authenticated through the public PKIX ecosystem and does not require DNSSEC.

The mechanisms are not interchangeable. RFC 8461 explicitly states that MTA-STS validation must not override a failing DANE validation when both are in use. A deployment that supports both must preserve the stricter failure semantics required by the applicable DANE state rather than using MTA-STS as a fallback around it.

MTA-STS is therefore best treated as persistent transport policy, not merely a switch that says TLS is available. Its security effect comes from the combination of an authenticated HTTPS policy, constrained MX identities, certificate validation, sender-side caching, and refusal to downgrade delivery while a valid enforce policy applies.