DMARC is in the email tech news once more, following Yahoo’s decision to publish a DMARC policy on some of their domains, telling recipient systems to reject mails which do not have valid origin information. I’ve previous written here about DMARC in the context of its privacy implications, covering mailing-list disclosures and then revisiting, for bug interactions making matters worse.
In addition, my name has come up in various circles because of a patch to Mailman which I contributed to. However, it seems that my stance may be being inadvertently misconstrued based solely on the evidence of having contributed to that patch. (My thanks to those who have drawn my attention to this happening.)
My first stance boils down to “when making changes, first do no harm”.
As a fundamental security matter, given a mailing-list, one subscriber should not be able to take actions which cause another subscriber to be unsubscribed. Nor should domain postmasters be able to cause this to happen. This is a Denial of Service attack and should be treated as such.
What the path is to stability is open to some debate, but when large mail operators (who have competing market products) shrug their shoulders and decide it’s not their problem and proceed to take action causing collateral damage to others, this is clearly not a constructive path forward and does not lead to finding a common ground to work with others.
The next three paragraphs are technical background, written because there is a lot of misinformation about what DMARC actually does.
Some technical terminology: “DKIM” is a method for provably attaching an origin domain to an email, as long as the mail is not modified, which lets reputation accrue to a domain rather than just to IP addresses, and DKIM signing will typically improve the deliverability of email from legitimate senders, including making it easier for mail from new locations to get through, as long as the mail is attached to the same domain (even if with a different selector key). “ADSP” was a method for domain owners to say what should happen to messages purporting to be from that domain where the message lacked valid DKIM signatures. “DMARC” is much the same thing, but more hand-waving around allowing “SPF” (sender-IP whitelisting) to be used instead of DKIM, support for privacy-violating feedback reports (see the two previous posts) and support for percentage rollouts (which might work better if everyone using DMARC records actually looked at the current DNS records, instead of periodically snapshotting some records into internal static configuration and then ignoring updates, for nine months in my first-hand experience).
There are two types of passing on of messages: those which change the content
and those which don’t. Simple forwarders don’t, and so are transparent to
DKIM. Mailing-lists typically do make modifications, including those which
are all but legally mandated – making sure that unsubscribe information is
clearly visible, in a world where Mail User Agent vendors have still not
implemented clean controls for taking advantage of RFC 2369
List-Unsubscribe: headers (fully formalised in 1998). The most common
solution to avoid legal risk is to use “footers” at the bottom of each mail
through a list and these footers invariably break DKIM, as this sort of
modification is what DKIM is designed to prevent. (End-users often clamour to
also have the
Subject: headers changed, because too many are stuck using
inadequate mail-clients which can’t adequately filter and segregate messages
based on list identity headers, so this also breaks most DKIM signatures).
Mailing-lists become the
Sender: for mails which they propagate, but the
human who the mail is
From: remains unchanged. Common mailing-list software
manages this just fine, although some mail-clients continue to not show the
Sender:. Such mail-clients can dangerously mislead their users as to
message origin, but for non-technical reasons are wide-spread. The DMARC
folks decided that it was more important to ensure that mail sent to people
using such limited mail-clients couldn’t mislead those people and tied DMARC
strongly to the
From: header, penalizing those who play by the rules. Game
theory has some interesting things to say about the end results of such
Thus it is that a DMARC policy record makes a lot of sense for domains sending transactional email, where the messages are sent directly to some account-holder, to prevent forgery of such, but that same policy record is deeply flawed when applied to a domain from which humans send email and participate in public mailing-lists.
The basic security problem is that if you have a list with 1000 subscribers
and 400 of those are using one of the big providers who actually implement
DMARC policy checking and pay attention to the published policies, then when
the mail is passed through the list in the normal way, DMARC verifiers will
reject the message as having a broken signature for something claiming to be
From: a domain with a DMARC policy, and so the mail-systems for 400 of the
1000 subscribers will reject the mail. When a subscriber’s mail is repeatedly
rejected, any sane mailing-list management software will disable the
subscription of that recipient. Failure to do so is at best very impolite.
In March of 2013, after seeing the impact of someone from a well-known tech company sending mail to a mail operators mailing-list, where that company had published a DMARC rejection policy, Jim Popovitch and I both decided that the right minimal approach was to not allow mails into the list from domains which publish such policy. This is basic security protection for the other list subscribers, to shield them from harmful activity impacting their subscriptions. As Jim is the operator of said mailing-list, he made the changes to ensure this rejection happened. I took his changes and made the DNS handling somewhat more robust. This applied to software called “Mailman”, which is a common choice for mailing-list management software.
This modification is entirely about ensuring that those who misuse DNS records for their domain (which is theirs to control and use – their domain, their rules) can not cause harm to others.
The patch pushes back the immediate problem, and has very recently been merged into the main codebase of Mailman.
However, I do think that mailing-lists should probably reconsider how they work. Yet, every avenue suggested carries problems larger than those purported to be solved by DMARC.
So, the short-term fix is to protect the integrity of the communication links between the mailing-lists and other subscribers and default to rejecting messages from domains which publish reject-triggering DMARC policies. This is a remediation step, preventing immediate harm, but is not a long-term solution. My contributing to the patch was not meant to imply that it is the end goal.
Yet this reject-mails-from-DMARC-reject-policy-domains behaviour must be the default for any mailing-list which adjusts content and which doesn’t take any other DKIM-handling action, simply to limit the damage which can be done.
For me, a concern with changing the way mailing-lists operate is that this
will require some changes to clients, when some of the worst of those clients
are webmail interfaces provided by the same people pushing DMARC. They
“simplify” clients, but then use the simplicity of the user interface to argue
for why standards must constrain the
From: domain. Gmail is a notable
exception here, amongst the web interfaces, in providing visible sender
This pattern of behaviour does not bode well for anything other than settling
into a long-term game of making list-operators jump through arbitrary hoops
which can be moved. Further, when we start overloading everything into the
From: record, whether those changes are individually justified or not, we
start down a path which parallels that of web-applications which run
everything over port 80 to bypass firewall rules, until there’s no point in
having firewall rules.
Mailing-list operators understand this, they’ve seen these sorts of tactics before, so seeing the behaviour repeat does, quite predictably, raise defensive hackles. Any vendor who has dealt with email professionally must surely have understood the behavioural impact and its reasons; to not have understood this would be a sign of incredible dysfunction.
There are interesting things that can be done, including those which improve behaviour for the mailing-lists. For instance, a mailing-list could track whether inbound mails from a given subscriber are DKIM-signed, with a DMARC or ADSP policy record in DNS (no matter whether that policy record asks for rejection). Since these signatures are applied by mail transfer systems, not by humans or software in MUAs, once the existence of those signatures is reliably observed for a domain, the MUA could block at ingress those messages which lack a signature, perhaps shunting it for moderation. This would avoid end-users having to learn PGP to have some explicit security around their interactions with managing their list subscriptions or sending messages to the list. The ability to turn this off, in the user interface, would allow for cascading lists, which are very much the exception rather than the rule, but can still be handled.
Ironically, a record similar to DMARC but intended only for those confident that they’re the direct receivers, thus not most end-user mail-systems, would help here: it could protect any system which takes automatic action based on emails, whether a mailing-list or a user-assistance system or in-feed for passing messages to other forms of media. Anything which would not normally be subscribed to external mailing-lists (and where those setting up such subscriptions could manually whitelist those particular lists).
Mailing-lists can also use DKIM, to carry reputation as a list, but asserting the identity as that of the Sender, whether as a domain or a full address.
Various other actions have been proposed, including rewriting the
header to use a trusted domain, but while these can in the short term work
around the deficiencies of DMARC, they all have unpleasant side-effects. One
of which concerns the legality of claiming authorship of something written by
someone else. The identity expressed in the
From: header is the author and
owner of the content of the mail, that in the
Sender: is merely who passes
the mail on. Indeed, what mailing-list operator wants to claim
responsibility for the content written by others, rather than just
responsibility for having passed it on to those who have subscribed to receive
such content? In the USA, in the short-term they can say goodbye to
DMCA Safe Harbor.
But DMARC as it is requires that the standards state one thing, but actual
recipient behaviour be modelled on comprehensive knowledge of which senders,
as seen by a given end recipient, are legitimate mailing-lists. This has
negative consequences for new lists not provided by established players, or
new subscriptions to lists not previously locally seen. The only players for
which this is not an issue are those who are so large that across their
userbase, 99.9% of all mailing-lists will already be seen, so that they can
have comprehensive knowledge of which sender IPs/resend-DKIM should be
whitelisted to exempt the DMARC
p=reject. This is a distorting competitive
advantage which amplifies the power of the large players. How unsurprising
that they would promote this technology.