[colug-432] Why my list mail is often tossed or winds up in spam folders...

[Rob Stampfli]

On Mon, Jan 11, 2016 at 08:39:49AM -0500, Scott Merrill wrote:
> GMail thinks your message is spam.
> 
> "Why is this message in Spam? It has a from address in cboh.org but
> has failed cboh.org's required tests for authentication."
> 
> 
> 
> ---------- Forwarded message ----------
> From: Rob Stampfli <rob944 at cboh.org>
> Date: Sun, Jan 10, 2016 at 10:16 AM
> Subject: Re: [colug-432] Troubleshooting Suggestions: File Truncation Issue
> To: Central OH Linux User Group - 432xx <colug-432 at colug.net>
> 
> 
> On Sun, Jan 10, 2016 at 08:15:02AM -0500, William E. T. wrote:
> > If we iterate over that process thousands to millions of times, we'll see
> > the following happen:
> >
> > open("main.pre.tsidx", O_RDWR|O_CREAT|O_EXCL|O_TRUNC, 0600) = 3
> > write(3, "\33[?1049h\33[?1h\33=\33[1;61r\33[?12;25h\33"..., 29549) = 29549
> > close(3)                                = 0
> > link("main.pre.tsidx", "main.tsidx")    = 0
> > unlink("main.pre.tsidx")                = 0
> > stat("main.tsidx", {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
> 
> I presume that the "tsidx" is a unique identifier?  Are these files
> being produced serially by only one process, or are there several
> processes running simultaneously?  My first inclination is to suspect
> you are getting an occasional collision in file names between two
> independent processes.
> 
> Also, it would be interesting to do a stat() of main.pre.tsidx before
> it is unlinked (i.e., the "link" step may be a red herring here).
> 
> Rob

Yup.  It's my DMARC specification:  I found a surprising number of
spammers were using my domain to proliferate their spam, trying to
make it appear as if it was legitimate email coming from cboh.org.
Because of this assault, my real cboh.org email was increasingly
being viewed with suspicion and relegated to the recipient's spam
folder if not outright rejected.  So, I instituted DMARC on cboh.org.

The DMARC specification uses two vectors (one cryptographic, the other
DNS based) to ensure that any mail from my domain only originates
from a legitimate source -- in this case, my servers.  If a message
from cboh.org fails all of the DMARC checks, I request the message
be rejected, and usually the email providers accede to the request,
although they are free to deviate and handle any email as they see
fit.

The problem is that DMARC doesn't interact well with mail exploders.
Indeed, it breaks the traditional way of handling list mail, which
retains the sender's email address in the resent list email, but
alters the message (changes the Subject line, adds a footer, etc.)
in a way that it no longer passes the DMARC check.  This is a well-
known problem which has had to be addressed by all mail exploders
including the big boys like YahooGroups.  It is one of the reasons
I stopped managing my own email lists.

Colug still adheres to the old standard of leaving the From: line
intact, as described above.  Google, OTOH, recognizes that there is
a remailer being run on the colug servers, so it grants these servers
a bit of a reprieve and overrides my "reject outright" request,
bumping the email up to the next level of acceptance, which is to
accept it but treat it as probable spam.  (I actually got feedback
from gmail indicating that that is exactly what they did in this
case.)  I've been running DMARC for over a year now, and colug is
about the only place left where there is a significant problem with
my legitimate emails.  I'm not extremely active here, so I've
elected to just live with it.

Whether you consider DMARC as a useful tool or the work of the devil,
it appears to be here to stay.  And if you're still interested,
google "DMARC breaks lists" and you'll get an eyeful.

Rob