[Twisted-Python] Mailgun Email Service Replacement Proposal

[Glyph]

Hello Twistors,

As many of you know, Twisted's email infrastructure for the past few years has been generously provided by Mailgun.  This was a huge upgrade to our sender reputation over running our own Exim instance at the time, and I'm tremendously grateful for all the effort it's saved the project.

However, Mailgun has never been a super close fit for what we actually want as a project (see for example https://github.com/glyph/mg2dsn <https://github.com/glyph/mg2dsn> which papers over some of the differences between what they provide and what we actually want), and over the years there have been several hiccups in service as Mailgun tweaks its pricing and their plan-upgrade machinery doesn't know what to do with our discounted account.

Right now, I'm dealing with a minor issue where our "plan" (which, again, Mailgun has generously given us for free) no longer includes the ability to process incoming email... so, for example, this email is only reaching you because the pre-existing inbound routes are grandfathered in, and continue to work (whew!); however, I can no longer add new @twistedmatrix.com email addresses nor change the configuration of existing ones.

Now, for this particular issue I've opened a ticket with Mailgun and hopefully they'll be kind enough to extend their support to us yet again, but in the longer term, it feels like we might want to go back to hosting our own thing; ideally a thing that allows us to dogfood Twisted and maybe learn interesting things about our SMTP support.

What we need is not trivial, but it's also not too complicated.  Critically we do not need to host a full-featured mail storage service, as we only do forwarding to and from other services.  So what we need is a thing that can:

listen on TLS ports with a certificate
generate an RSA key for DKIM
spit out something we can plug into our twisted.names configuration instead of this - https://github.com/twisted-infra/braid/blob/1df63c5d8b44e079487be2f0bf099108a77872e5/services/t-names/zones/twistedmatrix.com#L58-L63 <https://github.com/twisted-infra/braid/blob/1df63c5d8b44e079487be2f0bf099108a77872e5/services/t-names/zones/twistedmatrix.com#L58-L63>
distinguish between inbound-forwarded and outbound-sent email
authenticate users to send for particular addresses (i.e. set "From" and "Sender" headers, and confirm consistency with MAIL FROM & auth, reject if anything doesn't match; with a caveat for messages forwarded via mailman); sign these outbound messages with https://pypi.org/project/dkimpy/ <https://pypi.org/project/dkimpy/>
forward inbound messages which have DMARC alignment
maybe run spambayes and junk stuff that's obviously spammy before forwarding if we want to get fancy
DKIM sign messages on their way through
take over port 25, somehow talk to mailman (either via talking to Debian's Exim on some alternate port or by running mailman's receipt scripts itself).

Basically, a signing / authenticating MX relay.

Anyone interested in attempting to write such a thing with Twisted? :)

-glyph
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20201002/ec86c826/attachment.htm>