[Twisted-Python] newbie problem with SMTPClient

Jean-Paul Calderone exarkun at divmod.com
Mon Mar 2 16:22:28 EST 2009


On Mon, 2 Mar 2009 15:57:17 -0500, Dave Britton <dave at davebritton.com> wrote:
>I love the idea of twisted but I think I must have a twisted learning disability, as I have gotten nowhere in what ought to be a simple matter.
>
>I need to send out emails to small groups from my apache server running a python cgi using mod_python, but my hosting service doesn't have a MTA. Instead of learning how to install and configure exim I thought I would use twisted to make a simple mail client. I started with the tutorial example that appears at:

It may be that installing exim is actually a better idea.  In order to get
reliable message deliver, you'll need to handle transient failures.  That
means persisting state over time (as long as several days) and performing
redelivery attempts.  However, if you don't mind losing outgoing messages
when there is a transient failure...

> [snip]
>
>This nicely looks up the right MX record and sends out an email, just what I need. Now I want to expand it to allow me to give it a list of email addresses to send the message to (not just call this same routine multple times, which seems wasteful and slow and doesn't use twisted's power to process the multiple emails in multiple threads), but I'm having terrible trouble figuring out how to do that, which tells me I'm missing a paradigm somewhere, there's something I'm not getting.

There is a reason that just calling the top-level function you've written
once for each email might be wasteful, but I don't think it's the reason
you're thinking.

If you just naively call getMailExchange(host).addCallback(cbMailExchange)
once for each email, then you will get parallel processing.  Almost as soon
as you call most Twisted APIs, they'll start an operation (to be precise,
many of them start it immediately - before they even return - and others
start it when you allow program execution flow to return to the reactor).
So if you call getMailExchange in a loop, each iteration of the loop will
start a new operation and they'll all run in parallel.  That seems like
just what you're after.

The reason this isn't the most efficient solution is that your list of
email addresses might contain two addresses which have the same mail
exchange host.  In this case, you could connect to that mail exchange
once and address your message to both addresses and then deliver the
message body to the mail exchange just once.  This is why getMailTo
returns a list.

So I think your other questions aren't directly relevant to this problem,
but I'll answer them anyway, since they're good questions.

>
>Trouble 1 is figuring out the right way to pass additional parameters to callbacks. Is this right:
>
>Dosomething(that-returns-a-deferred).addCallback(Then-do-the-next-thing, extra-parameter1, extraparameter2)
>
>The function Then-do-the-next-thing() will receive the deferred returned results from Dosomething() as its first argument, and extra-parameter1 and extraparameter2 as the next two. That is as if calling:
>Then-do-the-next-thing(Result-returned-by-Dosomething(),extra-parameter1, extraparameter2). Have I got this correct?

Yes, this is right.

>So, if this is right, then where do I want to put the additional argument that contains the next email address to send, if I iterate through the list and hand each one to the email sending process like this:
>
>elist=['addr1 at domain.com','addr2 at nextdomain.com'...]
>for e in elist:
>.... e2={'mxhost':'','toaddr':e}
>.... getMailExchange(e2).addCallback(cbMailExchange)
>

I'm not sure why you switched to a dictionary here.  Ignoring that, the
biggest potential problem with this code snippet is that it creates a
Deferred (the one returned by getMailExchange) and then drops it on the
floor (albeit after adding a callback).  You'll almost certainly want to
build up a list of Deferreds in cases like this, and then use something
like twisted.internet.defer.gatherResults to find out when they've all
fired.  Otherwise, you don't really know when your list of operations has
completed.

As to where to put extra arguments for tracking which email addresses to
send the message to next, I don't see why you'd want that in this case.
Your for loop iterates over all the addresses, so none of your callbacks
should need to know about any more addresses.  That is to say, by the end
of the for loop, there are no more addresses to which the message needs to
be sent - you've started sending to all of them already.

>In the tutorial, getMailExchange() is passed just the domain of the addressee, and the sending out of the email happens when the callback returns the MX exchange. I changed that to split the email address, so now it returns both the full address and the mx:
>
>def getMailExchange(addr):
>.... host=addr.split('@')[1]
>.... def cbMX(mxRecord):
>.... .... return [addr,str(mxRecord.name)]
>.... return relaymanager.MXCalculator().getMX(host).addCallback(cbMX)
>
>
>At this point I can't figure out how to get the email address passed to wherever it needs to go. And I don't know really where it needs to go... yikes.

Likely nothing in this code cares.  However, the address is going to be
passed to whatever callback is added to the Deferred this function
returns - because you've included it in the list returned by cbMX.

>
>In the tutorial the actual email address is hard coded into the class SMTPTutorialClient(smtp.ESMTPClient) as a class attribute, mailTo. I need to change that to be variable.
>
>How do I get this value (of mailTo) changed for each of the instances created by smtpClientFactory = SMTPClientFactory()? I think I must be confused about the roles of Factories and Protocols.

This is probably simpler than you expected - just override __init__ to
accept any extra data you need, and then save that data as an instance
attribute.

>I can't seem to figure out a way that works to get the email address passed into the system as a variable. Rather than waste people's time by describing my various failures, I thought I'd just ask for suggestions about the right twisted way to do it.

Another API you might want to look at is twisted.mail.smtp.sendmail.  It's
probably rather unfortunate that SMTP client tutorial doesn't cover that
function.

Hope this helps,

Jean-Paul




More information about the Twisted-Python mailing list