[Twisted-Python] Some beginner questions about "twisted.names.client" and ".tac" environment
exarkun at divmod.com
Sat Dec 17 19:55:04 EST 2005
On Sun, 18 Dec 2005 01:05:32 +0100, Jesus Cea <jcea at argo.es> wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Jean-Paul Calderone wrote:
>> You probably want to move most of your program out if "dns.tac" and into
>> an importable Python module. Code defined inside .tac files lives in a
>> weird world where some surprising rules apply.
>Any documentation about that?
Hmmm, not that I know of. The problems are all essentially consequences of the fact that .tac files are run using the built-in execfile() function (or equivalent), rather than being loaded as modules.
>> It's best to keep the
>> .tac file as short as possible. Generally, you just want to create an
>> Application and give it some children, importing from modules the
>> definitions of all classes and functions needed to set this up.
>I reduced the "dns3.tac" to:
>from twisted.application import service
>application = service.Application("DNS test")
>All logic is in "dns3.tac". Basically the same code posted in my
>I have the very same problem with "reactor.stop()": repeated exceptions,
>only stoppable using "kill -9".
>I don't get the "give it some children" point. Could you post some
You want to tie your application logic to a Service subclass. To start with, I'd try something like this, in dns3.py:
from twisted.application import service
# Copied the top-level code from the original dns.tac
pendientes = 
redirigidos = 
f = open("domain_list")
for i in f :
aun_pendientes = len(pendientes)
concurrencia = 1000
for i in pendientes[:concurrencia]:
Then, in dns3.tac after application is defined,
from dns import DomainResolver
This will delay the execution of your startup code until twistd is totally ready and the reactor is fully initialized.
>> Passing (60,) as the timeout might not be the best idea. This will
>> cause the DNS client to send one request and then wait 60 seconds for a
>> response. If either the request or the response is dropped (as often
>> happens with UDP traffic), you will never get a result, and you will
>> have to wait 60 seconds to discover this fact.
>I'm resolving several thousand of domains, so one minute more or less is
>not an issue. I don't want to retransmit since I'm using 127.0.0.1, so
>losing request (in the same machine) should be rare (udp backlog
>Some domains takes a long time to resolve. So if I use an small value I
>load the server and get "Unexpected message (XXXXX) received from
>('127.0.0.1', 53)", caused because the DNS server gets a late answers
>and my code already give up.
Okay, it sounds like you know what you're doing here :)
>> To customize the server used by the resolver, you may want to create
>> your own resolver instance, rather than relying on the defaults guessed
>> by the resolver automatically created in the client module.
>Also a good point. Done and working fine.
>How can I easily use the cache resolver?. My problem is updating the
>cache when I get a response thru the network. In a long running daemon,
>caching DNS when I'm serving several hundreds of email for day woul dbe
>a big win.
>Maybe with my own overloaded cache class, but seems an obvious addition
>to standard twisted.names. :-? Maybe next release :-)
I think the current caching resolver was a step in the wrong direction. A cache should probably *wrap* another resolver, not whatever weird thing it is doing now.
If you write such a thing, it'd be great if you could submit it for inclusion :)
>> Hmm. The non-existence of the domain is hidden by the very last step in
>> performing the lookup. The Resolver class has a method, filterAnswers,
>> which is used to turn a DNS response into the three-tuple of lists which
>> all the lookup* methods return. You may want to subclass Resolver and
>> override filterAnswers to behave differently when the `message' argument
>> it is given has an `rCode' attribute equal to twisted.names.dns.ENAME,
>> which indicates the name requested does not exist.
>That seems doable but an ugly hack :-). Perhaps a future "twisted.named"
>release could include a flag to easily differenciate between missing RR
>and nonexistent domain. Any hope?.
I think flags like this are ugly hacks as well. I completely agree that subclass/override is not a great way to get this functionality, but I'd like to think of a cleaner way to offer the new feature while still maintaining backwards compatibility.
>>> 3. How can I stop this ".tac"?. If I do "reactor.stop()", I get an
>>> infinite error, repeated forever:
>> reactor.stop() is the correct way to end the program. If you still have
>> this problem after you have split the program into multiple files,
>> please post again.
>Program splitted. Same problem :-/
>The dode is basically the same that in my previous email. Moved 99% of
>code to "dns3.py". The dns3.tac" is trivial:
Try the service class I used above. If you still see the exception, it may indicate a bug in Twisted's UDP support. If this case, could you attach the new version of the whole program? I'll take a closer look and see if I can nail down the exact cause.
More information about the Twisted-Python