[Twisted-Python] Some beginner questions about "twisted.names.client" and ".tac" environment

Jean-Paul Calderone exarkun at divmod.com
Sat Dec 17 19:55:04 EST 2005


On Sun, 18 Dec 2005 01:05:32 +0100, Jesus Cea <jcea at argo.es> wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>Jean-Paul Calderone wrote:
>> You probably want to move most of your program out if "dns.tac" and into
>> an importable Python module.  Code defined inside .tac files lives in a
>> weird world where some surprising rules apply.
>
>Any documentation about that?
>

Hmmm, not that I know of.  The problems are all essentially consequences of the fact that .tac files are run using the built-in execfile() function (or equivalent), rather than being loaded as modules.

>> It's best to keep the
>> .tac file as short as possible.  Generally, you just want to create an
>> Application and give it some children, importing from modules the
>> definitions of all classes and functions needed to set this up.
>
>I reduced the "dns3.tac" to:
>
>=====
>
>from twisted.application import service
>
>application = service.Application("DNS test")
>
>import sys
>sys.path=["."]+sys.path
>import dns3
>
>=====
>
>All logic is in "dns3.tac". Basically the same code posted in my
>previous message.
>
>I have the very same problem with "reactor.stop()": repeated exceptions,
>only stoppable using "kill -9".
>
>I don't get the "give it some children" point. Could you post some
>sample code?.

You want to tie your application logic to a Service subclass.  To start with, I'd try something like this, in dns3.py:

  from twisted.application import service

  class DomainResolver(service.Service):
      def startService(self):
          # Copied the top-level code from the original dns.tac
          pendientes = []
          redirigidos = []
          f = open("domain_list")
          for i in f :
              pendientes.append(i)
          aun_pendientes = len(pendientes)
          concurrencia = 1000
          for i in pendientes[:concurrencia]:
              resolucion(i)
          pendientes=pendientes[concurrencia:]

Then, in dns3.tac after application is defined,

    from dns import DomainResolver
    DomainResolver().setServiceParent(application)

This will delay the execution of your startup code until twistd is totally ready and the reactor is fully initialized.

>
>> Passing (60,) as the timeout might not be the best idea.  This will
>> cause the DNS client to send one request and then wait 60 seconds for a
>> response.  If either the request or the response is dropped (as often
>> happens with UDP traffic), you will never get a result, and you will
>> have to wait 60 seconds to discover this fact.
>
>I'm resolving several thousand of domains, so one minute more or less is
>not an issue. I don't want to retransmit since I'm using 127.0.0.1, so
>losing request (in the same machine) should be rare (udp backlog
>overflow, basically).
>
>Some domains takes a long time to resolve. So if I use an small value I
>load the server and get "Unexpected message (XXXXX) received from
>('127.0.0.1', 53)", caused because the DNS server gets a late answers
>and my code already give up.
>

Okay, it sounds like you know what you're doing here :)

> [snip]
>
>> To customize the server used by the resolver, you may want to create
>> your own resolver instance, rather than relying on the defaults guessed
>> by the resolver automatically created in the client module.
>
>Also a good point. Done and working fine.
>
>How can I easily use the cache resolver?. My problem is updating the
>cache when I get a response thru the network. In a long running daemon,
>caching DNS when I'm serving several hundreds of email for day woul dbe
>a big win.
>
>Maybe with my own overloaded cache class, but seems an obvious addition
>to standard twisted.names. :-? Maybe next release :-)

I think the current caching resolver was a step in the wrong direction.  A cache should probably *wrap* another resolver, not whatever weird thing it is doing now.

If you write such a thing, it'd be great if you could submit it for inclusion :)

>
>> Hmm.  The non-existence of the domain is hidden by the very last step in
>> performing the lookup.  The Resolver class has a method, filterAnswers,
>> which is used to turn a DNS response into the three-tuple of lists which
>> all the lookup* methods return.  You may want to subclass Resolver and
>> override filterAnswers to behave differently when the `message' argument
>> it is given has an `rCode' attribute equal to twisted.names.dns.ENAME,
>> which indicates the name requested does not exist.
>
>That seems doable but an ugly hack :-). Perhaps a future "twisted.named"
>release could include a flag to easily differenciate between missing RR
>and nonexistent domain. Any hope?.

I think flags like this are ugly hacks as well.  I completely agree that subclass/override is not a great way to get this functionality, but I'd like to think of a cleaner way to offer the new feature while still maintaining backwards compatibility.

>
>>> 3. How can I stop this ".tac"?. If I do "reactor.stop()", I get an
>>> infinite error, repeated forever:
>>
>> reactor.stop() is the correct way to end the program.  If you still have
>> this problem after you have split the program into multiple files,
>> please post again.
>
>Program splitted. Same problem :-/
>
>The dode is basically the same that in my previous email. Moved 99% of
>code to "dns3.py". The dns3.tac" is trivial:

Try the service class I used above.  If you still see the exception, it may indicate a bug in Twisted's UDP support.  If this case, could you attach the new version of the whole program?  I'll take a closer look and see if I can nail down the exact cause.

Jean-Paul




More information about the Twisted-Python mailing list