[Twisted-Python] Finding peers.
Tim Allen
twisted-python@twistedmatrix.com
Mon, 7 Jul 2003 23:38:38 +1000
--Apple-Mail-24--178975944
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
delsp=yes;
charset=US-ASCII;
format=flowed
On Sunday, Jul 6, 2003, at 23:59 Australia/Sydney, Jody Winston wrote:
> I'm also interested in using Rendezvous in twisted. I don't have much
> time at the present to both learn Rendezvous and the internals of
> twisted but I'll pass on two links if you haven't seen them. The
> first link is http://dotlocal.org/ and they are "Software developers
> interested in implementing zeroconf, multicast dns and dns service
> discovery". The second link is a Python implementation of Rendezvous,
> PyRendezvous, which can be found at..
> http://radio.weblogs.com/0105002/stories/2003/01/06/
> multicastDnsServiceDiscoverForPython.html
I just spend a day or so reading about ZeroConf and Twisted's DNS
support, and hacking together the attached code.
The quick, quick summary is this: ZeroConf has two important parts:
DNS-SD and mDNS.
DNS-SD (DNS based Service Discovery) is the easy part - it's just a
standard way of organising DNS records such that a set of DNS queries
can create a useful list of things to connect to. 'Implementing' DNS-SD
support is not at all difficult, providing there exists an API to
request PTR and SRV records, and providing you can add them to your DNS
server.
mDNS (Multicast DNS) is more complex, and seems to require
operating-system cooperation. Regular DNS queries to the magic address
224.0.0.251 on port 5353 UDP are declared to be multicast queries, and
link-local. Also, any attempt to resolve any address ending in '.local'
should be done via multicast. With multicast DNS come some different (I
think - I don't know DNS all that well) behaviours:
- When you send a request that you've already got a reply for, you
should include the
list of names you already have, so those services know not to
respond a second time.
- Single requests can have multiple replies.
- You can have a 'continuous request'.
I haven't read the mDNS spec in detail, but the interaction between
Twisted and the operating system's resolver seems complex. It'd be nice
if Python had a DNS API more extensive than gethostbyname(), but it
doesn't.
The attached files are a first draft at a DNS-SD client library for
Twisted. dns_sd.py contains the meat, and dns_st_client.py is a thin
wrapper based on doc/examples/dns-service.py dns_sd.py also contains a
quick hack to make Twisted.Names issue mDNS requests. The end result is
that on my Powerbook G4 running Mac OS X 10.2.6, I get the following
output:
$ ./dns_sd_client.py http tcp local
{'instance': u"Tim Allen's Web Site",
'servers': [('grundoon.local', 80)],
'serviceInstance': "Tim Allen's Web Site._http._tcp.local",
'settings': {'path': '/~st/'}}
{'instance': u'Grundoon',
'servers': [('grundoon.local', 80)],
'serviceInstance': 'Grundoon._http._tcp.local',
'settings': {}}
Note that my laptop's name is 'grundoon' and my login name is 'st'.
The only obvious wart in the code as it stands is that the spec says
that the 'instance' name can contain any Unicode character, while
Twisted's DNS code seems to choke on DNS name components that contain
periods. For the record, the DNS-SD spec says that embedded periods
should be written '\.', and backslashes obviously '\\'.
It occurs to me now that perhaps I should have written a patch for the
Twisted DNS code and submitted it along with this message, but it's
getting late and I wish to get some sleep before I go to work tomorrow.
:)
--Apple-Mail-24--178975944
Content-Disposition: attachment;
filename=dns_sd.py
Content-Transfer-Encoding: 7bit
Content-Type: application/octet-stream;
x-unix-mode=0644;
name="dns_sd.py"
"""
DNS-Based Service Discovery
This is a simplistic Twisted-based client for the DNS-SD system currently
detailed at: http://files.dns-sd.org/draft-cheshire-dnsext-dns-sd.txt
DNS-SD is an imporant part of ZeroConf, or what Apple calls (with typical
marketing flair) Rendezvous. Given a service (say, 'http'), a protocol (in this
case, 'tcp') and a domain (say, 'example.com') DNS-SD defines a system of
queries that should return all the appropriate servers (assuming the servers
and the DNS server all co-operate).
A quick example stolen from the above documentation:
$ nslookup -q=ptr _ftp._tcp.dns-sd.org.
_ftp._tcp.dns-sd.org name=Apple\032QuickTime\032Files.dns-sd.org
_ftp._tcp.dns-sd.org name=Microsoft\032Developer\032Files.dns-sd.org
_ftp._tcp.dns-sd.org name=Registered\032Users'\032Only.dns-sd.org
TODO:
- Seperate the 'find service instances' step from the 'get service instance
details' step, so clients can pick a service, store it in a config file, and
look up the details when they need them.
"""
import codecs
from twisted.internet import defer
from twisted.names import client
# The DNS server 224.0.0.251:5353 (UDP) is a magical value which means
# 'multicast DNS'. At least on MacOS X 10.2.6, it Just Works.
defaultResolver = client.Resolver(servers=[('224.0.0.251', 5353)])
def findServices(serviceName, protoName,
domain="local", timeout=None, resolver=defaultResolver):
"""Find all the servers in domain serving the given protocol.
@type serviceName: C{string}
@param serviceName: A service name, as seen in /etc/services. For example,
'http', 'imap4', 'ssh' and so forth.
@type protoName: C{string}
@param protoName: The name of the protocol used - 'tcp' or 'udp'.
@type domain: C{string}
@param domain: The DNS domain in which the queries will be performed. If
none is supplied, the default "local" will be used.
@type timeout: C{None} or C{list} of C{int}
@param timeout: The query is tried len(timeout) times, and each attempt is
given the appropriate number of seconds in which to complete. When the last
timeout expires, the query is considered failed.
@type resolver: A subclass of C{twisted.names.common.ResolverBase}.
@param resolver: This resolver will be used to do all the name resolving.
If this parameter isn't supplied, a default resolver will be used that does
multicast-DNS queries.
@rtype: C{Deferred} that will return a C{list} of C{dict}s. Each C{dict}
will have the following keys:
- C{serviceInstance}: The complete DNS name that should be resolved
(ask for the SRV record) whenever you want to connect to it.
- C{instance}: A Unicode string containing the human-readable name of
this service instance.
- C{servers}: A C{list} of (C{hostname}, C{port}) tuples. These should
all implement the given service. Note that this list could be
different each time you connect, so if you're going to save something
in a settings file, you should save C{serviceInstance}.
- C{settings}: A C{dict} of service-dependant settings. This is
generally the minimum extra information required to connect to
a service - for example, a http service might have a setting 'path'
that gives the path a client should request.
"""
# This is the deferred that will be fired once all the results are in.
masterD = defer.Deferred()
# Convention seems to dictate that the domain of discovered services won't
# have a trailing '.' so strip it for string-comparison purposes.
if domain[-1] == '.': domain = domain[:-1]
resultDomain = '_%s._%s.%s' % (serviceName, protoName, domain)
d = resolver.lookupPointer(resultDomain, timeout)
d.addCallback(__processPointerRecords, resolver, timeout, resultDomain,
masterD)
d.addErrback(masterD.errback)
return masterD
def __processPointerRecords( (answers, auth, add), resolver, timeout,
resultDomain, masterD):
if not len(answers):
masterD.errback("No pointer records found!")
return
dlist = []
resultSuffix = "." + resultDomain
for record in answers:
serviceInstanceName = record.payload.name.name
# We can't just split at the first '.' because the service instance
# name can contain '.'s.
if serviceInstanceName.endswith(resultSuffix):
instanceNameUTF8 = serviceInstanceName[:-1 * len(resultSuffix)]
instanceName = codecs.getdecoder('UTF8')(instanceNameUTF8)[0]
else:
# This should never happen.
instanceName = serviceInstanceName
def addExtraNames( (ans, auth, add),
sn=serviceInstanceName, ins=instanceName):
return (ans, auth, add, sn, ins)
d = resolver.lookupService(serviceInstanceName, timeout)
d.addCallback(addExtraNames)
dlist.append(d)
d = defer.DeferredList(dlist)
d.addCallback(__processServiceRecords, resolver, timeout, masterD)
d.addErrback(masterD.errback)
def __processServiceRecords(records, resolver, timeout, masterD):
dlist = []
for (answers, auth, add, serviceInstanceName, instanceName) in \
[r[1] for r in records if r[0]]:
servers = [ (a.payload.target.name, a.payload.port) for a in answers]
def addExtraNames( (ans, auth, add),
sn=serviceInstanceName, ins=instanceName, sv=servers):
return (ans, auth, add, sn, ins, sv)
d = resolver.lookupText(serviceInstanceName, timeout)
d.addCallback(addExtraNames)
dlist.append(d)
d = defer.DeferredList(dlist)
d.addCallback(__processTextRecords, masterD)
def __processTextRecords(records, masterD):
res = []
for (answers, auth, add, serviceInstanceName, instanceName, servers) in \
[r[1] for r in records if r[0]]:
textDict = {}
for textRecord in [a.payload for a in answers]:
for textItem in [i for i in textRecord.data if len(i)]:
parts = textItem.split('=', 1)
if len(parts) == 1:
key = parts[0].lower()
value = 1
else:
key = parts[0].lower()
value = parts[1]
if not textDict.has_key(key):
textDict[key] = value
resultDict = {
"serviceInstance": serviceInstanceName,
"instance": instanceName,
"servers": servers,
"settings": textDict,
}
res.append(resultDict)
masterD.callback(res)
--Apple-Mail-24--178975944
Content-Disposition: attachment;
filename=dns_sd_client.py
Content-Transfer-Encoding: 7bit
Content-Type: application/octet-stream;
x-unix-mode=0755;
name="dns_sd_client.py"
#!/usr/bin/python
import sys
import pprint
from twisted.names import client
from twisted.internet import reactor
import dns_sd
def printResults(results):
for r in results:
pprint.pprint(r)
reactor.stop()
def printFailure(failure):
print failure
reactor.stop()
try:
service, proto, domain = sys.argv[1:]
except ValueError:
sys.stderr.write('%s: usage:\n' % sys.argv[0] +
' %s SERVICE PROTO DOMAIN\n' % sys.argv[0])
sys.exit(1)
d = dns_sd.findServices(service, proto, domain)
d.addCallbacks(printResults, printFailure)
reactor.run()
--Apple-Mail-24--178975944--