[Twisted-Python] Naive questions about how to do it twisted way.

Jia Liu ayuer.python at gmail.com
Wed Jun 28 05:07:14 EDT 2006

  I have been follow twisted for like half a year now, I have read the doc
and the oreilly book, but i still cann't do some simple work in the twisted
way. Could someone give me a hint?
  Like today, i write this little code to follow my friends list on my msn
space, and get their friends list, so on and so on, and maybe at last I can
build a social relationship graph with it. The code is actually really
navie, here it goes. Sorry for the ugliness of my code, please help me make
it prettier:)
  Thanks. Appologize for my poor English.
Jia Liu

import re, sys, socket
import urllib
from sets import Set

socket.setdefaulttimeout = 10
re_buddy_pre = re.compile("href=\"(?:http://)?([^\.\"]+)\.spaces.msn.com",
re.IGNORECASE)  # http://nick.spaces.msn.com
re_buddy_post = re.compile("spaces.msn.com/members/([^\"\/]+)",
re.IGNORECASE) # http://spaces.msn.com/members/nick/

all_buddies = Set()
relation_dict = {}

def get_buddy(page, buddy=None):
    #print page
    print buddy
    global all_buddies
    buddies_of_this_page = Set()
    for re_buddy in [re_buddy_pre, re_buddy_post]:
        for m in re_buddy.finditer(page):
    relation_dict[buddy] = buddies_of_this_page
    return buddies_of_this_page

def got(url, buddy=None):
    f = urllib.urlopen(url)
    return f.read(), buddy

URL = "http://ayueer.spaces.msn.com/"
(page, buddy) = got(URL, "ayueer")

buddies_of_this_page = get_buddy(page, buddy)
next_to_visit = buddies_of_this_page.difference(all_buddies)

    while 1:
        for buddy in next_to_visit:
            url = "http://"+buddy+".spaces.msn.com/"
                (page, buddy) = got(url, buddy)
                buddies_of_this_page.update(get_buddy(page, buddy))
        next_to_visit = buddies_of_this_page.difference(all_buddies)
except KeyboardInterrupt:
    print all_buddies

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20060628/0c2b379b/attachment.htm 

More information about the Twisted-Python mailing list