[Twisted-Python] Some problems with twisted

Pieter Claerhout Pieter.Claerhout at Creo.com
Tue Dec 2 03:05:21 EST 2003


Hello all,

I'm having some problems with getting a certain app running under Twisted.
I've attached
the code I have so far. There are a few things that are not working the way
they should
(because I probably did something wrong). Twisted is still pretty new for
me.

1. If I run this script from the command prompt, e.g.:

   C:\>python sample.py
   
   It does start and performs the processing as defined in the
ProcessingService, but it
   doesn't seem to start the WebAdminService. When I try to connect to the
web interface
   by surfing to http://localhost:9119/, it fails to connect.

2. During the execution of the doProcessing function, it's impossible to
stop the
   application with e.g. CTRL+BREAK. Is there a way around this?

3. The code from the doProcessing service is basically blocking the whole
application 
   when it's running. The goal of this application is that it does two
things:
   
   1. It grabs files from a list of servers every hour and performs some
calculations on
      these files. The results will be stored in a database (SQLite or
MySQL).

   2. The application will provide a web interface to administer the
application and
      display the results by getting these from the database.

   Since downloading from the internet is not always as reliable as we would
like it to
   be, it's possible that if a number of the servers are not online, the
systems slows
   down considerably. It would be a waste of time anyway to let the
application wait
   for this single server when it can continue with the other ones in the
meantime.

   What I would like to find is a way to get the data from these servers as
fast as
   possible and do the calculations on them without blocking the system.

   What is the normal way of doing things like this in Twisted. Should I use
threads to
   do this, or is there another way?

Thanks a lot in advance,


pieter

[ pieter claerhout | pieter at yellowduck.be | www.yellowduck.be ]


#---- START OF CODE ----
import time
from twisted.internet import app
from twisted.internet import reactor
from twisted.web import static
from twisted.web import script
from twisted.web import server

class ProcessingService( app.ApplicationService ):

    def __init__( self, serviceName, serviceParent=None ):
        app.ApplicationService.__init__(
            self, serviceName, serviceParent=serviceParent
        )

    def startService( self ):
        reactor.callLater( 0, self.doProcessing )

    def doProcessing( self ):
        for url in [ 'http://www.yellowduck.be/', 'http://www.beachshop.be/'
]:
            print 'Processing %s' %( url )
            time.sleep( 5 )
        reactor.callLater( 360, self.doProcessing )

class WebAdminService( app.ApplicationService ):

    def __init__( self, serviceName, serviceParent=None ):
        app.ApplicationService.__init__(
            self, serviceName, serviceParent=serviceParent
        )

    def startService( self ):
        root = static.File( 'htdocs' )
        site = server.Site( root )
        self.serviceParent.listenTCP( 9119, site )

def main():
    myapp = app.Application( 'myapp' )
    ProcessingService( 'processor', myapp )
    WebAdminService( 'webadmin', myapp )
    myapp.run()

if __name__ == '__main__':
    main()
#---- END OF CODE ----




More information about the Twisted-Python mailing list