[Twisted-Python] Sending large amounts of data over Pb

Gabe Rudy rudy at goldenhelix.com
Tue Apr 25 18:38:19 EDT 2006

> I'm not sure how you are using RemoteCache, but the general technique here
> is to use a pager.  Take a look at twisted.spread.util.Pager, and the two
> examples/tools that come with it, StringPager and FilePager.
Very nice, I think that will be very usefull. It may put some holes in my 
abstraction with the current way I use RemoteCache, but I could imagine some 
design changes that would work.

Here is the idea I might try (if you care to comment)

Background: RemoteCachedTable represents some table view of a server-side 
database, thus on the client side it might go something like this (very 
simplified and NOT TESTED):

from twisted.spread import pb, util
class TableDataCollector(util.CallbackPageCollector):
    def __init__(self):
        self.table = None

    def remote_gotPage(self, page):

    def remote_endedPaging(self):

class RemoteCachedTable(pb.RemoteCache):
   def setCopyableState(self,state):
        self.headers = state['headers']
        self.dataIsSmall = state['dataIsSmall']
        if self.dataIsSmall: #Ship the table data if its small, <100 rows
            self.data = state['data']
        else: #We receive the data through a Pager/Collector
            self.data = []
            self.collector = state['collector'] #A TableDataCollector instance
            self.collector.table = self

   def updateUi(self, finished):
        pass #Notify UI of changes to self.data

   #various observe_* methods go here

Obviously the server side stuff is not shown, but this depicts my intent :)

I don't think I have that right though. The "Server" here can not create the 
TableDataCollector instance and pass it back with the state dictionary can 
it? Somehow my "Client" needs to create the instance and send it to the 
server, at which point the server creates the StringPager (or TablePager), 


More information about the Twisted-Python mailing list