[Twisted-Python] breaking distrib.ResourceSubscription

Kevin Turner acapnotic at twistedmatrix.com
Tue Sep 24 18:11:41 EDT 2002

On Tue, 2002-09-24 at 06:57, Glyph Lefkowitz wrote:
> ResourcePublisher instance has no attribute 'remote_wasModifiedSince'

I'm surprised that this is in the logs much, since it should only happen
for distributed servers that set a Last-Modified header.  Are personal
servers serving that many static files?  I would have thought that would
be handled by ~/public_html.

> I thought that I'd mentioned this over IRC before, but let me stress it: I
> think that twisted.web.server is implementing *WAAAAY* too much of HTTP right
> now. 

Interesting position.  I know you're not a fan of complete support for a
protocol when you believe the protocol itself is broken, and I'll admit
that nobody actually uses ETags, but I would hardly say that
Last-Modified and HEAD are obscure or unnecessary.

> First of all, this is killing performance -- you have a minimum of 4 or 5
> PB server/client interactions now for each conditional request instead of 2.
> Those method calls are not cheap.

Recognized.  Although I thought I only increased the count by one, not
two or three?

Also take into consideration that these calls only happen for the
*conditional* requests.  Resources that don't supply ETag or
Last-Modified cache validation headers will probably never be bothered
with them.  And if the condition fails, callRemote("request") never
happens, nor does all the work that goes with it.  So overall,
estimating its impact on performance isn't as simple as grep --count

> Second, this is exploding the complexity of
> the webserver: its goal should be to get to the endpoint resource *as simply as
> possible* and then invoke endpoint behavior on that resource.  If clever
> header-parsing things need to happen, then we need to make a lower-level
> 'request' method and make *it* clever, not do request processing before the
> request is dispatched.

This I agree with.  I knew something was wrong when I had three remote
methods in ResourcePublisher all independently doing getResourceFor.  It
would be more symmetric if Request.process was invoked on the distri --

ggrngh.  What is Request.process anyway.  Is it just supposed to do
site.getResourceFor and pass itself off?  Yes.  All the rest of the junk
in there is stuff I added (though the HEAD/supportedMethods stuff was
quite some time ago) with the possible exception of the
util.Unauthorized block.  And it's probably all logic that belongs in
web..Resource rather than Request.

(which is just re-iterating what you've been telling me.)

So how do we fix it?  You're suggesting that Request.process call
Resource.request, which contains most of the stuff currently in
Request.process, and Resource.request calls Resource.render if it's not
a conditional request?

> Third, and not least, this has broken backwards compatibility (albeit in a
> mostly-harmless way) at a time when we should be focusing the most on backwards
> compatibility. 

Sounds like we need an inter-version test harness.  Runs peer1 under
version X and peer2 on Y, or starts server on version X, pickles it, and
re-starts under Y, etc.

> I'm sending this to you directly instead of the list because I've had enough
> political headaches recently and I didn't want you to think that I'm publicly
> lambasting you :).  I appreciate the effort that went into this feature
> implementation, even if it was misguided.

God help us if we can't critique the codebase in the developer's forum.
I appreciate the feedback; I *told* you that I didn't think my changes
had been tested "a lot".

> If you think some of this
> information is useful or that my views on twisted.web's architecture are not
> very clear (or require public discussion), feel free to edit and re-post.


 - Acap

The moon is waning gibbous, 91.9% illuminated, 17.5 days old.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://twistedmatrix.com/pipermail/twisted-python/attachments/20020924/2ed35414/attachment.pgp 

More information about the Twisted-Python mailing list