[Twisted-Python] breaking distrib.ResourceSubscription
glyph at twistedmatrix.com
Tue Sep 24 20:28:33 EDT 2002
On 24 Sep 2002 15:11:41 -0700, Kevin Turner <acapnotic at twistedmatrix.com> wrote:
> On Tue, 2002-09-24 at 06:57, Glyph Lefkowitz wrote:
> > ResourcePublisher instance has no attribute 'remote_wasModifiedSince'
> I'm surprised that this is in the logs much, since it should only happen
> for distributed servers that set a Last-Modified header. Are personal
> servers serving that many static files? I would have thought that would
> be handled by ~/public_html.
The tracebacks I was seeing were for wikis and viewcvs; either moin is setting
those headers or it's serving the images / static data from a personal server
> > I thought that I'd mentioned this over IRC before, but let me stress it: I
> > think that twisted.web.server is implementing *WAAAAY* too much of HTTP right
> > now.
> Interesting position. I know you're not a fan of complete support for a
> protocol when you believe the protocol itself is broken, and I'll admit
> that nobody actually uses ETags, but I would hardly say that
> Last-Modified and HEAD are obscure or unnecessary.
Nor would I. After much discussion with itamar I conceded that HEAD ought to
cause the body of the request to be eaten there before being sent to the
What I mean here is that the actual module, twisted.web.server, is being too
smart about stuff like Last-Modified and ETags. HTTP should be fully
implemented, but parts of the responsibility lie elsewhere.
twisted.web.Resource subclasses can implement those themselves if they want to,
because the 99.9% cases should be handled: static.File will handle filesystem
content and woven.* will handle dynamic content. We can also provide a helper
for mix-in that does conditional-rendering stuff.
> > First of all, this is killing performance -- you have a minimum of 4 or 5
> > PB server/client interactions now for each conditional request instead of 2.
> > Those method calls are not cheap.
> Recognized. Although I thought I only increased the count by one, not
> two or three?
Looking at it now, I guess ETags and last-modified support are exclusive. (Is
this correct, protocol-wise?) However, each call that requires an answer is
minimum of 2 interactions (request/response). Plus most resources will want to
do a .write().
> Also take into consideration that these calls only happen for the
> *conditional* requests.
The only heuristic I have for how often these happen are the once-per-minute
tracebacks I was seeing while bots were crawling the website yesterday. So it
looked quite often. :)
> > Second, this is exploding the complexity of the webserver: its goal should
> > be to get to the endpoint resource *as simply as possible* and then invoke
> > endpoint behavior on that resource. If clever header-parsing things need
> > to happen, then we need to make a lower-level 'request' method and make
> > *it* clever, not do request processing before the request is dispatched.
> This I agree with. I knew something was wrong when I had three remote
> methods in ResourcePublisher all independently doing getResourceFor. It
> would be more symmetric if Request.process was invoked on the distri --
> ggrngh. What is Request.process anyway. Is it just supposed to do
> site.getResourceFor and pass itself off? Yes. All the rest of the junk
> in there is stuff I added (though the HEAD/supportedMethods stuff was
> quite some time ago) with the possible exception of the
> util.Unauthorized block. And it's probably all logic that belongs in
> web..Resource rather than Request.
HEAD definitely ought to still have a little bit of logic in Request, since it
should discard any body that the resource attempts to return. However, the
rest of what you're saying is definitely...
> ([...] just re-iterating what you've been telling me.)
> So how do we fix it? You're suggesting that Request.process call
> Resource.request, which contains most of the stuff currently in
> Request.process, and Resource.request calls Resource.render if it's not
> a conditional request?
See above for my suggestion. I think that Resource is too low-level for this
sort of thing (since the default implementation of wasModifiedSince is going to
be 'return 1', why invoke it at all by default?). I don't feel strongly that
my suggestion is better than this one though, if you think that dynamic
resource implementors are going to want to use this functionality a lot.
> Sounds like we need an inter-version test harness. Runs peer1 under
> version X and peer2 on Y, or starts server on version X, pickles it, and
> re-starts under Y, etc.
Yes. This is something that I think radix hacked up manually a little while
back, but it needs to be added to accepttests.
Sounds like we're in general agreement, then...
| <`'> | Glyph Lefkowitz: Traveling Sorcerer |
| < _/ > | Lead Developer, the Twisted project |
| < ___/ > | http://www.twistedmatrix.com |
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 189 bytes
Desc: not available
Url : http://twistedmatrix.com/pipermail/twisted-python/attachments/20020924/879e7dc8/attachment.pgp
More information about the Twisted-Python