[Twisted-Python] Couched 0.1: CouchDB twisted client
glyph at divmod.com
glyph at divmod.com
Fri Sep 5 19:12:13 MDT 2008
On 5 Sep, 11:25 pm, thostrup at gmail.com wrote:
>2008/9/5 Jean-Paul Calderone <exarkun at divmod.com>
>>On Fri, 5 Sep 2008 09:48:08 +0200, Henrik Thostrup Jensen <
>>thostrup at gmail.com> wrote:
>>>[snip]
>>>Twisted web is currently not an option (I need request pipelining and
>>>http
>>>1.1).
Please feel free to use twisted.web2 if it is meeting your needs
currently. However, our goal remains to focus maintenance effort on
twisted.web and provide a gradual, compatible evolution to a better
webserver. The HTTP 1.1 features that you need will eventually be added
to twisted.web, and I hope that the split with paisley can be resolved
at that point.
I personally don't use couchdb or have any interest in its bindings, but
it sounds like you have some other misconceptions about twisted and its
development process which I'd like to correct.
>>Which other HTTP 1.1 features do you need?
>Furthermore I need the server to support large uploads (last i checked
>the
>640K limitation was still there) and downloads, and with reasonable
>efficiancy.
640k limitation? In twisted.web? As far as I know there's no such
thing; the only 640k limitation I know of anywhere in Twisted is in PB,
and that's just on individual message size. A brief skim through the
twisted.web client and server code doesn't reveal any obvious size
limitations that I can see. (Is there some 640k limitation in the 'cgi'
module?)
If there is such a limitation it should be more clearly documented
somewhere.
True, twisted.web doesn't provide a graceful way for resources to handle
streaming file uploads as they arrive, but for dedicated servers it is
certainly *possible*; Request.handleContentChunk says it's "not for
users", but you could easily write a few simple patches which would make
it "for users", or just subclass it and ignore the docstring. Nothing
in twisted.web is really that hard to fix.
>Ticket #1956 fixes some of these things. I'm not quite sure how it
>handles
>errors (which can happen both ways), but I've just had a quick look.
>It's
>been a whlie since i've had a look at all this. I write a bit about it,
>which I've placed here: http://www.cs.aau.dk/~htj/prodcons.txt
Ticket #1956 is still kind of stuck defining the problem.
If you have comments or thoughts on #1956, please feel free to add them
as comments. Text files lying around the web are not as helpful as
discussions on tickets.
Your text file suffers from the same problems as the existing discussion
on the ticket, though: a lack of concrete use-cases and example code.
There are lots of proposed solutions but not a lot of proposed problems.
(I am certainly a guilty party here, I really should describe the
problem in more detail.) We all feel there is some inelegance, but
there is no example of some code that we can all agree is gross and
would be fixed by some better API.
Please come up with an example - the shorter the better - of a
deficiency in the streaming API and put it on the ticket. Hopefully
that will further the discussion there.
>IMHO, getting (and deciding on) a proper producer/consumer interface
>need to
>happen, to make whatever web things twisted will have usefull. Changing
>the
>producer/consumer might be massive API break, however it is a needed
>one.
No.
1. We will never break compatibility with an API this old and this
stable. When we do start to move over to a new API, the transition will
be particularly gradual and slow given the core nature of this API and
the fact that pretty much everything uses it. Even if it were only
around for one release, see
http://twistedmatrix.com/trac/wiki/CompatibilityPolicy
2. It's not clear what the new API would be. Every suggestion proposed
thus far has some issues, and there's no clear winner. Unless someone
takes the time to really thoroughly explain the problem and describe
what effect the solution would have, with examples, then we're not going
to make any progress.
3. There isn't anything that you *can't do* with
ITransport/IProducer/IConsumer/IProtocol. It can be slightly awkward
sometimes, but I have yet to see an example where you really can't make
something work. For example, IConsumer doesn't have .finish(), but
that's fine, because you probably mean IProtocol.connectionLost(...)
anyway. Some users of these interfaces don't provide nice ways to get
data in and out, and yes, a uniform and consistent way to deal with
slinging data between endpoints (for example, a standard, featureful
object to remix a 'protocol' into a 'transport' and vice versa) would
probably help with that. Given that the deficiency creates momentary
inconveniences rather than impossibilities, there's no way we're going
to break *every single twisted program ever* just to make it slightly
more convenient.
4. Given that the semantics are already there, it will be very easy to
evolve new code to call old interfaces, provide adapters which wrap
around things and generally make it easy to have some hypothetical shiny
new interface cooperate just fine with the crufty old ones.
More information about the Twisted-Python
mailing list