[Twisted-web] Web2 and Half-Baked Reinvention of Wheels

[L. Daniel Burr]

Hi Ed,

On Mon, 02 Jul 2007 17:21:13 -0500, Ed Suominen <general at eepatents.com>  
wrote:

> I have been debugging an odd problem that apparently is arising in
> web2.stream.FileStream. One thing I noticed while poring over the source
> is that its read() method will try to instantiate SendfileBuffer if you
> try to use its sendfile option. However, there's no reference To such a
> class anywhere in the module or anywhere else in the Twisted source
> tree, for that matter. I understand that sendfile isn't supported, but
> it's horrible coding practice to let that just sit there in the source.
>

I know you are a regular in these parts, but isn't your tone a trifle
accusatory?  I can't speak for any of the core devs, but posts like
this don't tend to motivate people to fix code; rather, I find they
tend to lead to people like myself responding about the tone of the
post ;)

> As requested by Dialtone, and to borrow liberally from what he wrote in
> an IRC chat with me today, web2.stream code is pretty badly tested and
> someone should do something about it. (I would also add that it is
> pretty badly documented, but that problem is widespread in Twisted's
> source, despite my ranting about it for years and offering bounties for
> people to contribute docstrings.)
>

OK, I'll grant that it is a shame nobody has taken you up on the
bounties, but are you really surprised that "ranting about it for
years" hasn't produced much of a result?

> Dialtone says that there are numerous other problems about streams. He
> says, and I wholeheartedly agree, that they are mostly a substitution
> for something that already exists in twisted, namely Producer/Consumer.
> (Please guys, stop chasing butterflies and direct your considerable
> talents toward improving what already works!) Dialtone says that glyph
> once wrote a replacement for the CompundStream using Producer/Consumer
> paradigm; instead of streams, we could use functions that combine
> different producers or consumers.
>

Right, I vaguely recall a sandbox module, called consprod.py, which
was supposed to be a brief sketch of how to implement HTTP in the
consumer/producer vein.  It definitely wasn't a replacement, just a
proof of concept, with no tests or docs.

What is it that makes you think that twisted devs are, as you put
it, "chasing butterflies"?  Specifically, what code should the
twisted devs be improving instead of web2?

> I can't believe that the world's greatest networking platform *still*
> doesn't have an adequately supported HTTP server!
>

This is overly dramatic, to be sure.  The supported HTTP server is
twisted.web, and it receives bugfixes when needed.  There is a
mega-ticket describing what needs to be done in order for web2
to fully replace twisted.web.  That this work hasn't been completed
has to do with developer priorities, and the fact that twisted.web,
along with nevow, is perfectly adequate for all sorts of production
websites, and has been for some time.

I know you've produced plenty of code yourself, so I'm really curious
as to why you'd adopt such a confrontational tone, when you must
realize that such tactics seldom produce the result you desire.

L. Daniel Burr