[Twisted-Python] making twisted provide good feedback (was Re: Having reactor run at the main thread...)

glyph at divmod.com glyph at divmod.com
Thu Nov 6 15:22:53 EST 2008

(Moving this off the twisted-web list because it is of more general 
interest and contains nothing web-specific.)

On 09:48 am, jml at mumak.net wrote:
>On the other hand, there's a related invalid argument that gets used a
>lot by library and framework authors: "if we provide X, people might
>misuse it, so we should not provide X". This argument is also bogus
>(that way lies Java). Provide safe, well-documented alternatives for
>the abusers and let those who want to shoot themselves in the foot do
>so: sometimes they might actually know better than you.

I find it peculiar that you think this is Java's disease.  My 
perspective is that Java is unpleasant because it is remarkably tedious, 
redundant, and inconsistent, not because it lacks functionality.  If 
anything, it's a mess of functionality, and there are too many ways to 
do the same thing.

If you want to shoot yourself in the foot (or face, as the case may be) 
there's always the com.sun hierarchy, which despite being full of 
warnings signs and sparsely documented, is all public (at the language 
level) and accessible from Java programs which wish to go that way.  In 
that sense it's relatively hacker friendly.  For more fun with foot- 
shooting there's also AccessibleObject, which lets you call private 
methods and access private fields.

I think the whole issue here is very complex, and so I'd like to repeat 
and perhaps more clearly explain my own views.

I work at the bleeding edge of lots of software and am frequently 
frustrated when the authors of libraries that I use don't bother with 
"edge" cases that are necessary to the applications that I write. 
Twisted should provide excellent support for valid use-cases, no matter 
how obscure.

On the other hand, it's equally frustrating to use a library where the 
incredibly obscure "you probably don't need this" and the 99%-of-the- 
time positive-path APIs are listed, without any particular ordering, in 
one giant flat list.  It's particularly frustrating to google for "How 
do I (X)" and get a solution which uses an apparently reasonable API to 
do (X), only to discover months after using it that it's actually 
unsupported and you're not supposed to touch it and now your program is 
going to break horribly with the next minor version of the library.

This isn't just a problem with documentation, because in reality nobody 
reads documentation.  If you're writing a program, you're going to read 
_just_ enough of the API doc and do just enough groveling around with 
dir() and pydoc to get yourself to the point where something works. 
When it works, you will probably consider that sufficient proof that you 
used the APIs correctly, unless you had to type something that _looked_ 
particularly gross in the course of doing it, or you got a bunch of 
warnings out of your program when you ran it.  This is the whole point 
of DeprecationWarning: somehow, your library code itself has to 
communicate with the the developer who is invoking it and tell them 
useful things about their use or misuse.

I feel like Twisted has more of the latter problem than the former. 
There are a lot of methods which aren't really private 
(addReader/removeReader) that don't differentiate themselves as more or 
less private or refer you to the more portable and general APIs that you 
probably want (pauseProducing/resumeProducing).  However, (modulo global 
reactors, of course) Twisted does support a lot of edge cases very 

Speaking from personal experience, there are a lot of people who show up 
in #twisted looking at 
when they should be looking at 
when they should be looking at 
In the long term, we need to write both better documentation and more 
code that gets these people back onto the right track.

I know I've used a few libraries which provided exactly this kind of 
feedback and it was very satisfying and educational.  It's really an 
art, one that's difficult to master, and I don't know that I really know 
the right way to do it in the general case.  The danger that comes along 
with doing it wrong is very nicely explained here: 
http://blogs.msdn.com/oldnewthing/archive/2008/10/06/8969399.aspx - if 
you make an API which doesn't work except for the "special" cases, then 
everyone just starts using the "special" version.

More information about the Twisted-Python mailing list