[Twisted-Python] documentation / kqueue / feedback
glyph at divmod.com
glyph at divmod.com
Wed Apr 16 11:49:49 EDT 2008
On 01:15 pm, dr.pythoniac at googlemail.com wrote:
>Seeing there is doc. available, I digged through it. After all TM seems
>worth the effort. From what I see, I love TM.
Great! The rest definitely doesn't sound like you love it though ...
>Links are often broken, though. Often seemingly useful doc. is old,
>old. Even current doc. isn't up to date.
Since the 8.0 release, we have had quite a few problems with links,
specifically the links from the core documentation to the API
documentation on the website. We are working to fix this. However, if
you spot broken links, please feel free to report them in the bug
Saying that there are "often" broken links is just about as pointless as
saying that you "often" find bugs. Without specifics, this is not
useful information; the reason we have not fixed specific problems is
not because we believe there are no problems anywhere.
>They talk about "resource trees", yet I don't find them in the doc. I
>putchild() but all examples and docs indicate, that you can do it only
>within the root resource.
You can do putChild to any resource, as long as you understand its
lifecycle. A "resource tree" is simply a tree (
http://en.wikipedia.org/wiki/Tree_structure ) of resources (
Our documentation assumes, and will continue to assume, a certain level
of familiarity with basic computer-sciency jargon; if you feel that a
particular phrase is too obscure, feel free to submit a patch
elaborating on it. However, the audience of the bulk of the Twisted
documentation is fluent Python programmers. If you want to write
something for folks who are new to Python or programming in general, a
separate set of documentation would be better than trying to re-explain
everything in-line. For example, explaining that we are referring to a
directed graph of IResource providers would be fine, but adding an
explanation of graphs or how python references or dictionaries work
>OK, a little later, I
>have my first web server running an my first rpy works, spitting out
Sounds like the docs are doing their job, then :).
>I'm wondering, why of all things "rpy" ? python won't generate sth.
>"rpyc" for that, which translates in lost speed, I assume. No
"rpy" to mark the file as a specific kind of python file that needs to
be run within the web server to produce a resource. Implemented in this
manner, python would not generate a "pyc" file even if it were called a
".py"; it is not a module that you import.
As far as "lost speed" is concerned, it's not entirely clear that that's
true, because the times when Python would have to check for the .rpyc
are entirely different than the way that it does when you are importing
modules. Twisted could emit .rpycs as an optimization if you did a
benchmark, discovered it did indeed matter, and submitted a patch. This
is, however, hardly the highest priority for twisted.web developers
right now :).
>I'm on FreeBSD. Of course, I want to use kqueue. Now, another "funny"
>journey begins ...
>Intense googleing, and I mean "intense". Reading through years of this
>mailing list. No positive result.
Fair enough. The kqueue reactor doesn't work and there's no
documentation saying that it doesn't, except a failing buildbot:
http://buildbot.twistedmatrix.com/builders/freebsd-py2.4-kq - also,
Twisted's support for platforms is not clearly explained.
>I have the clear impression that twisted is something in between a toy
>brilliant product. It's hackers, however, leave much to be desired in
>of usefulness. As it is, it's a great and promising hobby but not a
Many, many people (including me!) disagree.
I understand that you're frustrated, but this does nothing but serve to
weaken your position. You have some valid criticism, but when I read
"twisted is not useful", I think that you are simply mistaken.
>Sorry, but doc strings don't replcae a halfway decent documentation and
>reasonable tutorial, considering the highly complex matter.
First google hit for "twisted tutorial":
Not perfect, perhaps, but I think it could definitely be described as a
"reasonable tutorial" to core Twisted concepts.
>Sorry, but documentation that is often enough outdated and sometimes
>broken links is next to useless.
If you were truly sorry, you would be contributing patches which fix the
broken links and updates the documentation to be more recent.
>Kqueue seems to be vital to an event driven approach like TM, yet there
>multiple versions of pykqueue floating
>around, none of them properly working (and I didn't fumble around. I
>plain simply used the FreeBSD port), some of
>them with some mor and some with some less "annotations" (I refuse to
>call that doc.)
Kqueue is most definitely _not_ "vital" to something like Twisted. It
is an optimization of a very specific part of a Twisted application on a
very specific platform. *Most* realistic Twisted applications will be
bottlenecked on application code long before something like kqueue (or
epoll) will help. Working from the amount of support from the
community, most Twisted applications also don't run on FreeBSD.
I apologize if this response seems too harsh, but I am annoyed by people
seeing a pet feature (variously: good ideas like HTTP/1.1, kqueue, and
Cocoa GUI support, bad ideas like "block on a Deferred", and core API
thread safety) which isn't too important in Twisted and then claiming
it's "vital" to the project's success. The project is succeeding, so
clearly it is not vital: QED. It may be important to you: that's great.
Do some work to support it. Once it's supported, buildbot will listen
to its tests and it won't break again.
For those projects which really do benefit from high multiplexing volume
with something like kqueue, Twisted offers an extremely abstract API
where the multiplexing mechanism is independent of the application code.
You can, at any moment, replace the reactor and your application will
keep working. So even if kqueue support does not exist at all, you can
add it with Twisted much more easily than if you wrote your own select()
You might say, in fact, that Twisted is vital to a platform-specific
tool like Kqueue, because otherwise almost nothing will use it.
>- How about getting 1 version of pykqueue properly running and into TM
Yes, how about that? Maybe you could contribute to one of the numerous
kqueue tickets already in the tracker.
>- How about freezing some TM version (like 8.0) and updating/matching
>(Of course, experiments are funny and intriguing, but quite some of us
>here need sth. stable for everydays work)
Twisted is very, very stable. The next version (8.1) will break only
code that is using APIs which have been outdated for literally _years_.
There is no need to "freeze" anything, just update the documentation to
what is most recent and do a new release including that documentation.
>- How about writing some complete docs and tutorials? Sth. along the
>of "My first web server wit TM" (instead all those - sometimes seeming
>contradict each other - snippets)
Yes, how about that? You can write documentation on your own site, and
we will link to it. Or, you could contribute to the core documentation.
The bottom line: complaining like this is not very useful. If you have
time to help, then contributing patches and responding to reviews in the
tracker is helpful. If you don't have the time to actually do any work,
then describing specific problems (in as much detail as possible!) is
useful. If you don't have time even for _that_, then cash donations to
the foundation are useful.
If you are interested in getting stuff in Twisted fixed, though, writing
rambling complaints serves to do nothing but reduce our developers'
already scarce motivation, make us think our work is not appreciated,
and encourage us to spend time writing rambling responses like this one
rather than fix the problems you're talking about.
(Which is not to say this has been a _complete_ waste of time. If one
out of every fifty people I write a message like this to understands
what I'm saying and becomes a long-time contributor, then it's probably
More information about the Twisted-Python