[Twisted-Python] Freevo 2.0, Kamaelia, pyevent, Eventnet/LGT: what's going on?

[ph!l chr!stensen]

well, for some reason I'd like to add my 2/100ths of a dollar to this  
discussion...

On Oct 20, 2005, at 7:10 PM, Antoine Pitrou wrote:
> Oh, I have too. But only using the basic networking stuff, because I'm
> not confident enough to touch the high-level things.
>
> Parts of the Twisted API are marked as "API unstable". I don't think I
> dreamed it, since I've read it in the doc ;)
> Also I've seen several times some answers on this very list which were
> of the sort "actually this module is not maintained anymore" (or
> "experimental").
>

i can't see this as anything but a good thing. open source software  
is as much of a learning opportunity as a way to get good software  
without paying for it. since twisted is under continuous development  
(and always will be, i hope), there's always going to be material  
like this as part of the codebase.

i'd much rather have one more option to consider when implementing a  
system, even if it's not stable. as far as getting information about  
deprecated features after-the-fact, that's one of the (comparatively)  
small costs of using otherwise very robust open source software.

> There is a whole lot of methods which are not documented at all. Also,
> there is usually no comprehensive documentation for a specific
> functionality. Not to mention the annoying interface system which  
> means
> the documentation for e.g. twisted.internet.reactor.ListenTCP is
> actually to be found elsewhere in the class hierarchy...

this is another cost of having an excellent platform freely available  
to you, developed by people in what is usually spare time. the battle  
of document vs. implement has been raging in every serious  
programmer's mind as long as the craft has been around, and it's not  
going to stop any time soon.

then again, let's think of how often the first response to a new  
language or platform is 'well, let me see some sample code.' there's  
a reason for this, and it's that no amount of documentation is going  
to give you the necessary grasp on a complex system.

obviously this is an area where twisted needs improvement, but it's  
not alone in the open source world in this respect.

> It is proprietary in the sense that it is, AFAIK, implemented only by
> Twisted.
> Contrast this with e.g. XMLRPC which has lots of implementations.  
> XMLRPC
> is quite mediocre, but at least I know that mostly everyone can  
> find an
> implementation for his/her favourite programming language.
> I won't ever use PB for something for which I want others to be  
> able to
> write clients easily (without forcing them to use Twisted). It's a
> simple matter of simplifying interoperability. For some people/ 
> projects
> it matters very much, for others it doesn't matter at all.

well, not to mince words, but PB isn't proprietary in any sense,  
simply by definition. as far as xml-rpc is concerned, if you find it  
an acceptable replacement for PB, you probably didn't need to use PB  
in the first place.

personally, PB has saved me countless hours of coding when  
implementing complex client/server apps which must maintain state  
while providing two-way communication for (in my case) extensible  
protocols. as to whether i'm concerned about 'forcing' someone to use  
twisted to write their own client, it goes without saying that  
there's a myriad of other examples where you're required to implement  
clients or other modules in a particular language. at least PB gives  
you 2 choices, with more to come.

> There is a terminology problem, since I don't understand why something
> named "Protocol" should be destroyed at every connection. Or it should
> be, quite logically, renamed "Connection". Also this doesn't appear in
> the API docs (again).
>
> For me, a "protocol" is something like HTTP or SMTP. It is "eternal",
> I'm not expecting it to be created/destroyed at every connection (that
> would be like calling an int a float, or whatever).
>
> There is no documentation on the magic that really happens when
> something is triggered (for example when a TCP session is  
> established).
> Which methods are called, which objects are created, in which order,
> etc. Unexplained magic is annoying in software development.

again we're getting into matters of personal taste. first of all,  
Protocol should *not* be named connection, because that's not what it  
is. protocols happen 'above' the connection, and there's a host of  
other functionality that needs to be encapsulated in a connection  
object that have nothing to do with the protocol.

furthermore, there's no scenario where you can logically think a  
Protocol object represents the actual protocol itself, since protocol  
is by definition an intangible thing. if i have an instance of a  
Protocol object, it's not much of a stretch to assume that is an  
implementation. furthermore, as implementations of protocols are wont  
to behave differently for different clients, creating a worker object  
for each one makes plenty of sense to me.

> It's not that I don't want to learn. It's that the more you use
> framework-specific constructs, and the higher level those constructs
> are, the more your software project is stuck with this framework and
> difficult to reuse/adapt/port/interoperate with. Of course, some
> projects (especially proprietary or internal ones) are less  
> sensitive to
> this than others.

this is hardly relevant. this is an absolute fact of life in the  
software development world, and is definitely just as common during  
development of proprietary software as well as public projects. in fact,
i'd say the biggest problem with modern development practices is that  
there is too much fear of change. sometimes you make a bad design,  
and sometimes you choose an inappropriate 3rd party library.

however, it is your responsibility as a developer to research a  
project enough to make an informed decision. the only concern you  
should have is whether the framework works, and has a big enough  
community to support you.

the strongest card in twisted's deck is the community that is growing  
around it. i have never found any member of the twisted team  
unwilling to help with a problem, and there's never been a revision  
or refactoring that cost me more than a day's work to make  
adjustments for.

i've just completed work on a custom application for one of my  
clients who insisted upon a number of completely unreasonable time  
constraints for the project. twisted allowed me to complete this work  
well within my schedule, and provided me with a variety of 'free'  
features that are simply not available in other frameworks. in  
addition, i was able to get one of my coworkers up and running and  
collaborating with me, **despite the fact he had never even written a  
complete application in python**.

in the final analysis, a project like twisted is going to have the  
strongest appeal to a certain kind of programmer. if your first  
priority is to get the job done without regards to modularity or  
possibility of future expansion, and you don't want to take time  
becoming familiar with the very powerful but very complex twisted  
framework, then you are right to choose other APIs.

if, however, you are looking to add a tool to your kit that will  
serve you well time and time again, becoming involved with the  
twisted community and the framework it supports will only pay off for  
you.

it's a fact of life, you don't get something for nothing.

-phil