Jabber (was Re: [Twisted-Python] Closing Jabber connection)

[Glyph Lefkowitz]

Let me reply to the last thing you said first:

Ralph Meijer wrote:

> There is no support for server-to-server connections and the server side
> of client-to-server and server component connections, yet. Having that,
> would be really great, though.

 > While writing this, I decided I'd like to work on all this and help is
 > of course appreciated.

Awesome.  I hope that my feelings about packaging and distribution don't 
discourage you!  Words badly needs a dedicated maintainer and Jabber 
would be a great place to start, and probably not a bad model to 
structure the whole messaging system around.

> On Tue, May 03, 2005 at 07:30:19PM -0400, Glyph Lefkowitz wrote:

>>Ralph Meijer wrote:

>>>On Tue, May 03, 2005 at 03:17:34PM -0500, Peter Saint-Andre wrote:

>>>>As I said, I know of several folks in the Jabber community who
>>>>are quite interested in building a reference XMPP/Jabber server
>>>>in Python, and I'm wondering if Words is the place to point them.

>>>Besides that, one could argue that XMPP/Jabber doesn't belong in Words
>>>alltogether because IM is just one possible use of XMPP/Jabber.

>>One could argue that, but one would also be wrong :)
>>
>>Any protocol can be repurposed.

> I can see your point and understand that people thinking of Jabber
> believe it must be in Words, but my initial remark on it not belonging
> there has more to do with that most of the current code doesn't belong
> where it is now.

> Yes, Jabber was originally designed for chat. I'd like to give a little
> bit of background to let you see where I'm coming from, and then giving
> my view about how the current organisation is not ideal. I hope you'll
> forgive me for the length of it.

I still don't understand why you think Jabber can be a chat project that 
grows beyond that scope but you don't think that Words can be a chat 
project that grows beyond that scope :).

> XMPP Core defines (...)

> XMPP IM defines  (...)

> Then, the Jabber Software Foundation (JSF) defines a set of Jabber
> Enhancement Proposals. (...)

Okay... I did know most of that ;)

> In Twisted, the current implementation of what is called Jabber is split
> between twisted.xish.xmlstream and twisted.words.protocols.jabber. In
> xmlstream, the basic concept of an XML stream is covered, and hooks for
> doing any kind of stream authentication. Everything else is in the
> jabber module.

This is the core problem, I think.

At this point, twisted.xish is simply support for the Jabber module in 
twisted.words, with no real legitimate life of its own.  I believe it 
should be moved into twisted.words as a (admittedly largeish) protocol 
support module rather than having a separate installer and a separate 
project page.  Factoring of XMPP IM/XMPP Core within Words would be a 
good idea, but this should not affect installation and deployment 
concerns, because there is no reason to use them separately right now.

We should separate the project back out if people start clamoring for a 
separate distributable, but I seriously doubt this will be the case.

> The jabber module only supports the legacy authentication method (not
> XMPP!) for setting up client-to-server connections (client side only),
> and there is no support for TLS, SASL or even error conditions. Recently
> I did implement XMPP stringprep profiles, though. The client module also
> has a raw implementation of abstracting the <iq/> stanza, by allowing
> you to wait for the response to a query. Higher level support for the
> other two stanzas, roster handling or blocking communication is not
> there. There is also a framework for building server-side components,
> but that is defined in a JEP, not by XMPP.

Obviously the implementation needs to be brought into line with more 
recent versions of all these specifications.  No argument there :).

> My feeling is that the current implementation needs a lot of work, and
> I'd like to split that between implementing XMPP Core and XMPP IM. I
> could agree that the latter would belong in Words, but the former
> doesn't

Again.  I disagree.  You spent a lot of text above talking about a lot 
of things which were allegedly supposed to back up this point, but I 
don't see how they're related.  Jabber is a chat protocol which has been 
expanded to a wider array of application domains, Words is a project to 
enable expanding chat protocols to handle a wider array of application 
domains.  We have already made a sort of policy decision at a global 
level to bundle protocol logic in the application modules that it 
supports, i.e. twisted FTP protocol and HTTP protocol support belong in 
twisted.web, NOT in some common protocol-abstractions-only package.  You 
might have an application which uses only the XMPP core parts of Words, 
but those sorts of applications are, in my experience, *extremely* rare, 
and should not distort the marketing of the project and normal 
experience of people downloading the code or hacking on it for the first 
time.  Internal factoring is a separate question to packaging; if I view 
them in the light of internal factoring, your ideas all seem good.  If I 
view them as packaging, they're all bad.

> and that means that most of what is currently in Words needs to
> go to somewhere else. Maybe that somewhere-else is xish, but I didn't
> really like the moving xmlstream there, either. Maybe it'd be good to
> have a new twisted subproject XMPP that implements XMPP Core, and moving
> xmlstream there.

To reiterate, what is currently in Words needs to stay in Words, and 
things which support it that are not otherwise used needs to move there 
next to them.  Twisted subprojects are organized around some set of 
application functionality that can be provided by the core.  Twisted 
Words' provided application is a multiprotocol chat server and client, 
with chatterbot support.  Like all Twisted projects, the code can be 
used at any level, whether it's to load into the provided application as 
a plug-in or to develop a new application on top of only a subset of a 
package.

There is a cultural feeling in the Jabber community, I think, that 
Jabber is somehow unique and should be segregated and separated from all 
the other "legacy" protocols that are "just chat".

All chat protocols support a range of functionality.  IRC really is used 
as a message routing platform, in some cases in actual serious 
applications.  As far as I know OSCAR is used both for chat and for some 
back-end message delivery at AOL.  Even just on the chat client, it has 
a range of extensions which is pretty impressive.  MSN's protocol is 
used for more than just messenger.  All these protocols have a message 
format and a chat layer and an extension layer, and equal support for 
all of them should not be precluded.  Jabber is not a beautiful and 
unique snowflake.  It is the all-singing all dancing chat of the world ;)

That said, Jabber does have advantages over these protocols in many 
ways.  As I said above, it would not bother me to see the bulk of the 
Twisted Words architected around it.  Still, it would be a mistake to 
break the project up into multiple pieces first.  A key hint that this 
is a bad place to break the project is that the design for both halves 
is going to be done by one person as one effort, rather than building an 
XMPP core application and then waiting a few months to a year and then 
building an XMPP IM extension to that - the useful bits are up in XMPP 
IM, and XMPP core is just support code for that.  It certainly may be 
useful support code for a wide range of other applications but it does 
not stand on its own yet.