[Twisted-Python] Re: [Twisted-web] CPUShare-Twisted

Andrea Arcangeli andrea at cpushare.com
Sat Jan 21 23:52:18 MST 2006

On Sat, Jan 21, 2006 at 11:55:44PM -0500, glyph at divmod.com wrote:
> If you've pending patches that have not been applied, would you please 
> consider instead to agitate for those patches on the mailing lists, and add 

These are the very old ones (ignore the web2 part that is recent).


IIRC you said that disconnecting isn't present in all protocols, not sure if
you added disconnecting to all protocols. There was a discussion to add
more names, but I didn't see any fix going into SVN trunk. I need it in
tcp and these fixes are good enough for my usage (call it dirty fix but
it works for me). The other is a fundamental feature I need from the
proxy, it's backwards compatible so it doesn't add up any risk but I
never received feedback on it. Without it, it's impossible to safely
stack one server on top of the other without losing the client info.

I'm in discussion how to add it to web2 with headers instead of path

> information to them in the tracker?  Discussing an abandoned patch can cast 
> some light where it is needed.  Putting it into the CPUShare fork rather 

I agree, but I always needed the fix since day zero. I had to maintain
it for so long. The fact I'm exporting the tree in a public way can't make
things worse as far as SVN integration is concerned. If something it can
make things better, since with a public tree there may be more interest
from your part in reducing the diff to nothing ;).

> than the bugtracker virtually guarantees that no core Twisted developer 
> will ever look at it.  Not that you can't do both - I just want to make it 
> clear that we're going to keep working off the bugtracker and SVN, not off 
> of random mercurial (or git or bzr or monotone - not to pick on hg) 
> repositories run by users.

Sure I understand that.

> Also, Andrea, would you mind making sure that your codebase is visually 
> distinct from Twisted in some way that is identifiable when it is run?  My 
> main concern here is that if CPUShare-Twisted is to become popular, it 
> seems likely that users might confuse it with the actual Twisted.  Since 
> you have loudly declared your animosity towards filing bugs, I assume 
> CPUShare-Twisted will not have its own bugtracker, and since you are 
> opposed to test-driven development, [..]

My developmnt is generally test-driven. I'm only opposed to unit-test
mandatory development to fix bugs or add new features. Writing unit-test
isn't the only way to test code. It's nice to have a unit test, even a
simple one, but it shouldn't be mandatory.

> [..] I am sure that it will be full of bugs. 

Time will tell. Since you made your prediction I'll make mine. I'm sure
axiom is wasted time in its current API (at least as far as twisted is

I don't see how you can advertize axiom saying "We do plan to add some
later, and perhaps also support other databases in the future.". Sure
you can add it, but if you do it, the whole axiom api will fall apart
unless you want to make synchronous queries over the network. The only
two deferreds you have are during startup and in the testsuite, just
grep for the word Deferred. Making synchronous sql queries in the
twisted async model is unacceptable for anything serious.  Infact even
sqllite queries are obviously unacceptable once the db grows beyond the
size of the cache (and for sure you can't scale the queries over
different servers to have more ram-cache when using sqllite).

Ironically axiom current api would have a chance to work well with
threads, with twisted single threaded async model not.

There are good python storage packages to use with twisted and threads
(heck there is even pgasync if you're religious about not using
threads, why do you think people like me, Valentino and the pgasync
author care about using non-blocking solutions while doing queries to
the db?). I can't imagine why you insist on making your inferior
solution with a design that can't work well with twisted.

I'm feeling guilty for risking hitting the harddisk for a few msec when
people clicks on the mailing list archives, and infact I keep two
webservers exactly to avoid hurting the scalability of the ssl one.

> I would prefer it if we could avoid asking the Twisted team to deal with 
> those bugs as well as actual bugs in Twisted, when users discover the only 
> Twisted bugtracker is on twistedmatrix.com.


> The best option would be to avoid using the same module name, but I 
> understand that tracking trunk would be made more difficult with a diff 
> that touched every import.  Perhaps instead you could change the version 
> from SVN-Trunk to 'HG-CPUShare', so that the CPUShare-ness of the code is 

Ok, I'll make this change right away. I already did that for the web2 side.

> visible at the beginning of any logs, and change the names of the command 
> line tools to e.g. cpushare-twistd and cpushare-mktap?  This would save 
> time since developers could reject bugs immediately upon seeing one of 
> those commands or the telltale log message rather than having to identify 
> that the line numbers are all wrong before doing so.

This is a very fair requirement (changing the version is trivial).
However I don't see much point in changing the commands if the module
name is the same. Either I change both, or none. I mean, changing
cpushare-twistd won't prevent a python app importing the reactor to get
mine instead of the official one, so you would still risk to get bogus
bugreports unless I rename both. I think I can rename the module later
if some people uses it, right now I've no idea if people will use it, or
if I'll remain the only user (clearly as long as I'm the only user I
don't need to rename the module ;).

More information about the Twisted-web mailing list