exploratory coding and the purpose of UQDS (Re: [Twisted-Python] Re: [Twisted-commits] r20094)

glyph at divmod.com glyph at divmod.com
Mon Apr 30 13:08:59 EDT 2007


I'll respond to a few specific things JML said here, but I want to get 
across my personal view of the purpose and benefits of the process we're 
using.  I specifically told JML to make a branch and continue with these 
changes before he got started, and I still think that was the right 
thing to do.

UQDS was instituted on Twisted because tracking changes to Twisted was 
becoming too difficult.  Some changes were critical bugfixes, some were 
random whitespace twiddling, and some introduced regressions.  It was 
designed in the first place (for Divmod) because we were having similar 
issues in a different context.

In other words, not all change is progress.  The goal of UQDS was to 
provide a way to encourage and ensure that all changes to *trunk* are, 
in fact, progress, and Twisted is always improving, so that we don't 
need to burn our energy tracking down bugs or gratuitously bad design 
decisions after they've been merged to trunk and people have begun 
depending on them.  I think it has been very successful at meeting this 
goal.

It's true that the UQDS document on the Divmod wiki does specify that 
the lifetime of a task in UQDS begins with the creation of a ticket. 
For the context that I originally wrote "If it's not in the tracker, you 
shouldn't be working on it.", i.e. the full-time developers at Divmod, 
that makes perfect sense.  In a commercial environment, management needs 
to have a clear idea of where all the resources of the company are being 
deployed.

I would go so far as to say that it makes sense for Twisted as well, 
insofar as UQDS manages our workflow.  However, Twisted SVN does things 
other than UQDS (the trunk/doc/fun/Twisted.Quotes exception and the 
sandbox, to name two notable things) and I don't think that every branch 
needs to be backed by a ticket.  I'd rather have non-ticketed branches 
that we can delete as prototypes than a profusion of vague tickets which 
only their author understands.  (In other words, SVN branches are not 
always necessarily part of the UQDS workflow, but tickets are.)

It's fine by me if there is tons of bad, broken code that gets checked 
in to branches (as long as it is then deleted - see below).  Sometimes, 
the only way to learn how to implement a feature correctly is to 
implement it incorrectly.  Sometimes the only way to figure out what 
feature you're implementing is to mess around for a while and see what 
kind of code you write.  This is especially true of refactoring.

The policy on having tickets applies to anything wanting a review, and 
therefore, anything that hopes to go to trunk.  If you have done some 
work in a branch which you want reviewed, you still need to explain, for 
the reviewer's benefit, what it is that the change is supposed to 
accomplish in a ticket.  Again, this is especially true of refactoring - 
it's fine to noodle around for a while trying to discover a better shape 
for the code, but once you've found one, it's important to clearly 
express *why* the new shape is better rather than simply different. 
Just because the description eventually needs to be written doesn't mean 
it needs to be the first thing that happens, though.

There is one caveat to creating branches for experimentation.  I think 
it's rude to leave dead branches around.  This can be mitigated by using 
ticket numbers, because someone else can come along later, notice that 
the branch refers to a closed ticket, and delete it.  Open branches have 
a cost: "svn ls svn://svn.twistedmatrix.com/svn/Twisted/branches" 
should, at a glance, give an indication as to what's in progress in 
Twisted right now, and dead branches obscure that view and make it 
difficult to get a feel for what's going on.  Right now there are 133 
entries there, and it's probably time for some pruning.  So I think it's 
reasonable to say that if you leave a ticket-number-free branch around 
for more than a month, don't be surprised if someone else deletes it. 
(But please delete it yourself before it comes to that.)

However, deletion or rejection should not necessarily be seen as a 
failure, either of the developer or of the process.  The whole *point* 
of using branches as temporary development lines is that, sometimes, 
those lines end without reaching trunk.  Each rejected branch should be 
a learning experience.

Another important thing to keep in mind, as long as I'm talking about 
how to use and not use branches: don't *ever* deploy Twisted from a 
branch, and this goes triple for a branch created for experimentation. 
Such branches are still at the mercy of a reviewer, and may be 
substantially changed or deleted.  Eventually, *all* branches will be 
deleted, as they are merged to trunk and become obsolete.

On 08:42 am, jml at mumak.net wrote:
>>This time, I decided to not file a ticket, because I did not want to
>>prematurely problem of concisely defining my work. I am being
>>chastised (albeit gently) for such work, and I will probably stop
>>working on this code.

Please don't.  This kind of clean-up can be very important.  I don't 
know if the particular changes in this branch are, but certainly the 
changes which can come out of exploratory coding are useful.
>However, I will note:
>- The time penalty of discussion is significant to me because of my 
>timezone.

An important feature of UQDS is its asynchrony.  Although branches have 
to be reviewed before they're merged, to the extent that changes do not 
directly depend upon one another, it's critical that someone be able to 
sit down and work on feature X, then feature Y, then feature Z, without 
stopping each time to wait 24 hours for a review and then again for the 
second review.  With 24 hours (or more!) of context-switch latency, the 
inability to pipeline requests could completely kill development speed.

This asynchrony should apply to all aspects of the process, including 
discussion of requirements and design goals.  The key feature of a 
heavyweight (read: broken) process is that a developer who is itching to 
do some work has to sit around and wait for permission from a committee 
to do so.
>- I really do think discussion is good. However, I think that planning
>as a group is not always worth the cost.

Enforcement by consensus is great.  Design by consensus (i.e. committee) 
is, unfortunately, terrible.  I never intended UQDS to require that. 
The best sort of group design is when one person has an idea and 
champions it and the group offers criticism and analysis.  Having the 
idea in the first place can require writing some code.
>My goal is more complex than making Twisted the best software
>possible. My goal is to be able to sit down and work on Twisted even
>if I don't have a fully specified, fully approved goal in mind.

As I see it, irrespective of their priorities, the latter is a 
prerequisite for the former.  If, as an open source project, we can't 
harness the power of people's idle time, I think we're doomed.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20070430/1a903516/attachment.htm 


More information about the Twisted-Python mailing list