exploratory coding and the purpose of UQDS (Re: [Twisted-Python] Re: [Twisted-commits] r20094)
glyph at divmod.com
glyph at divmod.com
Mon Apr 30 13:08:59 EDT 2007
I'll respond to a few specific things JML said here, but I want to get
across my personal view of the purpose and benefits of the process we're
using. I specifically told JML to make a branch and continue with these
changes before he got started, and I still think that was the right
thing to do.
UQDS was instituted on Twisted because tracking changes to Twisted was
becoming too difficult. Some changes were critical bugfixes, some were
random whitespace twiddling, and some introduced regressions. It was
designed in the first place (for Divmod) because we were having similar
issues in a different context.
In other words, not all change is progress. The goal of UQDS was to
provide a way to encourage and ensure that all changes to *trunk* are,
in fact, progress, and Twisted is always improving, so that we don't
need to burn our energy tracking down bugs or gratuitously bad design
decisions after they've been merged to trunk and people have begun
depending on them. I think it has been very successful at meeting this
It's true that the UQDS document on the Divmod wiki does specify that
the lifetime of a task in UQDS begins with the creation of a ticket.
For the context that I originally wrote "If it's not in the tracker, you
shouldn't be working on it.", i.e. the full-time developers at Divmod,
that makes perfect sense. In a commercial environment, management needs
to have a clear idea of where all the resources of the company are being
I would go so far as to say that it makes sense for Twisted as well,
insofar as UQDS manages our workflow. However, Twisted SVN does things
other than UQDS (the trunk/doc/fun/Twisted.Quotes exception and the
sandbox, to name two notable things) and I don't think that every branch
needs to be backed by a ticket. I'd rather have non-ticketed branches
that we can delete as prototypes than a profusion of vague tickets which
only their author understands. (In other words, SVN branches are not
always necessarily part of the UQDS workflow, but tickets are.)
It's fine by me if there is tons of bad, broken code that gets checked
in to branches (as long as it is then deleted - see below). Sometimes,
the only way to learn how to implement a feature correctly is to
implement it incorrectly. Sometimes the only way to figure out what
feature you're implementing is to mess around for a while and see what
kind of code you write. This is especially true of refactoring.
The policy on having tickets applies to anything wanting a review, and
therefore, anything that hopes to go to trunk. If you have done some
work in a branch which you want reviewed, you still need to explain, for
the reviewer's benefit, what it is that the change is supposed to
accomplish in a ticket. Again, this is especially true of refactoring -
it's fine to noodle around for a while trying to discover a better shape
for the code, but once you've found one, it's important to clearly
express *why* the new shape is better rather than simply different.
Just because the description eventually needs to be written doesn't mean
it needs to be the first thing that happens, though.
There is one caveat to creating branches for experimentation. I think
it's rude to leave dead branches around. This can be mitigated by using
ticket numbers, because someone else can come along later, notice that
the branch refers to a closed ticket, and delete it. Open branches have
a cost: "svn ls svn://svn.twistedmatrix.com/svn/Twisted/branches"
should, at a glance, give an indication as to what's in progress in
Twisted right now, and dead branches obscure that view and make it
difficult to get a feel for what's going on. Right now there are 133
entries there, and it's probably time for some pruning. So I think it's
reasonable to say that if you leave a ticket-number-free branch around
for more than a month, don't be surprised if someone else deletes it.
(But please delete it yourself before it comes to that.)
However, deletion or rejection should not necessarily be seen as a
failure, either of the developer or of the process. The whole *point*
of using branches as temporary development lines is that, sometimes,
those lines end without reaching trunk. Each rejected branch should be
a learning experience.
Another important thing to keep in mind, as long as I'm talking about
how to use and not use branches: don't *ever* deploy Twisted from a
branch, and this goes triple for a branch created for experimentation.
Such branches are still at the mercy of a reviewer, and may be
substantially changed or deleted. Eventually, *all* branches will be
deleted, as they are merged to trunk and become obsolete.
On 08:42 am, jml at mumak.net wrote:
>>This time, I decided to not file a ticket, because I did not want to
>>prematurely problem of concisely defining my work. I am being
>>chastised (albeit gently) for such work, and I will probably stop
>>working on this code.
Please don't. This kind of clean-up can be very important. I don't
know if the particular changes in this branch are, but certainly the
changes which can come out of exploratory coding are useful.
>However, I will note:
>- The time penalty of discussion is significant to me because of my
An important feature of UQDS is its asynchrony. Although branches have
to be reviewed before they're merged, to the extent that changes do not
directly depend upon one another, it's critical that someone be able to
sit down and work on feature X, then feature Y, then feature Z, without
stopping each time to wait 24 hours for a review and then again for the
second review. With 24 hours (or more!) of context-switch latency, the
inability to pipeline requests could completely kill development speed.
This asynchrony should apply to all aspects of the process, including
discussion of requirements and design goals. The key feature of a
heavyweight (read: broken) process is that a developer who is itching to
do some work has to sit around and wait for permission from a committee
to do so.
>- I really do think discussion is good. However, I think that planning
>as a group is not always worth the cost.
Enforcement by consensus is great. Design by consensus (i.e. committee)
is, unfortunately, terrible. I never intended UQDS to require that.
The best sort of group design is when one person has an idea and
champions it and the group offers criticism and analysis. Having the
idea in the first place can require writing some code.
>My goal is more complex than making Twisted the best software
>possible. My goal is to be able to sit down and work on Twisted even
>if I don't have a fully specified, fully approved goal in mind.
As I see it, irrespective of their priorities, the latter is a
prerequisite for the former. If, as an open source project, we can't
harness the power of people's idle time, I think we're doomed.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Twisted-Python