Changes between Version 202 and Version 203 of ReleaseProcess


Ignore:
Timestamp:
03/07/2016 02:30:44 AM (22 months ago)
Author:
hawkowl
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • ReleaseProcess

    v202 v203  
    11[wiki:ContributingToTwistedLabs Contribute] > [wiki:TwistedDevelopment Development] > Release Process
    22
    3 [[PageOutline(1-3,, inline)]]
    4 
    5 This document describes the Twisted release process. Although it is still incomplete, every effort has been made to ensure that it is accurate and up-to-date. There are plans to eventually move this document into the source tree (#4543).
    6 
    7 This process has only been tested on Linux or OS X, so we recommend that you do the release on Linux or OS X.
    8 
    9 {{{
    10 #!html
    11 <b style="color: red">
    12 Thinking about editing this document?  DON'T.  The only time changes to this document are allowed is during a release.  The only changes that may be made are changes that are tested by the release engineer and found to actually work.
    13 </b>
    14 }}}
    15 
    16 If you want to make other changes to the release process, follow the normal Twisted development process (contribute release automation software that has documentation and unit tests demonstrating that it works).
    17 
    18 = Outcomes =
    19 
    20 By the end of a Twisted release we'll have:
    21  * Tarballs for Twisted as a whole, and for each of its sub-projects
    22  * Windows installers for the whole Twisted project
    23  * Updated documentation (API & howtos) on the twistedmatrix.com site
    24  * Updated documentation on Read The Docs
    25  * Updated download links on the twistedmatrix.com site
    26  * Announcement emails sent to major Python lists
    27  * Announcement post on http://labs.twistedmatrix.com
    28  * A tag in our Subversion repository marking the release
    29 
    30 = Prerequisites =
    31 
    32 To release Twisted, you will need:
    33  * Commit privileges to Twisted
    34  * Access to dornkirk.twistedmatrix.com as t-web
    35  * Permissions to edit the Downloads wiki page
    36  * Channel operator permissions for `#twisted`
    37  * Admin privileges for Twisted's PyPI packages
    38  * Contributor status for http://labs.twistedmatrix.com
    39  * Read The Docs access for the Twisted project
    40 
    41 = Version numbers =
    42 
    43 Twisted releases use a time-based numbering scheme. Releases versions like `YY.MM.mm`, where `YY` is the last two digits of the year of the release, `MM` is the number of the release in the year, and `mm` is the number of the patch release.
    44 
    45 For example:
    46  * The first release of 2010 is `10.0.0`
    47  * The second release of 2010 is `10.1.0`
    48  * If 10.1.0 has some critical defects, then a patch release would be numbered `10.1.1`
    49  * The first pre-release of `10.0.0` is `10.0.0pre1`, the second is `10.0.0pre2`
    50 
    51 Every release of Twisted includes the whole project, the core and all sub-projects. Each of these has the same number.
    52 
    53 Throughout this document, we'll refer to the version number of the release as '''$RELEASE'''. Examples of $RELEASE include `10.0.0`, `10.1.0`, `10.1.1` etc.
    54 
    55 We'll refer to the first two components of the release as '''$API''', since all releases that share those numbers are mutually API compatible. e.g. for `10.0.0`, `$API` is `10.0`; for `10.1.0` and `10.1.1`, `$API` is `10.1`.
    56 
    57 The change-versions script automatically picks the right number for you. Please retrieve it after you run it.
    58 
    59 = Overview =
    60 
    61 To release Twisted, we
    62  1. Prepare for a release
    63  1. Release N pre-releases
    64  1. Release the final release
    65 
    66 = Prepare for a release =
    67 
    68  1. Check the milestone for the upcoming release
    69     1. Get rid of any non-critical bugs
    70     1. Get any critical bugs fixed
    71     1. Check the [https://twistedmatrix.com/trac/wiki/ReleaseManagerNotes release manager notes] in case anyone has left anything which can only be done during the release.
    72  1. Check for any [http://twistedmatrix.com/trac/query?status=new&status=assigned&status=reopened&type=regression&order=priority regressions]
    73  1. Read through the `INSTALL` and `README` files to make sure things like the supported Python versions are correct
    74     1. Check the required Python version.
    75     1. Check that the list matches the current set of buildbots.
    76     1. Any mistakes should be fixed in trunk before making the release branch
    77  1. Choose a version number.
    78  1. File a ticket
    79     1. Assign it to the upcoming release milestone
    80     1. Assign it to yourself
    81     1. Call it "Release $RELEASE"
    82  1. Make a branch (`mkbranch release-$RELEASE-4290`, using mkbranch from twisted-dev-tools)
    83 
    84 
    85 = How to do a pre-release =
    86 
    87  1. Check [http://buildbot.twistedmatrix.com/boxes-supported buildbot] to make sure all supported platforms are green (wait for pending builds if necessary).
    88  1. If a previously supported platform does not currently have a buildbot, move from supported platforms to "expected to work" in `INSTALL`. (Pending #1305)
    89  1. In your Git-SVN-enabled Git repo, fetch and check out the new release branch.
    90  1. Run `./bin/admin/change-versions --prerelease`
    91  1. Commit the changes made by `change-versions`
    92  1. Run `./bin/admin/build-news .`
    93  1. Commit the changes made by `build-news` - this automatically removes the NEWS topfiles (see #4315)
    94  1. Bump copyright dates in `LICENSE`, `twisted/copyright.py`, and `README` if required
    95  1. `git svn dcommit --dry` to make sure everything looks fine, and then `git svn dcommit` to push up the changes.
    96  1. Make a temporary directory for the tarballs to live in (e.g. `mkdir /tmp/twisted-release`)
    97  1. Run `./bin/admin/build-tarballs . /tmp/twisted-release/`
    98     1. ''Note'': `build-tarballs` does not produce exactly the same output when run multiple times, even when nothing else has changed.  If a problem is encountered that requires `build-tarballs` to be re-run (either during the pre-release or later during the release), care must be taken to avoid releasing two or more '''different''' versions of the tarball.
    99  1. Copy `NEWS` to `/tmp/twisted-release/` as `NEWS.txt` for people to view without having to download the tarballs.
    100     1. `cp NEWS /tmp/twisted-release/NEWS.txt`
    101  1. Upload the tarballs to `twistedmatrix.com/Releases/pre/$RELEASE` (see #4353)
    102     1. You can use `rsync --rsh=ssh --partial --progress -av /tmp/twisted-release/ t-web@dornkirk.twistedmatrix.com:/srv/t-web/data/releases/pre/<RELEASE>/` to do this.
    103  1. Write the pre-release announcement
    104     1. Read through the `NEWS` file and summarize the interesting changes for the release
    105     1. Get someone else to look over the announcement before doing it
    106  1. Announce the pre-release on
    107     1. the twisted-python [wiki:TwistedCommunity#MailLists mailing list]
    108     1. on IRC in the `#twisted` topic
    109     1. in a blog post, ideally `labs.twistedmatrix.com`
    110 
    111 == Pre-release announcement ==
    112 
    113 The pre-release announcement should mention the important changes since the last release, and exhort readers to test this pre-release.
    114 
    115 Here's what the `$RELEASEpre1` release announcement might look like:
    116 
    117 {{{
    118 Live from PyCon Atlanta, I'm pleased to herald the approaching
    119 footsteps of the $API release.
    120 
    121 Tarballs for the first Twisted $RELEASE pre-release are now available at:
    122  http://people.canonical.com/~jml/Twisted/
    123 
    124 Highlights include:
    125 
    126  * Improved documentation, including "Twisted Web in 60 seconds"
    127 
    128  * Faster Perspective Broker applications
    129 
    130  * A new Windows installer that ships without zope.interface
    131 
    132  * Twisted no longer supports Python 2.3
    133 
    134  * Over one hundred closed tickets
    135 
    136 For more information, see the NEWS file.
    137 
    138 Please download the tarballs and test them as much as possible.
    139 
    140 Thanks,
    141 jml
    142 }}}
    143 
    144 A week is a generally good length of time to wait before doing the final release.
    145 
    146 
    147 = How to do a final release =
    148 
    149 == Prepare the branch ==
    150 
    151  1. Have the release branch, previously used to generate a pre-release, checked out
    152  1. Run `./bin/admin/change-versions`
    153  1. Add the quote of the release to the README
    154  1. Make a new quote file for the next version: `git mv docs/fun/Twisted.Quotes docs/historic/Quotes/Twisted-$API; echo '' > docs/fun/Twisted.Quotes; git add docs/fun/Twisted.Quotes`
    155  1. Commit the version and README changes.
    156  1. Submit the ticket for review
    157  1. Pause until the ticket is reviewed and accepted.
    158  1. Tag the release
    159     * e.g. `svn cp svn+ssh://svn.twistedmatrix.com/svn/Twisted/branches/releases/release-$RELEASE-4290 svn+ssh://svn.twistedmatrix.com/svn/Twisted/tags/releases/twisted-$RELEASE`
    160     * A good commit message to use is something like "Tag $RELEASE release"
    161 
    162 == Cut the tarballs & installers ==
    163 
    164  1. Create a new staging area for the release (e.g. `mkdir /tmp/twisted-release`)
    165  1. Using a checkout of the release branch or the release tag (with no local changes!), run `./bin/admin/build-tarballs . /tmp/twisted-release/`
    166  1. Build Windows MSI
    167     1. http://buildbot.twistedmatrix.com/builders/windows7-64-py2.7-msi
    168     1. For "Branch" specify the release branch, e.g. "branches/releases/release-$RELEASE-4290"
    169     1. Download the latest `.whl` files from from http://buildbot.twistedmatrix.com/builds/twisted-packages/ and save them in the staging directory
    170  1. Sign the tarballs and Windows installers. (You will need a PGP key for this - use something like Seahorse to generate one, if you don't have one.)
    171     1. MD5: `md5sum Tw* | gpg -a --clearsign > /tmp/twisted-release/twisted-$RELEASE-md5sums.txt`
    172     1. SHA512: `shasum -a 512 Tw* | gpg -a --clearsign > /tmp/twisted-release/twisted-$RELEASE-shasums.txt`
    173     1. Compare these to an [http://tmrc.mit.edu/mirror/twisted/twisted-10.1.0-md5sums.txt example of twisted-$RELEASE-md5sums.txt] - they should look the same.
    174 
    175 == Update documentation ==
    176 
    177  1. Get the dependencies
    178     * Pydoctor (use the branch "twisted" from [https://github.com/twisted/pydoctor])
    179     * Epydoc (`python-epydoc` in Debian)
    180  1. Build the documentation
    181     1. `./bin/admin/build-docs .`
    182       * This doesn't seem to build the book?
    183     1. `cp -R doc /tmp/twisted-release/`
    184  1. Run the `build-apidocs` script to build the API docs and then upload them (See also [wiki:APIDocs] and #2891).
    185     1. Copy the `pydoctor` directory from the twisted branch into your Git checkout.
    186     1. `./bin/admin/build-apidocs . /tmp/twisted-release/api`
    187     1. Documentation will be generated in a directory called `/tmp/twisted-release/api`
    188  1. Update the Read The Docs default to point to the release branch (via the [dashboard https://readthedocs.org/projects/twisted/]).
    189 
    190 == Distribute ==
    191 
    192  1. Create a tarball with the contents of the release directory: `cd /tmp/twisted-release; tar -cvjf ../release.tar.bz2 *`
    193  1. Upload to the official upload locations (see #2888)
    194     1. `cd ~; git clone https://github.com/twisted-infra/braid`
    195     1. `cd braid;`
    196     1. `virtualenv ~/dev/braid; source ~/dev/braid/bin/activate; cd ~/braid; python setup.py develop;`
    197     1. `cd ~/braid; fab config.production t-web.uploadRelease:$RELEASE,/tmp/release.tar.bz2`
    198  1. Test the generated docs
    199     1. Browse to http://twistedmatrix.com/documents/$RELEASE/
    200     1. Make sure that there is content in each of the directories and that it looks good
    201     1. Follow each link on  http://twistedmatrix.com/trac/wiki/Documentation, replace current with $RELEASE (e.g. 10.0.0) and look for any obvious breakage
    202  1. Change the "current" symlink
    203     1. Upload release: `fab config.production t-web.updateCurrentDocumentation:$RELEASE`
    204 
    205 == Announce ==
    206 
    207  1. Update [wiki:Downloads] pages
    208     1. The following updates are automatic, due to the use of the [https://raw.github.com/twisted-infra/twisted-trac-plugins/master/twisted_trac_plugins/release_macro.py ProjectVersion] [WikiMacros wiki macro] throughout most of the [wiki:Downloads] page.
    209       1. Text references to the old version to refer to the new version
    210       1. The link to the `NEWS` file to point to the new version
    211       1. Links and text to the main tarball
    212     1. Add a new md5sum link
    213     1. Add a new shasum link
    214     1. Save the page, check all links
    215  1. Update PyPI records & upload files
    216     * http://pypi.python.org/pypi/Twisted/
    217       * Edit the version. *Make sure you do this first.*
    218       * Upload tarball, MSIs and wheels
    219  1. Write the release announcement (see below)
    220  1. Announce the release
    221     1. Send a text version of the announcement to: `twisted-python@twistedmatrix.com, python-announce-list@python.org, python-list@python.org, twisted-web@twistedmatrix.com, twisted-jabber@ik.nu`
    222       * Note: you need to be subscribed to some of those lists to be able to send, like `twisted-jabber@ik.nu`.
    223     1. http://labs.twistedmatrix.com
    224       * Post a web version of the announcements, with links instead of literal URLs
    225     1. Twitter, if you feel like it
    226     1. #twisted topic on IRC (you'll need ops)
    227  1. Merge the release branch into trunk, closing the release ticket at the same time.
    228     * For now you need to add a .misc NEWS fragment to merge the branch.
    229  1. Close the release milestone (which should have no tickets in it).
    230  1. Open a milestone for the next release.
    231 
    232 == Release announcement ==
    233 
    234 The final release announcement should:
    235  * Mention the version number
    236  * Include links to where the release can be downloaded
    237  * Summarize the significant changes in the release
    238  * Consider including the quote of the release
    239  * Thank the contributors to the release
    240 
    241 Here's an example:
    242 
    243 {{{
    244 On behalf of Twisted Matrix Laboratories, I am honoured to announce
    245 the release of Twisted 13.2!
    246 
    247 The highlights of this release are:
    248 
    249  * Twisted now includes a HostnameEndpoint implementation which uses
    250 IPv4 and IPv6 in parallel, speeding up the connection by using
    251 whichever connects first (the 'Happy Eyeballs'/RFC 6555 algorithm).
    252 (#4859)
    253 
    254  * Improved support for Cancellable Deferreds by kaizhang, our GSoC
    255 student. (#4320, #6532, #6572, #6639)
    256 
    257  * Improved Twisted.Mail documentation by shira, our Outreach Program
    258 for Women intern. (#6649, #6652)
    259 
    260  * twistd now waits for the application to start successfully before
    261 exiting after daemonization. (#823)
    262 
    263  * SSL server endpoint string descriptions now support the
    264 specification of chain certificates. (#6499)
    265 
    266  * Over 70 closed tickets since 13.1.0.
    267 
    268 For more information, check the NEWS file (link provided below).
    269 
    270 You can find the downloads at <https://pypi.python.org/pypi/Twisted>
    271 (or alternatively <http://twistedmatrix.com/trac/wiki/Downloads>) .
    272 The NEWS file is also available at
    273 <http://twistedmatrix.com/Releases/Twisted/13.2/NEWS.txt>.
    274 
    275 Many thanks to everyone who had a part in this release - the
    276 supporters of the Twisted Software Foundation, the developers who
    277 contributed code as well as documentation, and all the people building
    278 great things with Twisted!
    279 
    280 Twisted Regards,
    281 HawkOwl
    282 }}}
    283 
    284 
    285 = When things go wrong =
    286 
    287 If you discover a showstopper bug during the release process, you have three options.
    288  1. Abort the release, make a new point release (e.g. abort 10.0.0, make 10.0.1 after the bug is fixed)
    289  1. Abort the release, make a new pre-release (e.g. abort 10.0.0, make 10.0.0pre3 after the bug is fixed)
    290  1. Interrupt the release, fix the bug, then continue with it (e.g. release 10.0.0 with the bug fix)
    291 
    292 If you choose the third option, then you should:
    293  1. Delete the tag for the release
    294  1. Recreate the tag from the release branch once the fix has been applied to that branch
    295 
    296 
    297 = Bug fix releases =
    298 
    299 Sometimes, bugs happen, and sometimes these are regressions in the current released version. This section goes over doing these "point" releases.
    300 
    301  1. Ensure all bugfixes are in trunk.
    302  1. Make a branch off the affected version.
    303     1. eg. {{{svn cp svn+ssh://svn.twistedmatrix.com/svn/Twisted/branches/releases/release-$API.0-7844  svn+ssh://svn.twistedmatrix.com/svn/Twisted/branches/releases/release-$API.1-7906 -m "Branching to $API.1"}}}
    304  1. Cherry-pick the merge commits that merge the bugfixes into trunk, onto the new release branch.
    305  1. Go through the rest of the process for a full release from "How to do a pre-release", merging the release branch into trunk as normal as the end of the process.
    306     1. Instead of {{{--prerelease}}} when running the change-versions script, add the patch flag, making it {{{--patch --prerelease}}}.
    307     1. Instead of waiting a week, a shorter pause is acceptable for a patch release.
    308 
    309 
    310 = Open questions =
    311 
    312  * How do we manage the case where there are untested builds in trunk?
    313  * Should picking a release quote be part of the release or the pre-release?
    314  * What bugs should be considered release blockers?
    315    * All bugs with a type from the **release blocker** family
    316    * Anybody can create/submit a new ticket with a release blocker type
    317    * Ultimately it's the RM's discretion to accept a ticket as a release blocker
    318  * Should news fragments contain information about who made the changes?
    319 
    320 = Bugs mentioned on this page =
    321 
    322  * [ticket:2888 Automate uploading tarballs to TMRC]
    323  * [ticket:4353 Automate uploading pre-release tarballs]
    324  * [ticket:4315 Convenience command for removing news fragments]
    325  * [ticket:2891 Automate building & uploading API docs for website]
    326  * [ticket:2380 Automate building & uploading howto docs for website]
    327 
    328 = See also =
    329 
    330  * [http://twistedmatrix.com/trac/query?status=new&status=assigned&status=reopened&component=release+management&order=priority release management tickets]
    331  * [http://twistedmatrix.com/trac/query?status=new&status=assigned&status=reopened&milestone=regular-releases&order=priority regular-release tickets]
    332  * ReleaseAutomation
    333  * [http://doc.bazaar.canonical.com/bzr.dev/developers/releasing.html Releasing Bazaar] -- release documentation for another project that does time-based releases
     3This page is superseded by the version in our official documentation: [http://twisted.readthedocs.org/en/latest/core/development/policy/release-process.html].