Get rid of references and code specific to Python 2.3

[jesstess]

Some of these comments can just get removed. Some might require removal of code as well.

twisted/conch/test/test_cftp.py:        # Python 2.3 compatibility fix
twisted/lore/lint.py:# parser.suite in Python 2.3 raises SyntaxError, <2.3 raises parser.ParserError
twisted/python/_release.py:            "Version should be in a form kind of like '1.2.3[pre4]'")
twisted/python/components.py:    warnings.warn("components.backwardsCompatImplements doesn't do anything in Twisted 2.3, stop calling it.", ComponentsDeprecationWarning, stacklevel=2)
twisted/python/components.py:    warnings.warn("components.fixClassImplements doesn't do anything in Twisted 2.3, stop calling it.", ComponentsDeprecationWarning, stacklevel=2)
twisted/python/reflect.py:                    execName == importName): # python 2.3, no cleanup
twisted/python/reflect.py:        # Necessary for cleaning up modules in 2.3.
twisted/python/syslog.py:# These defaults come from the Python 2.3 syslog docs.
twisted/python/util.py:    return a negative value.  Python 2.3 shares this behavior, but also
twisted/python/util.py:    To use this function safely you must use the return value. In Python 2.3,
twisted/spread/jelly.py:Python 2.3; this can be accomplished by using L{twisted.python.compat.set}.
twisted/spread/jelly.py:        Python 2.3), and 'module' types, as well as basic types.
twisted/test/test_reflect.py:        # Make sure that this behavior is *consistent* for 2.3, where there is
twisted/test/test_reflect.py:        Test references search through a deque object. Only for Python > 2.3.
twisted/trial/runner.py:    differences between unittest in Python 2.3, 2.4, and 2.5. These
twisted/trial/runner.py:        In Python 2.4, doctests have correct id() behaviour. In Python 2.3,
twisted/trial/test/test_doctest.py:        doctest in Python 2.3.
twisted/trial/test/test_doctest.py:        # doctest reports failures as errors in 2.3
twisted/trial/test/test_loader.py:            "Python 2.3 import semantics make this behavior incorrect on that "
twisted/trial/test/test_loader.py:            "was imported will succeed on Python 2.3, whereas it will fail on "
twisted/trial/unittest.py:        # we implement this because Python 2.3 unittest defines this code
twisted/trial/unittest.py:# Support for Python 2.3
twisted/trial/unittest.py:    # Python 2.3's TestSuite doesn't support iteration. Let's monkey patch it!
twisted/web/http_headers.py:    # Python 2.3 DictMixin.setdefault is defined so as not to have a default
twisted/web/http_headers.py:    # like dict.setdefault on Python 2.3. -exarkun
twisted/web/test/test_http_headers.py:            "Python 2.3 does not support keyword arguments to dict.update.")
twisted/words/protocols/jabber/sasl.py:# Python 2.4. For Python 2.3 compatibility, the legacy interface is used while
twisted/words/protocols/jabber/xmpp_stringprep.py:                  "recommended you upgrade to Python 2.3.2 or newer if you "
doc/core/development/policy/coding-standard.xhtml:    example, Python 2.3's <code>sets</code> module was deprecated in Python 2.6
doc/core/development/policy/test-standard.xhtml:folder. This option requires Python 2.3.3 or newer.</p>

[rockstar]

Remove mentions of Python 2.3 and code where applicable.

[jesstess]

(In [33790]) Branching to 'python-2.3-5386'

[jesstess]

(In [33792]) Apply 5386.patch by rockstar.

refs #5386

[jesstess]

Thanks for working on this, rockstar! Some comments:

twisted/python/util.py

• In mergeFunctionMetadata, merged always needs to be set to g now instead of only in the else block, or merged won't be defined if a TypeError is raised.
• merged.__dict__.update(g.__dict__) seems like a copy-paste error since merged is already based on g, and commenting it out doesn't result in any test failures.
• The code removals result in some unused imports that can be deleted:

  $ pyflakes twisted/python/util.py
  twisted/python/util.py:5: 'inspect' imported but unused
  twisted/python/util.py:6: 'types' imported but unused
  

twisted/spread/jelly.py

• Can you linewrap the textual changes to less than 80 characters (as described in the ​coding standard).

twisted/test/test_reflect.py

• The way I read this test is that the second assertRaises in test_importExceptions is for confirming that reflect.namedAny will propagate Exceptions correctly in Python 2.3 even though there isn't post-failed-import cleanup. ​http://www.python.org/getit/releases/2.4/highlights/ seems to back this up. Do you agree? If you do, the second assertRaises can go away with the comment.
• test_deque is for Python > 2.3, so it should stay, but the comment about being for > 2.3 can go away.

twisted/trial/runner.py

• DocTestCase is referenced in several places still. It would be cleanest to take care of removing it and its references in a separate removal ticket, where it gets its own NEWS file. Can you undo these deletions for now and open a removal ticket?

There are some lingering references to Python 2.3 that can be removed:

• twisted/web/test/test_http_headers.py: "Python 2.3 does not support keyword arguments to dict.update.")
• twisted/spread/jelly.py: Python 2.3), and 'module' types, as well as basic types.

Please also add a NEWS file as described in ​http://twistedmatrix.com/trac/wiki/ReviewProcess#Newsfiles.

Please make future patches against the branch for this ticket.

Here are the ​build results so far.

Thanks!

[rockstar]

Ugh. Sorry for the kamikaze contribution. I was more tired than I thought the other night. :)

I've made the requested changes directly into the branch you created. The ​Build results are here.

I've also filed ticket #5554 after re-instating DocTestCase it twisted.trial.runner

[Jean-Paul Calderone]

Some more feedback, mostly about extra things I think you can remove, but also about a few things I think we need to be careful to preserve:

1. twisted/test/test_reflect.py
   1. The deleted assertion about ZeroDivisionError being raised by namedAny("twisted.test.reflect_helper_ZDE") should be restored. It's great that the implementation for that behavior is now simpler, but we still need to verify the behavior is provided.
   2. test_deque is mis-indented, so it won't ever be run
   3. Also, collections.deque was added in Python 2.4, so it should always import. The module doesn't need to protect its import with an except ImportError, nor skip the test if it can't be imported.
2. twisted/trial/unittest.py
   1. I'm a bit worried about the side-effects of deleting TestSuite.run. However, all our tests pass, so I guess I shouldn't be. It might be nice if you could convince jml or lifeless (irc nicks) to comment on this part of the patch, though.
   2. The line iter(pyunit.TestSuite()) is now sitting in the module all by itself. It has no side-effects, it should be deleted.
3. twisted/python/util.py
   1. The merged name is actually pointless now. The function can just use g throughout.
   2. This also means that to use the function safely, it is no longer necessary to use the return value. It is the same as the function object passed in as an argument. So, the docstring does not need to claim that the return value must be used anymore.
4. twisted/spread/jelly.py
   1. I think that, in allowInstancesOf, dropping Python 2.3 support means that the 'class' value being passed to allowTypes is superfluous. There will only be a builtin "classobj" type to allow, never a builtin "class" type. If this is right, we should get rid of the 'class' parameter. We should make sure we have good unit test coverage for these cases though (not just delete it and hope).
5. twisted/lore/lint.py
   1. I think you should leave parserErrors defined, but define it as just (SyntaxError,). It's a public name, even if it's a terrible one. We don't want to break any code that was importing this.
6. twisted/words/protocols/jabber/xmpp_stringprep.py
   1. The entire code block beneath if sys.version_info < (2,3,2): can go, can't it?
   2. This seems to imply that some of the later code guarded by the if crippled: check can go as well.
7. twisted/trial/test/test_doctest.py
   1. I'd lean towards keeping test_id which is deleted in the branch. Perhaps DocTest behaves fine in all our currently supported Python versions, but keeping the check will let us know if that continues to be the case going forward.
   2. The other change in this file, to just compare against the length of failures, is cool. Specification tightening. :)
8. The news fragment in topfiles/ doesn't follow the style guide. I might go for a .misc here anyway, for ease. However, if you like, you could make a news fragment for _each_ subproject and mention the main area of code cleanup for each subproject in each of those files.

Thanks!

[Jean-Paul Calderone]

Oh, one more thing, there's a doc error now: ​http://buildbot.twistedmatrix.com/builders/documentation/builds/1954/steps/api-documentation/logs/stdio

[ralphm]

I saw r33901 that removes the Python <2.3.2 stuff for XMPP stringprep. As we are also dropping 2.4 support, you might also want to address the try/except import for unicodedata in this patch, as this block is being dedented now.

[Jean-Paul Calderone]

Presumably that change should be addressed in the patch for the ticket that drops Python 2.4 support.

[ralphm]

Replying to exarkun:

Presumably that change should be addressed in the patch for the ticket that drops Python 2.4 support.

Of course, but that would surely create a merge conflict in both directions. That's why I suggested it here.

[Jean-Paul Calderone]

Yea. There's going to be merge conflicts. Probably more than a few.

[rockstar]

Yeah, I've already run into a few of them already, which is why I haven't re-submitted for review just yet. I expect that things will mellow out in a few days and I'll be able to get this patch completed.

[Thijs Triemstra]

(In [36883]) Branching to 'python-2.3-5386-2'

[Thijs Triemstra]

(In [36889]) merge forward, address review comments, add news files. refs #5386

[Thijs Triemstra]

Replying to exarkun:

1. twisted/trial/unittest.py
   1. I'm a bit worried about the side-effects of deleting TestSuite.run. However, all our tests pass, so I guess I shouldn't be. It might be nice if you could convince jml or lifeless (irc nicks) to comment on this part of the patch, though.

Removed __call__.

1. The line iter(pyunit.TestSuite()) is now sitting in the module all by itself. It has no side-effects, it should be deleted.

Fixed.

1. twisted/python/util.py
   1. The merged name is actually pointless now. The function can just use g throughout.
   2. This also means that to use the function safely, it is no longer necessary to use the return value. It is the same as the function object passed in as an argument. So, the docstring does not need to claim that the return value must be used anymore.

Fixed.

We can also ignore the reference to Python 2.3 for unsignedID that has been deprecated recently (#5544).

1. twisted/spread/jelly.py
   1. I think that, in allowInstancesOf, dropping Python 2.3 support means that the 'class' value being passed to allowTypes is superfluous. There will only be a builtin "classobj" type to allow, never a builtin "class" type. If this is right, we should get rid of the 'class' parameter. We should make sure we have good unit test coverage for these cases though (not just delete it and hope).

This could be done in a new ticket?

1. twisted/lore/lint.py
   1. I think you should leave parserErrors defined, but define it as just (SyntaxError,). It's a public name, even if it's a terrible one. We don't want to break any code that was importing this.

Fixed.

1. twisted/words/protocols/jabber/xmpp_stringprep.py
   1. The entire code block beneath if sys.version_info < (2,3,2): can go, can't it?
   2. This seems to imply that some of the later code guarded by the if crippled: check can go as well.

Fixed.

1. twisted/trial/test/test_doctest.py
   1. I'd lean towards keeping test_id which is deleted in the branch. Perhaps DocTest behaves fine in all our currently supported Python versions, but keeping the check will let us know if that continues to be the case going forward.

Fixed.

1. The news fragment in topfiles/ doesn't follow the style guide. I might go for a .misc here anyway, for ease. However, if you like, you could make a news fragment for _each_ subproject and mention the main area of code cleanup for each subproject in each of those files.

Added a .misc file for each project that received changed.

Please review.

[Jean-Paul Calderone]

Thanks.

I think that, in allowInstancesOf, dropping Python 2.3 support means that the 'class' value being passed to allowTypes is superfluous.

This could be done in a new ticket?

Sure.

1. The second hunk in twisted/spread/jelly.py seems like an undesirable removal to me. It's providing potentially interesting information, I don't think it should be removed.
2. Also, there is code that can be deleted from twisted/spread/jelly.py that has been left in place. _set will never be set to None on a supported version of Python now; its definition is now redundant (always the same as the builtin set) and all of the code that is conditional on it not being None no longer needs to make the checks it makes.

I forced a ​build, hopefully it will be green. Please re-submit once the jelly changes have been made. Thanks again.

[Thijs Triemstra]

The windows build failed with:

[ERROR]
Traceback (most recent call last):
  File "C:\buildslave\twisted\winxp32-py2.7\Twisted\twisted\trial\runner.py", line 562, in loadPackage
    module = modinfo.load()
  File "C:\buildslave\twisted\winxp32-py2.7\Twisted\twisted\python\modules.py", line 383, in load
    return self.pathEntry.pythonPath.moduleLoader(self.name)
  File "C:\buildslave\twisted\winxp32-py2.7\Twisted\twisted\python\_reflectpy3.py", line 262, in namedAny
    topLevelPackage = _importAndCheckStack(trialname)
  File "C:\buildslave\twisted\winxp32-py2.7\Twisted\twisted\python\_reflectpy3.py", line 209, in _importAndCheckStack
    reraise(excValue, excTraceback)
  File "C:\buildslave\twisted\winxp32-py2.7\Twisted\twisted\conch\test\test_cftp.py", line 17, in <module>
    from twisted.conch import unix
  File "C:\buildslave\twisted\winxp32-py2.7\Twisted\twisted\conch\unix.py", line 17, in <module>
    import fcntl, tty
exceptions.ImportError: No module named fcntl

twisted.conch.test.test_cftp

[Thijs Triemstra]

Replying to exarkun:

1. The second hunk in twisted/spread/jelly.py seems like an undesirable removal to me. It's providing potentially interesting information, I don't think it should be removed.

Fixed.

1. Also, there is code that can be deleted from twisted/spread/jelly.py that has been left in place. _set will never be set to None on a supported version of Python now; its definition is now redundant (always the same as the builtin set) and all of the code that is conditional on it not being None no longer needs to make the checks it makes.

1. Removed the imports and checks, inc the deprecation warning, introduced 5 yrs ago (r22570).
2. decimal was introduced in 2.4 so that conditional import and it's code could be removed as well..

Also:

1. fixed the import in test_cftp and test_filetransfer, picked up by the win32 buildslave
2. restored the parserErrors exception in lore.lint
3. added a test for the exception handling in util.mergeFunctionMetadata. Also silenced the AttributeError when trying to merge to a read-only __module__.

Forced a new ​build.

[Tom Prince]

1. I don't think removing the serialization of sets.Set and sets.ImmutableSet is a compatible change. (Removing the *un*serialization is, since that code is never exercised)
2. set and frozenset in twisted.python.compat are no longer needed. The import logic for them should be removed here, and perhaps a ticket filed to deprecate them.
   • We don't need to point people at twisted.python.compat.set anymore (in jelly.py)
3. I don't think parserErrors should be *used* in lore, just defined.
   • A comment that it is there for compatibility, and perhaps a ticket to deprecate it might be warranted.

Please resubmit after addressing the above comments.

[Thijs Triemstra]

(In [37118]) address review comments. refs #5386

[Thijs Triemstra]

Replying to tom.prince:

1. I don't think removing the serialization of sets.Set and sets.ImmutableSet is a compatible change. (Removing the *un*serialization is, since that code is never exercised)

Restored the serialization.

1. set and frozenset in twisted.python.compat are no longer needed. The import logic for them should be removed here, and perhaps a ticket filed to deprecate them.

Removed the imports from test_jelly.

• We don't need to point people at twisted.python.compat.set anymore (in jelly.py)

Removed reference.

1. I don't think parserErrors should be *used* in lore, just defined.

Reverted back to SyntaxError.

Forced a new ​build.

[Tom Prince]

1. Please remove the conditional import of set and frozenset in twisted.python.compat (but leave the aliases there).
2. test_oldImmutableSets and test_oldSets have odd indentation. (reported ​here).

Please address the above issues then commit after a green buildbot run.

[Thijs Triemstra]

(In [37121]) address review comments. refs #5386

[Thijs Triemstra]

Replying to tom.prince:

1. Please remove the conditional import of set and frozenset in twisted.python.compat (but leave the aliases there).

Done. Opened #6297 to remove it's usage and deprecate it.

1. test_oldImmutableSets and test_oldSets have odd indentation. (reported ​here).

Fixed.

[Thijs Triemstra]

(In [37124]) Merge python-2.3-5386-2: Get rid of references and code specific to Python 2.3.

Author: rockstar, thijs Reviewer: jesstess, exarkun, tom.prince Fixes: #5386

[Tom Prince]

(In [37129]) Revert r37124: LookupsTestCase.test_importExceptions failing on python 3.3

test_importExceptions is failing on Python 3.3 only because a statement was modified to clean up Python 2.3 code which introduced an issue in test_reflectpy3.LookupsTestCase. This wasn't caught initially because the buildslave for python 3.3 wass down.

====================================================================== FAIL: test_importExceptions (twisted.python.test.test_reflectpy3.LookupsTestCase) test_importExceptions

---

Traceback (most recent call last):

File "twisted/trial/_synctest.py", line 328, in assertRaises

result = f(*args, kwargs)

File "twisted/python/_reflectpy3.py", line 273, in namedAny

obj = getattr(obj, n)

AttributeError: 'module' object has no attribute 'reflect_helper_IE'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "twisted/trial/_synctest.py", line 1185, in _run

runWithWarningsSuppressed(suppress, method)

File "twisted/python/util.py", line 1065, in runWithWarningsSuppressed

return f(*args, kwargs)

File "twisted/python/test/test_reflectpy3.py", line 290, in test_importExceptions

reflect.namedAny, "twisted.test.reflect_helper_IE")

File "software/twisted/svn/Twisted/trunk/twisted/trial/_synctest.py", line 335, in assertRaises

failure.Failure().getTraceback()))

twisted.trial.unittest.FailTest: <class 'AttributeError'> raised instead of ImportError:

Traceback (most recent call last):

File "twisted/trial/_synctest.py", line 1216, in _runFixturesAndTest

if self._run(suppress, todo, method, result):

File "twisted/trial/_synctest.py", line 1185, in _run

runWithWarningsSuppressed(suppress, method)

File "twisted/python/util.py", line 1065, in runWithWarningsSuppressed

return f(*args, kwargs)

File "twisted/python/test/test_reflectpy3.py", line 290, in test_importExceptions

reflect.namedAny, "twisted.test.reflect_helper_IE")

--- <exception caught here> ---

File "twisted/trial/_synctest.py", line 328, in assertRaises

result = f(*args, kwargs)

File "twisted/python/_reflectpy3.py", line 273, in namedAny

obj = getattr(obj, n)

builtins.AttributeError: 'module' object has no attribute 'reflect_helper_IE'

Reopens: #5386 Refs: #6298

[Thijs Triemstra]

(In [37190]) fix tests for py3.3, refs #5386

[Thijs Triemstra]

Forced a new ​build with the fix from #6298. Unfortunately the 3.3 slave is down atm..

[Tom Prince]

It is now back up, and appears to be green.

[Thijs Triemstra]

Replying to tom.prince:

It is now back up, and appears to be green.

Was this a review cause the keyword's still there.. :)

[Tom Prince]

1. twisted.python.test.util.TestMergeFunctionMetadata.test_moduleBuiltinTypeNotMerged doesn't seem to make sense. Please file a ticket.
2. twisted/spread/jelly.py: In the docstring that references python2.3, classobj should be the primary name, and class mentioned parentheically (reversing the current situation).
   • On further considertion, there is some weirdness here, so I'll file a ticket to deal with it.
3. doc/core/howto/debug-with-emacs.xhtml has a reference to 2.3. It probably wants to be replaced with python -m pdb
4. doc/core/development/policy/coding-standard.xhtml references set/frozenset in compat. It doesn't appear that there are any modules that currently fall in to that category. Thus, we probably want to keep that example, but mention that it is historical.

Please resubmit for review after addressing 1+3+4.

[Thijs Triemstra]

Replying to tom.prince:

1. twisted.python.test.util.TestMergeFunctionMetadata.test_moduleBuiltinTypeNotMerged doesn't seem to make sense. Please file a ticket.

Removed the test. It was to catch an uncovered line or 2 in mergeFunctionMetadata but it's not that important for a new ticket i think.

3. doc/core/howto/debug-with-emacs.xhtml has a reference to 2.3. It probably wants to be replaced with python -m pdb

Fixed.

4. doc/core/development/policy/coding-standard.xhtml references set/frozenset in compat. It doesn't appear that there are any modules that currently fall in to that category. Thus, we probably want to keep that example, but mention that it is historical.

Rewrote that part.

1. There were also 2 fallbacks used for collections.deque imports that were also removed
2. the crippled attribute in twisted/words/protocols/jabber/xmpp_stringprep.py was public and since there might be people out there using it it was restored and deprecated instead.

Forced a new ​build.

[Tom Prince]

1. twisted.lore.lint.parserErrors should be deprecated.

[Tom Prince]

2. The various deprecated attributes should be mentioned in the appropriate news files.

And, please merge, after doing that (with test).

[Thijs Triemstra]

(In [37860]) Branching to 'python-2.3-5386-4'

[Thijs Triemstra]

(In [37861]) merge forward, resolve conflicts. refs #5386

[Thijs Triemstra]

(In [37914]) Address review comments, refs #5386

[Thijs Triemstra]

(In [37915]) Merge python-2.3-5386-4: Get rid of references and code specific to Python 2.3.

Author: thijs, rockstar Reviewer: jesstess, exarkun, tom.prince Fixes: #5386