Opened 4 weeks ago

Last modified 4 weeks ago

#9730 defect new

Twisted crashes when parsing XMPP XML stream with Python 3.8

Reported by: Jérôme Poisson Owned by:
Priority: highest Milestone: Python 3.8 Support
Component: words Keywords: review
Cc: Branch:
Author:

Description

Hello,

When parsing an XML stream with Python 3.8, twisted.words.xish.domish import ExpatElementStream is crashing with the following traceback:

        Traceback (most recent call last):
          File "/home/goffi/venvs/sat_principal/lib/python3.8/site-packages/twisted/python/log.py", line 103, in callWithLogger
            return callWithContext({"system": lp}, func, *args, **kw)
          File "/home/goffi/venvs/sat_principal/lib/python3.8/site-packages/twisted/python/log.py", line 86, in callWithContext
            return context.call({ILogContext: newCtx}, func, *args, **kw)
          File "/home/goffi/venvs/sat_principal/lib/python3.8/site-packages/twisted/python/context.py", line 122, in callWithContext
            return self.currentContext().callWithContext(ctx, func, *args, **kw)
          File "/home/goffi/venvs/sat_principal/lib/python3.8/site-packages/twisted/python/context.py", line 85, in callWithContext
            return func(*args,**kw)
        --- <exception caught here> ---
          File "/home/goffi/venvs/sat_principal/lib/python3.8/site-packages/twisted/internet/posixbase.py", line 614, in _doReadOrWrite
            why = selectable.doRead()
          File "/home/goffi/venvs/sat_principal/lib/python3.8/site-packages/twisted/internet/tcp.py", line 243, in doRead
            return self._dataReceived(data)
          File "/home/goffi/venvs/sat_principal/lib/python3.8/site-packages/twisted/internet/tcp.py", line 249, in _dataReceived
            rval = self.protocol.dataReceived(data)
          File "/home/goffi/venvs/sat_principal/lib/python3.8/site-packages/twisted/words/xish/xmlstream.py", line 93, in dataReceived
            self.stream.parse(data)
          File "/home/goffi/venvs/sat_principal/lib/python3.8/site-packages/twisted/words/xish/domish.py", line 797, in parse
            self.parser.Parse(buffer)
          File "/build/python/src/Python-3.8.0/Modules/pyexpat.c", line 407, in StartElement
            
          File "/home/goffi/venvs/sat_principal/lib/python3.8/site-packages/twisted/words/xish/domish.py", line 810, in _onStartElement
            for k, v in attrs.items():
        builtins.RuntimeError: dictionary keys changed during iteration

note that this was working as expected with Python 3.7.

I've made a small script which reproduce the issue, the value in buffer comes from real XMPP stream:

#!/usr/bin/env python3.8

from twisted.words.xish.domish import ExpatElementStream


buf = (b"<?xml version='1.0'?><stream:stream from='sat-pubsub.tazar2.int' xmlns:stream='"
       b"http://etherx.jabber.org/streams' id='c2f2fecf-fb57-4a30-bbfe-0e9b5a587a8e' "
       b"xmlns='jabber:component:accept' xml:lang='en'>")

elem_stream = ExpatElementStream()
elem_stream.parse(buf)
print("test passed")

This makes Twisted unusable for XMPP with Python 3.8, so I set the priority as highest.

Thanks!

Change History (6)

comment:1 Changed 4 weeks ago by Jérôme Poisson

Summary: Twisted crash when parsing XML stream with Python 3.8Twisted crashes when parsing XML stream with Python 3.8

comment:2 Changed 4 weeks ago by Jérôme Poisson

Summary: Twisted crashes when parsing XML stream with Python 3.8Twisted crashes when parsing XMPP XML stream with Python 3.8

comment:3 Changed 4 weeks ago by hawkowl

Milestone: Python 3.8 Support

Hi, thanks for the bug report. We don't officially support Python 3.8 yet, so I've added it to the 'support 3.8' milestone. I'd suggest using 3.7 for the time being :)

comment:4 Changed 4 weeks ago by Jérôme Poisson

comment:5 Changed 4 weeks ago by Jérôme Poisson

Keywords: review added

comment:6 Changed 4 weeks ago by Jérôme Poisson

Hello hawkowl, thanks for the feedback, I'll apply the fix in my own code until a Python 3.8 compatible version of Twisted is available, as it seems to be enough for me. I hope the PR I've made is fine.

Note: See TracTickets for help on using tickets.