Unicode header in twisted.web.wsgi causes "undefined" behavior

[etrepum]

If the startResponse function in a wsgi app is called with a header key or value that is unicode, then the response ends up as UCS2 or UCS4 garbage. I believe the reason for this is that deep down in twisted.internet.tcp.Connection a buffer object is created from the unicode object.

>>> buffer(u'die in a fire')[:]
'd\x00i\x00e\x00 \x00i\x00n\x00 \x00a\x00 \x00f\x00i\x00r\x00e\x00'

I would expect that unicode in the headers either get coerced to ascii first, or raise an exception. Coerced to ascii would be more consistent with other WSGI containers, such as paster, but less consistent with the unicode philosophy in Twisted. The behavior is undefined in the WSGI spec.

It appears there is an explicit check for unicode in twisted.internet.abstract.FileDescriptor.write but not writeSequence, which is what ends up getting used in this case.

[exarkun]

See also #3896