[Twisted-Python] _dumbwin32proc and PIPE_NOWAIT

Manlio Perillo manlio_perillo at libero.it
Sat Sep 30 07:06:03 EDT 2006

glyph at divmod.com ha scritto:
> [...]
> "threads are bad"

But they are used in twisted.enterprise.adbapi, and as I can see several
developers prefer to use threads to interact with "foreign" APIs instead
of develope native asyncronous API.

>> The solution is simple.
>> Windows *do not* supports POSIX ;-).
> It does, however, support standard IO, at least well enough for a large
> chunk of applications.  Although it's not as well supported as on
> "POSIX" platforms, there are plenty of programs that produce output
> which Twisted can happily parse semi-asynchronously using the current
> strategy.  Why on earth would you want to remove it?  It serves a
> purpose (cross-platform multi-process communication and control), it
> works, it's tested.

This is true, for sockets it works well, but for pipes it uses an
inefficient implementation.

> [...]

>> One needs to write a specializd abstract.FileDescriptor class, that use
>> the Overlapped structure for I/O.
> You mean like twisted.internet.iocpreactor.abstract.ConnectedSocket? 
> Have you even read this code?

Yes, but as the name suggests it works only for Sockets(?).

The problem is simple:
if I want to do asyncronous I/O with Windows I don't have an unique

Only sockets are really integrated with the reactor.
serialport has some specialized code for win32eventreactor integration,
but if I want to use, as an example, named pipe, I have to write my own
(but it seems this is an issue for broken POSIX implementations, too).

>> [snip more oblique allusions to various win32 APIs]
> Frankly, I don't think you know what you're talking about.  

This is very likely ;-)

> If you know
> enough to make iocpreactor, or even win32eventreactor, work on Windows
> with SSL and GUI support, great, do that and contribute it, and we can
> discuss the patch.  If not, we're not going to break the *only* working,
> tested Windows reactor because you think one day it should be different.

Is win32eventreactor not working and not tested?
You did not mention it in the "removing unsupported reactors in twisted"

>> This should allow one to easily use asyncronous I/O support of Window,
>> not only for sockets, but on all (not many...) 'handles' that support
>> the asyncronous I/O API.
> If it's easy, feel free to do it :).

I have write it, now stdio works ;-).

Unfortunately it is not very efficient since:
- it uses _pollingfile (not a real porblem)
- it reads raw keyboard input and have to do all low level stuff like
  '\b' handling, '\r' conversion, and so

Here is the main code.
I suspect such a thing will never be accepted into Twisted ;-):

class _PollableConsoleReader(_PollableResource):
    # XXX TODO add support for Window events (resize)

    def __init__(self, con, receivedCallback, lostCallback):
        self.con = con
        self.receivedCallback = receivedCallback
        self.lostCallback = lostCallback
        self.cp = "cp%d" % win32console.GetConsoleCP()

        # We need this
        self._stdout = win32console.GetStdHandle(

    def checkWork(self):
        finished = 0
        fullDataRead = []

        info = self._stdout.GetConsoleScreenBufferInfo()
        Y = info["CursorPosition"].Y

        while 1:
                n = self.con.GetNumberOfConsoleInputEvents()
                if n == 0:

                records = self.con.ReadConsoleInput(n)

                # We need to process input
                # XXX check me
                # XXX TODO rewrite in C(?), now this is very inefficient
                for record in records:
                    if record.EventType != win32console.KEY_EVENT \
                            or not record.KeyDown:

                    char = record.Char
                    if char == '\b':
                        # We need to handle this
                        info = self._stdout.GetConsoleScreenBufferInfo()
                        rowSize = info["MaximumWindowSize"].X
                        cursorPosition = info["CursorPosition"]

                        # Move the cursor
                        if cursorPosition.X == 0:
                            if cursorPosition.Y > Y:
                                cursorPosition.Y -= 1
                                cursorPosition.X = rowSize - 1
                            cursorPosition.X -= 1

                        self._stdout.WriteConsoleOutputCharacter(' ',
                    elif char == '\0':
                    elif char == '\r':
                        char = '\n'

                    char = char * record.RepeatCount
                    data = char.encode(self.cp)
                    self._stdout.WriteConsole(data) # Do echo

            except pywintypes.error:
                # XXX this should not happen
                finished = 1

        dataBuf = ''.join(fullDataRead)
        if dataBuf:
        if finished:

        return len(dataBuf)


The ConsoleWriter does not need to support asyncronous I/O, so it is a
simple wrapper for ConsoleWrite.

This code still have some problems[1]...
For now I have added this to _pollingfile module.

The _win32stdio module can simply check if the stdin is attached to a
terminal, so it (hopefully) can support both pipes and consoles.

Many things can be added, like an emulation of POSIX terminal, callbacks
for resize events, support for focus in/out, colored output, history
editing, ...

How can be tested such a thing?

Regards  Manlio Perillo

More information about the Twisted-Python mailing list