[Twisted-Python] _dumbwin32proc and PIPE_NOWAIT

[Manlio Perillo]

glyph at divmod.com ha scritto:
> [...]
> 
> "threads are bad"
> 

But they are used in twisted.enterprise.adbapi, and as I can see several
developers prefer to use threads to interact with "foreign" APIs instead
of develope native asyncronous API.

>> The solution is simple.
>> Windows *do not* supports POSIX ;-).
> 
> It does, however, support standard IO, at least well enough for a large
> chunk of applications.  Although it's not as well supported as on
> "POSIX" platforms, there are plenty of programs that produce output
> which Twisted can happily parse semi-asynchronously using the current
> strategy.  Why on earth would you want to remove it?  It serves a
> purpose (cross-platform multi-process communication and control), it
> works, it's tested.
> 

This is true, for sockets it works well, but for pipes it uses an
inefficient implementation.

> [...]

>> One needs to write a specializd abstract.FileDescriptor class, that use
>> the Overlapped structure for I/O.
> 
> You mean like twisted.internet.iocpreactor.abstract.ConnectedSocket? 
> Have you even read this code?
> 

Yes, but as the name suggests it works only for Sockets(?).

The problem is simple:
if I want to do asyncronous I/O with Windows I don't have an unique
'interface'.

Only sockets are really integrated with the reactor.
serialport has some specialized code for win32eventreactor integration,
but if I want to use, as an example, named pipe, I have to write my own
support
(but it seems this is an issue for broken POSIX implementations, too).


>> [snip more oblique allusions to various win32 APIs]
> 
> Frankly, I don't think you know what you're talking about.  

This is very likely ;-)

> If you know
> enough to make iocpreactor, or even win32eventreactor, work on Windows
> with SSL and GUI support, great, do that and contribute it, and we can
> discuss the patch.  If not, we're not going to break the *only* working,
> tested Windows reactor because you think one day it should be different.
> 

Is win32eventreactor not working and not tested?
You did not mention it in the "removing unsupported reactors in twisted"
thread.

>> This should allow one to easily use asyncronous I/O support of Window,
>> not only for sockets, but on all (not many...) 'handles' that support
>> the asyncronous I/O API.
> 
> If it's easy, feel free to do it :).
> 

I have write it, now stdio works ;-).

Unfortunately it is not very efficient since:
- it uses _pollingfile (not a real porblem)
- it reads raw keyboard input and have to do all low level stuff like
  '\b' handling, '\r' conversion, and so


Here is the main code.
I suspect such a thing will never be accepted into Twisted ;-):


class _PollableConsoleReader(_PollableResource):
    # XXX TODO add support for Window events (resize)
    implements(IPushProducer)

    def __init__(self, con, receivedCallback, lostCallback):
        self.con = con
        self.receivedCallback = receivedCallback
        self.lostCallback = lostCallback
        self.cp = "cp%d" % win32console.GetConsoleCP()

        # We need this
        self._stdout = win32console.GetStdHandle(
	                   win32console.STD_OUTPUT_HANDLE
                        )

    def checkWork(self):
        finished = 0
        fullDataRead = []

        info = self._stdout.GetConsoleScreenBufferInfo()
        Y = info["CursorPosition"].Y

        while 1:
            try:
                n = self.con.GetNumberOfConsoleInputEvents()
                if n == 0:
                    break

                records = self.con.ReadConsoleInput(n)

                # We need to process input
                # XXX check me
                # XXX TODO rewrite in C(?), now this is very inefficient
                for record in records:
                    if record.EventType != win32console.KEY_EVENT \
                            or not record.KeyDown:
                        continue

                    char = record.Char
                    if char == '\b':
                        # We need to handle this
                        info = self._stdout.GetConsoleScreenBufferInfo()
                        rowSize = info["MaximumWindowSize"].X
                        cursorPosition = info["CursorPosition"]

                        # Move the cursor
                        if cursorPosition.X == 0:
                            if cursorPosition.Y > Y:
                                cursorPosition.Y -= 1
                                cursorPosition.X = rowSize - 1
                        else:
                            cursorPosition.X -= 1

                        self._stdout.SetConsoleCursorPosition(
                            cursorPosition
                         )
                        self._stdout.WriteConsoleOutputCharacter(' ',
                                                      cursorPosition)
                        continue
                    elif char == '\0':
                        continue
                    elif char == '\r':
                        char = '\n'

                    char = char * record.RepeatCount
                    data = char.encode(self.cp)
                    self._stdout.WriteConsole(data) # Do echo

                    fullDataRead.append(data)
            except pywintypes.error:
                raise
                # XXX this should not happen
                finished = 1
                break

        dataBuf = ''.join(fullDataRead)
        if dataBuf:
            self.receivedCallback(dataBuf)
        if finished:
            self.cleanup()

        return len(dataBuf)

   ...

The ConsoleWriter does not need to support asyncronous I/O, so it is a
simple wrapper for ConsoleWrite.

This code still have some problems[1]...
For now I have added this to _pollingfile module.

The _win32stdio module can simply check if the stdin is attached to a
terminal, so it (hopefully) can support both pipes and consoles.


Many things can be added, like an emulation of POSIX terminal, callbacks
for resize events, support for focus in/out, colored output, history
editing, ...


[1]
How can be tested such a thing?



Regards  Manlio Perillo