[Twisted-Python] Windows spawnProcess - Child process inherits files and sockets (POSIX does not)

exvito here ex.vitorino at gmail.com
Thu Apr 7 11:09:04 MDT 2016


On Thu, Apr 7, 2016 at 4:57 AM, Oliver Palmer <oliverpalmer at opalmer.com> wrote:

> Owner of pywincffi here, I'd certainly welcome a PR or two for pywincffi
> with the necessary functions/constants/etc so you don't have to manage that
> code and I'd be happy to help write it too.  I think the consensus is
> Twisted is going to eventually replace calls into pywin32 with calls into
> pywincffi rather than implement all of that code inside of Twisted itself.
> I've already started doing this in a couple of places,
> twisted.python.lockfile being the one I'm actively working on because it's
> simpler to start with.  But the code inside of dumbwin32proc.py and
> _win32handleinherit.py are both high on my list to convert too so it
> probably makes sense that we work on this together if you're open to it.

Certainly. Thanks for reaching out.

> Looking at your code, some of it could be put into pywincffi already.  It
> would need more tests and some additional code so the API calls are closer
> to what's already in the project (type checking, default arguments,
> documentation, etc) but overall it seems like you've already done the major
> work of understanding how it all fits together.  The other advantages of
> putting this code into pywincffi is testing and releases are easier because
> the project is using AppVeyor to test all PRs and build the wheel files for
> most major Python versions including both 32 and 64 bit variants.  From
> Twisted's perspective, it's just a dependency on another library.

Agreed. I'll be happy to follow your guidance in that.

>> Just for kicks I compared the performance of the ctypes vs cffi
>> implementation: ...
>
> Have you tried a comparison between out-of-line modules and those using
> dlopen?  I imagine they'd end up being pretty similar in the end performance
> wise but I am a little curious.  In pywincffi I started out using dlopen but
> moved away from it because I needed to write some extra code which couldn't
> be included from a header.  The other advantage I saw is that you don't have
> to rely on the DLL being present and/or Windows being able to locate it so
> you can include code which might only be available if you have some extra
> library installed.

I did not and I confess that I haven't fully grasped (yet) the
different cffi "approaches", if I may call them so.

>> It is mostly based on SysInternals information at
>> http://forum.sysinternals.com/howto-enumerate-handles_topic18892.html. There
>> are many other references across the web to those undocumented options and
>> data structures. But none of those come from formal Microsoft documents,
>> that I could find.
>
> Some of the people on that site either have contacts within Microsoft or
> have worked for Microsoft at one point so I usually trust what's there if
> it's the only source.  The other place I often look is the ReactOS project
> where they've managed to reverse engineer quite a bit of the Windows kernel
> which can either hint at the info you need or validate what you already
> know.

AFAICT, SysInternals was bought by / integrated with Microsoft: tteir
tools are now available under Microsoft domains (example: Process
Explorer at https://technet.microsoft.com/en-us/sysinternals/processexplorer.aspx).

>> Wrapping up: I'm really not sure how to more forward with this: not only
>> does pywin32 not provide the needed APIs, but also one of those APIs --
>> documented -- is being used in an undocumented fashion.
>
> IMHO (again, with some bias), I think implementing the calls you need in
> pywincffi is the first step.  If the calls are undocumented it would also be
> a good place to do the necessary research, testing and development I think
> in isolation from Twisted itself so it's clear we're going in the right
> direction.  Once that's done a patch set for Twisted, which calls into
> pywincffi, can be opened and tested across the supported platforms.  This
> makes the patch set smaller but also makes it easier to understand what if
> anything the new code breaks.

Sounds like a sane and safe approach.

> Regardless, even if you don't want to go the route of putting this into
> pywincffi thanks for working on this because it helps in some of the work
> I'm doing too.

Thank you to, again. I will issue PRs against pywincffi such that the
APIs are available. They most probably won't be up to pywincffi's
standards/requirements but I expect we can cooperate under that
context and go from there.
-- 
exvito




More information about the Twisted-Python mailing list