[Twisted-Python] Direct access to main server class

[glyph at divmod.com]

On Tue, 29 Aug 2006 02:55:03 +0200, Terry Jones <terry at jon.es> wrote:

>It does make me wonder though, still. What if you're writing a Twisted app
>that you intend to license for commercial companies to run on their
>intranets: You install the Python source and presumably the unit test code
>gets run at some point, and maybe remains on the server. Then you're
>showing the world how to access the innards of your service without going
>through the approved authentication. Same comments apply to simply putting
>the code online and letting people see how to get in to any instance they
>can get local (user) access to. What can be done to prevent this? The most
>obvious thing seems to be to verify the uid of the process trying to create
>an authenticated user, and have both your unit tests and twistd run under
>this uid.  If you don't do something about this, a normal user on the
>system can simply import your module, create an instance, and start making
>method calls (possibly unsuccessfully, due to file system permissions on
>databases, etc). Maybe they wont get far, but it's an issue: especially if
>the system has been built to allow this, and the unit tests show precisely
>how to pull it off.

If your system is so insecure that anyone with some know-how can run arbitrary Python code in it, it is a pretty straightforward step to extract all the source code from the server directly.  Consider this simple Python quine:

    # quine.py
    print file(__file__).read()

If you think you can hide Python code by compiling it to bytecodes, have a look at http://www.crazy-compilers.com/decompyle/ -- that should quickly disabuse you of that notion.

Moreover, any security scheme that relies, in any way, upon the fact that your code is hidden is broken by design.  From wikipedia: "computer science professors consider security through obscurity so obviously wrong that the matter doesn't need proof or study".

However, even disregarding that, _some_ code in your system is instantiating authenticated users and calling methods on them.  Assuming that your code is packaged and distributed (tests included, so as to make testing of deployments easier), anyone who can read the tests can read the rest of the code, and discover the same entrypoints.

So, I disagree in the strongest possible terms: it *isn't* an issue, and nobody should ever consider it as such when writing tests.  The correct way to write unit tests is one unit at a time, and it should not be polluted by misguided security concerns.

Apologies if this sounds too harsh.  I'm not just trolling, I promise :).  We have a lot of tests in the Twisted suite which bend over backwards to instantiate things through obtuse higher-than-unit-level interfaces (which often aren't really themselves public), rather than testing functionality directly.  These tests are all overcomplex, error prone, and generally contain race conditions and fail intermittently for unrelated reasons.

It is a systematic problem which we have spent literally years fixing, and we're not done.  JP Calderone, Chris Armstrong and I have almost daily conversations now about how to reduce this coupling and "heal" the Twisted suite so that tests will fail only when they're failing.  In other words, make the tests more "unit-y" and less like integration tests.  The idea you're proposing here seems to be enthusiastically attempting to doom a future project to a worse version of this syndrome :).  Programs like Twisted which interface heavily with the OS are hard enough to write good test for when you're only worrying about legitimate difficulties with the tests... if you start introducing concerns like "maybe we're revealing too much in our test suite" it becomes impossible.