[Twisted-Python] tor hidden service endpoint parser returns a deferred

Glyph glyph at twistedmatrix.com
Mon May 5 23:31:31 MDT 2014


On May 5, 2014, at 6:52 PM, David Stainton <dstainton415 at gmail.com> wrote:

> OK... I decided that txtorcon's endpoint constructor should only take
> string arguments besides the reactor arg...
> because serverFromString passes only str args to the endpoint constructor.

I strongly suggest that this is wrong.

The endpoint constructor should do _construction_.  The endpoint parser should do _parsing_.  The task of parsing is of taking strings and producing meaningful values.

You may want to have the endpoint parser actually live in a separate method in txtorcon rather than putting it into the plugin itself, and leave the plugin simply the task of plugging in, but the parsing method should not be called __init__.

(Also, don't use asserts.  This is probably indicative of a weakness in the parsing API we've provided, but you should likely raise something like UsageError.)

> I wrote the endpoint parser to be as simple as possible.
> The endpoint's `listen` method now handles the tor configuration and
> launch logic... and returns a deferred which will
> fire with an IListenPort.

Great.

> The pip method definitely works... my code works but prints traceback
> with this error:
> "Unexpected error while writing cache file"
> Do you as well?
> 

What exactly is 'usewithtor'?

> usewithtor pip install twisted



> Running the example code:
> cd examples
> python launch_tor_endpoint.py
> Unexpected error while writing cache file
> Traceback (most recent call last):
>  File "/home/human/virtenv-txtorcon/local/lib/python2.7/site-packages/twisted/internet/endpoints.py",
> line 1465, in serverFromString
>    return _serverFromStringLegacy(reactor, description, _NO_DEFAULT)
>  File "/home/human/virtenv-txtorcon/local/lib/python2.7/site-packages/twisted/internet/endpoints.py",
> line 1393, in _serverFromStringLegacy
>    nameOrPlugin, args, kw = _parseServer(description, None, default)
>  File "/home/human/virtenv-txtorcon/local/lib/python2.7/site-packages/twisted/internet/endpoints.py",
> line 1380, in _parseServer
>    for plugin in getPlugins(IStreamServerEndpointStringParser):
>  File "/home/human/virtenv-txtorcon/local/lib/python2.7/site-packages/twisted/plugin.py",
> line 209, in getPlugins
>    allDropins = getCache(package)
> --- <exception caught here> ---
>  File "/home/human/virtenv-txtorcon/local/lib/python2.7/site-packages/twisted/plugin.py",
> line 181, in getCache
>    dropinPath.setContent(pickle.dumps(dropinDotCache))
> exceptions.AttributeError: 'ZipPath' object has no attribute 'setContent'
> I have set up a hidden service, advertised at: <<class
> 'twisted.internet.tcp.Port'> of twisted.web.server.Site on 55141>

This traceback is actually harmless, if annoying.  It's trying to generate the plugin cache, which is explained here:

<https://twistedmatrix.com/documents/current/core/howto/plugin.html#auto3>

> http://mqrcnytlnh4xynmh.onion:80
> locally listening on IPv4Address(TCP, '127.0.0.1', 55141)
> 
> and txtorcon's setup.py output:
> python setup.py install


> running bdist_egg
 ^ This right here is your problem.

There are a couple of solutions, in order of increasing complexity:

Don't build eggs.  Why are you building eggs?  Eggs are unfortunately a huge problem magnet.  I don't think 'pip install' builds eggs; I don't manually run setup.py anywhere any more because I assume pip understands these concerns better than I do.
Set zip_safe in your setup.py to False, which will turn your egg into a directory rather than a directory.
Figure out a way to hook the _build_ step specifically so you can run getPlugins() to generate the dropin cache before zipping up your zip file.  Twisted plugins actually do work within zip files, but nobody has bothered working out the setuptools incantation to get them to generate at the appropriate time for building an egg.  This is extra tricky because you only want to generate this dropin.cache file if the dropin is going to live inside an egg; otherwise, it would conflict with the communal dropin.cache file generated at install time.  This option will be the most confusing and frustrating for you, but I hope you choose it, because it would be GREAT to have a Right Way™ to address this problem in a Twisted plugin project's setup.py :).

> zip_safe flag not set; analyzing archive contents...

It's probably best to set zip_safe one way or the other though, even if you want to set it to 'true'.  Letting setuptools guess like this will, unsurprisingly, get you unpredictable results.

-g
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20140505/f17e4a33/attachment.html>


More information about the Twisted-Python mailing list