[Twisted-Python] tor hidden service endpoint parser returns a deferred

David Stainton dstainton415 at gmail.com
Tue May 6 05:38:13 MDT 2014


> The endpoint constructor should do _construction_.  The endpoint parser
> should do _parsing_.  The task of parsing is of taking strings and producing
> meaningful values.

Ah yeah I see what you mean. OK... I changed it:
https://github.com/david415/txtorcon/commit/1e96d550c40bef1be1b45c3c975c0da0bf6a2949

> You may want to have the endpoint parser actually live in a separate method
> in txtorcon rather than putting it into the plugin itself, and leave the
> plugin simply the task of plugging in, but the parsing method should not be
> called __init__.

Done.

> (Also, don't use asserts.  This is probably indicative of a weakness in the
> parsing API we've provided, but you should likely raise something like
> UsageError.)

OK... I changed that part too.

> What exactly is 'usewithtor'?
>
> usewithtor pip install twisted

It's torsocks. It performs an LD_PRELOAD hack to force programs to use tor.
Here's some torsocks info:

https://trac.torproject.org/projects/tor/wiki/doc/torsocks
https://gitweb.torproject.org/torsocks.git
https://lists.torproject.org/pipermail/tor-dev/2014-April/006649.html

Torsocks is useful... but a terrible hack... and this reminds me:

I'm hoping that the effort to make Tahoe-LAFS more Tor-friendly will
result in well documented
solutions. I would hope that this would make it easier for other
developers to make their twisted apps more Tor-friendly as well.
I do not think we have all the solution details worked out yet...

We'd like to stop using torsocks with Tahoe-LAFS and instead use
twisted endpoints...
which is why I ported the Foolscap client side to use twisted endpoints.
(str4d already ported the Foolscap server side to use endpoints)

https://tahoe-lafs.org/trac/tahoe-lafs/ticket/517

This trac ticket discusses various ideas to make Tahoe more Tor friendly...
including this endpoint related idea that I've been thinking about
lately in trac comment 16
(https://tahoe-lafs.org/trac/tahoe-lafs/ticket/517#comment:16) by leif
:

"""My current thought is that Tahoe should have an "always_use_tor"
option which causes all furls to be rewritten as tor: endpoints before
being passed to foolscap. If this option is not enabled, tor
connections will be made if a tor endpoint is used, but not otherwise.
So, users of mixed tor/non-tor grids can leave it disabled if they
prefer faster connections to non-tor nodes."""

This is necessary for a tahoe client wanting to use a public grid over
tor... because the grid's introducer node is going to
announce tcp endpoints not tor endpoints.

However I was thinking that the "only_tor" mode for tahoe (mentioned
here https://tahoe-lafs.org/trac/tahoe-lafs/ticket/517#comment:15 )
could be implemented in tahoe... it could filter endpoint descriptors
by calling clientFromString or serverFromString and then checking the
object type.... or is this a terrible way to do it? I'm not sure. The
thing is... tahoe wouldn't normally
know anything about endpoints because it uses Foolscap which could use
endpoints (it does in my branch
https://github.com/david415/foolscap/tree/endpoint_descriptors_server2).

> Don't build eggs.  Why are you building eggs?  Eggs are unfortunately a huge
> problem magnet.  I don't think 'pip install' builds eggs; I don't manually
> run setup.py anywhere any more because I assume pip understands these
> concerns better than I do.

Ah OK yeah it works just fine when installed with pip.

> Figure out a way to hook the _build_ step specifically so you can run
> getPlugins() to generate the dropin cache before zipping up your zip file.
> Twisted plugins actually do work within zip files, but nobody has bothered
> working out the setuptools incantation to get them to generate at the
> appropriate time for building an egg.  This is extra tricky because you only
> want to generate this dropin.cache file if the dropin is going to live
> inside an egg; otherwise, it would conflict with the communal dropin.cache
> file generated at install time.  This option will be the most confusing and
> frustrating for you, but I hope you choose it, because it would be GREAT to
> have a Right Way™ to address this problem in a Twisted plugin project's
> setup.py :).

OK... I'll try to get it working that way... but for now a pip install
seems to work fine.

> zip_safe flag not set; analyzing archive contents...
>
>
> It's probably best to set zip_safe one way or the other though, even if you
> want to set it to 'true'.  Letting setuptools guess like this will,
> unsurprisingly, get you unpredictable results.



More information about the Twisted-Python mailing list