[Twisted-web] Question regarding custom contextFactory for twisted.web.client.Agent

[Jason Litzinger]

> This sounds eminently sensible, and in fact sounds like Twisted, or Treq at least, really ought to do something like this on our end with some kind of small LRU cache, keeping the most-used N hostnames (where N defaults to some small number, say, 20).  I'm surprised to hear it is such a big optimization, but surprises like that are entirely the point of performance testing!
Thanks for taking a look!

I like the idea of an LRU for the last N hosts.  Given I have something
prototyped that I will expand on, that isn't much of a stretch.  If
you think it might be useful to Twisted then I can try to iterate the
patch until it is either rejected or accepted (following the well
documented steps for contributing of course), thoughts?

If it fits more in treq then I'm more than happy to share the code, but,
since I'm not currently using treq the amount of effort I can
realistically put towards it isn't as high, though that project is very
appealing.

On the size of the performance increase -- I was surprised as well, and
there's always the possibility I made an error, but it was consistent
across the five tests I ran.  Regardless, I'll be doing more profiling as
I work toward the final implementation.
> 
> Nevertheless, thanks for asking!  I really wish people would ask questions about client TLS more often before doing something potentially insecure :).  In this case, the client connection creator is used to produce client connections anyway, and the connections are not shared, so there should be no worrisome mingling of state between connections.
Agree.  I've come to believe that assumptions mixed with TLS/security
related issues are my worst enemy.

-Jason

* Completely unrelated - I still owe on my committment to ticket
  6597...yeah, life happened.  Luckily I saved my notes now that I'm
  finding some time here and there.