[Twisted-web] Question regarding custom contextFactory for twisted.web.client.Agent

Jason Litzinger jlitzingerdev at gmail.com
Fri May 12 14:56:51 MDT 2017


Hello,

I have an optimization question in the realm of "before I do something
foolish..." regarding the contextFactory passed to twisted.web.client.Agent.

Background:

I have (several) twisted applications running on a Cortex-A5 system, one of
which interfaces with a web service using the twisted.web.client.Agent
API*.  The
problem I've observed is that requests to this web service result in a higher
than desired CPU load.  To understand the source of the load I isolated the
server interactions and profiled.

Investigation:

The profile data revealed the load in question stemmed from TLS related
calls, specifically optionsForClientTLS, which is called once per connection.
In looking (briefly) at what optionsForClientTLS does, it seemed
(perhaps wrongly)
that there was an opportunity for optimization in my specific case.

The optimization is to create a custom contextFactory that caches the connection
creator since the host is not changing.  I prototyped an implementation and the
load was reduced by a factor of ten.

Question:

Is it safe/sane to reuse a connection creator if the host to which I'm
connecting is
not going to change?

Thanks,
-Jason Litzinger

* Technically this code is inside an open source third party SDK, but were I in
  their shoes I'd write it the same way, so I've no issues with their
implementation.



More information about the Twisted-web mailing list