validate against platform-trusted certificate authorities by default for HTTPS URLs in twisted.web.client.Agent

[Glyph]

HTTPS client connections from Agent.request should use the platform's native list of certificate authorities as the trust root by default.

This ticket is only for fixing Agent to actually point at the certificate store API we come up with; there is another ticket for actually implementing that store which is really the hard part here.

[Itamar Turner-Trauring]

I am skeptical that this will happen anytime soon. A more realistic, short term goal is to do what requests does: ship the Mozilla CA pack, with a integration point which packagers can point at their own local CA storage location.

[Glyph]

Replying to itamar:

I am skeptical that this will happen anytime soon. A more realistic, short term goal is to do what requests does: ship the Mozilla CA pack, with a integration point which packagers can point at their own local CA storage location.

I agree, as my recent comments on the ticket related to hostname verification suggest; maybe we should have an interim ticket for doing that.

However, I think it's important to have realistic expectations here: aside from Linux, as I noted on the ticket for the hard part of this problem "local CA storage" is a database with an API to query it, not a filesystem location. Tools to manage trust don't necessarily keep a filesystem directory that we can list in sync.

[Glyph]

Update description to mention the other ticket, which is where the bulk of the work is and where the comments explaining what should be done are.

[Jean-Paul Calderone]

The ticket description could still do a better job of explaining what needs to be done. For example, as is, I might just say that it is already possible to point an Agent at a certificate store using WebClientContextFactory and the pyOpenSSL Context APIs. What do you think is necessary beyond that to resolve this ticket?

[Glyph]

Replying to exarkun:

The ticket description could still do a better job of explaining what needs to be done. For example, as is, I might just say that it is already possible to point an Agent at a certificate store using WebClientContextFactory and the pyOpenSSL Context APIs. What do you think is necessary beyond that to resolve this ticket?

Hmm. Good point. The issue here is that this always ought to be done by default; as it was phrased previously this was not clear at all.

[Glyph]

Struggling with a resolution status for this: the functionality still isn't quite there, but the API is; however, there's no longer any work left to do in Agent to make this happen, we just have to make platformTrust smarter.