Opened 2 years ago

Last modified 7 days ago

#6371 enhancement new

Support native Windows trusted CA database for SSL certificate validation

Reported by: itamar Owned by: aronbierbaum
Priority: normal Milestone:
Component: core Keywords:
Cc: adi@… Branch: branches/windows-trust-root-6371
(diff, github, buildbot, log)
Author: glyph Launchpad Bug:

Description

This was originally part of #5446, where Glyph wrote:

On Windows - and this is purely from a quick glance at the reference documentation, so take it with a grain of salt - I believe the right way to do this is to use CertOpenSystemStore with the string "CA", or possibly "ROOT", or maybe both, and then do CertEnumCertificatesInStore or maybe just PFXExportCertStoreEx to dump the certs into a format we can import into OpenSSL.

Attachments (3)

windows_trust_root.patch (4.4 KB) - added by aronbierbaum 8 months ago.
windows_trust_root2.patch (4.4 KB) - added by aronbierbaum 8 months ago.
windows-trust-root-6371-2.patch (4.4 KB) - added by aronbierbaum 8 months ago.

Download all attachments as: .zip

Change History (16)

comment:1 Changed 2 years ago by exarkun

There is a Python library to help out with this too:

https://bitbucket.org/tiran/wincertstore

This may be easier than writing C, or Cython, or using ctypes or cffi. Or maybe not, I haven't investigated much. But it's something to look at I guess.

comment:2 Changed 16 months ago by itamar

wincertstore is written with ctypes.

comment:3 Changed 16 months ago by aronbierbaum

I have been working on an implementation that uses for ctypes also. In the spirit of not duplicating effort, is anyone else also working on a solution?

comment:4 Changed 16 months ago by itamar

Assuming wincertstore does what it says it does, presumably that's who you should talk to. Unless you want to add something to Twisted, in which case using wincertstore seems like a good start.

Changed 8 months ago by aronbierbaum

comment:5 Changed 8 months ago by aronbierbaum

  • Keywords review ssl added

I have added an initial pass at adding support for certificate verification on Windows using the wincertstore package. Please provide feedback and required changes.

Changed 8 months ago by aronbierbaum

Changed 8 months ago by aronbierbaum

comment:6 Changed 6 months ago by glyph

  • Keywords ssl removed

We use keywords for workflow, and ssl is not a workflow state. Therefore, removing that keyword.

comment:7 Changed 6 months ago by glyph

  • Author set to glyph
  • Branch set to branches/windows-trust-root-6371

(In [43870]) Branching to windows-trust-root-6371.

comment:8 Changed 6 months ago by glyph

  • Keywords review removed
  • Owner set to aronbierbaum

Hi aronbierbaum! Thanks very much for this contribution - it's an important building block of Twisted's client TLS subsystem, and I'm glad someone is working on this :).

Unfortunately this first cut at the patch looks like it's not in great shape:

  1. there are a number of coding-standard violations.
  2. some of the tests are failing on some platforms
  3. wincertstore should really be included in the list of setuptools extras for the tls extra on Windows.

However, the general idea of using wincertstore is pretty good, and it looks like the implementation is basically correct. I'm looking forward to re-reviewing this!

comment:9 Changed 4 months ago by adiroiban

  • Cc adi@… added

comment:10 Changed 5 weeks ago by glyph

A bug report in treq indicates that we really need to come up with a solution for this or people can't really use TLS on Windows; setting up a certificate bundle is incredibly subtle and undocumented.

https://github.com/twisted/treq/issues/94

comment:11 Changed 4 weeks ago by glyph

The specific issue of "empty trust store" as opposed to windows API bindings for real trust roots filed here.

comment:12 follow-up: Changed 10 days ago by akgood

Two notes:

  1. As of Python 2.7.9 (along with recent versions of 3.x), wincertstore is not needed; that functionality is available as "ssl.enum_certificates()" in the standard library.
  1. However, both wincertstore and the Python stdlib implementation have a significant caveat:

Starting with Vista, Microsoft began shipping only a very minimal set of root CA certificates with Windows. Microsoft does trust many other authorities, but for these, Windows relies on the "Update Root Certificates" feature: http://technet.microsoft.com/en-us/library/cc749331(WS.10).aspx

"... if the application is presented with a certificate issued by a certification authority in a PKI that is not directly trusted, the Update Root Certificates feature (if it is not turned off) will contact the Windows Update Web site to see if Microsoft has added the certificate of the root CA to its list of trusted root certificates. If the CA has been added to the Microsoft list of trusted authorities, its certificate will automatically be added to the set of trusted root certificates on the user's computer."

Critically, this update mechanism is only invoked if you're using (e.g.) CryptoAPI functions to validate a specific chain; if you just ask Windows to enumerate the certificates it knows about, it won't pull anything down from Windows Update.

(Some concrete numbers: on a clean installation of Windows 8.1, running certmgr.msc shows 18 certificates listed in the "Trusted Root Certification Authorities"; by contrast, OS X comes with over 200 trusted roots).

As far as I understand it, the only 100%-accurate way to validate certificates against Windows' trust policy would be to hook the OpenSSL verify callbacks to retrieve the leaf and intermediate certificates provided by the server, then use CryptoAPI functions (e.g. perhaps CertGetCertificateChain?) to have Windows perform the actual chain validation.

(I filed a bug report about this in Python as https://bugs.python.org/issue20916)

Last edited 10 days ago by akgood (previous) (diff)

comment:13 in reply to: ↑ 12 Changed 7 days ago by glyph

Replying to akgood:

Two notes:

Thank you very much for these highly informative implementation notes, akgood.

ideally, eventually, we can just use the CryptoAPI/NG functions for doing TLS directly, but until we do, hooking up the verify callbacks in the manner described seems like the best option. Except... presumably that will block (and the verify callback has to block, I think?)

Note: See TracTickets for help on using tickets.