Opened 5 years ago

Last modified 4 years ago

#9209 defect new

platformTrust Phase 2

Reported by: Glyph Owned by:
Priority: normal Milestone:
Component: core Keywords:
Cc: Branch:
Author:

Description (last modified by Glyph)

At the time #5446 was fixed, I think the assumption was that we'd use the default OpenSSL APIs to do things, and then either we'd implement something smarter or OpenSSL would just generally get better about doing those things.

Neither of those things have happened.

The current state of the art is:

  • On Linux, if you have ca-certificates (or equivalent) installed, everything works great.
  • On macOS, if you have brew install -ed the version of openssl that your version of cryptography happens to be linked against, into the default location (/usr/local/) then homebrew's formula will have run its script to dump the macOS trust store into the default OpenSSL location and things will work great. Until recently that was brew install openssl, now it's brew install openssl@1.1 (I don't know the exact version where it switched off the top of my head).
  • On Windows, basically nothing ever works out of the box. You have to manually get yourself a trust store and set some environment variables.

So: not as bad as it was, but still not a great situation!

There have been a number of ecosystem developments in the meanwhile.

For "phase 2", I think that we should make twisted[tls] depend upon certifi, conditionally depend upon certitude on macOS and then platformTrust should use logic something like this:

  • If `SSL_CERT_FILE` or `SSL_CERT_DIR` is explicitly set, use the current, default OpenSSL behavior.
  • If the platform is linux and an heuristic check for "is something like ca-certiifcates installed", then use the default OpenSSL behavior as well; trust the platform if it looks like there's anything there to trust.
  • If the platform is macOS, try to use certitude.
  • If all the above checks failed, try to import certifi and use it.

Certifi should be unconditionally required by the tls extra so that it's always available as a fallback, even though it hopefully wouldn't be used much.

It would be helpful to log some messages when making these initial determinations as well, to help users diagnose verification issues. (It would be super useful if we could actually collect metrics on this, but that's probably a discussion for another day.)

Change History (6)

comment:1 Changed 5 years ago by Cory Benfield

So, certitude is an interesting library. It's probably fine, but I have thus far suggested that people not rely on it because it doesn't have enough testing for me to be confident that it works: specifically, that it doesn't fail open. If someone can work out how to port a whole load of X.509 validation tests to it, that would make me a lot more confident.

Also, it works fine on Windows. The description is just out of date.

comment:2 in reply to:  1 Changed 5 years ago by Glyph

Replying to Cory Benfield:

So, certitude is an interesting library. It's probably fine, but I have thus far suggested that people not rely on it because it doesn't have enough testing for me to be confident that it works: specifically, that it doesn't fail open. If someone can work out how to port a whole load of X.509 validation tests to it, that would make me a lot more confident.

What would your recommended course of action be, then? Just use certifi everywhere for now? Don't add the dependency on certitude, but use it if it's explicitly installed? An environment variable? My concern is that what's happening now is that no TLS domain would validate at all, which makes people fall back to plaintext. Few bugs could be worse than this.

Also, it works fine on Windows. The description is just out of date.

Even the released version?

comment:3 Changed 5 years ago by Kali

#6934 covers the use of certifi. Something I don't quite get is why the two first cases would need to be handled separately in twisted code.

I was under the assumption that:

  ctx.set_default_verify_paths()

will already try several fallbacks, according to the documentation of PyOpenSSL issue 642

comment:4 Changed 4 years ago by Glyph

Description: modified (diff)

comment:5 Changed 4 years ago by Benoit Gagnon

Just wanted to highlight a patch/fork for certifi called python-certifi-win32 [1] that allows it to extend its certificate store with those found in the Windows registry [2]. This includesroot CAs deployed through entreprise group policies.

[1] https://pypi.python.org/pypi/python-certifi-win32 [2] https://github.com/certifi/python-certifi/pull/54

comment:6 Changed 4 years ago by Glyph

Thank you very much for the heads up. A patch to integrate this would obviously be awesome :).

Note: See TracTickets for help on using tickets.