Introduction of Additional Certificate Authorities

Knowledge Solutions
, , , ,
1

Problem Statement

Cloudflare will introduce new Certificate Authorities (CA) for signing their certificates that may affect some tenants.

Symptoms / Troubleshooting

Most tenant admins will not see any changes. However, applications may encounter errors when trying to access the tenant in the following circumstances:

  • integrations with AWS Cognito
  • applications using certificate pinning
  • applications running on older operating systems and application runtimes that may not have the new certificate in the truststore
  • applications using OpenSSL 1.0.2
  • some Xamarin applications

Solution

To resolve any challenges with these warnings or errors,

  • check your operating system and application runtime truststores
  • validate your application does not use certificate pinning
  • ensure that you are using a recent version of OpenSSL (> 1.0.2)

In addition,

Encryption and Certificate Authorities

In simplified terms, in order to establish a secure, encrypted session (via Transport Layer Security “TLS”) and make successful API requests, first a client must be able to verify the identity of the server. Verification is performed by checking the server name against a list of certificates in what’s called a TrustStore. Clients trust these Certificate Authority (CA) root certificates in the TrustStore by default.

This list of trusted CAs is not static and changes over time. New CAs can be introduced by major cloud, service, and security vendors. In some cases, CAs are removed from the list due to security incidents. Security best practice is to keep the list of Certificates up to date.

CA Root Certificates in the TrustStore

Auth0 is currently using LetsEncrypt and will soon adopt Google Trust Services (in 2023) as Certificate Authorities.

The follow root certificate public must be present. Most modern operating systems have these installed, older operating systems or unpatched older operating systems might not have them.

What Name Location
LetsEncrypt (current) ISRG Root X1 http://x1.i.lencr.org/
Google Trust Services (upcoming in 2023) Trusted Google root CA bundle trusted Google root CA bundle

LetsEncrypt CA details can be found here.

Google Trust Services (GTS):

  • was included in initial announcement as a new Certificate Authority. Auth0 is not using it as a CA currently.
  • is intended to be offered as alternative and backup to LetsEncrypt however a final decision is still pending
  • We have included GTS root CA details for completeness to enable customers to verify their operating systems and app runtime are compatible. GTS root CAs are present in any modern trust store.

How do I prepare for these additional Certificate Authorities?

Check your operating system and application runtime truststores

Operating systems have a CA truststore and some application runtimes - like Java and Node.JS - have their own truststore independent from operating system. In both cases, the truststores need to have the appropriate root CAs.

Recommendation:

  • Ensure operating systems and application runtimes are running recent versions (not end of life!) so their truststores are up-to-date
  • verify of “ISRG Root X1” and “Google Trust Services” root CAs are present in your truststores

Validate your application does not use certificate pinning

Some customer applications implement certificate pinning (explained in detail here): they contain configuration or application logic that insists on specific certificate details like the certificate’s serial number, fingerprint, or Common Name.

Certificate pinning is not a best practice. Instead, the application validate a connection is secure by relying on TLS and certificate validation (using an up-to-date truststore).

Recommendation:

  • verify your application logic relies on certificate validity and not specific certificate details

Test an authentication request using Auth0

All Auth0 tenants using the hostname pattern <tenant>.<geography>.auth0.com are using certificates signed by the new Certificate Authorities.

Test your application by attempting to fetch your tenant’s JWKS keyset and confirming your application runtime retrieves them successfully.

Operating Systems and Application Runtimes that Require Action

LetsEncrypt may not be in the truststore of older operating systems and application runtimes. According to Certificate Compatibility, this affects:

  • macOS < 10.12.1
  • iOS < 10
  • Mozilla Firefox < 50
  • Ubuntu >= intrepid / 8.10 and < Precise Pangolin / 12.04
  • Debian >= squeeze / 6 and < jessie /8
  • Java 8 >= 8u101 and < 8u141
  • Java 7 >= 7u111 and < 7u151
  • NSS >= v3.11.9 and < 3.26
  • Amazon FireOS (Silk Browser) (version range unknown)
  • Cyanogen > v10 (version that added ISRG Root X1 unknown)
  • Jolla Sailfish OS > v1.1.2.16 (version that added ISRG Root X1 unknown)
  • Kindle > v3.4.1 (version that added ISRG Root X1 unknown)
  • Blackberry >= 10.3.3 (version that added ISRG Root X1 unknown)
  • PS4 game console with firmware >= 5.00 (version that added ISRG Root X1 unknown)

Additional Information for Auth0 Customers

Certificate expiry error

OpenSSL 1.0.2 results in a certificate expiration error while all certificates are actually valid. This version of OpenSSL has a bug (does not honor cross signing) that causes some root CAs to not be evaluated when determining whether a connection is secure or not.

Follow up actions:

  • Upgrade OpenSSL to recent version, or
  • Workaround: remove expired certificate “DST Root CA X3” from trust store.
3 Likes