Continuous Integration within Auth0

Help
1

Note: Moved this question into its own topic from What is the #1 concept you wish you understood better in auth? - #13 by lee3 (question asked by @lee3)

Continuous Integration within Auth0

For us, we’ve managed to get a beautiful SSO implementation into Production with shared users spanning our 3 SaaS applications, and simultaneously enabling Social authentication channels for our users. This has been a massive win for us and our customers, and we’re on to the next step: Running down edge cases that we didn’t udnerstand we needed to cater for in our original Solution Architecture.

As we’re a Continuous Integration shop, we really liked the Tenant concept that Auth0 offers, as we have PreProduction tenants that align with our Integration and Staging environments, with us manually maintaining Configuration (Applications, Connections, Rules, Hooks etc) across these tenants/environments.

Now that we’re post-Go Live, the main thing we’ve noticed we missed in our CICD research for Auth0 was the ability to pull a snapshot of Users and User Metadata from a Production Tenant into our PreProduction tenants. So here we go:

CURRENT STATE - We’ve successfully gone live with Auth0 and have both Production and PreProduction tenants where we have manually processes that mirror the configuration between these environments.

CHALLENGE - Our CI/CD pipeline has a requirement for all internal systems to have a like-for-like PreProduction environment. Currently our PreProduction tenant matches our Production tenant in configuration (due to our manual process) but diverges in Users and User Metadata ever since launch. This divergence of Users and User Metadata causes problems with our automated testing when introducing change, as well as replication of Production bugs in PreProd when replicating issues specific to individual users.

GOALS - Duplicate the Users and User Metadata from the Production tenant with a target PreProduction Tenant . This is most practical as a point-in-time snapshot that we replicate into the PreProduction tenant, which we’ll manually trigger when performing a rebuild of our Staging environment with Production data.

NOT GOALS

  • No need to replicate passwords, Auth0 ID or other data that we cannot reasonably access or control. Our CICD architecture will obviously just have to work around these limitations
  • No need to make this real-time, we’ll just kick this off when we need to refresh User Content in a given PreProduction tenant
  • No need to synchronise configuration (Applications, Connections, Rules, Hooks, Emails etc) as part of this solution, given the rate of change here is low and we can do this by hand without too much overhead

QUESTIONS

  1. Does Auth0 have any established best practice for us to try and reduce the distance between our Prod and PreProd tenants for companies building and maintaining a CI pipeline?
  2. Does Auth0 have any existing tools that will assist us to achieve this?
  3. There are definitely Auth0 customers that are trying to achieve similar goals to this. Can you point us to any examples, or any libraries they have developed?
2

Great question, well articulated!

You have two main options I would recommend for CI/CD:

  1. I know you said you don’t really need this for your other configuration, but I would still recommend the Auth0 Deploy CLI for configuration syncing between environments. The documentation has some best practices around integrating with your CI that I won’t repeat here.
  2. You should be able to use the Auth0 Management API to write a script that can automate the syncing of users. We don’t have an out of the box user sync, capability, but we do provide all of the API endpoints you need to write a simple script to do this for you. NOTE: you will need to use the client credentials flow to authenticate your script since it is a Machine-to-Machine script. This will be a very powerful script so you will need to protect the client secret with care, using a CI/CD tool that can store secrets safely and has limited access to it.

WARNING: I would not recommend syncing users that are not for test! You need to be wary of syncing user data to a test environment that has more people who have access to the data and therefore more likely that someones private information is leaked. This can also violate privacy laws. Auth0 has a specific article on how to comply with GDPR, and though GDPR is specifically an EU regulation, many of these concepts exist in other countries as well in one form or another including the US.

6 Likes
3

Thanks very much. We’d found the Deploy CLI and the Management API, but the credentials flow was a solid tip.

I’m onboard with not syncing users unnecessarily outside of production for Privacy purposes, these are good points.

Was there anything that directly addressed question 3 out there? I’m pretty sure we’re not the first to embark on this journey…

4

I’m not aware of any specific examples for migrating user’s from one tenant to another using the management API. We have docs on how to search and create users, but not this specific scenario.

Closest thing might be our User Import/Export Extension