Password hashes export data format

Knowledge Solutions
1

Question: I received a password hashes export file from Auth0. What can I do with it.

Answer:

As part of Auth0’s support procedures, customers can request an export of password hashes.

You can use the file to move your users to a different system, or to move them to a different tenant or database within Auth0.

Data format

The provided file is in NDJSON format (New line delimited JSON). If you open it with a text editor you will see one user object per line. E.g.:

{ ... one user... }
{ ... another user ... }
{ ... and so on ... }

Each user object will have some properties. Most of them are self-explanatory, but it’s good to clarify a couple of them related to the user id. A sample single user would look like this (formatted for better readability):

{
  "_id":{"$oid":"5dea9f9c82dd7c0e76e4ec93"},
  "email_verified":false,
  "email":"nico9090@nico.com",
  "passwordHash":"$2b$10$.qHPp/srqo1NAAAAAvlkmOdqAbH2Rg0qPv2Txj3ZwXfjJnewSjc4m",
  "password_set_date":{"$date":"2019-12-06T18:36:12.412Z"},
  "tenant":"nico-sabena",
  "connection":"Username-Password-Authentication",
  "_tmp_is_unique":true,
  "alt_id":"euclid"
}

Every user will have an _id field, while the alt_id is optional. You’ll see an alt_id for users coming from a custom DB (where the custom DB script provides the user id) or when you create regular DB connection users with the Management API v2 and you provide your own user id.

In the above case, the user identifier is likely to be something like auth0|euclid.If the alt_id field wasn’t present, then the full user_id would be auth0|5dea9f9c82dd7c0e76e4ec93.
The connection parameter is important if you requested an export from multiple databases, as that’s the field that indicates where the user came from.

Can I use this file to directly import files to Auth0 (e.g. with the Users Import extension)?

No, not really. The format accepted by the Bulk Import Users API (or extension) is different. The full specification can be found at Bulk User Import Database Schema and Examples. Some important things to consider:

  • The size of the import file is limited to 500kb
  • The expected format is a JSON array, not NDJSON.
  • If you want to import the original user ids, you’ll need to provide a user_id field with the value found in the exported user’s alt_id (if present), or the $oid value.

E.g. for the sample user above:

[
  {
    "user_id":"euclid", // if you want to keep the same user id
    "email_verified":false,
    "email":"nico9090@nico.com",
    "password_hash":"$2b$10$.qHPp/srqo1NAAAAAvlkmOdqAbH2Rg0qPv2Txj3ZwXfjJnewSjc4m"
  },
  {...} // other entries in the array, if you have more users
]

If you wanted to keep the user id and this user did not have an alt_id, you’d do:

[
  {
    "user_id":"5dea9f9c82dd7c0e76e4ec93", // if you want to keep the same user id
    "email_verified":false,
    "email":"nico9090@nico.com",
    "password_hash":"$2b$10$.qHPp/srqo1NAAAAAvlkmOdqAbH2Rg0qPv2Txj3ZwXfjJnewSjc4m"
  },
  {...} // other entries in the array, if you have more users
]

Using jq to convert data

While you can convert data as you see fit, one tool that you might find useful is jq.

This command will read an input file (input.json), pipe it through a couple of jq filters, and write it in an output file that’s ready to import (output.json):

cat input.json | jq '{user_id: (if has("alt_id") then .alt_id else ._id."$oid" end), email, username, email_verified, password_hash: .passwordHash}' | jq -s 'del(.[][] | nulls)' > output.json

The first jq pass picks relevant fields, and the second one removes null values and transforms the input into one big array (there’s probably better ways to do this, apologies in advance to jq experts :slight_smile: )

The above command will pick the alt_id as the user id, and use the _id.$oid if alt_id is not available.
Alternatively, you could decide to just use the alt_id if present and let Auth0 assign a random user_id if the alt_id is not provided by replacing ._id."$oid with null:

cat input.json | jq '{user_id: (if has("alt_id") then .alt_id else null end), email, username, email_verified, password_hash: .passwordHash}' | jq -s 'del(.[][] | nulls)' > output.json

What if I want to move metadata or other information too?

Things get a bit more complicated if you want to move additional user information like metadata and keep passwords. The password hashes export provided by Auth0 Support does not include the user metadata fields. You’ll need to export the data yourself separately ((the regular bulk user export from the extension or Management API v2) ) and merge that data with the password hashes provided by Auth0 support, so that the final import file has all the data. E.g.:

[
  {
    "user_id":"5dea9f9c82dd7c0e76e4ec93", // if you want to keep the same user id
    "email_verified":false,
    "email":"nico9090@nico.com",
    "password_hash":"$2b$10$.qHPp/srqo1NAAAAAvlkmOdqAbH2Rg0qPv2Txj3ZwXfjJnewSjc4m",
    [...], // other properties like these:
    "given_name":"Nicolás",
    "app_metadata": {
      [...]
    },
    "user_metadata": {
      [...]
    }

  },
  {...} // other entries in the array, if you have more users
]
12 Likes