Migrate users in bulk from external database

[sureshkrishnan-mr]

We are migrating users from Magento into Auth0, and we are bulk importing users using following endpoint /api/v2/jobs/users-imports

We want to migrate users into Auth0 without users having to change their password.

I have migrated few customers into our Auth0 test environment and I noticed when a user tries to login, they are being asked to change their password.

Password hash in the external db is stored like below - md5 algorithm
Hyee6a46f901259d56e3813d2c4d5eaf6e:450y86JkmHru6KtZo4O07CSHuwy

Below are the technical details.

AUTH0 LOGS :

{
“date”: “2021-02-10T20:23:44.987Z”,
“type”: “fp”,
“description”: “Password change required.”,
“connection”: “MigrationDryRun”,
“connection_id”: “con_5RFvdwjNSUjvLOQG”,
“client_id”: “l7OCjTSTzRhogBPoyOQD3luoNHXl3TqO”,
“client_name”: “All Applications”,
“ip”: “31.124.202.113”,
“user_agent”: “Chrome 88.0.4324 / Mac OS X 10.15.7”,

“details”: {
“error”: {
“message”: “Password change required.”,
“reason”: “Verification failed for the provided custom_password_hash: {‘algorithm’:‘md5’,‘hash’:{‘value’:‘6196a46f901259d5…’,‘encoding’:‘hex’},‘salt’:{‘value’:‘MzU3SGxRMHk…’,‘encoding’:‘base64’,‘position’:‘prefix’}}”
}
},
“user_id”: “auth0|943d6810114f4d10a55086f8”,
“user_name”: “xxxx.yyyyy@gmail.com”,
“strategy”: “auth0”,
“strategy_type”: “database”,
“log_id”: “90020210210202345525000825933440351445052253934865874978”,
“_id”: “90020210210202345525000825933440351445052253934865874978”,
“isMobile”: false
}

PAYLOAD:

[
{
“user_id”: “943d6810114f4d10a55086f8”,
“email”: “xxxx.yyyy@gmail.com”,
“given_name”: “xxxx”,
“family_name”: “yyyy”,
“name”: “xyxyxy”,
“custom_password_hash”: {
“algorithm”: “md5”,
“hash”: {
“value”: “8iisddd6f901259d56e3813d2c4d2cfaf398ee71740e68252003d2f733eee”,
“encoding”: “hex”
},
“salt”: {
“value”: “MMMMEEE3SGxRMHk4NkprbUhydTZLdFpvNE8wN0NTUGFrUVQ=”,
“encoding”: “base64”,
“position”: “prefix”
}
},
“app_metadata”: {
“magento_imported”: true,
“external_pwd”: “74747a46f901259d56e3813d2c4d2cfaf398ee71740e68252003d2f07445eaf6e”,
“magento_id”: “35079”
},
“email_verified”: true
}
]

The hash and salt is constructed using the logic:

if ‘:’ in r[‘password_hash’]:
r[‘custom_password_hash’] = {
‘algorithm’: ‘md5’,
‘hash’: {
‘value’: r[‘password_hash’].split(‘:’)[0],
‘encoding’: ‘hex’
},
‘salt’: {
‘value’: base64.b64encode(bytes(r[‘password_hash’].split(‘:’)[1], ‘utf-8’)).decode(‘utf-8’),
‘encoding’: ‘base64’,
‘position’: ‘prefix’
}
}
r[‘app_metadata’] = {
‘magento_imported’: True,
‘external_pwd’: r[‘password_hash’].split(‘:’)[0],
‘magento_id’: str(r[‘entity_id’])
}
del r[‘password_hash’]
elif ‘$2y$’ in r[‘password_hash’]:
pass_hash = r[‘password_hash’]
r[‘password_hash’] = pass_hash.replace(‘$2y$’, ‘$2a$’)
r[‘app_metadata’] = {
‘magento_imported’: True,
‘external_pwd’: pass_hash,
‘magento_id’: str(r[‘entity_id’])
}

I think there is something wrong in salt value which I tried to explore but could not find any root-cause.
Your inputs will be much appreciated to find the root-cause.

[sgo]

Hi @sureshkrishnan-mr ,

I was not able to import your sample payload, I believe this is because the hash value you have provided is not hex-encoded as per your chosen encoding.

Can you please check how you are generating and encoding the hash value for your import payload, and try again? If you are still having an issue, can you please provide another sample payload and we can take another look.

[sureshkrishnan-mr]

@sgo This is the original payload which failed…i manually modified hash before posting…can you please check using the below payload?I will check other details meanwhile

[
  {
    "user_id": "943d6810114f4d10a55086f8",
    "email": "mehdi.kimakhe@gmail.com",
    "given_name": "mehdi",
    "family_name": "kimakhe",
    "name": "Mehdi Kimakhe",
    "custom_password_hash": {
      "algorithm": "md5",
      "hash": {
        "value": "6196a46f901259d56e3813d2c4d2cfaf398ee71740e68252003d2f07445eaf6e",
        "encoding": "hex"
      },
      "salt": {
        "value": "MzU3SGxRMHk4NkprbUhydTZLdFpvNE8wN0NTUGFrUVQ=",
        "encoding": "base64",
        "position": "prefix"
      }
    },
    "app_metadata": {
      "magento_imported": true,
      "external_pwd": "6196a46f901259d56e3813d2c4d2cfaf398ee71740e68252003d2f07445eaf6e",
      "magento_id": "35079"
    },
    "email_verified": true
  }
]

Password hash in the database for the user in the payload : 6196a46f901259d56e3813d2c4d2cfaf398ee71740e68252003d2f07445eaf6e:357HlQ0y86JkmHru6KtZo4O07CSPakQT

This how we are generating the hash value for the payload

r['custom_password_hash'] = {
                'algorithm': 'md5',
                'hash': {
                    'value': r['password_hash'].split(':')[0],
                    'encoding': 'hex'
                },
                'salt': {
                    'value': base64.b64encode(bytes(r['password_hash'].split(':')[1], 'utf-8')).decode('utf-8'),
                    'encoding': 'base64',
                    'position': 'prefix'
                }
            }

[sgo]

Thanks @sureshkrishnan-mr, however we would need to know the real values involved to test this properly. Could you please create a fake user on your original system, and share with me the original hash and salt, the associated password you used for the fake user, and a sample import payload you would use to try to import it into your tenant?

What would also be very useful to know is the code that is used currently to verify your passwords. For example, you are currently using a prefix, but your shared code grabs the first part for the hash value and the second for the salt if I am reading it correctly. Is it the opposite way around on your original DB?

You can send this information in a direct message to me if you prefer not to post these details.

[sureshkrishnan-mr]

@sgo I have messaged you the requested details.Please let me know if you need more details.

[Saltuk]

Hi @sureshkrishnan-mr I’m looking into this now. I have received the files you had shared with @sgo

[konrad.sopala]

Thanks for helping with this one Saltuk!

[Saltuk]

The issue seems to be related to the legacy identity provider using different hash algorithms for different users. With a sample user we have verified that import to Auth0 works. I will mark this questions as resolved however we will continue from the internal support ticket I have opened for the remaining issues with the migration.