User migration to autho takes too much time

Help
1

We are evaluating auth0 and very first use case that we have is: migrating existing users to auth0. After doing some experiments, it looks like importing users to auth0 is very time consuming process. Roughly it takes 5 mins to import 1.5k users. Auth0 does also have limitation of 500kb file and two parallel imports at the same time. A single user data

{"email":"DB9TXVDCRGMPJD25UTOB@example.com","email_verified":true,"user_id":"280f980b-aaf6-4dc4-883b-ead05b2edede","blocked":false,"custom_password_hash":{"algorithm":"bcrypt","hash":{"value":"$2a$16$6dtwML7afpVuixQkPttpCu5mub26hYXwACcEOCSq0mUPWbNQ6APky"}},"mfa_factors":[{"phone":{"value":"+91XXXXXXXXXX"}}]}

Given this, it might take 4-5 days to import 3Million+ users.

We have following question

  1. Is our understanding correct?
  2. Does these limits are enhanced for enterprise edition? If not then is there any realistic way to import users?

—> We do not want to go for lazy migration
—> importing through auth0 extensions has the same challenges.

2

Hi @rajivk

Make sure you are testing this in a tenant flagged as production. Dev tenants will be much slower.

Keep in mind that you can run two import jobs at a time.

If you have a private cloud instance, you can get better import times.

3 days sounds long to me, but it will take a while to do 3 million users, several hours at least.

John

2 Likes
3

Also test with full jobs (close to the 500K limit) to avoid job overhead

1 Like
4

@john.gateley

We flagged the tenant as production. And it still takes the same amount of time. does the limit get enhanced only in case of paid subscription?

5

Hi @rajivk

Honestly, I don’t know about the paid subscription part. Again, test with several files of close to 500K for a more accurate estimate.

John

1 Like
6

@john.gateley here is the link Tenant Settings for subscription part.

Yeah, we have been testing files of size close to 500KB. We can test with several files but it accepts only two jobs at a time, so we have to wait for at least 1 jobs to end and then try again. We have observed time is almost fixed ~5 minutes every time.

When you say, it will take several hours for users, how did you calculate? is there any rough numbers with you?

7

Hi @rajivk

First, I wouldn’t stress about the import users speed. This is a one time cost that happens while you are integrating Auth0 with your platform. Once it is done you don’t need to do it again. And there are other options, like automatic migration or pass through databases. 3 million users could take several hours, but again, this is a one time cost.

You can (and should) write a script that handles the job submission, so that you aren’t manually creating the import jobs.

Because of the differences in users (different fields, sizes etc) there isn’t an accurate estimate I can give you. The last time I personally did a large import job was quite a while ago, so I’m not sure my numbers are still accurate. But I can say that 3 million users will be hours rather than minutes.

John

2 Likes