Search v3 1000 result limit

The docs state:

Auth0 limits the number of users you can retrieve (1000). If you exceed this threshold, please redefine your search, use the export job or User Import / Export extension.

Does this mean per request, or is it in total (I will never be able to paginate beyond the 1000th result for a query which returns more than 1000 results)? We’re approaching the 1000-user threshold and I want to make sure we can exceed it fine, but the alternatives seem extremely poor (export job/extension) from an application perspective.

1 Like

So according to the documentation you’ll only be able to search through the first 1000 records of a query, however you can use the export job to grab all your users to a file.

Endpoint: POST /api/v2/jobs/users-exports

For more info you can check out the API Explorer and then select Jobs

OK thanks. Unfortunately the exports aren’t very useful when you want to display large numbers of users using up-to-date information in real time.

Are you using the API or the UI to view the info?

@jerdog Is there any management api endpoint to get users that is listed outside the limit of 1000, one of my search query result exceeds 1k limit , currently I’m getting the first 1k user data , how do I get rest of the users ?

:wave: @aminul can you describe your use case for wanting to search for 1000+ users synchronously? Are you looking to display these on a dashboard list? Depending on your use case we could do a full user export using export job endpoint or the Import/Export Extension . We could even try searching for users created after a certain time (since last sync let’s say) so we don’t perform a search for every user. It will really depend, let me know what you are looking to do and we can work on a solution that works for you.

1 Like

hi, @kimcodes Thanks for your reply :slight_smile: , here is our user case : We are maintaining an cache of auth0 user data so that we don’t have invoke auth0 management api all the time(it help us to speed up the performance a lot), User info that are included into the cache is obtained through v3 user search api endpoint and filtered based on some application specific lucene query, I think because of that limitation of search api endpoint currently we are getting only first 1000 users at our user cache ,
do you think we should use the user export job to manage our scenario ?

@aminul I apologize for the delay. I believe at this time you may want to user the user expert job like you mentioned. Is that what you continued with as your solution?

1 Like

@kimcodes currently we are joining results from multiple search queries to get the final list of Users (that’s kind of slow but works for now ) , In the next version we will start using that export job if needed.

Thanks for your time and attention :slight_smile:
Aminul

1 Like

When I use the import/export extension, this also appears to be limited to 1000, although this statement in the V3 User Search API implies that this shouldn’t apply.

“When you query for users with the list or search users endpoint, you can retrieve maximum 1000 users. If you exceed this threshold, redefine your search. If you need a full export of your users, use instead the export job or the User Import / Export extension.”

Can you confirm if/how I can get more than 1000 users exported using the Extension.

Also, can you confirm that the Export Job has no limits

Thanks

Please see my follow-on questions in the thread below - the extension also seems to be limited to 1000? i.e. when we enter the details, the button says “Export 1000 users”.

Additionally, to those questions, can you filter the results using the extension e.g. to export a range based on created_at?

Thanks

Hello, I am new to Auth0 and as the documents says i am trying to fetch all the users from Auth0 account (which is around 300). I am using Management API’s UserEntity’s list() method and passing the filter with search engine v3 as shown below:

	UserFilter filter = new UserFilter().withSearchEngine("v3");
	Request<UsersPage> request = mgmtApi.users().list(filter);

However, i am still getting only 50 records.

@sbrewell I believe it is the case that using the extension should not be limited to the 1000 users. What version of the extension are you using? (The extension’s version should be next to the extension’s name in the ‘Installed Extensions’ tab.)

Yes, the other option is to use POST /api/v2/jobs/user-exports endpoint of the Management API v2 directly to get an export. This should allow you to overcome any limitations while exporting all your users.

@supriya.vyas what query as you specifying with the UserFilter (via withQuery(String query) )? Are you using the Auth0 java client library?

Yes I am using the Auth0 Java client library.
I am passing the connection name in ‘with query’
UserFilter filter = new UserFilter().withQuery(“identities.connection:"”+connection+“"”).withSearchEngine(“v3”);

However, in 1 connection there are only 30 members. So this works properly.

But, in another scenario I want all the members irrespective of the connection name which totals to more than 200.
UserFilter filter = new UserFilter().withSearchEngine(“v3”);
This scenario fails as only first 50 members are listed.
TIA

@kimcodes - it is version 2.4 - up to date?

Can some one correct me if i am Wrong. User Import/Export extension can be used to download all the User records in to a file. but can not query by passing the values in URL after adding Import/Export extention??

2.4 should be the latest, yup! And you’re still experiencing a limit to 1000 users? I believe the extension can do up to 100K users so this is a bit odd. Are you using the extension or the export job?

What are you looking to query?

I am querying last login details from [-3days to current date]. But I received all records (2000+) after removing “search_engine=v3” from Url. Thats means is it taking v2 as default ??