Hi, as none of us developers at our company are psychic, we’d actually have more docs on the Management API - Jobs. What are the possible states of jobs? When receiving results of the users export job, are they wrapped or not?
I kinda hate myself for not iterating over all users using their signup date. This seemed so stupid! Not so now when working with jobs is so tedious. Why isn’t it fully included in the Management SDK? Why is getJob of type ?
Working with the APIs is frustrating. What does the limit property of page stands for? If it’s the opposite of start, why is it not named end? Btw guys, limit can also be lower, so start could be the same.
You can go ahead and ignore my rant while I spend my time figuring out how jobs work. Or should I spend the time migrating my database to competition?
I’ve used the jobs endpoint quite a bit so I’d be happy to help you out. I agree that the docs could use some work, but I find that is almost always the case (docs can always be improved!)
I’m not sure what all the possible jobs states are, but I am usually only interested in pending, processing, and completed. There’s a failed job as well of course. I usually just iterate on /api/v2/jobs/{id} until I see completed.
I’m not a JSON expert but the resulting file is not, I think, a valid JSON file. Rather each line is a single JSON object representing a single user. Personally I’d prefer if the entire file was formatted as a list object so I can easily slurp it into python. As it stands my scripts iterate over each line, importing each line as an individual JSON object / dict.
limit limits the number of entries in the response. E.g., limit=100 means you will only get back the first 100 results.
I’m writing a migration of our data connected to the users (data stored outside auth0). So I just need to iterate over all users and migrate their data one by one, since the key to the data is store in the user_metadata. Unfortunately there’s the 1000 users per query limit. First I weighed on trying to load them using last_login day-by-day, then I decided that using the export would be less pain. Unfortunately the export and the job was not documented enough to write it just out of the docs without trying.
@jerdog
The point where the docs are incomplete is the /api/v2/jobs/users-exports endpoint - the format of the json output is undocumented. /api/v2/jobs/{id} also lacks an enumeration of possible job states. While writing this I ran into this document I had no idea existed: User Search Best Practices. So I guess just adding some information from the document into API docs would be nice.
While I’m at it, it is confusing why reading the result of the export job need’s create:passwords_checking_job scope while creating the job itself needs just read:users which is included by default. I suggest to move the result reading into GET /api/v2/jobs/users-exports/{job_id}/results. That way that could be correctly scoped.
I totally understand! I handle all the manual data manipulation on my team and the 1,000 result query limit (and 10,000 result Import/Export Extension limit) forced me to start doing exactly the same thing… extract jobs for large queries and massaging the results with python. I certainly understand the point of those limits but it increase the amount of manual work quite a bit.
I now have a small ‘library’ of export job JSON files and associated python scripts (for JSON to CSV conversion) for common use cases, which helps a bit, but it is still a lot of work and ad-hoc requests are pretty common where I work so my library is only useful part of the time. And that is just for reporting! Updates are another thing altogether.
@markd wow, sounds like a lot of pain. I don’t actually understand the limit of the queries, since they’re pageable and paging should mitigate the load. What’s the difference in loading first 1000 users than the last 1000? But anyways I’m running into issues with the token from https://manage.auth0.com/#/apis/management/explorer. Even though is states
The token includes all the scopes that are granted to the API Explorer Application , by default it can invoke all the Management API endpoints
the token does not have create:passwords_checking_job. It does not work in the app or in the API Explorer itself.
I submitted feedback requesting the format of the /jobs api query results be updated to match what the /users endpoint returns (a list object instead of a “text file of JSON strings”).
Thanks @markd, that’s what I’ve already tried. Also thanks for posting the feedback.
But I found the solution. Each client has APIs tab in which each API is granted permissions. For some reason the API Explorer Client missed some permissions. I can’t believe how long it took me to find it.
That’s a great find! I just had a look at my API Explorer Client and it does not have all the scopes assigned either. I just assumed it would have everything. Looks like some or maybe all of them are new scopes related to new services (new since I created this tenant, like custom domains).
Yes, that bit is very confusing. /api/v2/jobs/{id} is used to get the status of an import or export job, while /api/v2/jobs/{job_id}/results is only for password checking jobs. Feels like the /jobs endpoint is maybe very new and hasn’t received much love yet.
