Currently, when submitting user import jobs via the Bulk Import API, jobs can remain in a “pending” state indefinitely if the shared regional import worker infrastructure is experiencing a large overall queue size.
Because there is no visibility into the current queue load prior to (or during) the job submission, we rely on continuously polling the job status endpoint to determine when our jobs are picked up. During periods of high regional load, this extended pending time combined with our polling triggers global rate limit warnings in our Auth0 logs.
We are requesting an endpoint, response header, or metric that exposes the current load or queue depth of the shared import worker infrastructure. This would allow tenants to preemptively check system congestion before submitting large jobs.
We use the Bulk Import API to manage user imports, and we need our automated systems to process these jobs reliably. Currently, when the shared worker queue is backed up, our system blindly polls for status updates, which wastes our internal resources.
If we could preemptively monitor the current queue load, we could build smarter integrations that dynamically configure both our job submission rates and our polling intervals based on actual system status. This feature would significantly improve our developer experience, prevent unnecessary rate-limiting errors, and reduce excess API traffic on Auth0’s end.