When testing role migration script, I got HTTP 429, “Global limit has been reached”.
I understand APIs are rate limited, so I implemented a rate_limit procedure, which is ran after every API call to wait when I am close to the limit.
The logic is like
(After an API call)
If HTTP header x-ratelimit-remaining < 3 then
Sleep until x-ratelimit-reset
I understand that theoretically we can have API calls from other applications, so this is not 100% reliable. Please ignore this possibility for this post.
This did not work. I got HTTP 429 after a call returned x-ratelimit-remaining=993. I am sure nothing else made 993 calls between my 2 calls.
Is this because management API has a tight rate limit, and x-ratelimit-remaining is based on the maximum possible limit? If yes, why return this header? The value useless most of the time.
And then the only solution of HTTP 429 is to retry after x-ratelimit-reset FOR EVERY API call?
I understand that you are receiving HTTP 429 “Global limit has been reached” errors due to rate-limiting boundaries of the Management API, so lets clarify what is happening here.
To answer your question directly: this happens because Auth0 enforces multiple layers of rate limits simultaneously using a token bucket algorithm. Specifically, your role migration script hit a burst limit while it was running, even though your overall sustained rate limit had plenty of capacity left.
Here is a breakdown of why this occurs and how you can adjust your script.
Burst Limits vs. Sustained Limits
Auth0 rate limits are divided into two concepts: the burst limit (the maximum number of rapid requests you can make in a fraction of a second) and the sustained limit (your total allowance over a longer window, like a minute). Because scripts like a role migration tool fire off multiple requests very rapidly, your script essentially emptied the instantaneous burst bucket before it had a chance to refill. Even though your minute-level allowance was fine, the sudden spike triggered the 429 error.
The header isn’t useless, but it only tells half the story when running high-speed scripts. The headers returned on successful HTTP 200 responses are only tracking your larger, sustained per-minute bucket, which is why it accurately reported 993 remaining. Because the smaller burst limits refill on a sub-second basis, standard HTTP headers cannot update fast enough on successful calls to warn you that the burst bucket is draining.
Absolutely not. If you wait for the minute-level reset timestamp on every call, your migration script will take much longer than necessary. Because the burst bucket refills continuously (usually within a second), your wait time after a 429 is actually very short.
To fix this in your script, you can replace your current logic with these two adjustments:
Add a small, fixed delay: Introduce a slight delay directly in your script’s loop, such as a 100ms or 200ms sleep between each API call. This naturally throttles your script just enough to stay under the burst limit threshold, preventing the 429 errors from happening in the first place.
Rely on theRetry-Afterheader, that you can check more about in our Back-Channel Login flow - Status Check documentation. If your script does hit a 429, do not look at the x-ratelimit-reset timestamp. Instead, configure your error handling to read the Retry-After HTTP header included in the 429 response itself. This header will tell you the exact number of seconds (often just 1 or 2) you need to pause before the burst bucket has recovered.
You can read more about how the token bucket algorithm handles these burst versus sustained scenarios in the Rate Limit Use Cases documentation.
I hope this helps and if you have further questions please let me know!
Best regards,
Remus