Global limit has been reached

Help
,
1

Hi there,

We are experiencing rate limit issues and would like to understand why they are happening.
We have a free tier tenant for testing and we quite regularly hit the Global rate limit error, which is quite odd because we are not even reaching the described 300 request/minute global rate limit.
The error can easily be reproduced by performing several simultaneous user update requests (through the Management API) and then force refresh an access_token using a valid refresh_token for the current user’s session.

We are using the Next.js Auth0 SDK and it reports an HTTP 500 error, not an HTTP 429 error we would expect for the rate limit issue:

OPError {error: 'access_denied', error_description: 'Global limit has been reached', message: 'access_denied (Global limit has been reached)', name: 'OPError', stack: 'OPError: access_denied (node_modules\openid-client\lib\client.js:997:22)'}

The response returned from the Auth0 OAuth Token endpoint is:

requestUrl: 'https://<tenant>.auth0.com/oauth/token'
statusCode: 500
statusMessage: 'Internal Server Error'
headers: {
  'X-RateLimit-Limit': 30
  'X-RateLimit-Remaining': 29
  'X-RateLimit-Reset': 1643756409
}
body: {error: 'access_denied', error_description: 'Global limit has been reached'}

According to the Rate Limit documentation, the global rate limit is 300 requests/minute, is that correct? Do both Management API and Authentication API calls count towards the global rate limit?

According to this page the Management API has 2req/sec limit and up to 10 request burst, and we see this when we perform more than 10 requests quickly within a second, but why does a subsequent token exchange fail with global rate limit error?

Any help is greatly appreciated to understand what is happening. If you need any further information, log or tenant ID, feel free to ping me.

Thank you,
Mark Czotter

3

Hi @mczotter,

Welcome to the Auth0 Community!

That is how I interpret the docs: The limit is global for the tenant and not per endpoint. The global rate limit applies to all Authentication API endpoints.

From our docs: Using the Management API for free and trial tenants is restricted to 2 requests per second (with bursts of up to 10 requests ). Exceeding these values triggers an HTTP 429 error, but the error message states, “Global limit has been reached.” These are in addition to those indicated in the rate limit response headers.

It sounds like the crux of the issue is that you are hitting the management API rate limit. Do you not see this error if you don’t breach the management API rate limit?

4

Hi @dan.woda,

Thank you for your quick reply.

Yes, we understand that is a global per tenant limit and not per endpoint.
Our problem is that the API responds with an HTTP 500 response “Global limit has been reached error” when another endpoint hits its own lower rate limit (in this case the 10 request burst).
We see the HTTP 429 rate limit errors for the rest of the user metadata update requests, but for the Auth Token exchange, we see an HTTP 500 error, which we think is an issue with the API.

Cheers,
Mark

5

I am not able to recreate the behavior you are describing. I can see this error after the management API limit is reached:

{
  statusCode: 429,
  error: 'Too Many Requests',
  message: 'Global limit has been reached',
  errorCode: 'too_many_requests'
}

But if I request an auth token directly after this request, I get it successfully. This is the behavior I would expect, given they are separate APIs.

I’m also trying to understand the scenario where you would request a new token immediately after a 429. Can you describe the use case for requesting a new token directly after receiving the 429 error?

Can you please provide a code snippet or screen grab of the behavior you are describing? If it contains sensitive information, feel free to DM it to me.

6

Yes, that error is displayed nicely from the Update User endpoint, which we can use to slow down requests, etc.

I’m also trying to understand the scenario where you would request a new token immediately after a 429. Can you describe the use case for requesting a new token directly after receiving the 429 error?

We have a very short-lived accessToken, ~3 minutes, that holds permission data from the user’s app_metadata. Administrators are able to change this information for many users at once, for example revoking permissions to certain resources via a bulk change. This is where the many Update
User call comes from. If a user’s accessToken expires shortly after the change and they request new data, they often get the mentioned HTTP 500 error, but don’t know why exactly. We understand that information like this would be better somewhere else, but at this stage our service can only rely on the metadata information stored in Auth0.

I’ll try to create a simple code snippet that reproduces the problem and get back to you.

Thanks,
Mark

1 Like
7

I wonder if tweaking the access token lifetime would help to alleviate this issue. For example, if these bulk change requests typically take 5 minutes to complete, a token lifetime of 6-7 minutes could prevent a new token from needing to be requested for a bulk update, without compromising the short-lived security of the token.

That would be great, thank you!

8

Additionally, here is the script I was using to test:

const axios = require('axios');

const getToken = async () => {
  const options = {
    method: 'POST',
    url: 'https://{DOMAIN}/oauth/token',
    headers: { 'content-type': 'application/json' },
    data: {
      client_id: '{CLIENT_ID}',
      client_secret: '{CLIENT_SECRET}',
      audience: 'https://{DOMAIN}us.auth0.com/api/v2/',
      grant_type: 'client_credentials',
    },
  };
  try {
    const res = await axios(options);
    console.log('Token Request: ',res.status)
    return res.data.access_token;
  } catch (error) {
    console.log(error);
  }
};

const getUsers = async (token) => {
  const options = {
    method: 'GET',
    url: 'https://{DOMAIN}/api/v2/users',
    headers: { authorization: `Bearer ${token}` },
  };
  axios(options)
    .then((response) => {
      console.log(response.data);
    })
    .catch((error) => {
      console.log(error.response.data);
      // Attempt to get a token on an error
      getToken()
    });
};

const testRateLimit = async () => {
  const token = await getToken();
  for (let i = 0; i < 11; i++) {
    await getUsers(token);
  }
};

testRateLimit();
1 Like