Hi @rohan.dsouza
Welcome to the Auth0 Community!
You are asking whether Auth0 log IDs expire or become invalid, and whether you can reliably use a log ID in the from parameter of the /api/v2/logs endpoint to fetch logs with IDs greater than the specified value without worrying about expiry. You tested this in your free tenant and were able to successfully retrieve logs using any log ID in the from parameter.
Log IDs do not expire and remain valid indefinitely. You can use any log ID in the from parameter of the /api/v2/logs endpoint to retrieve logs with IDs greater than that value. However, the endpoint will only return logs that are still within your tenant’s log retention period. Log IDs themselves never become invalid, but the log entries they reference may be deleted based on your subscription’s retention policy.
Root Cause:
Auth0’s log retention is based on subscription type, not on log ID validity:
- Log IDs are permanent identifiers — Log IDs are assigned sequentially and never expire. They serve as checkpoints for pagination and remain valid references even if the logs they point to have been deleted.
- Log retention is subscription-dependent — The actual log entries are retained for different periods depending on your subscription plan:
- Free plan: Typically 2-3 days
- Developer plan: Typically 7 days
- Professional/Enterprise plans: Longer retention periods (varies by plan)
from parameter uses checkpoint-based pagination — The from parameter is designed for checkpoint-based pagination. It retrieves logs with IDs greater than the specified value, regardless of how old that log ID is.- Behavior with old log IDs — If you use a log ID from a log entry that has already been deleted (outside the retention period), the endpoint will:
- Accept the log ID without error
- Return logs with IDs greater than the specified value
- Only return logs that are still within the retention period
- Return an empty result if no logs exist with IDs greater than the specified value within the retention period
How the from Parameter Works:
Scenario 1: Log ID within retention period
from=LOG_ID_123 (created 1 day ago)
→ Returns all logs with ID > LOG_ID_123 that are still retained
Scenario 2: Log ID outside retention period (deleted)
from=LOG_ID_456 (created 10 days ago, now deleted)
→ Returns all logs with ID > LOG_ID_456 that are still retained
→ May return empty if no newer logs exist within retention period
Scenario 3: Log ID in the future (doesn't exist yet)
from=LOG_ID_999 (not yet created)
→ Returns empty result (no logs with ID > LOG_ID_999)
Recommended Usage Pattern:
For reliable log retrieval with checkpoint-based pagination:
// Step 1: Get the first batch of logs
const firstBatch = await fetch(
'https://YOUR_DOMAIN/api/v2/logs?take=100'
);
// Step 2: Extract the last log ID from the response
const logs = await firstBatch.json();
const lastLogId = logs[logs.length - 1].log_id;
// Step 3: Use the last log ID as the checkpoint for the next batch
const secondBatch = await fetch(
`https://YOUR_DOMAIN/api/v2/logs?from=${lastLogId}&take=100`
);
// Step 4: Repeat until no more logs are returned
Important Considerations:
- Log IDs are sequential — Log IDs increase over time, so using an older log ID will retrieve all newer logs (within retention).
- Retention period is the limiting factor — You cannot retrieve logs older than your subscription’s retention period, even if you have their log IDs.
- Empty results don’t mean invalid ID — If you get an empty result, it means either:
- No logs exist with IDs greater than the specified value
- All logs with greater IDs have been deleted due to retention policy
- The log ID is in the future (not yet created)
- Checkpoint method is preferred for older logs — The documentation recommends using the checkpoint method (with
from parameter) when you need to retrieve logs beyond the 1,000 search result limit.
- Search criteria vs. checkpoint pagination — There are two ways to query logs:
- Search criteria (time range, type, etc.): Limited to 100 results per request, 1,000 total results
- Checkpoint pagination (using
from parameter): No result limit, but only returns logs within retention period
Best Practice for Continuous Log Retrieval:
If you’re building a system to continuously retrieve logs:
// Store the last log ID you've processed
let lastProcessedLogId = null;
async function fetchNewLogs() {
const url = lastProcessedLogId
? `https://YOUR_DOMAIN/api/v2/logs?from=${lastProcessedLogId}&take=100`
: `https://YOUR_DOMAIN/api/v2/logs?take=100`;
const response = await fetch(url, {
headers: {
'Authorization': `Bearer ${accessToken}`
}
});
const logs = await response.json();
if (logs.length > 0) {
// Process logs
logs.forEach(log => {
console.log(log);
});
// Update checkpoint for next retrieval
lastProcessedLogId = logs[logs.length - 1].log_id;
}
return logs;
}
// Call periodically
setInterval(fetchNewLogs, 60000); // Every minute
Limitations:
- Retention period is hard limit — You cannot retrieve logs older than your subscription allows, even with valid log IDs.
- No guarantee of completeness — If logs are deleted due to retention policy, you may miss events that occurred outside the retention window.
- Real-time consistency — If you’re querying very frequently, you may occasionally miss logs due to timing issues.
Solution for Long-Term Log Storage:
If you need to retain logs beyond Auth0’s retention period:
- Use Log Streaming — Configure Auth0 Log Streaming to send logs to external services (Splunk, Datadog, etc.)
- Use Extensions — Auth0 provides extensions that automatically export logs to various destinations
- Implement periodic export — Regularly export logs to your own database or storage system
Log IDs never expire and remain valid indefinitely. You can safely use any log ID in the from parameter to retrieve newer logs, but the endpoint will only return logs within your subscription’s retention period.
Kind Regards,
Nik