Hi,
We are currently exploring options for exporting Auth0 logs in a batched (block) format, rather than relying on real-time streaming via webhooks or integrations. Our goal is to quarterly retrieve and store logs as a single consolidated file in cloud storage (such as Google Cloud Storage), both for cost optimization and compliance purposes.
From what we understand, Auth0 primarily supports real-time log export via streaming integrations and the Management API with pagination. However, due to API rate limits and the need to optimize storage operations (e.g., reduce write operations to cloud buckets), we are looking for a more native or efficient way to export logs in batches.
Is there a recommended approach or native feature that supports scheduled, batched log exports directly from Auth0?
Thanks!!
Hi @pedro.lagomarsino
Thank you for reaching out to us!
There are a couple of way of going about streaming or exporting the logs from your tenant, if you would be interested in more details and a list of all Log Streaming services that are supported by Auth0, please check out our documentation on Log Streams. Streaming through a supported, third party provider could by more fit for your needs, as each offer different functionality.
As you are looking for a more in-house solution, please first keep in mind the Log Data Retention period, as it’s dependant on the tenant’s subscription level and Auth0 will only retain logs up to a maximum of 30 days.
This being said, currently the best way of exporting logs would indeed be through the Management API, despite it’s limitations ( see our documentation for more details ). Another tool, while not as powerful, but which could be of interest is the Real-time Webtask Logs Extension. While this relies on the user's input/request which can include sensitive information and thus requires care when handling the logs, it does work in real time.
If the external providers that are supported by Auth0 do not satisfy your needs, I do encourage you to submit a Feature Request, which are all monitored by our Product team and on which other users can cast their vote. If there is enough interest in a specific feature, it can accelerate it’s development and could potentially be added.
Hope this helped!
Gerald