Fetch more than 1000 logs with a filter using checkpoint pagination

[archan]

Hi,

I am looking for a way to grab the most recent logs from the management API that fall under certain log types. I want to be able to filter the logs that come from checkpoint pagination, but the search filtering seems to only be in place when using the regular pagination, which is limited to 1k total records. Is there a way to add a filter to the checkpoint pagination query?

[dan.woda]

Hi @archan,

I would suggest leveraging an outside service for this. You can accomplish it by creating Log Streams and connecting to an external provider for filtering/monitoring or by leveraging the Retrieve Log Events Using the Management API to export the logs and then filter after the fact.

Hope this helps!

[archan]

The solution I’ve landed on is to create another query after 1000 events, starting from the last event, and to iterate through. Would this be acceptable or does the API block the new query as well?

[dan.woda]

What do you mean does it block the new query? If you show exactly the request you are trying to make I can test it on my side.

[archan]

Here’s the Python code I have written