Welcome to the Auth0 Community!
I am sorry about the delayed response to your questions!
When setting the Idle Session Lifetime inside the tenant settings, the specified time should be the one after which an user accessing your application should have their session terminated and forced re-authentication. This is also mentioned in our documentation:
Idle Session Lifetime → Timeframe (in minutes) after which a user’s session will expire if they haven’t interacted with the Authorization Server. Will be superseded by system limits if over 4,320 minutes (3 days) for Essential or Professional or 144,000 minutes (100 days) for enterprise plans.
So as mentioned in this article as well, interaction with the authorization server specifically refers to requests to /authorize, meaning that you will have to include silent authentication to keep the user logged in, precisely how you already mentioned above. The scenario is also described here in our Session Lifetime Limits documentation.
If you need further clarification on the matter, let me know!
Kind regards,
Remus