Thank you for the additional information. Based on this knowledge article → Inactivity Timeout and Refresh Token Exchanges
Interaction with the authorization server specifically refers to requests to
/authorize,including silent authentication. Refresh token exchanges do not count as an interaction in this context. Therefore, the user session will not remain active if the only activity is refresh token exchanges.
To extend the session timeout, you’ll need to make a call to /authorize the endpoint with prompt=none.
Thanks
Dawid