Hey there @vitalyk !
Hmm, alright!
Are you aware of a cookie mechanism on the SP side allowing the user to browse through the content? There is a short doc here describing different mechanisms that determine the user login session (the native purpose of this doc was to describe what’s happening after blocking a user, but it would give you also a context for your use case).
Let’s also check the behavior following these steps:
(EDIT: After clearing the browsers cookies):
- Please log in to your SP on the Firefox,
- Open a new tab and, while having the browser developer tool (Network tab) open, please provide the SP URL (simply opening your app in a new tab)
- Please search for an authorization call.
What does this call look like? What cookie (if any) is attached to the request? Are you redirected to any login page? Did you have to provide credentials once more?
You can also repeat the steps after reopening the browser.
(One more thing I can think of is the federated log out (which means if you log out on the SP side, it would also log you out form the IdP side).
Please let us know your findings so we know more about what’s happening! 
Also feel free to ask if you have any other questions on the topic!
PS:
A refresh token rotation mechanism is available, but its purpose is to return a new access token (with SAML, IdP returns assertions). I need to investigate if a client app that is set to talk SAML can use this mechanism.