Customs claims are lost after refresh token

I have an application where users can belong to several companies. For each company a user can have different permission.
I use post login action to redirect user to a page with the liste of user’s companies, user choose a company then during onContinuePostLogin , we enrich access token with companyId and permissions. It works, my problem is when refresh token happen, all my custom claim are lost.
I could store/retrieve from metada or a custom DB but not ideal IMO.