Problem with last stage of SAML authentication

I have troubles running SSO authentication. The workflow looks like this:

1.User on website is being redirected to auth0 page and chooses connection from options presented.
2.After choosing the SAML connection is redirected to IdP to login, in this case Ping Identity.
3.After successful login is redirected back to auth0 and logged into app there.
4.Then redirected back to the website where it lands with code parameter in URL.

–I tried exchanging that code parameter for access token - I get the token using /oauth/token endpoint but it comes with empty payload, does not work with my API.

–I already have working password authentication with auth0, but the same auth0 library that recognises user as authenticated after using password, does not recognise same way SSO users after final redirection to website.

–SSO login does not redirect using URL with state parameter, only code parameter. The password authentication attaches both.

During the SSO process, monitoring logs look like this:
Screenshot 2024-08-09 at 13.23.42

I am not sure if the code delivered in parameter should be used that way.

The password authentication works fine with 0auth library in the background. The SAML authentication does not. I have a feeling I am doing something wrong in the last bit, but at this point I run out of ideas.