Not getting refresh token or ID token when user is part of multiple organizations

I am using the Authorization Code Flow for my application with Auth0 Organizations, where the login flow is set for Business Users > Prompt for Credentials.

I am hitting the /authorize endpoint correctly with scope=openid offline_access. When I test with a user who is only part of one organization, the organization picker does not show up, which is expected. When I hit the oauth/token endpoint with the resulting authorization code, I correctly get a refresh_token and id_token in the response.

However, when I test with a user who is part of multiple organizations, and I pick one via the organization picker, I suddenly lose refresh_token and id_token in the response. The access_token I get does indicate org_id is set correctly though, so I know that the user has been authenticated successfully through the selected organization.

Why am I not getting a refresh_token or id_token with a user who is part of multiple organizations. Is there some additional configuration that I am missing? Any help appreciated, thanks.