It looks like if we specify the organization ID in the React component we use, SSO works.
<Auth0Provider
organization='org_xxxxxxxxxxxxxxxx'
domain={auth0Url}
clientId={auth0ClientId}
audience={auth0Audience}>
...
</Auth0Provider>
I am guessing this is all because the cookie on the Auth Server (as explained in this article) is organization specific.
Unfortunately, we don’t know what org the user belongs to prior to login. We are exploring different approaches that might solve this for our case, but the solutions listed in this comment would probably also help us out a ton: Support/Replace Organization prompt screen with "choose organization" during login - #17 by adam.housman
Is there a reason the cookie needs to be organization specific? Any suggestions that would make it so SSO just works in our use case, without having to specify the organization?