Hey @Katje,
Welcome to the Auth0 Community!
While I understand the issue of having an API behind a reverse proxy, it’s not clear to me your specific issue 
In particular,
But that AddOpenIDConnect() part with the callback is a challenge. It means my website needs to understand some of the JWT while I want the Web API to handle it all.
What JWT do you refer to? The ID token or the access token?
If you are talking about the ID token, you shouldn’t send it to the Web API. The ID token is intended for the client (the web app in your case)
If you are talking about the access token, your web app shouldn’t decode it, because it’s intended for the Web API. I understand that you can’t access the .well-known URL from your API in your scenario. In this case, you can validate the access token by providing the security key after manually getting it from the .well-known URL (see this doc for more info)