The relying party from the perspective of OpenID Connect (OIDC) in your scenario would be the client application as the API gateways is more an OAuth2 resource server that authorizes requests based on the presence of a bearer token.
It’s also a inherent characteristic of bearer tokens that anyone in possession of that token can perform an authorized request. There are efforts around token binding that would prevent the token from being used by a third-party, however, this is not widely available yet.
In relation to the specification you linked to, although I’ve already seen discussions about it internally this is not yet available in Auth0 and I’m also unable to provide you with a definitive timeline/outcome.
However, you have to have in mind that even if your API gateway received a session end termination notification the API gateway itself is not maintaining any user session (it’s authorizing requests based on bearer tokens) so it would not be straightforward to achieve what you really want (preventing a third-party from obtaining and reusing a bearer token).
The mitigation available for that scenario at this point is for you to use short-lived access tokens that can be renewed by the client application while the user is logged in. This would mean that after logout the application or a third-party would no longer be able to obtain more tokens and the one available in local storage would also have such a limited lifetime that the attack in question would be more difficult and even if achieved would be of limited duration.
Another approach is for you to issue access tokens with a unique identifier (see this reference doc on how to include custom claims) and then upon client application logout request to the API gateway to blacklist the access token with the unique identifier in question. This way the API gateway could reject future requests containing the blacklisted identifier. However, this implies the API does more than just validate a bearer token, it would also need to check it against a blacklist.