In a scenario that involves user authentication, according to OpenID Connect (OIDC), and API authorization, according to OAuth2, there will be, in general, at least two tokens issues; an ID token and an access token.
The ID token (per OIDC will always be represented as a JWT token). The signature algorithm used to sign the ID token can be configured at the client level; this token will carry information about the authenticated user and is meant to be validated and parsed by the client application itself.
The access token issued to a target API/resource server at this time will also be represented as a JWT, although OAuth2 does not impose that so the format could in theory be another. The signature algorithm to sign the JWT access token can be configured at the API level; this token is issued to the client application, but the application does not parse it, it only uses it to send to the target API as means to authorize the request. It’s the target API that will validate and parse the JWT access token.
In conclusion, it’s possible and acceptable to use different signing algorithms for the ID token issued for the web application to consume versus the JWT access token issued for the API to consume.