Cors error with angular ng serve + .Net with cookie authentication flow

We’re using the cookie auth flow with .net and angular and the Auth0.AspNetCore.Authentication nuget package to setup the integration with auth0.
We have angular app configured to be served out of backend and in that scenario we don’t have any issues during the authentication.
But when run the client app with ng serve then we run into this CORS issue when the client app is being redirected to auth0 for login:

Access to XMLHttpRequest at 'https://dev-eis.us.auth0.com/authorize?client_id=eVsTLW9GmpqrDLsqvNH4qShCWu4TuMl1&redirect_uri=https%3A%2F%2Flocalhost%3A44363%2Fcallback&response_type=id_token&scope=openid%20profile%20email&response_mode=form_post&nonce=638090793172114039.OWI0NzYyZmMtZDhiOS00OTc3LTlhOGQtYTVhYTk3Y2ExOWNkOThkNmZjY2QtODQ0MC00OWRiLWFkYzMtZTgxYTUxODZkNTg2&auth0Client=eyJuYW1lIjoiYXNwbmV0Y29yZS1hdXRoZW50aWNhdGlvbiIsInZlcnNpb24iOiIxLjAuMCJ9&state=CfDJ8JgzcQ08u8FFjpWwkm5zEN1ir5f5kSt11fQfzTcBGKKWXyyADnzzrrwrYPuf0qduOKiC2PI5Gu9RQalNtNgD5vOO4TQz6yc1pEfhloixuY1XlIay89tB8bbfabsXkADUxUVCcljopjBpt9J-V49aC9TwaR5aCcKDXU1BNTyvbUH7X-lOXLAGSFF4idz7Hc4GsZxSid4e1h9dcexPJJVLRSGN7NqEAe2EG6mociT5WHnvDPdg8nlQydoaJ6xWjl3pEKuxWZDTYo4EO_80zcEZA5rTb2b9S-1J1iRp4kLidRBJi8k5BPKgz24brduRbwsQAQ&x-client-SKU=ID_NETSTANDARD2_0&x-client-ver=6.10.2.0' (redirected from 'https://localhost:4200/account/user') from origin 'https://localhost:4200' has been blocked by CORS policy: The 'Access-Control-Allow-Origin' header has a value 'https://localhost:4200' that is not equal to the supplied origin.

Everything on auth0 side looks configured properly. We have allowed this URL as the allowed in CORS config.

Any ideas on how we can resolve this?