ASP.NET Core 2: Intermittent Correlation Failed Errors

  • Which SDK does this apply to? Auth0.AuthenticationApi v5.4 / Lock 11.3
  • Which version of the platform are you facing this error on? .NET Core 2
  • Was this code working before? Have you made any changes in the dashboard recently? I think this error has always occurred intermittently.
  • HAR: I will keep trying on this. Because of the intermittent nature of the problem, this is a little bit tougher. I do have a HAR where the intermittent problem does not occur… Let me know if you would like me to include that. Otherwise, I’m sure I will get this eventually.

I am seeing intermittent Correlation Failed errors in production. The code we’re using comes straight out of the ASP.NET Core 2 quickstart samples.

For the most part, this happens in Chrome, although when I try this on Safari (on a phone) it seems much more prevalent… Mobile is not our target right now, but I would say Safari seems to be pretty much broken right now.
To reproduce this, when I go to my site after several hours (maybe a whole day?) of inactivity, It shows the “you last logged in with…” dialog, and then when I click my email address, it crashes.

Here’s the trace, if that helps:

Exception: Correlation failed.
Microsoft.AspNetCore.Authentication.RemoteAuthenticationHandler+d__12.MoveNext()
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
System.Runtime.CompilerServices.TaskAwaiter.GetResult()
Microsoft.AspNetCore.Authentication.AuthenticationMiddleware+d__6.MoveNext()
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
System.Runtime.CompilerServices.TaskAwaiter.GetResult()
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware+d__7.MoveNext()

I have also seen an “oops! something went wrong!” page once or twice.

I understand this issue may be cookie related, so I thought I would also include my cookies:

.AspNetCore.Correlation.Auth0.9vfQ2_WqWLP6U1UXAi29WBtXqsuo97f2txMiHCiovP0 | N
.AspNetCore.OpenIdConnect.Nonce.CfDJ8DczvQ50mOVLnddvitCTG7Kp9Mc2yKSennC7oggfuaXdypiCvst-PX2Hva-N9UxUe20wXELi1G6LHg8W_XHmCtbv9MN3lROrFKfrEaTf8MPue32mGPFLqMjiPIGJTuKlAxaC4r-kQMIHH1_MJzZV44mrQ5b1Jq8tPXgT0vZcAeC-QblWwgFWJN8PzdG0b2NOCog3p-0QQPVkuZr_D2v65roPEi-1nstgExcxlme_w_zzH98Fit3FcXKBPXq6TQY0oVfTHWpKLyPUj1dPg1WLCBc | N
ARRAffinity | 7055382eb098285fb6aecb1d7b22ab0e092973bf2838304a05303973781a3ebb

Here’s my config:

using System;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Authentication.Cookies;
using Microsoft.AspNetCore.Authentication.OpenIdConnect;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.IdentityModel.Tokens;

namespace Keystone.Web.Config.IoC
{
    internal static class Auth0Config
    {
        public static void ConfigureAuth0(IServiceCollection services, IConfiguration configuration)
        {
            services.AddAuthentication(options =>
                {
                    options.DefaultAuthenticateScheme = CookieAuthenticationDefaults.AuthenticationScheme;
                    options.DefaultSignInScheme = CookieAuthenticationDefaults.AuthenticationScheme;
                    options.DefaultChallengeScheme = CookieAuthenticationDefaults.AuthenticationScheme;
                })
                .AddCookie()
                .AddOpenIdConnect("Auth0", options =>
                {
                    // Set the authority to your Auth0 domain
                    options.Authority = $"https://{configuration["Auth0:Domain"]}";

                    // Configure the Auth0 Client ID and Client Secret
                    options.ClientId = configuration["Auth0:ClientId"];
                    options.ClientSecret = configuration["Auth0:ClientSecret"];

                    // Set response type to code
                    options.ResponseType = "code";

                    // Configure the scope
                    options.Scope.Clear();
                    options.Scope.Add("openid");
                    options.Scope.Add("profile");
                    options.Scope.Add("email");

                    // Also ensure that you have added the URL as an Allowed Callback URL in your Auth0 dashboard 
                    // e.g. http://localhost:5000/signin-auth0 
                    options.CallbackPath = new PathString("/signin-auth0");

                    // Configure the Claims Issuer to be Auth0
                    options.ClaimsIssuer = "Auth0";

                    // Saves tokens to the AuthenticationProperties
                    options.SaveTokens = true;

                    //Signature validation for HS256 signed token

                   options.TokenValidationParameters = new TokenValidationParameters
                   {
                       NameClaimType = "name",
                       RoleClaimType = "https://schemas.quickstarts.com/roles"
                   };

                    options.Events = new OpenIdConnectEvents
                    {
                        // handle the logout redirection 
                        OnRedirectToIdentityProviderForSignOut = (context) =>
                        {
                            // This is for "flowing the API token" which we are apparently not using.
                            //context.ProtocolMessage.SetParameter("audience", "{YOUR_API_IDENTIFIER}"); 

                            var logoutUri = $"https://{configuration["Auth0:Domain"]}/v2/logout?client_id={configuration["Auth0:ClientId"]}";

                            var postLogoutUri = context.Properties.RedirectUri;
                            if (!String.IsNullOrEmpty(postLogoutUri))
                            {
                                if (postLogoutUri.StartsWith("/"))
                                {
                                    // transform to absolute
                                    var request = context.Request;
                                    postLogoutUri = request.Scheme + "://" + request.Host + request.PathBase + postLogoutUri;
                                }
                                logoutUri += $"&returnTo={ Uri.EscapeDataString(postLogoutUri)}";
                            }

                            context.Response.Redirect(logoutUri);
                            context.HandleResponse();

                            return Task.CompletedTask;
                        }
                    };
                });
        }
    }
}

Also: I am currently on a free account, and we aren’t using SSL/HTTPS yet.

I’ve created a HAR file showing the problem: Dropbox - File Deleted

@brianm_accounts I am getting someone to take a look into this

@brianm_accounts looks like you got a response in the Github issue:

https://github.com/auth0-samples/auth0-aspnetcore-mvc-samples/issues/35#issuecomment-396962186

@jerdog Thank you for having someone take a look at this. The github response really did contain anything that will help be resolve this… I hope someone will look at the HAR file I included. I am using the configuration from the quickstart, so if I am having problems, I must not be alone.

:wave: @brianm_accounts just wanted to follow up and see if you were able to solve your intermittent errors? I saw the github issue was closed, but I wanted to check here to see.

No, it isn’t resolved. Before my project goes into production I will have to either solve this, or switch to a different auth provider, unfortunately.

It seemed like I might get better support if I signed up for a paid account - on the fence between doubling down that way or walking away. What do you think Kim?

:wave: @brianm_accounts thank you for getting back to me and letting me know. I’d like to see what can be done from my side before anything. I apologize I’ve let this issue slip through. I didn’t have enough time to look at the complete issue yet to know what needs to be done to work through to a solution , so I am going to work with my team here to do just that. When do you need to have the project in production?

@brianm_accounts The GH mentions that your app might be behind a web farm. Have you tried configuring the Data Protection to use a common key ring? When using the default settings each instance gets its own, isolated key ring, so if one node receives a cookie or state generated in another node, it will fail decrypting it.

See Configure ASP.NET Core Data Protection | Microsoft Learn for various options for using a shared key ring.

You also mention periods of inactivity in your app when this happens. When there is such inactivity period, does the app stays put in Auth0’s login page? I.e.:

  • User stops using the app
  • App session lost or some other event causes a redirect to Auth0 for a new token.
  • User is no longer there so the long wait happens at Auth0’s login page
  • After many hours, user clicks on “Last time you logged in with …”
2 Likes

I thought I had replied to this, but I must not have, sorry! I have probably 30 days. Technically, it is already in production but it isn’t in use. Yet.

@nicolas_sabena Thanks for the reply. It is hosted in Azure App Services, so it might be possible that, after a period of inactivity, the code could end up running on different hardware… It seems unlikely that it would occur quite so frequently, but I’m willing to try the common key ring idea using blob storage.

It seems like you would see this issue a lot with Azure customers though. Am I the only one with this problem?

As for your question about whether the user says on the Auth0 login page, the answer is no. I am typically closing the browser, leaving work, and maybe trying again in the morning with (probably) a new instance of Chrome.

Is it possible that this could have something to do with me fiddling with timeouts? I had to do some kind of odd sign-up flows (invite only user creation which involved bastardizing elements of forgot password), and I think I made the lifetime of something or another go for multiple days. At this point I’m not sure what exactly I changed, that was some time ago. :slight_smile:

Is it possible that this could have something to do with me fiddling with timeouts? I had to do some kind of odd sign-up flows (invite only user creation which involved bastardizing elements of forgot password), and I think I made the lifetime of something or another go for multiple days. At this point I’m not sure what exactly I changed, that was some time ago.

It’s difficult to say, but unlikely. But I’ve been checking the source code for the .Net OpenIdConnect middleware (which we use out-of-the-box in the quickstarts, without putting anything on top), and my previous guess for the decryption failure might be incorrect.

When you do the “authentication challenge” in the /Account/Login controller (or wherever), the middleware will:

  • Create a CorrelationId, a random and unguessable string, that is stored in a cookie.
  • Create a state string that contains encrypted properties related to the authentication event (values that are needed when processing the authorization server response). Among these properties, the CorrelationId is included.
  • Create a nonce (a cryptographically secure random value) and store it, encrypted, in a cookie, if applicable.
  • Redirect to the authorization server to request a token, providing the state in the request.
  • [User completes all the interactive tasks]
  • [The authorization server sends a response back to the application, with the state exactly as it was provided originally as part of the response]
  • Among the checks that are performed on the callback processing, the OpenIdConnect middleware will:
    • Decrypt the state. Verify that it can be decrypted correctly. If this fails you will get a Message state is invalid or similar error.
    • Check that the CorrelationId found in the state matches a cookie with that value. If this fails you will get a Correlation failed. error.
    • Decrypt the nonce stored in a cookie, if applicable.
    • Validate the received id_token if applicable, checking that the nonce in the token from the response matches the nonce retrieved in the step above.

Sorry for the blob of text there. But, as you can see, the Correlation failed. means that, basically, the middleware could not match a correlation id retrieved from the (properly received and decrypted) state in the response with a correlation id stored locally in the browser (as a cookie).
The state was properly decrypted, so no issues in the cryptographic keys (and no need to implement a key ring for now).

The correlation cookie has a timeout of 15 minutes, so the full authentication flow is expected to finish in that time. Is it possible that the sign-up flow takes more than that?
Another option would be that, for some unexpected user action, more than one response with the same state is returned to the application, because the correlation cookie is removed after being validated the first time. Returning the same state twice might happen if the user clicks on the back button at some point and reposts the form with the authentication results.

@nicolas_sabena

Hmm, I am definitely not waiting 15 minutes. The flow is: I open the browser, type in the URL for the site. The version of the challenge comes up that lets me just click on my email address to continue, and a few seconds later I select brianm@getkinetiq.com … Redirections occur (I can see a few urls flash through the address bar), and then the site comes up with that Correlation Failed error.

Note the problem only happens when we’re in that mode where I can click my email address to continue, and only about half the time. I don’t know what that flow is, but to be very clear, I never have to enter my password. If there’s a way to turn it off, that might actually solve the problem…!

@nicolas_sabena

By the way, that blob of text was very helpful and taught me a lot about auth very quickly, so thanks.

So my question to you is, what’s happening with the CorrelationId cookie in the case where that “continue as brianm@getkinetiq.com” flow occurs?

Would it be possible to generate a new .HAR file with a repro? Make sure that the “Preserve log” option is selected, to see the whole traffic (starting at the application, before the redirect to /authorize). (The .HAR file provided before only has the POST to /signin-auth0, so most of the context is missing).
Make sure you don’t enter any passwords during the flow and, if you do, remove them from the .HAR file before attaching.

@nicolas_sabena Yes, I will work on that. It might take a few days since the problem only occurs after a long period of inactivity, and even then only rarely.

Actually… I should be able to make that happen more often by changing an auth timeout. I bet it defaults to 24h… Any idea what the easiest way to do that would be, given the .NET Core 2 Auth0 starter project?

I took a look and I see several settings it could be. Also, I suppose this could be an Auth0-level setting…

It seems that you are looking to change the expiration in the ASP.Net session ticket, which would be like this:

  .AddCookie(options => {
      options.ExpireTimeSpan = new TimeSpan(0,2,0); // 2 minutes in this example
  })

Hope that works.

@nicolas_sabena

Well, I am going to do more tests, but I have tried to repro this at least 5 times and so far I haven’t gotten the issue. I will try 5 more times.

I am getting the prompt to click my email address to continue after the 2m cookie timeout.

Is it possible that reducing the cookie has somehow solved the problem? Maybe the cookie timeout I had was greater than some other auth0 timeout and that was causing problems…?

Is it possible that reducing the cookie has somehow solved the problem? Maybe the cookie timeout I had was greater than some other auth0 timeout and that was causing problems…?

I don’t think the Auth0 timeout has any particular effect in this scenario. If the Auth0 session is expired by the time the ASP.Net requests a new token, Auth0 will simply re-prompt for credentials, but the end result is the same: a response is sent to the application. In any case, I’m just guessing. But the error is happening at the ASP.Net side when the correlation cookie can’t be found.