Auth0 Home Blog Docs

Rules are not being called

It seems like not all tenants affected have been fixed already.

For those who still have problems it is advisableI to remove and re-create the application that is having problems. Going for this solutions will delete all the settings you had for certain app in the dashboard but you can easily manage that by restoring all the settings using deploy-cli to backup clients and restore.

We’re still working on having all tenants-affected fixed. Will let you know once I have any update to share!

@konrad.sopala
I followed your workaround, so I removed the application in the dashboard, and recreated it.
(The rules now work.)
From my perspective this is good enough, since the app hasn’t gone public. But I’m not too happy about the possibility for something like this happen in future production, having more than 20h before a workaround is provided.
EDIT: clarified that the rules now work

1 Like

@olabackstrom as you see once the first accident was reported we immediately escalated it and the responsible team started monitoring that. Also we communicated regularly on our status page what’s happening.

The engineering team needed to detect how many tenants were affected and immediately started evaluating what fix should be implementing having time / severity / etc in mind.

One more thing I’d like to add is that if someone is a paid customer then it’s best to open a support ticket as there’s always someone at the watchtower who will be able to help more immediately especially in outages scenarios.

I don’t want to assure you saying anything that didn’t happen but that’s how the process looked like. What I can do is to once more apologise for any inconvenience! If you’d like I can also relay additional feedback to appropriate team.

1 Like

@konrad.sopala Is it possible, that rules are still not executed when accessing the /userinfo endpoint?

Let me reach back to the team and see if it’s possible. Will get back to you shortly!

Quick update

The engineering team is in the process of writing a script to fix left tenants that were affected by this issue. A quick fix to get unblocked is to delete and recreate the affected application/client as I mentioned above.

@konrad.sopala thanks for the feedback, and of reminding me that for production use there’s a separate channel with support tickets that could be used in a similar situation, and thus get faster feedback & trouble resolution.

Indeed, I’m on a free tenant as of now, and I think Auth0 has made it very easy to climb onboard, excellent tutorials and documentation. And good posts from you in the forum!

1 Like

Thanks a lot! Glad to hear that! We’re aiming at your successful developer experience and we know we’re pushing forward but there’s till work to do, so alway open for your feedback :slight_smile:

Hello,
I have noticed that this has been marked as resolved on the status page http://status.auth0.com/incidents/tjss6l7ck7lb?u=ryv9gfd9c0rr
However I am still experiencing this. I have a rule to add a claim to the token and it is run correctly when using the “try” rule but doesn’t work when requesting through the api “/oauth/token”.
The clients were created just before 2019-03-06T17:09:07.
Is re-creating the clients the only way to resolve this issue still?

Let me check it internally @wosborne for you and get back once I have any news to share!

Heads-up

The fix has already been made and it’s currently under review for being deployed to production. The only fix for tenants that are still affected is to re-create the clients, but as I mentioned above you don’t need to lose all the settings provided that you use deploy-cli to backup clients and restore.

Sorry for any inconvenience!

Status update

The team is only a few checks from the final deploy to production. We need to make sure that everything stays safe for all our developers. It should be there really soon!

@konrad.sopala I’m currently running on a free tier so I can understand that there might be a fix that is being applied for a few days. I wanted to ask, if there’s a non-free tier in your pricing that is not affected by this error or that has already been fixed? What I mean is, sorry for putting it bluntly, if I were a paying customer, would it affect my rules as well?

1 Like

@wosborne see my response at custom claims not added to access_token despite Rule - Rules are not run for Client Credentials Flow. You should use Hooks instead.

Hey @scooletz1!

No paying customers were harmed in the making of this incident. We caught the issue before it affected paying customers. The fix for the issue should be applied today. Let me know if you have any questions!

Status update

The team is executing the fix right now.

Status update

All tenants have been fixed at this point.

Hi @konrad.sopala,

Sorry but I am on a paid plan for my tenant and due to this we have a production outage. Also support is very slow reported hours ago . Feel free: https://support.auth0.com/tickets/00425540

Thank you @mikael for reporting that!

I’ve just escalated it and pinged our developer support team. Someone should reach out to you very shortly!

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.