Unable to get response from .well-known/openid-configuration

[alz-taskquark]

Hi!

Starting a couple days ago I’m having trouble getting any response from the URL:
https://taskquark.eu.auth0.com/.well-known/openid-configuration

The Spring Boot application, when it starts locally in this machine, gets blocked trying to get the configuration request:

2020-07-19 05:51:05.430 DEBUG 23199 — [ main] o.s.web.client.RestTemplate : HTTP GET https://taskquark.eu.auth0.com/.well-known/openid-configuration
2020-07-19 05:51:05.440 DEBUG 23199 — [ main] o.s.web.client.RestTemplate : Accept=[application/json, application/*+json]

After a couple minutes it just timeouts. No error, no info:
…
Caused by: org.springframework.web.client.ResourceAccessException: I/O error on GET request for “https://taskquark.eu.auth0.com/.well-known/openid-configuration”: Connection timed out (Connection timed out); nested exception is java.net.ConnectException: Connection timed out (Connection timed out)
…
Caused by: java.net.ConnectException: Connection timed out (Connection timed out)

Updating Spring Boot, JRE, etc. solves nothing. And the very same Spring Boot JAR when deployed in prod or stage servers works perfectly.

Moreover, a simple cURL request from the console also gets locked with no response at all from auth0.com. Same block happens when using wget with the same URL.

So I’ve checked the Anomaly Detection at the tenant webpage and even whilelisted the IP, in case any development problem would have marked this IP as suspicious, but the problem persists.

Interestingly enough, in this same machine, Firefox is indeed capable of getting the JSON response from this URL with no problem. Also, a VM started in this host (using the same public IP) works correctly.

Everything points to this machine OS, but getting no error, response message, etc. just a blocked connection does not help to solve the issue.

My question is what condition could trigger this behavior from Auht0.com servers, i.e. blocking some connection from JRE, curl, wget but not blocking Firefox requests in a machine?

Thanks!
Alz

[john.gateley]

Hey Alz,

The fact that firefox and the VM can get the URL indicates it is not being blocked on Auth0’s side - there is no way to tell the difference between those calls and the ones being blocked.

This sounds like a firewall issue to me. I’m not sure what OS you are using, but I’d start there, checking system logs, or using tcpdump/similar to check the ip traffic to see what is being sent out/received.

John

[alz-taskquark]

Hi John,

Thanks for having a look at this.

I agree that some firewall was blocking the connections from this machine. But after checking the system, there is no specific firewall (iptables are clean) on this side.

A “strace” for a “openssl s_connect” simply showed the connection being made but the packets being dropped by your firewall. I mean the packets where not rejected, nor any error response was sent. just the classical DROP action of a firewall.

I just checked again today using the machine and now the connections are made perfectly as they were a week ago. No changes and the Spring Boot application loads the configuration:

2020-07-26 08:59:16.244 DEBUG 14845 — [ main] o.s.web.client.RestTemplate : HTTP GET https://taskquark.eu.auth0.com/.well-known/openid-configuration
2020-07-26 08:59:16.253 DEBUG 14845 — [ main] o.s.web.client.RestTemplate : Accept=[application/json, application/*+json]
2020-07-26 08:59:16.529 DEBUG 14845 — [ main] o.s.web.client.RestTemplate : Response 200 OK

and curl, wget, etc get the response immediately at this machine.

So I assume that for some condition these connections were set to silently drop packets, but now the normal behavior is recovered.

Problem solved. Reason unknown.

Thanks!
Alz.

[john.gateley]

That’s very weird.

I don’t know of anything on Auth0’s side that would do that.

Let us know if it happens again and we can debug further.

John