Quick question: are web crawlers allowed on a Auth0 protected site?
I ask this because I am interested on implementing SEO to my Auth0 protected site. As some of you may know, web crawlers (such as for Google and Bing) need to crawl your site in order to index it on the search engine. I was wondering if those types of crawlers are allowed to enter on an Auth0 protected site (for SEO purposes).
@dan.woda Hello again, I wondered if the Tenant’s Auth0 domain (ex: example-co.au.auth0.com) has a robots.txt file preventing Google from crawling the site. Just curious .