Problem statement
I would need to modify the tenant’s robots.txt https://TENANT_DOMAIN/robots.txt for our tenant in order to add a domain verification to a 3rd party service that we’re connecting with. We don’t need to crawl anything from Auth0. We only need the 3rd party app to find the verification code inside the robots.txt and then allow us to use the Auth0 login form. Is this possible?
Solution
Unfortunately, this file can not be modified as of the current design. This domain is supposed to be used only for Auth0 API endpoints (authentication API, Management API) which should not be crawled or indexed by search engines.