Problem statement
Auth0 host’s this file for a given tenant ‘https://tenant_domain/robots.txt’;. Would it be possible to change the contents of this file?
Solution
Modifying the contents of the ‘robots.txt’ file is not supported at this time. This domain is only to be used by Auth0 API endpoints (Authentication API, Management API) which should not be crawled or indexed by search engines.