I want to create a custom database connection for auth0 following this page Create Custom Database Connections.
where is the login script is executed. In auth0 service?
Can I connect to my localhost db or restful api?
The login script will be executed within the extensibility runtime associated with each Auth0 tenant so, yes, it will be executed within the service itself. This implies that whatever endpoint the script needs to connect to will need to be reachable over the Internet.
It should be possible for you to consider using a service such as (https://ngrok.com/) to make your localhost machine reachable over Internet.