Hi @subhajit.m , welcome!
Yes. Both websites would have to use the same Auth0-hosted login Page (New Universal Login page).
The two apps would need to use the /authorize endpoint and login with redirect (as a result of calling the /authorize endpoint, users will be redirected to the login page and after providing credentials, redirected back to your app).
If a user is already logged in to one app this way, and attempts to log in to the second app, they will be redirected to the Auth0 login page and then immediately redirected to the app without providing credentials.
One thing to take into consideration is that sso is cookies based so it will be a problem for users using browsers prohibiting third party cookies. Refresh token rotation can be a workaround. One of our community member has shared his set up that works in this scenario.
Please let us know if you have any questions!