Azure Ad, Common Endpoint Home Realm Discovery domains are vanishing

When using the enterprise connection azure AD we encounter problems.

When adding the user’s email domain to the enterprise connection they vanish after some time with no message and adding them makes the login work again for other than the custom domain.

Then after some time they vanish again. This only happens if we enable common login endpoint.
We need this to allow to login from multi tenant applications from azure.

Anyone has any idea why this happens and how to fix it?