That’s correct! When a user is presented a login page, he or she will have the option to either enter existing credentials, or use the ones already established with the other tenant.
Unlike social connections, you’re not going to get the button for free in lock. However, it’s very simple to add an element for your custom connection. Also remember that you have the ability to initiate login with the &connection parameter, so if you have a direct link from tenantA → tenantB, that may be a suitable option as well.