Case 1: this sounds like correct behaviour.
Case 2: this also looks like correct behaviour - at least technically, though probably not what you want. While a failed social login creates a new user, this user is created under the social connection, not the database connection.
When you create an new via Mgmt API though, this user is created in the database connection. So yes, it allows you to create a new user, which is correct (as there is no other user existing with the same email address in this connection yet).
See this example: my user exists twice but under different connections. This is the same case like yours.
If you don’t want to allow this case, you need to create a pre-user registration hook (see Pre-User Registration) that checks if a user with the same email address already exists across all connections i.e. as social connection.
Furthermore, you probably want to have the merging rule enabled, to merge users with the same email address: