SSO Dashboard Application

Hi there, I am using the SSO Dashboard to setup a login dashboard.

I have set up an Office 365 saml connection, tested it and get the “It Works!” Screen, but when I use the application with the same SAML Connection on the auth0 SSO Dashboard it prompts me to log into my office 365 account every time. I thought that once I logged into the dashboard itself and clicked the application, that I would automatically be logged in. Am I missing any settings or something?

When I click the application with the connection that I tested, I get the following message
also, I get this message currently,

Sorry, but we’re having trouble signing you in.

AADSTS50107: The requested federation realm object ‘urn:dev-*******’ does not exist.