Solved the issue, Thanks to this demo.
The problem was that I was using: quarkus.oidc.application-type=web-app in my application.properties file. I changed that to quarkus.oidc.application-type=service which is the default option, so you can actually just remove this line. Quarkus behaves differently based on this variable.