Issue with Service Workers and accessing files on the server

We have a NextJS website built for our clients, that requires you to be authenticated to access any page (this was accomplished using NextJS’s middleware). Now, we are running into a problem where we want to make our web app into a PWA using a service worker, but when the service worker goes to fetch the manifest.json on the server it is forced to try and authenticate which fails. The best solution would be to be able to share cookies with the service worker, but is there a workaround for this other than to disable authentication for that one specific file?