Yes, the audience should be that of the resource server or API the access token is meant to be used to access.