Implementing an API Gateway in ASP.NET Core with Ocelot

I have have been trying to implement IP Rate Limit on Ocelot with no success. The configuration below is not taking IP Address into account
“RateLimitOptions”: {
“ClientWhitelist”: [
],
“EnableRateLimiting”: true,
“Period”: “10s”,
“PeriodTimespan”: 1,
“Limit”: 1,
“HttpStatusCode”: 429
}

I tried this with a colleague, We both got rate limited even when we were not supposed to. We decided to log our IP address on the service behind the API Gateway (Ocelot), at least the service could pick up the different IP Addresses, maybe because we added this

“UpstreamHeaderTransform”: {
“X-Forwarded-For”: “{RemoteIpAddress}”
}

app.UseForwardedHeaders(new ForwardedHeadersOptions { ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto });

So now we are thinking of applying rate limit on the service behind. Which is really not idea as it means we are not getting the full benefits of using Ocelot for our API GW.

Can you please advise where we might be getting things wrong