OTP requests limit

We are using OTP codes for our users to login to our apps.
I am currently working on test automation. We have white listed the office IP in our test tenant but I seem to be running into some type of throttling/blocking. Seems to happen after 15-20 minutes of running the automation tests. It also seems to happen at times if I create a user ahead of the login using the APIs and request another OTP immediately after to login in the UI. Are there specific settings I can set to allow our automated tests to run without the throttles in our test tenant?