The scraping system we use has randomly assigned IP addresses, and has a built in 'back-off' mechanism to ensure responsible scraping which will not overwhelm your website with requests. As a result there is currently no need to whitelist our scraper.
What is the IP address of your scraper? Do I need to whitelist it?
Written by Ian Pierson
Updated over 12 months ago