>Providers like Oxylabs can be quite restrictive, preventing access to many of the common sites that scrapers choose to target.
Most of them seem pretty reasonable?
"Entertainment & streaming" - who's trying to scrape netflix's library?
"Banking and other financial institutions" / "Government websites" / "Mailing" - seems far more likely it'll be used for credential stuffing than for "scraping".
"Ticketing" - seems far more likely that it'll get used by scalpers than for scraping
The main targets of scraping - e-commerce sites (for price comparisons) and social media networks (for user generated content) are fine to scrape. Is there some use case I'm missing here? Is there a huge contingent of people wanting to scrape ticketmaster or bank of america?
I used the term "scrapers" pretty loosely, but yes, in many cases they are more bad actors than actual scrapers. However as they say the list may include other sites, I suspect Oxylab adds sites to the list at the site owners' requests (Amazon, Target, etc are likely to be on those lists)
Most of them seem pretty reasonable?
"Entertainment & streaming" - who's trying to scrape netflix's library?
"Banking and other financial institutions" / "Government websites" / "Mailing" - seems far more likely it'll be used for credential stuffing than for "scraping".
"Ticketing" - seems far more likely that it'll get used by scalpers than for scraping
The main targets of scraping - e-commerce sites (for price comparisons) and social media networks (for user generated content) are fine to scrape. Is there some use case I'm missing here? Is there a huge contingent of people wanting to scrape ticketmaster or bank of america?