Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Providers like Oxylabs can be quite restrictive, preventing access to many of the common sites that scrapers choose to target.

https://faq.oxylabs.info/en/articles/8826164-restricted-targ...

Additionally I believe that $4/GB is an introductory price. When I went into the Oxylab dashboard, it showed me $8.



>Providers like Oxylabs can be quite restrictive, preventing access to many of the common sites that scrapers choose to target.

Most of them seem pretty reasonable?

"Entertainment & streaming" - who's trying to scrape netflix's library?

"Banking and other financial institutions" / "Government websites" / "Mailing" - seems far more likely it'll be used for credential stuffing than for "scraping".

"Ticketing" - seems far more likely that it'll get used by scalpers than for scraping

The main targets of scraping - e-commerce sites (for price comparisons) and social media networks (for user generated content) are fine to scrape. Is there some use case I'm missing here? Is there a huge contingent of people wanting to scrape ticketmaster or bank of america?


I used the term "scrapers" pretty loosely, but yes, in many cases they are more bad actors than actual scrapers. However as they say the list may include other sites, I suspect Oxylab adds sites to the list at the site owners' requests (Amazon, Target, etc are likely to be on those lists)


Hmm that’s unfortunate. I’m actually scaling up a data journalism project this month which is why I’ve been looking at these.

I’m curious if you can suggest a happy medium between curl-impersonate on VPS (dirt cheap) vs residential proxy ($8/gb)?

Personally I’m not trying to hit any of those common sites.




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: