Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Sounds like pretty sophisticated filtering!

I’m working on a scraping project at the moment so looking at this too but from the other end. Super low volume though so pretty tame - emphasis on success rate more than throughput

I bought a 4G dongle for use as last resort if nothing else gets through. And also investigating ipv6



Using a 4G dongle makes it easier to hide in the crowd indeed. Since your traffic will go through heavily shared mobile IPs, probably with thousands of users behind them, anti-bot vendors won't/shouldn't block per IP, but per fingerprint/session cookie instead.


Ah hadn’t realised it’s the NAT. I thought it’s because the IPs are dynamic and rotate too much. Interesting.

Currently planning on doing a layered approach. Cloud IPs first etc.

Interesting challenge but also trying to be somewhat respectful about it since nobody likes aggressive bots




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: