I'd be rather surprised if IPv6 hasn't done some damage to the idea of IP blocking on the whole. It's possible, even as a residential Internet user, to request a /56 or /48 automatically with DHCPv6 Prefix Delegation. I have a /56 with Comcast. That's potentially up to 65536 /64 blocks, just from a residential user, so if you're going to attempt IP filtering for IPv6, it's got to be a lot smarter than swapping out your single-IP blocking for /64 blocking.
Doesnt CGNAT make these methods obselete though? All my webscraping is proxied through my phone and I rarely get IP blocked and im very aggressive even on CF protected sites.
That's neat, could you explain how you achieved that?
I tried a year or so ago and had to round trip to my Android over mobile data so it was too high latency for what I needed. If there's a way to connect to a phone on the same LAN/WiFi but scrape using its mobile network I would be very interested.
You can pick up a pulled laptop modem like the T99W175 from China for a low price, drop it into a USB enclosure (you’ll be limited to USB 2.0 speeds), and hook it up to OpenWrt. Or grab a GL.iNet GL-X3000 - the Quectel RM520N is already on board and runs over PCIe, so it’s quicker.
Then you can have basically unlimited IPs.
Android messes with your traffic far more than a bare modem (there's unavoidable NAT for one), and it has tighter thermal limits, so higher latency is expected.
You can run an ssh server in termux and run whatever programs you want from there.
There's several options for storage, as well:
- connect an external drive via USB (The /Android/media directory on both the internal and external SD card is generally accessible from both termux and other apps on the phone)
- if you're rooted you can mount a network storage in termux (or system-wide, but then you have to figure out sandboxing)
- if you're not rooted, mount in reverse (mount your phone storage over the network)
Yes, if you're interested its not too difficult, but I don't really want to put it here on HN (my business kinda relies on these methods, and I'm increasingly worried they will start working on this problem). If you email me at mynameisamodel@gmail.com I'll send you details.
It actually makes things a easier for both blocking and allocating (e.g. VPS hoster) side.
Once the "oh no, we can't afford that many unique allocations" excuse is away, algorithms that enforce quotas for every prefix size at the same time (with no excuses for CGNAT weirdness) stop being too ruthless.
You can distribute your addresses as needed, and I can track successful and failing attempts - at whatever distribution scheme you use. E.g. group your "unverified" or "trial" accounts at a larger prefix size, so they get each other blocked - but not your paying customers.
Good question, I checked just now and I am indeed getting only a /60. At some point in the past they gave me a /56 but no longer. I didn't notice the change because I have fewer than 16 networks, and networkd handled delegating /64s to them automatically.
