I have a dumb question: What if we put a simple password in front of every website that everyone knew, like "password". Upon click of login, the user agrees to the terms of service which exclude all automatic scraping.
I know this is a dumb idea, but I would love to know exactly why.
Legal agreements won't stop companies who don't care about legal agreements, they'll just add "popup.write('password').submit()" and move on. Multiply by 1000x, and you've solved nothing, while making the UX for normal users worse.
Law is only as useful as your ability to enforce it. Would you have the legal means (and will) to go after a random foreigner who still scraped ypur site after accepting the terms? Multiply by 1,000 times a day.
I know this is a dumb idea, but I would love to know exactly why.