There is no stopping AI data scrapers without also preventing users from accessing that data. Even with rate limiters, whatever that rate is tied to can be duplicated or faked such that the tech will affect normal users more than scrapers.

This will especially be the case once language models can truly pass the Turing test, though they are already most of the way there. Using proxies/VPNs and multiple accounts, give the AIs “browsing habits” and there won’t be any way to tell the difference between bots and real people. The rate that they can scan comments might still be lower in that case than it is now and legally riskier, but it will still happen.

Bots will probably have an easier time navigating the new site, too. They might even have to be limited in that regard so that they don’t look different from real people.