LLMs are incredibly powerful and useful tools, but offloading your writing to them oftentimes means you’re placing an undue burden on whoever you’re sending the generated output to.
To take it to the extreme, a “3x worker” is by definition relative, and someone malicious could achieve that by overloading a colleague with generated text that they themselves hadn’t bothered to read in full. You can stay at your baseline but be “3x” if you’re sufficiently able to sabotage your coworkers.
I’ve been on the receiving end of needing to review “documentation” that was generated almost entirely by LLM. It wasn’t anything that was done with malicious intent, but it’s still exhausting, and you need to exercise an even greater degree of caution because of the potential subtle mistakes or inaccuracies that a real human would never make.
I agree that sending someone a text you haven't read and understood is a sign of poor skills, much like forwarding someone else's message that has nothing to do with the issue being discussed. But we have to acknowledge that the market for speechwriters and corporate copywriters has existed for centuries. Surely you don't think a president is being dishonest when delivering a speech that someone else wrote for them?
I run some small websites, including a tiny forum that’s been a goldmine for scrapers. I had to significantly tweak some firewall rules and configuration after scrapers behind residential proxies suddenly accounted for over 99% of requests.
However, I also relaxed rules for automated traffic that was well-behaved, and I went out of my way to ensure that the Wayback Machine was able to hit everything. I should kick a small donation their way. They provide an incredibly valuable service and I love the benefit that I get from them just for personal side projects.
How do you separate the Wayback Machine from malicious bots that pretend to be the Wayback Machine? Are you whitelisting their IP blocks?
Because I get a ton of scraper requests that forge Googlebot, Bing, and Yandex user-agents that are totally not coming from their IP ranges. In fact, sometimes they all come from the same IP...
Sorry, not following. I thought the user agent is a string that the caller can set to anything. There is no immediate, reliable way to tell whether the string is correct.
Yeah, a real browser would produce certain patterns and never certain others. So in some cases one could clearly say it's not a human using a browser. But a scraper could also make efforts to mimic human browsing. Mostly the frequency of requests can tell with high probability tell it's not human. But such algorithms might occasionally give false positives for real users, exactly like it has obviously happened for the archive.
What google service are you referring to? Not sure whether the archove uses any of Google's tracking. I have pretty strong blocking of trackers and ads. But the archive works for me.
Google's scraper bot at least used to be behind IPs that you could identify via reverse-then-forward DNS. Not sure if that is still up to date, though.
Yep, verifying the IPs is still the way to go. You often see websites that do it wrong when you set your user agent to Google Bot and they give you a different version of the page without validating that.
As someone who hosts a few “tiny websites” that are “hiding behind Cloudflare”, it’s because of some incredibly misbehaved botnet traffic that’s otherwise persistently scraping via residential proxies.
This is a hobby for me. It’s for my enjoyment and it allows me to provide resources that others enjoy using. However, I’m not going to allow literally 99%+ of requests to be aggressive scrapers that won’t give up until you’ve got a heavy-handed solution in place.
I hate it too, but the alternative is even more consolidation, so unfortunately this is just the reality right now and you’ll have to get over it until when/if things improve.
> I hate it too, but the alternative is even more consolidation, so unfortunately this is just the reality right now and you’ll have to get over it until when/if things improve.
No, the actual alternative is learning to do it yourself. Unless you're a frequent target of UDP DDoS, there is almost nothing you can't defend against straight from the server itself.
Spending time learning how to configure your firewall, set rate limits in your web server, tune your application for caching and further rate limits, and should all this fail install a self-hosted captcha/proof-of-work fence yourself, is definitely a fair amount of efforts when those things aren't part of the core hobby.
Sorry, this just isn’t true. Some clients just don’t have the resources to pay for a dev playing whack-a-mole against residential proxy bots that are basically indistinguishable from legit users, but en masse can take down a VPS.
Rate limiting doesn’t work when individual IPs are only making a handful of requests.
So that’s why we’re using CF/Fastly. Too bad, I’d prefer not to, but there is just not an alternative at this point.
Noise and air pollution are valid concerns with a lot of these datacentres being built as well. Some of these recent datacentres have been built close to residential areas, and the health harms are far more worrying than the property valuations IMO.
To take it to the extreme, a “3x worker” is by definition relative, and someone malicious could achieve that by overloading a colleague with generated text that they themselves hadn’t bothered to read in full. You can stay at your baseline but be “3x” if you’re sufficiently able to sabotage your coworkers.
I’ve been on the receiving end of needing to review “documentation” that was generated almost entirely by LLM. It wasn’t anything that was done with malicious intent, but it’s still exhausting, and you need to exercise an even greater degree of caution because of the potential subtle mistakes or inaccuracies that a real human would never make.