On a server that used to get below 1k HTTP requests a day, I've had several cases of mass scraping from these botnets exceeding 100req/s. The web is getting harder and harder to use due to all the anti-bot mechanisms site operators had to put up.
A compromised device inside the White House network could presumably be doing something _much_ worse than participating in a DDOS against random websites.
And it's right there in the second line of the article:
> One of the proxy services is a Russian company whose founder’s web hosting business was ground zero for cyberattacks against election systems in Arizona and Illinois in 2016.
Surely a Russian company known for cyberattacks against the US having direct access to the White House network is far more serious than just DDOSing random websites. This is espionage at the highest level. (Again.)
>> A compromised device inside the White House network
A whitehouse network. It isn't like everything done in that building run through a single wifi router. It is likely an unprotected network for personal use with zero connecting to anything important.
If the progression of security over the last couple years has made anything clear, it's that letting the barbarians knock down a couple of walls because "well we have more walls" is not a winning defense strategy. The barbarians are very creative and will often exploit the inch you give in unexpected ways.
Great, they have some isolated network.
What are the chances it _never_ touches a device with a path to an important computer (e.g., one core router or switch at some point)? Good thing there haven't been any Cisco/Juniper/etc vulnerabilities ever.
What are the chances no important device _ever_ touches this isolated network? No important person ever gets annoyed at the security/filtering on the secured network and decides to connect to the public one instead? The secured one has some sort of issue, so their phone decides to connect to the guest network?
Are they going to compromise the military C&C network in the underground bunker? Probably not.
But it's probably more realistically damaging if they were to compromise some high level staffer's phone and using it as a listening device in a bunch of mid-level Whitehouse meetings.
Remember all the senior people are engaged in blatant corruption and illegality.
So you're looking at not just national security stuff, but the ability to get personal data like selfies of fucking goats or screenshots of Maltese & Azerbaijan bank transfers, etc. Makes it easy for nefarious parties to manipulate standing with POTUS and blackmail people. The only escape for many of these people is pardons.
Perhaps some of them annoyed the President's illegally-appointed mega-donor, if they spoke out against giving root access to all his unvetted junior flunkies.
The same junior flunkies who would do things like go to government agencies, create administrative accounts, which would then, often with minutes, be seen connecting from Russia and Eastern Europe with the correct credentials.
> Daniel J. Berulis, a cybersecurity architect with the NLRB, filed a whistleblower complaint alleging that within 15 minutes of DOGE team members creating new user credentials and demanding administrative access, multiple login attempts using those exact credentials originated from a Russian IP address
Oh come on, search-engines have gone downhill, but not that far downhill! You already know it involves "DOGE" and "Russia" and "credentials", that ought to be enough:
> Berulis alleged in the affidavit that there attempted logins to NLRB systems from an IP address in Russia in the days after DOGE accessed the systems. He told Reuters Tuesday that the attempted logins apparently included correct username and password combinations but were rejected by location-related conditional access policies.
It was enough and I appreciated it with the context, and conversation ensued. Not sure why I was scolded for doing so, that's what I meant by being able to have the conversation with AI - this whole interaction is defunct if we're going by the logic that it could be had another way.. I didn't want to go research it myself, I wanted to know where my peers are sourcing this novel-to-me information from.
Maybe the reaction stems from the low effort? "source?"
Personally, sometimes I find a short command like this in a discussion forum insulting to my effort.
Or maybe it's just that I still like sentences, and others are using text message conversational forms. idk. wtf.
That sounds like being deliberately obtuse and arguing in bad-faith. Are you going to "nuh uh" all the journalists and lawyers until you are personally given raw TCP dumps?
Behold, from the sworn affidavit of the DevSecOps guy with the top-secret clearance, as included in a whistleblower disclosure to Congress [0]:
> There were more than 20 such attempts, and what is particularly concerning is that many of these login attempts occurred within 15 minutes of the accounts being created by DOGE engineers.
So there it is, now don't go moving the goalposts.
> The same junior flunkies who would do things like go to government agencies, create administrative accounts, which would then, often with minutes, be seen connecting from Russia and Eastern Europe with the correct credentials.
They are young and think they are very smart. What could go wrong?
I applied to USDS in 2023, the process took nearly a year and ended with them just ghosting me after two interview rounds. Pay was capped (by law) below my target salary but at a level that I would accept as it would have been fully remote, but there was also a term limit on the job which meant I would have at most two years in the position.
It’s not just the pay, there’s a lot of structural elements that mean that the potential talent pool is less than desired.
There was a period of time where it seemed (as an outsider) that they were attracting good talent. This was when Megan Smith was CTO. https://en.wikipedia.org/wiki/Megan_Smith.
Or let me sum it up for you. The government doesn't have the money to employ top security experts and top security experts don't actually want to work for the government.
Sometimes, top security experts often can’t work for the U.S. government even if they do want to due to being ineligible for security clearance from checks notes gender and sexuality mistrust, non-zero drug usage lifetime to date, formerly grayhats, have ever sought mental health care, and/or anti-authoritarian viewpoints.
The track record for preventing people in positions of trust from seeking out mental health care is terrible. It's just brushing problems under the rug and waiting for them to turn rancid. Don't interpret not seeking help as the nonexistence of a problem, especially if their career depends on it.
Before the most recent elections, people voted for an unwell person to lead the country. So it's not just someone can obtain the position, but that people actually chose to put one in office. Twice. Although, it was less well known on the first one.
> The web is getting harder and harder to use due to all the anti-bot mechanisms site operators had to put up.
Indeed. I recently set up a bot on my local machine that I can interact with via Telegram. I gave it the ability to retrieve the contents of a URL via Playwright (with modifications). Basic things like getting the Readability view and getting the site as Markdown with links.
I was essentially mimicking the feature OpenAI etc offer where it can do a web search for you and get you the answer you seek. I just didn't want to pay a company for that feature.
It's not autonomous. It only acts when I tell it to. The actual search is via the Kagi API, but the URL contents is via Playwright. I also ensured it can't do lots of requests at a time. There's a mandatory (random) sleep of between 1 and 5 seconds between two requests.
In less than a week, I've been flagged. When I do a Google search (as a human), I have to prove I'm human. Many sites have simply blocked me - I can't buy stuff from various vendors[1], I cannot access several news sites, etc.
The thing is, I didn't really access that many sites with the bot! At most it's the equivalent of someone mildly addicted to the web. Yet they caught me quickly.
Obviously, I disabled the bot's capability to get site content. I wonder how long my IP will be in the blacklist.
I know some people will say "Good riddance" to the outcome and to what I did. I'd love to have a real, serious discussion on whether what I did was immoral. But whatever your thoughts on the matter, do ponder over the fact that the system has made it hard for average Joes like you and me to have this "feature", but if you pay a giant tech company a monthly fee, you can have it.
[1] Thankfully, Amazon didn't block me. In the past I'd avoid them, but now I'm in the dilemma where I don't want to buy stuff from sites that make it difficult for me to buy stuff from them!
I think that major gateways like Google and Cloudflare and Akamai are doing things like TLS-fingerprinting and smearing.
I normally use a VPN to access the Internet. It's an ordinary consumer-grade VPN, and I use it for pretty innocuous things like watching the international versions of Netflix or avoiding being geographically-profiled and charged more when I book travel. The VPN itself, I'm sure, is being used for all sorts of naughty stuff: when I look it up in whatismyip, it comes up as VPN, "abuse detected", and "residential proxy detected". I expect to get blocked occasionally on it and just pause the VPN then.
What I found surprising is that pausing the VPN doesn't always clear the CAPTCHAs. The next request will also fail, sometimes. For a while I was wondering if I have a residential proxy on my network, but I went hunting with lsof and bandwidth monitoring and a list of all the open connections through my router, and it seems clear. Yet even with VPN disabled, I'll get CAPTCHAs from Google and Cloudflare.
I suspect they're using browser/TLS fingerprinting and then smearing the results of that across bad IPs. If you connect to them through a known-bad IP, they not only will CAPTCHA you, but they will record a fingerprint of any data they can. Then if you connect again with that fingerprint, regardless of the IP, they block that too.
Ironically, if you are actually malicious, you can get around this pretty easily. Because you'll know not to use a fingerprintable browser, or to rotate fingerprints. So you have one completely separate identity on a given IP (that is not yours), and if it gets blocked, you roll up another completely separate identity on a different IP. But this is too much work for an ordinary consumer who just wants to keep their IP private. So you end up with this excluded middle where the only rational strategies are to either do exactly what the website expects you to do (with no funny business, even if it's innocuous), or to go whole-hog as the aggressor where you're determined to skirt their policies no matter what, and if you're doing that you might as well amortize the effort by extracting everything of value you can from them.
Yeah, I didn't want to put too many details as it would obscure my point. I was actually using a modified Playwright that's been designed to better act as a human. It works, in that I got fewer captchas etc than regular Playwright, but apparently not well enough.
That modified Playwright was based on Firefox, and that's what I use for my regular browsing. So that didn't help.
I can actually launch Chrome and not deal with this - but I'm not going to. In fact, what surprises me is that I can create a new Firefox profile and not get those captchas. I don't know how my using that modified Playwright as a bot got them hooked on to my Firefox profile. I suspect they had my FF profile as a "suspicious" entity even before I made the bot (I use a number of privacy/anti-cookie extensions), and the bot pushed the score over the edge.
I'll probably start fiddling with my profile - disabling extensions, until things seem to work. Thankfully, no one else in the household is impacted.
For now, I got around all this by just paying for the Kagi extract API. It's not as good as my modified Playwright had been, though. I thought of a VPN, but I suspect that won't last long (not a residential proxy, etc), and I don't want an ongoing monthly fee. I like Kagi's pay as you go option.
But as I said, it's a weird world where suddenly Amazon is coming out the "better" site/vendor. It's funny - in the old days I was pretty anti-Walmart, but a few years ago I started preferring them over Amazon (and even several local stores) for a number of reasons (better collection than local stores, not having to deal with fakes for online orders, etc).
I've not been liking the state of the web for years prior to the advent of LLMs, but this is feeling like a final straw where I accept "the web is dead". We need a fundamentally different protocol - the model of "Set up a server anyone can reach, put content out there, and then put up a lot of sophisticated technology so your server isn't hammered" is not tenable. The protocol itself needs to manage the load.
Expect to remain blocked, that's how defense works, some services plenty use will have is bot or known proxy exit node set to true, and some websites use the same defense services, request velocity is just one factor, if you don't have random human like scrolling and jitter with a bot, it's a dead profile walking, metaphorically speaking
My residential network has been on Google's shit list for a few years now, I don't think it will ever go away. Like others on this thread if I see a Google recaptcha that site won't get my business as I have to exit since the captcha is an infinite loop.
I just did this as well. I also added paywall bypass by default along with inline article chat. The internet is such trash these days.
I made my modifications at the network level and auto redirect a configurable set of sites to go to reader mode/paywall bypass by default with a MITM proxy and a custom CA cert I trust
That is entirely ISP dependent and many don't actually work like that. Your cable modem gets a specific IP for as long as Comcast or whoever sees fit, restarting does nothing on the WAN side.
On cable ISPs, oftentimes, changing the router's WAN interface's MAC address and then rebooting the modem will get you a fresh IP address, but rebooting the modem without first changing the router's WAN MAC address will get you the same IP address.
It depends on the DHCP part and your provider lease time. When you access the Internet through a modem and landline, it's an highly dynamical system, where it's difficult to guarantee every caller/user gets its own IP. Especially when IP sharing is used among the clients. Even more difficult it is with the mobile Internet. It's a constant connection loosing/reconnecting. Actually resetting and freeing up IPs for new devices, the whole time. With cable Internet (the TV cable) one is having a kind of stationary access point. The network provider can see it's topology and it never changes once a leafe has joined the network. I typically have the same access creds over months. Even when I restart my cable modem, it's still the same IP. When I was on modem, I could get a new IP every second of I wanted. Now I have to think twice, because I just can't get a new IP, even if is forcedly, not.
Doesn't work like that, only on mobile phones you can toggle airplane mode to get a new ip assigned and cgnat makes sure mobile ips are "trusted" , but they fingerprint you beyond ip address
1) The proxy isn't the scraper. There are plenty of scrapers not using residential proxies.
2) They're clearly talking from the perspective of the user and network admin. A residential proxy virus is in fact quite benign for both.
3) No matter what, DoS is quite a low priority in terms of security. And it's not even really DDoS, you'll know a real DDoS when you're on the receiving end.
Let's not get ahead of ourselves... If someone plants a bomb under your car, you drive to work, then it blows up while you're out for lunch, should you be prosecuted?
I'm all for creating consequences for people who negligently don't secure their systems and people get hurt because of it, but people who installed a mobile game that has a proxy library in it? Please, chill.
I'm a screen reader user (which is as "weird and unusual" as it gets when fingerprinting mechanisms are concerned), and I am not really noticing much difference versus 2022 or so. Sure, there's little bits of anubis here or there, and a few sites (which I know for a fact are experiencing targeted DDoS attacks unrelated to AI scraping) having annoying Cloudflare verification screens, but that's about it.
My setup (other than the screen reader) is as vanilla as it gets. Safari, AdGuard for Ad blocking, no VPN except work VPN when necessary, no PiHole, no settings changed. If you're using something more esoteric, maybe try going mainstream and see if that improves your experience.
I am not sure what you are doing to get "harassed" by anubis but I use privacy respecting browsers with ad blockers and vpns myself, and anubis is by far the least annoying, almost invisible even.
In comparison I get cloudflare recaptcha on every site that uses it, sometimes multiple times per session and sometimes not even letting me pass at all due to "error".
And if the website uses Google recaptcha then I close the tab because I know it will just end up in endless recaptcha hell.
Google too. 'We are detecting unusual traffic from your address'. I don't know how often I've seen this and there isn't a byte that goes in or out on this connection without monitoring so if it really would be 'unusual' I would hope I would be on it before Google told me, especially not from a session that's been active for days or weeks.
Come on... This thread has people saying their small website went from 1k requests a day to 100 requests per second. What would you have them do to mitigate this, I wonder? I would rather not have to have an anime girl picture stalling the website load for 1s, but I'd rather have the anime girl and see the website than no anime girl and no website.
Please substantiate this claim. What do you think makes it DRM? What kind of browser are you using that results in "harassment"? I use tor browser and other oddball firefox forks all the time, Anubis never gave me any trouble. I think it just requires you to compute some hashes.
It’s a software tool that specifically exists to allow content creators to enforce rules about how their content is used. It meets the dictionary definition of DRM.
It’s DRM that I think of as silly larping that doesn’t really bother me, but it’s clearly drm.
> It’s a software tool that specifically exists to allow content creators to enforce rules about how their content is used.
This is what I would like substantiated. I can go to a site behind Anubis, download the content on it, and do whatever I want with it. It doesn't do any kind of attestation of hardware or software. So - what makes it DRM?
While your server handles that, what does your CPU do with the other 99% of available cycles?
I run https://bunnystomp.com on an Always Free VM from a major cloud provider. It handles 1,930 req/s at 33 ms latency with no tuning/optimization. It's literally just python behind cloudflare.
You seem to know a lot about my server. From what I could see, all cores were pinned to 100%, but maybe my budget AMD CPU is hiding fantastical amounts of compute from the kernel. I'd love to know how to make use of it.
I have no idea what the site you linked does, but it doesn't look like a cgit instance to me; that's what got hammered on my end, and when hosting it, one very quickly finds out that many git operations are CPU-bound.
Funny. My site is a search engine and cgit is one of many tarpits that I block. Otherwise the index gets filled with garbage and the crawler hits TCP conn limits because the bytes trickle in too slowly.
If you want credibility from the get-go, you need to be up front about your application. I certainly took it to be a generic host for an unpopular site in your first comment.
Curl UI wrappers are just the weirdest thing. They’re so popular, they offer so little (compared to other paid services), and they all have the same evolution of simple -> require account -> sell data.
They just all seem like things that developers would easily ditch. But up until now, I was also still stuck on postman because my team/org uses it as a knowledge repository. It’s just classic enshittification. They’re using high switching costs to make as much money as possible without everybody leaving. Who
> From context, "vehicle" is clearly inferred to mean "automobile" rather than literally "vehicle".
From what context? At no point does the game say that, and from both the questions and the epilogue it's made clear that the game is about finding edge cases for what passes as "a vehicle" and "in the park".
The context of being a human and living in a society, which may perhaps be described as "common sense". The game references a real law disallowing motorized equipment. It then substituted a precise verbiage commonly used in the real world to an imprecise verbiage for the game. As humans, we naturally autocorrect their imprecise mistake and hone in on the intended precision, even unconsciously. It says nothing about whether any of the things are vehicles. Boats and bicycles are obviously vehicles. But, in the context of the question, anybody with common sense is inferring the intended meaning of automobiles that such a rule is intended to prohibit. In fact this is not limited to humans. Even compilers are capable of this, operating on a subtype when you pass an overly broad supertype to them in the right context.
The general interpretation of laws in countries with British-derived legal systems is that firstly, words have the meaning given referenced by the legislation itself, or failing that by a standard legal definition; otherwise words have their literal meaning unless they are ambiguous. If they are ambiguous, then the meaning is usually taken to be that understood by the notional "reasonable man" (and I should also add the meta-note that in most legal contexts, "man" is regarded as meaning "man or woman", except where it explicitly isn't, etc. etc.)
In this case, "vehicle" is defined by Wikipedia (and most other reference works) as "a machine designed for self-propulsion, usually to transport people, cargo, or both", but even that is ambiguous, as is "in the park", so at that point we look for the opinion of the "reasonable man". Whether skateboards, toy cars, horse, surfboards, aircraft above the park, drones etc. meet these definitions is the kind of ambiguity the quiz is looking for.
The problem I'm pointing out here is that it fails to actually explore that ambiguity. The majority cleanly decide that the 3 automobiles are out and everything else is in. There is interesting ambiguity, but it gets thrown out by the context; the context dictates things that are unambiguously vehicles, like boats and bicycles, are actually not violating the rule, because the rule in context actually means automobiles. You cannot explore the ambiguity around the meaning of "vehicle" with a context that dictates you actually meant automobile. At most you can make a point about type theory and type inference, but that was not the point the game was trying to make.
I would also note that the game is not formulated as a law. It is formulated as a colloquial rule - a law would never be presented as "no vehicles in the park". This lends itself to the contextual inference that narrows imprecise wording. If it were instead presented as "Article 5, Section 3: Vehicles are not permitted in the park. [Subsection: definition of vehicle]" it would likely do a better job of getting results that explore the ambiguous bounds of the definition rather than, as it currently does, getting results that clearly delineate at what is unambiguously an automobile rather than at what is unambiguously a vehicle.
This should've been submitted link really. It makes original technical-heavy article more approachable and provides the sources. TH/w offers nothing over it, and neither does the reddit discussion.
But that's temporary, and phones can roam to other carriers. Bricking large swaths of consumer devices is a much more permanent effect, especially for non-phones (think modern cars, PoS terminals, and other devices with SIMs)
I suppose ditching Apple is "two birds with one stone" :)
reply