Great read. It's the doorman fallacy from the doorman's perspective. If a doorman is replaced by an automated door, you miss out on the intangibles the doorman provides like security, customer triage/support, and similar.
AI used in a framework of human augmentation generally gets that for free. Used under the mistaken notion that humans can be easily replaced by AI gets you to the big complaints here.
(These are just my personal read on the topic -- the market is in AI frenzy where AI valuations are disconnected from productive capacity -- heavy on my mind and looking to get some writings out on the topic soon).
Maybe if this was Dickson's party in 2008. A massive political event in 2026 is the hottest midterm to date? You can discount Hanlon's razor. It's not an infinite excuse any more than the book Infinite Jest was either unending or funny.
Yeah and nothing convinces the undecided independents, who now are mostly paranoid schizophrenics, to vote for you like appearing to shut down free assembly
Highly relevant then. HN tends to discuss the things that are either meaningful today or what they are building in their garage that is often meaningful in 5-10 years.
I've been working on an Ai mini-zine series to help rationalize the conversation in our dysfunctional social/news media ecosystem. I include these rules on the back cover.
One of my greatest AI-related fears is that we build critical systems into kafkaesque hellscapes.
Yes, nukes would be bad to give control to an AI with no workaround. But so are customer service groups of fraud departments at banks. A kafkaesque hellscape is where a customer or beneficiary of your service cannot get the help they need by policy and by design. Like getting Google on the phone, getting a non-scripted human 10 years ago in a call center, or now getting a non-agentic interaction on the phone. The scripts dominate the interaction.
The IRS, for a time, would cut off your call if their wait system was too busy. That's a critical system in kafkaesque hellscape.
Kudos to zlandes86, you have highlighted WHY these hellscapes start to pop up in systems where the human is removed.
I am an AI user and builder -- I fully advocate for the human-augmented system, NOT the human-replaced system!
Whatsapp has this problem before AI. They scaled without regard for the feasability in handling customer service for everyone, so their solution is to just not do it. I have a personal phone number in Hong Kong which I can't set up for Whatsapp for some reason. No idea why, I've tried figuring it out, only to learn that Whatsapp basically has no customer support. And yet Whatsapp is treated as critical infrastructure in a lot of places these days (as in you have to coordinate on whatsapp).
At this point I might be grateful for an AI if they give it enough permission to tell me why my phone number is impossible to verify, or why they think I'm a malicious actor for using a VPN to protect my traffic.
WhatsApp has become my nemesis. I tried moving from Android to iPhone. The first time it banned me completely and I had to beg their AI to appeal the decision, because I didn't use the required "Move to iOS" app and simply just tried to move my phone number.
After that I tried to move my chats across. I spent several hours a day for ten or so days trying this task. Including moving and unencrypting the chat DB onto my desktop and having Claude scour it for any inconsistencies. Nothing. Had to give up. My Android phone keeps the chats and stays in a drawer with a power cable while my iPhone is now just a "companion device." Trillion dollar companies.
Still remember when WhatsApp cost a buck a year and was just simple chat? Those were the good days, until they got offered enough billions to turn it into the same ad-selling machine every successful app will eventually become unless the founders are truly uninterested in money.
Of course, but is was never about technical feasibility, but about distribution. People use the app that everyone else already uses. You see the problem.
Yep. So does AT&T. Real scenario that just happened to me yesterday:
Spectrum is charging me $110 a month for internet. I go to the AT&T site to check if they can provide fibre internet in my area. No matter what address I enter, it says to contact them. I tried neighbouring addresses and, at one point, I even gave AI a try to see if I could get tips. Nothing worked.
I called the number the AT&T coverage page asked me to call (apparently a customer service number). I was "first" in line, but it took them over 15 minutes to get to me. Weird but sure.
This isn't even the kafkaesque part.
I finally get on the phone with someone and he says "Hello, can I get your name?" I give him my first name. He says, "I need your full name to bring up the address." I say, "You don't have my address. I don't have an AT&T account and I'm calling to ask if you provide fibre internet in my area." He says, "I understand. But I need your full name to look up the address." I say, "I don't understand why you would need my full name." I gave him my name again.
He hangs up on me.
Not much different to a useless cycle of talking to a bot, especially when AT&T apparently encourages to move on as quickly as possible from a call if the rep knows they won't close a sale and as they try to improve their "average handle time"
I'm not saying AI wouldn't exacerbate this, and this is reason to believe that this scenario would only be worse under an AI-controlled environment. Certain parts of our society will just become even more closed off and shut people down even faster and with less human effort. Even if AT&T was able to somehow identify my real identity through some governmental totalitarian identity system, I don't think this would change
> The IRS, for a time, would cut off your call if their wait system was too busy. That's a critical system in kafkaesque hellscape.
No, that’s Little’s Law at work. With bounded parallelism and other queue values fixed, and a fixed average dequeue rate, the queue cannot accept infinite enqueued tasks.
So it pushes upstream to the next work pool, which is the humans who want in. You are mad, you call the next day, eventually you get in.
Is it fair? Fairness in queuing systems isn’t simple.
It can be two things. This is a necessary aspect of a queuing system when the queues grow large. It’s also a system that many people are legally mandated to interact with that isn’t given enough resources to handle the volume in a timely fashion.
This problem started before AI hit the mainstream. Make the company “more efficient” (i.e. cut costs) by outsourcing to contractors in areas with lower labor costs. Then, remove escalation paths, then remove abilities from them so that can’t do crazy things like refund a customer. Finally, automate the whole thing and be certain there’s no option available that could possibly return money to the customer.
It’s a symptom of the underlying ideology of the evangelists, that holds that some people are just unnecessary people and should get no affordance. What now will be shipped out for the general population is what has always been the case for minorities. The 99% are now the minority.
> The IRS, for a time, would cut off your call if their wait system was too busy. That's a critical system in kafkaesque hellscape.
Now you're just on hold indefinitely, and eventually the offices will close, so you can either hang up and wake up early to call or just stay on hold overnight maybe? I had to call them for something back in March, and even waking up and calling at like 7:01 AM (a minute after their phone lines opened) I was on hold for half an hour.
They've been underfunded so regular taxpayers will hate them more and supposedly support removing income taxes, and to starve them of auditors who can go after wealthy tax cheats.
Oh, I don't disagree with you. I'm pretty far left-wing and in general think that people with at least as much money as me should be paying high taxes. I would be in favor of higher taxes for wealthy people (which I realistically am included in, even if I'm not making millions of dollars or anything) and putting some of that towards funding for the IRS.
If you're like most Americans and pay taxes on time then there's absolutely no reason to be audited. The two possibilities are:
1. You are taking a lot of special deductions and not providing all the required documentation.
2. Or you're not reporting things accurately because there's some edge case in the tax code that you didn't understand but is easy for the IRS to catch (eg. sales of stock without cost basis reported).
Simplifying the tax code and funding the IRS more solves both of those because then they can focus on the real whales who are abusing the tax code the most.
Most likely 3. He has not gotten audited, he is just a temporarily-embarrassed billionaire, and a bunch of tax cheats convinced him that what would really improve his life is the government raising his taxes, because it isn't collecting theirs.
Yeah, that's one of the points of the EU AI Act, I don't have the specifics, but for some critical sectors decision making cannot be automated by AI and you are obligated to get a human explanation if you request it.
Yeah a lot of people worry of ai in high stake environment but a jev-like evaluating a resume or a university application is much more impactful and likely to happen
Well put, but I don't feel this is an AI problem as such - if anything it's become somewhat alleviated since chatbots were widely deployed, as I find I can usually persuade a chatbot to understand my position and escalate toa. human, or make a clear statement of corporate policy in declining to assist.
It's been heading this way ever since the outsourcing boom in the 90s. People under teh age of 40 today probably cannot appreciate how much it has changed; customer service in the US used to be really good (whether for specialist B2B stuff or things like utility service) because it was easy to get someone on the phone in a timely fashion, it was easy to explain your issue, and customer service reps were generally given the training and autonomy to solve the customer's problem rather than divert them or make excuses. There was an ethos that it was better to lose a bit of time/profit rather than lose a steady customer. Anecdotally, I feel that was in some ways correlated with stronger antitrust enforcement that fostered a more competitive spirit among service providers.
It should be noted that we call these things "kafkaesque" because Kafka himself feared them, at a time when neither computers nor AI were a thing. Unless we go full Skynet, no AI is building itself into critical systems. Humans are putting AI there, and they do so because society decides that they should be allowed to.
I feel like the first part of your post shouldn't be needed. But perhaps the word Kafkaesque will long outlive the memory of Franz Kafka, and future generations will curse the name of the unknown Kafkae who delighted in building such impenetrable systems. Perhaps he was an emperor of Byzant or a rival of Orwel.
Is the society you mentioned in the room with us? Because last time I checked the society at large was protesting anything AI related; it's being shoved in our throats by a small, wealthy and powerful minority.
agreed, but also that 'deserving' is dictated and nudged/influenced @ them/us; part of the entire thing is deception, and the less you need to keep doing to maintain the grift and maintain the 'wool over eyes', the more economic hustles are available to that group(s).
> The IRS, for a time, would cut off your call if their wait system was too busy. That's a critical system in kafkaesque hellscape.
I had to call the IRS this spring and it was hell. I repeatedly was on hold for 1+ hour(s) then get hung up on. Or someone would finally answer, I'd explain, they'd put me on hold for another hour, then just hang up on me. A total of 10+ hours collectively waiting to finally have my questions answered in 10 minutes.
That said, I'd love if the IRS had an AI, where I could send my AI to go deal with it on my behalf. Then if it truly requires a human in the loop, we schedule accordingly vs me on the phone for 10 hours.
Definitely not just an IRS problem. But the IRS certainly sits atop this hill of phone calls you dread making each year. And you only call if it is absolutely necessary contact them. And just like most phone calls I make to support people these days, you must know more than the person you will end up speaking with because they must have just started working there last week.
> build critical systems into kafkaesque hellscapes
This is already SOP in large companies that seek reductions in support and customer service overheads. It has been for a while.
The only plausible fix is legislation mandating a reasonable level of _human_ support for all customers and clients and accountability for maintaining it.
Plausible until you consider the political obstacles to getting something like that enacted, that is.
I am in no way making the case for this and can understand the million ways in this can go wrong, but my experience with AI so far has been to make things far LESS Kafkaesque, because it allows you to insert a "human" in the loop far more than our previous systems. Hand rolled software had to boil down to hand written conditionals that tried to account for edge cases, but if you find yourself in an edge case it didn't account for you often just hit a wall. An agent in the loop can immediately understand "oh I see how this doesn't exactly fit into that category but this is clearly ABC."
It feels like their insanely bad phone support is by design. And I think we all know it is. Google's support (and this was over a decade ago) was one of the first times I identified this bald-faced strategy of someone on the other end being hired to make sure they were given precisely zero means of helping me.
They tout their lack of regular customer service for Google accounts.
Unless you are trying to spend money. I get calls and emails every single day from Google which I block or ignore because I helped a friend with their Google Ads account.
Google enabled SMS 2FA on my main account out of the blue one day, with a phone number I had in 2004. There will never be a way back into that account.
Yes, I agree -- if we ever give up this decision making to ai models not only will their actions sometimes be misaligned, but there will be no one accountable.
The venn diagram between people worried that Comcast customer support and the welfare office will suck worse and people who are actively scheming up ways to use AI bureaucracy to "nudge" people toward the outcomes they want on other issues has so much overlap it would embarrass people if they had any shame.
There was a game that was Jev trolley problem PVP. The meta appears to be that you need to make the most mainstream journalism victim case. The more holocaust adjacent the better
I had a surgery denied this year by an American insurance company that almost definitely was using AI to make the prior authorization determination. I don't know how else I can explain the company's refusal to allow the surgeon to discuss the case with their medical director.
TBH, your average AI agent in 2026 is far, far more capable than anybody in tier1 or tier2.
The reason you need heavy scripting in those tiers is because the people you can afford to hire there are incapable of literally anything else. Keep in mind that your average HN user isn't your average customer support caller. The average customer support caller just wants to talk to somebody (yes I'm serious, this happens far more than you think), or has a routine issue which is trivial to fix but which they're nevertheless unable to fix themselves, think "router got unplugged, there's no internet, but the customer can't figure out why".
A well-designed AI system can be far, far more effective and pleasant than traditional CS, and a well-designed AI system scales well, so it's actually worth spending money on. AI turns good CS from a cost center to a profit center.
I agree with this. I also note that the number of systems that achieve this pleasantness are like happy families in Anna Karenina, and follow similar design principles like easy routing to a human.
Reinhard Müller, a German philosopher and mathematician has written extensively about this, warning people about these dangers. Essentially, people would be judged by patterns and prejudice, with no recourse to the logic of the decision or the criteria.
Hey, I've tried to respond to your comment twice now but it looks like the AI/LLM checks are overcapturing any sense of structured writing. So a little less structured. Carlotta Perez, an economist, wrote Technological Revolutions and Financial Capital in the early 2000s. https://en.wikipedia.org/wiki/Technological_Revolutions_and_.... The installation phase starts with "irruption" and ends with "frenzy" which I think think is where the market is now. Then comes the deployment phase of synergy/synthesis to maturity, where valuation of new tech realigns to productive capital. In between frenzy and synthesis is where the short term pain happens. It's related to the hype cycle but is distinct enough to call out.
----
Note - this comment is 100% written by a human. AI checkers or moderators, please do not remove.
reply