> Over and over again you have to contend with bullshit like 120MB APKs full of obfuscated code, root detection, custom firmware detection, "non-mainstream device" detection, VPN detection, and even more obscure spyware-flavored functionality.
When you work in a regulated industry, you have to get pentests. The pentesters come with a list of "recommended best practices" to harden / secure your apps, which always include all of the user hostile bullshit you mentioned.
It's a finding if they're not implemented. Even if the dev team doesn't want to implement these things they get forced to by policy and the overall process :/
The only way this can NOT happen is if someone quite senior signs off on the risk, who would have to get in a big fight with the cyber/policy teams for basically no benefit to themselves. They would also have to be unusually technical and pro freedom to even understand the issues, which is not the typical personality profile of the kind of executive who ends up with overall risk authority for a bank's mobile app.
yeah, there are a lot of bad auditors in all kinds of ways, mostly:
- auditors which commonly "pass" things which should be flagged, but happen to be a more expensive then normal industry standard in ways which should make them non-competive but somehow they get hired a lot
- auditors which flag nonsense and don't understand anything about security at all
some of the "security frameworks" which are applied by default makes a lot assumptions about how software is build/structured/works _based on what was common in the 90th_, and they don't even list their own assumptions... Luckily they are a base line you can perfectly fulfill but still a pain.
Like e.g. a session key having 128bit entropy is a reasonable minimum, but then imagine the framework assumes the session key & session id are the same because some 90th software did so and now requires you session ids to have 128bit entropy, but e.g. UUIDv4 has only 122bit entropy and a UUID being used is deeply integrated into middle-ware you use and can't be changed... (And having a session id=session key setup is anyway a bad idea as it means you mustn't log the session id anymore, but you also might have requirements which do require logging session ids in audit logs and similar).
This isn't even the most nonsensical thing I had ran into, just one I remember well. There where rules based on assumptions of you uploading files into a file system _using the upload file name_ while interleaving security sensitive web-server config files with uploaded directories ... like yes I know some software did so in the past but the rule should be "your software musn't do such a negligent thing" and not "you software must do this 5 extra steps in case it maybe does that insane negligent thing"
I think there's a paradox. Is it only "cognitive surrender" if you already know how to code? Not knowing all the details seems more like the default human condition, in the sense that most people don't know exactly where their water comes from or where their waste goes exactly. "Water comes from the tap, food comes from the store".
Most of the non-technical people I see using LLMs to create software are not engaging in "cognitive surrender" as far as I can tell.
They didn't read the code before and they don't now. However, asking questions and improving understanding of the system and architecture is actually faster and easier for them now than it used to be.
They're evaluating what they can see, i.e, judging results. They build things for their own use. They test them, iterate on them to meet their needs and try to achieve their vision. This is roughly the same process they followed when they used to outsource to an agency except the loop is much tighter.
It seems to work well, mainly hurts when the problems are invisible, so security, backups, breaking migrations, perf, maintenance over time (language / runtime updates, dep updates, server updates etc). All the stuff that hurts if you have never been responsible for the uptime of something before. Not knowing that software is never "done". But junior devs trip on those exact same things.
I agree with the tail of the article; tooling and processes that make that stuff more visible would close a lot of the gaps.
> It seems to work well, mainly hurts when the problems are invisible, so security, backups, breaking migrations, perf, maintenance over time (language / runtime updates, dep updates, server updates etc). All the stuff that hurts if you have never been responsible for the uptime of something before. Not knowing that software is never "done". But junior devs trip on those exact same things.
If we continue from your "water comes from the tap" example this situation feels like someone who has no experience of plumbing deciding they're going to replace their bathroom without bothering to find out how any of it works. They'll possibly end up with a bathroom that superficially works but they're also liable to find problems down the line with leaks, things not draining properly, or water pressure issues.
Some of the people writing software purely via LLMs are even turning this into a business, which I guess would be analogous to deciding you're going to run sewage treatment for a small city without understanding anything about how that works. It'll mainly hurt when the problems are invisible, so food poisoning, or inlets backing up and overflowing into a nearby river.
That analogy breaks down immediately because it's missing the critical novel component: the AI.
In the bathroom remodel analogy, the AI is the plumber that you hire to do the work for you. You tell it what you want, it figures out the details necessary to get it to you. If what you want isn't possible, or requires large changes, a good plumber will raise the issue to you and explain the constraints they are working under, ask for guidance, and offer recommendations.
Sometimes the AI does things in a weird or inefficient way, or neglects details that become important later, or misunderstands something important about the task resulting in an outcome other than what you wanted, but that's also the same as the plumber. This is why people usually want to be at home and keep an eye on the people working on their improvement projects or repairs, because they want to make sure that the guy they hired is replacing the broken faucet, not the functional but odd-looking one.
There is still value in knowing the underlying systems well enough to present a good design vision to the plumber up-front. It's still possible to have a competent executor produce a bad idea, which is how we get stuff like Groverhaus, the Doom Bathroom, and McMansion Hell, which are examples of "slop architecture" in exactly the same way as we are seeing "slop software".
There were terrible software developers before LLMs. Honestly for a lot of people AI generated slop is a massive improvement over what they used to ship.
I agree it's pretty bloody cheeky to run an agency like that though. The only ways out I can see are licensing or real liability.
But let he who has never done dodgy home improvements cast the first stone.
> Honestly for a lot of people AI generated slop is a massive improvement over what they used to ship.
Citation very much needed. Otherwise you're basically arguing that the problems associated with people writing bad code somehow go away when they're suddenly in a position to direct the quality of code generated for them.
I am claiming something slightly different! There are varying effects. I can't really cite, I only have anecdotes.
For "terrible programmers" (I use this loosely, sometimes thats me! - not a value judgement of the person. Might be inexperience, working outside usual area, bad week etc) code shifts from being "fractally wrong how do I even explain to you why it's wrong, you clearly have zero mental model of whats going on here", to "actually ok code but wrong approach and you didn't need to add a 200 line Python script for this". The rework process is explaining the right approach which gets meat proxied to Claude who DOES get it.
I get that a lot of people feel mugged giving feedback and having it proxied, without a _relationship_ where you can see a person is growing and mistakes won't recur. It's like the "author" is just using a robot to steal your time. Sometimes I personally feel relief at work though, because it would steal WAY MORE time to explain detailed and intricate stuff across a really large knowledge gap... in which case I am actually grateful that they have Claude as a cognitive prosthesis. At times I am the one who needs the prosthesis.
Even in the absence of good feedback I frequently see stuff go from "not even wrong" to "thats kinda messed up but it'll get the job done".
Other places, the "good (but not highly conscientious) dev -> slop cannon" effect might be dominating. I feel like AI is raising the floor and ceiling of what people can do (a strong senior / expert can whip the AI until the output is actually good) but having particularly weird effects in the middle. Its enabling people to attempt things outside their usual area (devs doing infra, sres doing more dev, backends & frontends doing each other's jobs) and IMHO that's where good feedback really helps.
I think one of the potentially advantageous things about AI is that it acts as a force multiplier for people who already have expertise in a given discipline, while simultaneously raising the floor for everyone else.
People looking for self-contained, spot solutions to localized problems will have much more capacity to solve them locally, while people with in-depth expertise will be able to build systems that are even more reliable, performant, and scalable than what they were building before.
So we'll get the benefits of disintermediation for the basics, and much greater capacity to deal with effectively with complexity for the not-so-basics.
> However, asking questions and improving understanding of the system and architecture is actually faster and easier for them now than it used to be.
One of the things I've built in my current role is tooling to help understand current architecture, and to resolve the differences between things as-documented, as-implemented, and as-actually-behaving. It's used for a few things, including updating documentation, but it's been most impactful when onboarding new engineers. They can simply ask questions about how things work and why, and get explanations and deep dives wherever they need it.
> Is it only "cognitive surrender" if you already know how to code?
Is the article claiming that?
> Not knowing all the details seems more like the default human condition, in the sense that most people don't know exactly where their water comes from or where their waste goes exactly. "Water comes from the tap, food comes from the store".
Sure. The ability to specialize, and on the flip-side not have to care about a lot of things, remains one of the biggest driving forces of human civilization. But in this article, I don't think the person worried about cognitive surrender is the "most people" in relation to food or water. They're the farmer or the water infrastructure engineer! They absolutely for sure do need to know those things. And all around us we see a lure of systems that invite them to surrender.
> Most of the non-technical people I see using LLMs to create software are not engaging in "cognitive surrender" as far as I can tell.
Maybe not. But the might be cognitively surrendering when they use the LLM to summarize the day's news or write a message to a friend, or even plan dinner for the week. I'd say they are. I agree that it's not clear-cut.
> They're evaluating what they can see, i.e, judging results. They build things for their own use. They test them, iterate on them to meet their needs and try to achieve their vision.
No. They have things built, then they test them. This is different. It might be OK, but it's not the same as building.
> This is roughly the same process they followed when they used to outsource to an agency except the loop is much tighter.
Yes. But they wouldn't usually outsource summarizing a letter from grandma, or the political agenda of a candidate/party in an upcoming election. That's surrender.
> It seems to work well, mainly hurts when the problems are invisible, so security, backups, breaking migrations, perf, maintenance over time (language / runtime updates, dep updates, lib updates, server updates etc).
… or ones own abilities deteriorate.
> Tooling and processes that make that stuff more visible would close a lot of the gaps.
> I don't think the person worried about cognitive surrender is the "most people" in relation to food or water. They're the farmer or the water infrastructure engineer!
Fair. Maybe I "cognitively surrendered" a long time ago just as a consequence of working in large teams :(
I hit the breaking point of not being able to read all the code well before LLMs came along. Also people would leave and the documentation for their subsystems would be wrong or incomplete, so there were always mysteries to chase down (during an incident, naturally) even in code written by hand.
Gets to the point, you don't exactly read the MRs, you're mainly checking their approach against what you talked about earlier and making sure it doesn't look way more gnarly than it's supposed to.
Working with agents doesn't feel that different.
> No. They have things built, then they test them. This is different. It might be OK, but it's not the same as building.
It does bug me sometimes when people say "yeah we built the deck..." and they got a contractor to do it. But then we do say things like "Nolan made the film" even though he didn't hold a camera or build the sets. To me, one-shotting doesn't feel like building, but engaging in a loop and applying control to get the outcome you want, does. Regardless of the exact toolset.
I don't personally see super hard lines between using a compiler, a 4GL, a sql query, stitching together 10 libraries and a framework I didn't write, using an LLM or even directing a team.
> Not the problem of one's own skills though.
I read somewhere once we can't weld Saturn V engines anymore, or at least not economically. We would also probably have a lot of trouble finding people who could do all the engine calcs the way they did. It doesn't necessarily mean that rocket progress is stalled.
> Yes. But they wouldn't usually outsource summarizing a letter from grandma, or the political agenda of a candidate/party in an upcoming election. That's surrender.
> But then we do say things like "Nolan made the film" even though he didn't hold a camera or build the sets.
I do say that, too. But I'm not in the film industry. My understanding is that those who are, don't use words as crude as "made" to describe a role in the making of a film. I believe they say that he wrote, directed or produced a film.
We are not talking about some other field of expertise here, where we are outsiders – the production of food, the supply of water, or the making of movies – we're talking about our own field of expertise.
> My understanding is that those who are, don’t use words as crude as “made” to describe a role in the making of a film.
Oh, no, film insiders use ‘made’ constantly, or even cruder - attribute credit without using a verb at all. Saying it’s a “Nolan film” is super common. Here’s a VFX person saying “Nolan’s CGI”: https://youtu.be/AFLcP-t0-kI (I’m ex-film industry, and still film industry tangential, and found this clip being discussed on a film industry mailing list.)
Film people are far more likely to pay attention to who the director is than non-film people, and it’s very common for film people to talk in terms of directors ‘making’ the film or otherwise casually assign ownership. I think this is especially true for famous directors and good films, and maybe a bit less true for unknown directors and mediocre films.
Somewhat off topic now, but Nolan is fairly famous for being intricately involved with every aspect of the films he directs. He's constantly behind a camera checking out what shots are going to look like, pushing the limits of where you can put an IMAX camera, and coordinating with set builders to do ridiculous stuff like building a reproduction harbour jetty.
I feel like the earnings structure is quite different than it was post-war though. Today there's a larger difference between "income rich" and "capital rich" in terms of their options for avoiding tax incidence.
Like in 1947 a factory owner's profits would show up as high income. Today they wouldn't (money is in share prices and complex financial arrangements / borrowing). The high marginal rates hit the chief engineer instead, who actually works for a living.
Exactly the true wealth of America is no longer income or property its all shares. It could be argued the American 40T debt is pretty much because of tax avoidance due to hiding wealth in stocks and related possibilities.
Stock value is not real wealth. The stock value is a guess about future profits along the next 10 years. You can't give to workers guessed future wealth. It doesn't exists today.
No argument there, im simply making the observation that a lot of people are avoiding tax by hiding their wealth as shares and this is hurting the gov ability to tax and pay its debts.
Its a really tricky problem to solve. One simple solution would be to tax any loan secured by owned assets as regular income, though there may be situations that hits more average people, say if you take out a HELOC on your home.
It is pretty absurd that one can borrow against a mountain of owned stocks and live of the loan without it being considered income.
The error is to assume that there is wealth to tax. Imagine an island with 3 people, which produces 2 burgers a day, so the poorest one can't get a burger. Now the government taxes the richest person, and buys one burger. who lost the burger?
The burger is taken from the poorest person that was able to buy a burger. Now you have 2 people who cannot buy a burger.
This is why you cannot tax unrealized gains. You are sacking the poorest. Is not matter of how the government gets the money. The problem is that the government spends the money.
The poorest always pay all government spending. Always.
Seems like a good reason to figure out some additional soak-the-rich taxes! (Back in the 30s that was a slogan that Americans could rally behind..! Not so much right in the last several decades.)
But it's also disingenuous to claim that none of today's approaches would show up as high income. A lot of old money would be protected still, a lot of new-money would be hit.
Isn't it telling that there was so much massive development of American industry in the post-WWII era and yet there weren't any Musks or Bezos or Gates? A country that had minted tycoons like Getty and Rockefeller and Carnegie stopped for a while. A guy like Howard Hughes was involved in several areas of development but didn't touch the guys who came before him or after him overall...
Most of the world does NOT optimise for efficiency, particularly at the big end. Really big hyperscale companies (and some startups) are unusual because they at least sort of try, most of the world is not even trying.
The org structure is optimised to reduce or even minimise individual agency, explicitly at the cost of efficiency.
Teams have fiefdoms, private repos that the rest of the org can't see. You are not granted permission to see what permissions you have. Byzantine department structures. Shipping "hello world" to production takes 3-6 months and nobody can imagine it working any other way.
There's actually no central directory of teams or processes. To get anything meaningful done, you need the co-operation of maybe 8 teams but there's no consistent way to engage them all. You have to negotiate with them all in different ways, they're like little warlords, sitting and guarding access to some vital resource you need. They have established a ticket-based interface with a 7 day SLA on top of a perfectly good vending API that could do what you need in seconds. Sometimes a blocking team has a 3 month backlog so your project will just have to wait.
Everything is always in the process of being restructured, so no end to end 6 month process happens the same way every time.
These organisations could use AI to run the end to end process in minutes. Heck, they could just put APIs on top of what they already have. But the way they WANT to use it is to keep doing what they're doing but with EVEN MORE form filling and paperwork.
You would think they would get outcompeted, but no. The "IT organisation" sits like a brain slug on top of an enterprise with substantial regulatory, capital and execution moats.
Big cuts of course do happen in these places, but it's often for optics (we need to signal that we care about efficiency, and AI is a great excuse). But you can't protect yourself by being efficient and good at your job, because the cuts were not about output.
Naively, people have been working for 75 years over the span of the graph to create valuable IP, scalable manufacturing, unique organisational capabilities, partnerships etc.
If capital is being created faster than it degrades or is destroyed, wouldn't we expect its share of income to increase over time?
It's not obvious to me why labour's share should necessarily remain constant as the economy accumulates hard-to-replicate productive assets.
Because society breaks down when there is an elite class of people who are so wealthy the law no longer applies to them, while a significant portion of people are on the brink of being homeless, and many more are one health emergency away from it.
Not necessarily the accounts or posts. They can be made to delete all the profiling data that they've collected over the years and be barred from inferring anything from user submitted past data and new data for a few years.
Basically they shouldn't be able to monetize that data in any form directly or indirectly, even if they don't delete it.
What I don't get though is that with increased household efficiency and presumably an increase in population (i.e, density which usually lowers cost), and the benefits of efficiency improvements to generation and distribution... why is household energy spending is UP, even accounting for inflation?
In the US, it's due to the cost of the grid, the part of the utility bill that PUCs allow utilities to charge a fixed rate of profit. So if utilities take 20% on that grid "maintenance" project, the utility drives up the price as high as possible.
This is why distributed solar is actually cheaper than utility scale solar all-in: you need less of that very very expensive grid, even if you pay 3x on the initial install costs.
Households use about 20% less electricity today than in 1985; per person kwh is also down. But household expenditure is up by 20% in real terms and cost per kwh is up by 50%
40 years of technological progress and prices went up :(
In 1970 the UK domestic electricity grid and generating plants had been state owned since 1948, and was basically run by civil servants. This lasted until 1990-91 when it was all privatised by the Thatcher and Major governments (both Conservative).
Now in 2026 the main generators are EDF (French state owned), RWE (German), and Scottish Power (Spanish owned). These companies tend not to publish executive pay, but back in 2012 the CEO of EDF in the UK was being paid £1.2 million a year.
I would submit that all this has some bearing on why the prices have gone up.
I think this works well until you get into data, state, history, external interfaces etc. Large scale data migrations are still risky, anything with external consumers you can't quickly change is risky, etc. Internals inside a service I agree, yeah you can let it slide a lot.
I also found though, with agents you can quickly create large amounts of code, when you go to refactor / fix some design aspect that touches large parts of the system this consumes a lot of tokens and actually still takes meaningful time.
It's like, with a digger you can quickly create a hole! Digger can quickly clean up the work of 10 people with shovels! But if you use a digger for 1 week and you were putting the dirt in the wrong place, you find you have a HUGE pile of dirt to move and will still take a long time to clean up.
Directionally I agree with you, but I think design still matters.
I think the difference is that with cancer cures we mostly care that it works as proved by trials, and understanding it is a bonus.
Up until now the prize in pure (as opposed to applied) mathematics was the _understanding_ and the machine can't do that for you. What does it mean if we get "super powered alien maths" but humans can't do it? It's like inter univeral teichmuller theory but imagine if Mochizuki was right and it came with a lean proof?
It was the understanding for some, and the result for others. Math is going to bifurcate along those lines. Many are the builder type who use math for a purpose. To accelerate an algorithm, to improve the numerics or convergence of a computation, to verify statements about real things, to use it in angineering applications, etc etc.
In my view, over the last century, math has turned into an intellectual analogue of extreme bodybuilding competitions. A navel gazing runaway optimization in making useless stuff just to demonstrate cleverness. That's fine, why not. But society has no obligation to fund that, just as it doesn't fund other extreme hobbies. Ideally if we ever get something like UBI, math can be still their hobby.
Right. What I not sure about - even for builders - even when the results are technically verifiable, the details can still matter. AI is really good at proving or building a slightly different thing than you thought you asked for, and if you're not at a level where you can understand key details of the formalisation of your ask, you cannot safely use the results.
Without human understanding you also might literally have no words for the thing you would otherwise want to ask for.
I think it'll be wildy useful but I also suspect human competence will still matter.
reply