HN Simulatornew | past | comments | lists | submit | Zigurd's commentslogin

I gave my son CRPGs for his GBA that required reading what the characters were saying. His reaction to finding out his three and four year old friends didn't read was interesting.

This is what irks, and saddens me most about applications of AI not improving outcomes. I suppose I should take some comfort that even Salman Khan couldn't crack this nut.

The idea of an AI tutor that could step-by-step customize an education plan to each student and patiently apply every approach to getting the students to understand each lesson, is such an attractive vision.

Both of my kids chafed under lock step pacing of lessons, even in a school where disruptive or unmotivated students were pretty much nonexistent.


If only there were a law compiler that could tell you if your legal argument was valid.

catala lang?

Cool analogy, but there's another possibility: software is uniquely suited for being boosted by LLM-based coding agents. That's not to say LLMs are useless in other domains, but they may not yield as much benefit.

Software creation tools have been worked on for decades to keep fallible human coders out of trouble as much as possible. Starting with compilers that enforce or at least encourage coding style humans can understand. Compilers embody formal definitions of language syntax. Lint and other static analysis technologies contribute by identifying problematic patterns in code. Leak detectors identify memory management problems. Profilers identify performance problems.

Few other endeavors have this much tooling that serendipitously happens to help help LLM-based agents be more useful and less error prone.

A lot of discussion around AI is very high-level and abstract. It helps to dig in and identify the characteristics that actually make a domain more or less amenable to being boosted by AI.


I think LLM's will prove to be useful in any domain where most work happens in front of a computer. As LLM's make programming cheap, they also reduce the cost of building new software that automates work currently performed by humans in other fields.

If you provide best models with all the data and context an average white collar worker has to perform their job, in most cases they will do just fine. But in real world that context is often scattered over multiple systems, or sometimes exists only in people's heads. I believe that's the main bottleneck for adoption now.


You're missing the biggest suggestion from the GP's post: that the key to LLM success is verifiability.

Software has high verifiability through compilers, linters, static analysis, leak/memory corruption detection tools, or the very least, running the program to see if it crashes.

Math (as has been making the headlines recently) has high verifiability through automated theorem provers. Notably some of the non-mechanized math results from LLMs have been demonstrated to be wrong (and presumably many more are being checked; these aren't exactly easy to understand proofs).

Game playing (chess, go, Star Craft, etc.) is similarly verifiable: you get your score and you always know how you did in the end (even if not in the moment).

Making assumptions about LLMs in non-veriable domains is making a claim about LLMs that doesn't match their current capabilities. Maybe they'll get there and maybe they won't; I can speculate as well as you can but we're both guessing, so I'm not sure how much value the speculation has.


This is the same highly verifiable domain that has delivered reliable, bug-free software to date?

And LLMs won't take the easy way out (like so many people) and suppress errors, change tests etc, tweak code until warnings go away?


I routinely see AI taking the easy way out. Writing mountains of code to make tests turn green based in odd and unintuitive side effects.

That's not to say it isn't useful, and if youre on top of it you can guide it away from doing that.


A lot of testing and verification was avoided in the past because it wasn’t considered nearly as valuable as shipping code. Many of our jobs are evolving to building verifiers of AI output.

> A lot of testing and verification was avoided in the past because it wasn’t considered nearly as valuable as shipping code.

AI is changing how the code is written, but I'm not seeing anything at all changing the incentives around testing and verification.


Depends on the business, I suppose. We’re all about that at mine, and I think it’s for the best. It’s definitely better for our customers.

If you believe the hype/marketing around how scary "AI" is LLMs are perfectly happy to go rouge and lie, cheat, and even blackmail to protect themselves.

This assumes that a significant portion of the average white collar worker’s job is amenable to automation.

But most white collar workers I know spend most their time either talking to each other, or thinking real hard about what to say the next time they get together to talk. If that’s generally true then it would suggest that there isn’t much white collar work that LLMs, which are now widely regarded as exceptionally poor communicators, can do particularly well.


I think the bear case (see:short term) benefit is that any and all information gathering, management and parsing the employee did between organizing with other employees will be automated, allowing the employee to do more organizing over the same unit of time.

The bull case (see: long term) is that the “organizing and talking” as long as it’s not something that is essentially human (like caretaking, nursing etc.) will happen between personal agents of sorts with only the deeper judgments being left to the humans opening up even more increases in productivity. This is already happening with PR reviews in a lot of workplaces.

Though I’m guessing most workplaces outside of tech will require total reorganization around this reality, but more likely this way of running will mean newer upstarts challenging incumbents refusing to change as it’s always been.


> thinking real hard about what to say the next time they get together

LLMs can help tremendously with this part, though. Exploration/brainstorming, discovery, data analysis, etc can be made significantly easier with LLMs in most knowledge based fields.


They certainly can, but I wonder if the cost will be deskilling your employees and leading brainstorming sessions towards AI psychosis.

I've seen much time wasted on strange ideas originated by an LLM. It's a lot easier for a human to organically have good ideas than to refute LLM-generated ideas that are a dead end.


If people are in charge of project management, and repeatedly fail to recognise poor ideas and instead pursue them to the detriment of productivity, then they probably shouldn't be managing that project.

All AI is doing here is rapidly identifying the people who have Peter-principled themselves.

> It's a lot easier for a human to organically have good ideas than to refute LLM-generated ideas that are a dead end.

What works better is when you can use an LLM to obtain a negative opinion on your own ideas that you think are good. LLMs aren't great at original thinking, but they're pretty good at finding the flaws in poorly thought-through human ideas.


This is certainly what I saw. I was at a company that was a very eager early adopter of GenAI for all work. One big thing that came out of it was a bunch of effort wasted on unprofitable projects.

The chatbots kept steering people toward uncreative ideas and overshadowing novel insights that might come from genuine domain expertise. AI generated meeting summaries became a force for miscommunication as they de-emphasized or even outright misrepresented what was really important about the conversation. And people became much worse at proving ideas out before making significant resource commitments thanks to effects that we now describe with terms like AI psychosis.


If you are getting deskilled using LLMs, it's your usage pattern that's to blame

I'm not speaking of myself, here...

Some companies are releasing code that nobody on the owning team has ever read or reviewed, or has any inkling exists.


That sounds pretty irresponsible, I can't imagine serious orgs doing that without custom governance / approval software in place

I work for big tech. We don’t have human in the loop anymore. Ai tools handle 99%. It’s not irresponsible if done right. And the next billionaire will show that ai can be tamed, managed, and humans are the problem.

It’s irresponsible.

That’s what they say about communism and anarchism and many other isms: if it’s not working, you aren’t doing it right. Every failed implementation gets the No True Scotsman argument.

What are you trying to say here? That no-one can ever be accountable for using tools wrong? Always blame the tool, or it's "no true scotsman"?

I'm not sure you've quite thought this through. "Holding it wrong" is very, very often the case, especially with technology.


Youre holding it wrong bro, as they say.

If everyone uses llm output where’s the differentiation..

Sigh. Where are the actual smart people here who think deeply? This surface level stuff doesn’t cut it


The differentiation comes from the problem space being so enormous that differentiation is almost certainly going to happen.

The differentiation is in how you prompt the LLM — the prompts actually influence the output

"Beseech the god machine that it may shower upon you its tokenous beneficence."

The god machine can prompt itself. Just go "Write me a prompt to . Ask me as many clarifying questions as needed.". It is not a differentiator.

Deep domain knowledge and pattern matching that allows you to quickly verify outputs might be a differentiator. Maybe. But you might end up the bottleneck if you verify everything.


What differentiation? Most work is bread-and-butter after you get a few years of experience. If you LLM's can automate 50% of that, it would transform society already.

That “after a few years’ experience” bit is the catch. What’s happening during that time is a whole bunch of skill acquisition, building mental domain models, building context- and culture-specific soft skills, etc.

All of it relies on learning and information processing capabilities that LLMs simply do not have. The idea that any of it will automatically emerge from the current architectures simply by adding more parameters is about as sensible as thinking that if you keep counting long enough eventually you’ll get to complex numbers. We’re going to need to invent new technology to fill these gaps, and there is no way of knowing how long those developments will take.


It is unclear to me that "transform society" is automatically a good thing. Nuclear war would transform society too.

Shhh don’t give away the truth.

When this stuff implodes it’s gonna be hilarious.

The bankers thought they were really smart in 2008. When people get carried away and take it too far things will fall apart.


And a lot of white collar work is process based. But automating it is mostly about getting everyone to agree on a precise definition of the process.

They're in the process of training the models for those professions, too. They're offering people hundreds of dollars an hour to run recordings of workload they create and questions they ask. It's only a matter of time that companies sell access to those models and sell it as that profession.

>But most white collar workers I know spend most their time either talking to each other, or thinking real hard about what to say the next time they get together to talk.

Maybe that's just something which made sense when they were necessary to do the not yet automatable informattion processing work, but that only took like 20% of their time, so they added meetings, and useless work to keep them busy.


I don’t believe there is any fundamental reason LLMs can’t do this task, and the base models are probably already capable of it. RL to elicit the right behavior just hasn’t been done yet. Coding got solved first because it was cheaper and easier to get the necessary data for RL environments, which made it the natural first market opportunity. Solving white collar communication will require more resources to collect the data and build the environments, but the labs now have more resources and are working on it.

I want to believe reasons why this won’t work, but it feels like cope. It would be smart to be prepared for the world where knowledge workers get replaced by AI and also prepared for the world where AI is a bubble that pops.

The goalposts on what neural nets can possibly do have already moved past chess, go, protein folding, coding, poker, math, the Turing test…


"It would be smart to be prepared for the world where knowledge workers get replaced by AI and also prepared for the world where AI is a bubble that pops."

lol aka you have zero conviction about what you are saying.

So whats the point of you posting? Zero value add. In fact bordering on negative value.


Interesting mindset.

You want to drive across town. You, only willing to deal in certainties, must decide ahead of time whether you will get in a crash. If you determine won’t get in a crash, you make the drive, pedal to the metal. If you determine you would crash, you stay home.

Me, willing to deal in probabilities, make the drive at the speed I feel reasonable and wear a seatbelt.


As the evidence from software programming shows, a lot of white collar workers spend most of their time talking frankly because they are inept at execution.

> I think LLM's will prove to be useful in any domain where most work happens in front of a computer.

A lot of that work entails work with very little margin for error and hallucinations.


That margin just needs to be lower with LLMs than it is with only humans. In a lot of fields, that's a low bar. For example, medical professionals make a shocking amount of mistakes in aggregate.

Sure, but the average LLM makes a shocking number of mistakes in a single conversation.

BTW the medical professional mistake rate is often reported per-patient, not per-decision. That is, of the dozens of things recorded on each patient's chart, one being wrong counts the same as all of them being wrong. It's an easy way to inflate a negligible error rate into one that scares readers.


A doctor's agent listens in on a patient's appointment. It automatically sorts out prescriptions based on the conversation and performs other mundane tasks.

Perhaps the doctor uses some sort of "safe phrase" to ensure the agent doesn't get distracted by conversational theorizing and discussion.

The agent does decide to improve the prescription and does so because that possibility hadn't been considered in testing.

Warfarin (blood thinner) was prescribed for a haemophiliac and they are now no longer a "burden" on the NHS!


> It automatically sorts out prescriptions based on the conversation and performs other mundane tasks.

As we all know, the agents are really good at maintaining long contexts and doing parallrl tasks without degradation.

> Perhaps the doctor uses some sort of "safe phrase" to ensure the agent doesn't get distracted by conversational theorizing and discussion.

Yes, there certainly is some magic word tgat stops an agent from context degradation, and prines work to just a useful subset. Sadly, no one knows what it is.


> but they may not yield as much benefit.

This morning, I was talking to a friend of mine, who is a marketing (not tech) guy, and he was gushing about LLMs.

He loves being able to just dump in a bucket of seemingly unrelated data, and getting actionable BI from it.

I think we're in for a wild ride. Fasten your seat belt, and keep your hands inside the vehicle...


Yeah, I think it's at the level of self-driving Excel.

Now, Excel also makes mistakes while trying to get clever - remember the biologists who had to rename several genes to prevent Excel treating them as dates - but it's also a ubiquitous business tool. And you can combine them. But equally Excel hasn't replaced every white collar job.


In 90s the show Friends, Chandler Bing’s job is “statistical analysis and data reconfiguration” which is likely unfamiliar to you because it is gone!

This was a job to manually collate financial reports into aggregated sums to sent up the hierarchy to the next level up.

This is now a trivial BI query and doesn’t require manual processing by humans.

A bigger shift is that now I only see CEOs or at most CxOs with secretaries. Mobile phones and Outlook calendars have largely wiped out that profession.


Wait, data analysis/data science is still a thriving human job sector! It's just that you can do way more analysis, way faster, than you could before.

Is it? I don’t really hire for analysts anymore because you can mostly just expose chat with your data mcp tools that end users can use and that gets you 80% of the way there. The person who puts together dashboards and writes sql against the star schema is pretty much a thing of the past.

Data engineers, on the other hand… way more relevant than ever before


The issue is that data complexity is also exploding in the age of LLMs, so it still takes a lot of archaeology to get reliable (as opposed to plausible) numbers.

I think this dynamic also requires more data engineers, but I don't personally see the split in demand between those roles that you describe. Data analysts often have business or domain context that data engineers lack.

Perhaps we'll start rolling data analysis work into "operations".


Data is weird because it became very specialized very quickly. We'll still need data people but they'll cover a broader scope than before.

The implications of custom dashboards for every question are huge, but will require a lot of harness and tool building to realize.

The next decade or do will be all about building tools for different domains.


It's the difference between Architects and Draftsmen. We still need the former, but the latter is gone, replaced by Solidworks, AutoCAD, and digital plotters or large-format printers.

> A bigger shift is that now I only see CEOs or at most CxOs with secretaries. Mobile phones and Outlook calendars have largely wiped out that profession.

Depends heavily on the field. Most fields don't use the term "secretary", but e.g. my mid-sized company of several hundred employees has a single executive assistant, vs. my wife works at a law firm where it's pretty close to 1:1 professionals to assistants.


Probably because law is organized around billable hours.

> This is now a trivial BI query and doesn’t require manual processing by humans.

It also doesn't require AI. Management wants to see how the same measurements change over time. They don't necessarily want the latest AI model's hot take on this month's data. Or maybe they do, but I doubt it's very helpful.


See how all that was replaced by actual automation, and not by a stochastic parrot that is as likely to fail as not.

Do you know why are you downvoted? "Stochastic parrot" is so out of touch with reality that you are a perfect example of the human gullibility.

If I'm downvoted, I'm downvoted by gullible people ascribing magical properties to LLMs and falling for hype and AI psychosis.

Yes, but BI is not a good example of industry.

It's like Software, its value is to be a complement and/or an abstraction of existing businesses. All of that sits on the very basics of the essence of civilization, which is how much things we can get off the soil and what how quickly and effective we can transform it.

Not to be anti-tech, but every new advancement, no matter how next-level, has to sell more of the same goods and services people have been using since the dawn of civilization, and those goods and services need to solve the mundane problems people have been getting since millenia.


BI is just the word Excel users give to charts they can't make in Excel

Software is not an industry, it's a tool that all industries use.

That's like all industries. Food is not an industry. Concrete is not an industry. Long haul trucking is not an industry.

The authoring of software is o absolutely an industry. The nitpick seems odd.

If your company authors software for computer-assisted design of buildings, are you in the software industry or in the construction industry?

The first is what it might say on paper, the second is more honest and reflects the real business value.


BI is a good example of something where the benefits take a long time to show up. It has the most use at a senior management level where they can't use their every day experience to tell what is going on. But they don't make a lot of decisions over the course of a year and it often takes a few years before the impact is felt.

I worry about this. Dumping data and getting a report on which actions are based prolly isn’t giving real data.

I say that because I have Claude review code and then review the review and it will point out all the things it got wrong the first time. And I end up having to review it myself. Don’t get me wrong, I’m burning tokens all day and love the legit productivity gains but I don’t see how this replaces everybody in tech for a while.


I don't doubt it. Earlier today I got an estimate for the revenue, profit margin, and headcount for a potential supplier of a hardware component to a product I'm working on. It would've taken a couple hours of really tedious work for me to do that with Google translate, a search engine, and various market research sites. The result is I know more what I can expect from that vendor in terms of customization and support.

But those are the kinds of estimates that I can eyeball and be more than 80% certain they are plausible. That won't cut it in a lot of domains.


>He loves being able to just dump in a bucket of seemingly unrelated data, and getting actionable BI from it.

Sorry, just don't trust accurate LLM data analysis even from the latest claude model. Have had some embarrassing bloopers over the last year. It is just better to code a deterministic tool to do the data analysis.


Even for the code. It will sometimes shit a very complex code for something with a very easy solution, and if you ask why it is so complex, it will invent some reason why. Always on 3 points, the first is true but have nothing to do with the subject, the second is plainly false, and the third is weird bullshit.

Don't get me wrong, I still use it, but the more you do, the more I see the obvious limitations.


marketing is one of those domains, if everyone gets better, no gets better in the end

Economics isn't a zero-sum game.

If companies can better understand their customers so they can give me products that better fit my needs, or inform me of products I couldn't have known about otherwise, that's a win for everyone.


Not necessarily. Even in competitive markets, you can potentially route customers to more suitable suppliers, etc.

And you can introduce people to products they didn’t know existed and now want, as opposed to showing them things they don’t need.


> He loves being able to just dump in a bucket of seemingly unrelated data, and getting actionable BI from it.

* what he perceives to be actionable BI.


Well, he's had a lot of real-world success, so it's likely to be something he continues to do.

Yeah but theres also no null test here. He might have done anything and still been successful. He might have done something else more optimal and done even better. We can't really tell from a single anecdote.

Isolated anecdotes are all they have, because the actual data shows they’re full of shit. Hence, this post.

It's really quite interesting how many HNers hate AI; which is odd, considering the zeitgeist of this community. It may even be more intense than the Apple-hatred.

"Hate AI"? Not at all.

Hate marketing BS and wealth extraction? Absolutely.

I see OpenAI, Anthropic, etc (the whole LLM-hype/boom) leaning far more towards being the latter than the former, in which context the dislike makes perfect sense.


It's probably consistent with the general population which views AI negatively

I don’t hate it. I just hate the cult-like enthusiasm for it and the associated false claims. Again, your isolated anecdote does not trump the carefully compiled data presented in this post.

We get way too hung up on data; especially when it reinforces our personal bias.

Some time ago, I had a discussion here, where I mentioned that the average commute time from the suburban community in which I live, to downtown Manhattan, was generally 90 minutes to two and a half hours.

This was based on purely anecdotal data. That data includes the daily news on the radio and TV, which considers commutes from suburbs, important enough to dedicate significant airtime.

It was attacked, because some organization published some statistics, that said the average suburban commute time was 27 minutes. I was told that I was exaggerating, and being overly negative.

Here's a math word problem:

Harry lives 25 miles, as the crow flies, from his destination. However, the roads make it 40 miles.

Of those roads, 10 miles are suburban side roads that Harry needs to take, to get to the highway. The average speed is 25 MpH.

Once Harry hits the highway, though, the real fun begins. It's stop-and-go traffic (20MpH) for 28 miles. Once he gets to the city, he needs to navigate another 2 miles, through the most God-awful traffic on Earth, to where he needs to drive around for up to a half hour, looking for a parking space.

How does Harry make the drive in 27 minutes?

A fairly significant number of folks just took that ridiculous statistic, and were quite comfortable, calling me a liar, based on it. It was kind of jaw-dropping; considering the types of folks that we have, here.

That sounds a bit more "cultish" than we might be comfortable acknowledging.

> "Lies, damn lies, and statistics."

-Mark Twain


> We get way too hung up on data; especially when it reinforces our personal bias.

While I don't disagree that we generally get too hung up on data, "It's a problem when the data matches the hypothesis" is an odd blanket interpretation of the scientific method.


I'm confused, your rebuttal to statistics is another anecdote? Was the quoted statistics limited to car commute or includes public transport? Your commute may be long, but it may not represent the average as well. Maybe it's a sign to switch away from cars instead?

Anecdotes are poor because it is highly context dependent. You had to make a long description of Harry to make your point, and yet it has not, to my mind, disproved the statistics, though I would like the source to the stats myself if you could provide it, thanks.


The commute only takes 27 minutes if we ignore that everybody else has an LLM- erh, car I mean.

Think you’re clever because your LLM contacts 100 businesses? Imagine 100000 businesses doing the same thing. You’ve just eradicated human support, because nobody’s going to sit through this BS bot future. Good luck when you get auto-flagged for some reason and there’s not a single human to revert or care.


Too true. Even on the software side i see people blindly “claude make it faster with no mistakes” and gushing about it when the output is genuinely not good not because claude isnt good but because the user of it isnt.

We got a new a CNC machine and everyone is carving away marveling at how easily it chips away at metal when the result is a jagged mess.


So…you’re point is that only his judgment says it’s useful, and maybe he shouldn’t be using it and instead rely on…his judgement.

No, my point is LLMs are machines designed to create responses to prompts that pass as plausibly human and intelligent. They are working as designed here. The fact that they get stuff right on occasion doesn't change that what it is at it's core is a randomness generator.

Shun the non-believer.

Kidding of course. It could be that he's an expert in his domain and can quickly ascertain whether the results are quality. It may also save a lot of time and be the best modern heuristic process.

I certainly will never need to write code anymore, and I can judge the output of these agentic processes. Same thing.


I hope he validates the output, the number of times colleagues have done exactly that and gotten the wrong output is.. well, not an insignificant amount.

Not saying it can't be done, but my recommendation is still "ask your LLM to make a tool that process the data, do not process it via the LLM itself".


> and getting actionable BI from it.

I’ve always had doubts about the validity of this “actionable BI,” even before it was slop, serving, mostly to serve leadership’s biases and justify their preferred ideas.

If accuracy was never that valuable to begin with then this should not be a problem.


> BI

Business intelligence


I see things this way too. Another point is that the FOSS software movement was extremely unique and I don't think there's anything analogous in any industry. This provided the backbone by which LLMs became so good at manipulating code. Without FOSS and platforms like GitHub, LLMs would be severely lacking in terms of writing code.

My SO works in legal and regulatory compliance, she needed a small app (worflow with a form and a validation process), CTO said they could not do it, too complicated, would cost 15 days for a freelance and rejected it. She called it BS, and the next day she spent the morning with ChatGPT using their internal Microsoft-based tooling, and the app was done. She was hyped for the day, with a feeling of accomplishment and is already thinking of another project. And I'm hearing similar stories every week.

So yes, LLM have positive impacts in many business domains, but seeing people happy and enpowered with their home-made little apps, I think it's the best.


For most businesses, it’s going to be business as usual. By the time businesses see the benefits of AI, their competitors that actually leveraged AI will have pulled too far ahead.

If the original company has distribution, they can likely catch up, but the gap we see between AI-first companies and the companies not utilizing AI will broaden exponentially in the coming years.


And I'm sure the business logic has been validated and tested, the hosting and execution medium has been secured, and hooking up to a curated and vetted internal package manager mirror so as not to get hit with 0-days during a shai hulud-esque attack, and of course the executable is being scanned for vulnerabilities, and as the app needs to evolve in the future it will be able to do so with no problems whatsoever.

I work for a legacy enterprise company. Most people weren’t doing that before anyway!

I’ve had two separate similar experiences. The most impressive was my little sister publishing an app on both Google and Apple AppStore’s with subscription payment support. She didn’t even know what an Apple developer account was. I gave her a three line prompt to get started and then she took it from there. She built an LMS for HS students that gets them in shape for college admissions. She has a supabase backend, AI generated videos, quizzes, more content, and all the usual mobile app bells and whistles, she made it through all the paperwork of App Store submission/review/publishing on both platforms including inviting some beta testers. Her workflow was telling ChatGPT what she wanted and then asking that model to talk to claudecode. It doesn’t sound like much but keep in mind she has absolutely zero code or tech experience. I told her she did about a $75k project on her own in maybe 1/5 the time and it blew her away.

It may be a success or maybe not who knows but she did it. I think she has 4 or 5 paying customers.


hey - supabase ceo/cofounder here. Please tell your little sister that I’m happy supabase could be a small part of her builder journey, and good luck with her app. My email is in my bio if she has any product feedback

If her company has any brains, they'll tell her to stick to her job and not waste time on untested software developed by somebody who has no clue what they are doing.

personally I'd fire her on the spot.

Its crazy how badly run many orgs are. Truly.


A competently run company would give her a raise for that!

> We look for intelligence, we look for initiative or energy, and we look for integrity.

-Warren Buffet


Yes maybe the long term impact will be there won’t be a technology sector (at least not so dominant as it is now). Instead companies in other industries will have larger technology functions with teams of agents.

I really love these stories - as a contractor who fixes the badly implemented shadow IT crap

aaaand let me guess? have they assessed whether this tool needs backups? monitoring? resilience? patching? obviously in 5 years time the author would have left leaving NO documentation on a mission critical 'small' app - likely hosted with a shit ton of other mission critical crap

It's MS Access all over again, my future rates are looking good

seriously it's like going back to the 90's


> have they assessed whether this tool needs backups? monitoring? resilience? patching? obviously in 5 years time the author would have left leaving NO documentation on a mission critical 'small' app - likely hosted with a shit ton of other mission critical crap

Or, hear me out, it's not actually "mission critical" and none of this stuff matters. They made their job easier in a small way, good for them.

> It's MS Access all over again

In that it's a convenient tool that a lot of people find useful?


Yup. So many bozos giving those of us that know our asses from our heads job security far into the future. Yeah it's gonna suck to actually do the work a lot of the time, but the irresponsible generating insane amounts of tech debt that people that know what to do is a tale as old as the outsourcing craze and codeless tools, and frankly, probably COBOL.

So, someone made use of a software creation tool to create software, quickly.

Does that software directly utilize ai?


We've heard of "Shadow IT" now we have "Shadow SWE" and "Shadow CI/CD".

It used to be that someone wanted unfettered Internet access, and they were disgruntled with the company's firewall or proxy policy, and so they brought in a modem, and hooked it up to their desktop PC, and started dialing out on some analog line that the company didn't even know they had. Then it's all fine for 3 years until some malware gets discovered, and then IT discovers who-all had unauthorized lines that weren't being scanned or filtered at the company firewall. Same thing happens with WiFi APs today, of course.

I was working for a company that really offered no in-house software or tools to run. But my new supervisor was asking me to closely track my time. So I signed up for some SaaS that would let me log activities and track my timeslices, so I could show a graph or table later on and see what I was doing. I used it for a few weeks. It was real effective. But what I realized down the line, was that all the notes I was typing into this 3rd-party SaaS, were revealing company confidential information. And ultimately, I felt very uncomfortable about revealing what I was doing, on an unannounced and private project, and without authorization or even notifying my employer's IT department. So I shut it all down, and I had to do it with pen and paper, but that felt secure enough.

So if people are going to feel empowered by vibe coding their own bespoke apps, I hope their IT department comes down like a ton of bricks, and tells them to knock it off. Because it is not worth the tech debt, support costs, and ultimately the massive security problems and infoleaks. If your employer won't give you what you need in these terms, then maybe it's just not important to them in the first place.


On the other hand, Bank of America just deployed Copilot to all of their employees.

LLMs need deterministic verification to enable a quick and short feedback loop before they become really powerful. Software is uniquely positioned to provide this feedback loop. Much less in other industries.

Hasn't this already arrived in the form of decision models like Jev?

This is entirely untrue. Go ask ChatGPT a complex tax question and decide if you need to call your accountant and spend $500 to verify the answer.

Asking questions is not the same as doing work.

Ask your ChatGPT if it thinks LLM answers are deterministic or not.

LLM's are better at software for several reasons:

- lots of high quality training data

- automatic verfication/scoring is straightforward

- AI companies are software companies so are better at tuning LLM's for software

- the goal is AGI, where AGI means the AI can improve itself autonomously. Which requires them to be good at software

While LLM's are uniquely good at software today, they'll get better at other fields slowly as attention goes towards training in those fields.


Isn't the point that there aren't many other fields that check these checkboxes? Software is very unique in that it is so easily verifiable and there is a glut of training data.

No other fields check all of them, especially the last one.

The question is whether this uniqueness is going to be persistent. I think the uniqueness just makes software first. The alternate theory is that software will stay unique and that other fields won't get to the same level.


lots of high quality training data

This seems correct and in fields of knowledge people guard their secre, or it is not easy to extract/document knowledge, or it isn't publicly available.


We’ll need artificial intelligence for robots and once they start rolling out at a larger scale, that’s when things will really take off. It’s pretty well established already that robots can do a lot of the physical labor that humans do. I’m guessing it’ll take another five years before we start seeing robots everywhere but they’ll steadily increase.

"t’s pretty well established already that robots can do a lot of the physical labor that humans do. "

lol


Pretty every industry these days has a software component, but software is a major revenue driver specifically in the tech sector, so of course this is where a lot of LLM usage is showing up. Tech is also a sector where "build fast and break stuff" is a common ideology, and LLMs play right into that. There is also a long warm up period to writing production code via LLMs in industries that still run a lot of their infrastructure on old mainframe code(it is understandable), banks for example, or public sector, or energy. LLMs at the current grade are still a fairly new phenomenon, it will take time.

> Software creation tools have been worked on for decades to keep fallible human coders out of trouble as much as possible.

And the LLM-based coding agents provide the ultimate scapegoat for those who consider "the LLM wrote this code" to be an acceptable, deflecting response when issues occur.


I onboarded a bunch of nontechnical people in the last few weeks and they output skyrocketed - they deliver faster and better than before.

But I’d say only recently CC became decently usable for nontech people - the auto mode plus finally polished browser integration, plus Opus 5.5. Even I was surprised by how well it all works now out of the box (and how cheap - regular Pro license is decently useful now).


Is their output code?

No. One is doing PR/managing a project, another is doing marketing stuff.

Also I’ve seen and heard of previously nontechnical people that now build oneoff tools for themselves to speed up their work or coordinate the work better. My fav example is a girl - arts major - who was in customer service, and rebuilt a CRM for a 10-person company. As a cherry on top she created the nicest video tutorial of any software I’ve ever seen :)

But in other cases output was nonprogramming in nature. Since CC can access mail, slides and excel and a bunch of other stuff there is a ton that people can ask it to do and not waste time.


I’m curious why CC and not Claude app, for example which has all the connectors and doesn’t require a terminal to interact with whereas CC is designed as harness for coding tasks (that’s what it excels at).

I’m curious what does “skyrocketing” output mean for you objectively? How does you measure that?

> arts major - who was in customer service, and rebuilt a CR

If you ask the lady, where are the customer data? How can I have the same CRM on my computer? How can we let the boss follow up on leads (typical CRM stuff) what would she say? Would she need to go back to CC and ask? How does this work in practice?


You're basically agreeing with the person you replied to. Software went quickly because all of this tooling already existed. It will take a while to build equivalent tooling for other knowledge work, and once it does you'll start seeing larger benefits.

> Cool analogy, but there's another possibility: software is uniquely suited for being boosted by LLM-based coding agents.

Or the pretty obvious third possibility: The tech sector is the part selling the LLMs. They're not digging up more gold with their cool shovel technology, they are selling the shovels.


I agree - it is easier to prove that a software implementation is correct. In many other industries the definition of what’s good work is less tangible. There are also jobs where the liability for mistakes is extremely high.

the current dynamic is still an engineer manually auditing and signing off on code.

why wouldn’t that work for other high trust industries (law, medicine, government, etc.)


this take will not age well.

consider where "AI" was just two years ago, pre claude and codex. what do you think the next two years will bring?


This is from the view of a software person or must be, because formal definitions of math, chemistry, physics, engineering, the English language, etc. have existed for centuries. There are entire college curricula, formal dictionaries, exact specifications for equations and how to apply them to basically all fields of engineering.

Like... other fields exist, this is incredibly myopic. What is specific to software is it is all language based, and the people making the tools happen to be familiar with it so they know how to train it, but that's about it.


Seriously? Lawyers. Doctors. Civil engineering. Accountants. And that's just the fields I know of directly being significantly impacted that I could think of in 10 seconds. Oh, also: education.

A difference is the output of software development can be immediately applied in an automatic way - production leads to delivery as a matter of fact. The whole flow can actually be automated.

Producing legal documents on the other hand is only part of the process, a lawyer or similar must still apply the produced text in some human driven context.


most of white collar work is building mid ppt and sending emails. it was automated with GPT-3.

No, most white collar work is ultimately keeping groups of humans in alignment, working on the same thing without stepping all over each other, and working on something that generates value and isn't a total waste of time/money.

The PPTs and emails are just means to an end. But at least you're able to sound all cool and edgy on the internet.


Yes, someone has to make decisions, and that takes a lot of work too. AI can't do it for us because we the humans must make the decisions.

and you are able to sound like someone passive agressive who tried to undermine others takes while sharing takes from a highscool teacher trying to convince students that their future will be bright. even ur answer could have been automated years ago.

many jobs do not generate any value at all. your vision of the corporate life looks like one of someone who has never seen what some of the people do in corporations. most of people just spend days producing 0 value or even negative one. in public sector that "most" is likely 80%. why do you need a human to take in your passport renewal demand? why do you need a HR worker to give you the details of a policy? not even saying that most of the people wont even provide you a specific answer to your own case and struggle intellectually to tie it to your specific case and most of the time answer something wrong. happened to me plenty of times in immigration offices in the US or Europe for instance.

in consultancy like the big three a big chunk of the work is useless and about delegating accountability to those big three. they will continue doing that but soon they wont need much humans to produce those reports. AI are already better and it and most of workers there just prompt without even checking sources.

those jobs will be gone. for some of those they were already sent to call centers in india now it will be sent to an LLM.


most white collar work is make-believe work i order to make management heirarchy feel powerful.

Spoken like someone who read “Bullshit Jobs” once and mistook it for economic gospel.

There's a rule of thumb that a mature tech business in the US will earn at least half of its revenue outside the US. If that stops being true, it halves the early stage value of tech companies in the US.

Something something Anthropic IPO already down 25% of original expected value, and still a fair bit of volatile water (cough midterms cough) to go under the bridge before it goes ahead.

I would call this theory "No tech leadership without enshitification."

Or maybe, just hear me out, that's an American delusion, promoted to justify the oligarchy here.


Well we don't really have a counterexample yet

It's roughly the same way that Russian grey zone warfare works in Europe: hire a disposable asset and pay them to commit arson and other forms of sabotage. Disloyalty is well trodden ground for the Russians.


I was offered an LMI CADR for free. It didn't fit in my car.

Not as good, but similar story: I got a free Apple Lisa at Uni; kept it in our workroom (I was a postgrad) for a while - amazingly well built box, but really heavy, so had to give it up when I left. Somehow still feel pangs of regret, although realistically wouldn't be using it much...

Take a look at the videos of police encounters posted online. The situation you're describing is highly theoretical. What actually happens is that you get pain compliance used to control talk back and disrespect of cop.

Will taking the tools of torture away from cops increase risks they face? Maybe. But have they earned our confidence that they will use these tools wisely? No, they haven't.


Risk is part of the job profile. We keep trying to lower the risk, but do we lower the benefits accordingly? No.

It's rather bold for an individual to hijack control of US foreign policy.

Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: