> the removal of all human leverage via their labor
I’m not convinced at all that SI will ever be able to produce the correct result with little input. The advancements in capability I’ve seen in models is that they understand my intention better, and follow industry best practices better, so their output is often acceptable. But I absolutely have to make my intentions known, as detailed as possible, or I will get the mediocre, standard solution that falls out of the weights.
I’ve been having trouble with models always steering towards certain solutions - they insert a queue here, a mutex there, a backoff over there. If I remove those, and let the model freely rebase the PR stack, then it will re-insert those mechanisms because it has been specifically trained to do exactly that. It’s essentially boilerplate, smoothly aligning my codebase to match their weights. Which is often good enough! They are training the models well. But individual problems require individual solutions.
My personal take on the human part in all this is: We get to decide, and the models obey. Our decisions are now super leveraged. So let’s make good decisions, and communicate them well. The layer of where decisions yield the most leverage, has been pushed upwards (I see it on the type and function signature level, others will see it even higher), but IMO it won’t be possible to push it up much further without losing truth to the decisions we need to make for a good product.
It’s just that the vibe coded stuff has so many assumptions baked in, that you will be constantly surprised about the behaviors your slopped codebase has. Doesn’t matter for small tools, where you’ll be delighted that clicking that one label copies it to your clipboard - nice one, Claude, that’s delightful!
It’s less nice when your web app launches that other service only when the other Tailscale node is up, because the agent interpreted your prompt as a hard access rule.
Lauren Tan from SpaceXAI/Cursor/Grok has been on a few podcasts recently. I was surprised how much info she shared, and she sounded quite honest about what works and what doesn’t, in the context of building grokbot. I wouldn’t be able to do all the verifications she does, with my token budget. But it doesn’t seem like rocket science either (pun not intended).
Maybe you can help me understand. What determines the flight level of understanding you need to function? Obviously you don’t need to know everything down to the last atom cluster (in objects), or down to every last line of assembly (in software). Why does a clean interface with a contract not suffice? What really distinguishes a low level library from a higher level one, in that sense?
Not the person you are responding to, but for me I really like to have a general understanding of how things work under the covers, but then have confidence that the contracts between me and those lower layers are rock solid, and that's simply not the case with LLMs.
Take software. In college I took computer architecture and digital logic design courses, and I thought they were immensely valuable in understanding how computers actually work and what software is actually doing. Sure, modern chip design is obviously several orders of magnitude more complex that what I studied, but I understand the basic concepts, and more importantly I have faith that chip designers do understand the nitty gritty details. Moving up the stack, I also had to build a rudimentary compiler in college. Again, modern compilers are a lot more complex, but I know that if something breaks, there is an identifiable bug either in my code or the compiler (or maybe even the chip). And importantly, while I may not have the skills to debug all the layers, when someone explains the bug to me, I can understand it in context (e.g. I'm not a chip designer but I thoroughly understand how Spectre is exploited and mitigated).
LLMs are nothing like that. Not even their builders understand the low level details. They're inherently stochastic systems, so people get slightly different results every time they're run. There is no "clean interface with a contract". And perhaps most importantly, many programmers are still expected to be responsible for their code, even when AI agents are generating so much of it that it's impossible to understand (or even read) it all. That's the thing that really stresses me out, when I'm responsible for a system but I don't really understand how it works.
To me the reasoning of understanding all the things you use in a rock solid way is hard to grasp.
You just typed some text and hit send. You trust that the combination of letters that form words and the combination of words that form sentences, arrive somewhere in a list of comments? But do you understand how those bytes (utf8 or utf16 or..) are transmitted (which endianness, how are they packaged, does it use sentinels or length codes), where do they end up (database, plain text), how is the text organised (alphabetical, by data, by points...)? In fact you are not sure about any of those. Still you typed and hit send. So how is that much different from writing a prompt and hit send?
So first of all, when LLMs first came on the scene, I did want to understand them better. I read the Attention is All You Need paper, I did Karpathy's transformer course, etc. And I think this basic understanding of LLMs and further reinforcement techniques makes me understand better how to use LLMs, e.g. what their limitations are, etc.
But most importantly, there are very different qualifications in my head for things I just use and then get on with my day, and tools that are integral to things I am building. I take the bus but I don't really understand how a diesel engine works. But if I'm building it, I'm responsible for it. In fact, you brought up some networking issues in your comment. For a long time when I was a software engineer (primarily a web developer) I felt that my knowledge of network engineering was lacking, so I specifically took some courses in network engineering to better understand how my code interacted with the network. You also brought up text encodings. Once at my job I did a full, rabbit-hole deep dive into text encodings and localization because we had frequent bugs related to localization, and I still find a lot of developers misunderstand a bunch of the important details in localization (e.g. the difference between an encoding and a unicode code point).
But sure, I don't need to understand every minutia of detail at the atomic detail. But I really like to know that I could, and in the meantime I have a enough of an understanding to build a complete conceptual model in my head.
Now consider the case where I send you a message (in a language that's not english) and it doesn't render as text on your screen. Reasonably, because I understand how network protocols work, I can assume that it's not a little/big endian mixup between our systems - the HTTP request would not have worked, then, as utf-8 and utf-16 are incompatible (unlike the weird utf-8 ascii mix). With HN, I can open up developer tools see what encoding it is sent as over-the-wire, and then I can ask you to look at the encoding on your end. I can say "the vibecoded browser you use is trying to interpret this as utf-16 but hackernews only sends utf-8". I can dial down the root cause of the problem and then address it.
Contrast that to a situation where I ask an LLM to use a tool and it can only do it like 98% of the time. How do you root cause that. How do you fix it such that that doesn't happen again?
Also consider: occasionally your support agent gives people free plane tickets.
Yes, we've made a lot of progress towards LLMs being fairly good, and adversarial systems and such do work, but it is asymptotic approach to 100%, never actually 100%
For me at least I have to be able to believe I can understand any part of the stack if I want in reasonable effort. I would happily dive into semiconductor physics if I happen to want to (I'm a former physicist so the atom level I pretty much already learnt at school).
Honest question: What is preventing you from understanding any part of the stack now?
You could potentially ask an LLM to explain it, but I would venture a guess that wouldn't feel like the right source to help you. You could use the LLM to help you identify source material written by humans in order to learn anything new. It doesn't have to "do all the work for you".
Modern AI systems are powerful. They are not omnipotent or omniscient. Understanding the details is still valuable, especially if you want to push the frontier of any known field.
In any event, the situation isn't hopeless. At least not yet. :)
You can ask frontier models with all the bells, whistles, language servers etc to give you every example of X. And it gives you 12 examples and says that's all of them. When you know damn well there are at least 40 (but not exactly how many). So you say no, you know it has missed some, such as X23, and X27. So it goes away and comes back again and says yes, there are 41 X, here they are. How much digging should you do to see if its right?
I have many times gone looking myself to find that it has still missed some. Asked it to go check for those, to look harder it goes oopsie and says now im sure ive gotten all of them (has it?)
All this to say, I have been burned, repeatedly, multiple times a day, for the last year+. I still use these tools, but I truly cannot understand how some people treat them as oracles that know everything about our codebases
Excellent points. I agree that frontier models get things wrong all the time. They are NOT oracles. I believe that we need to encourage our peers to become sophisticated operators of these systems, instead of treating the output like Moses and the 10 Commandments. Perhaps we need more specialists, because the major AI labs lack the incentives to venture down the long tail of knowledge. Agents can be a piece of the learning puzzle, despite their imperfections. Again, thanks for your reply. It gave me lots to think about.
You need to define the goal in a way that you will trust the completion. If you are not sure how you yourself would verify that there are 41, then you are in trouble. Verification must be deterministic, or it's worthless.
What the large models do really well nowadays, is that they won't lie to you if your deterministic verification fails. If you tell an OpenAI or Anthropic model that they need to run a certain `grep` or search or whatever command to verify, then they will do it. I haven’t seen them lie about this for a year, and trust them in this.
Surely you can see that for basically every example of this sort of problem, fully defining a deterministic check is the same as finding them all?
Like you're telling me if I had a script that printed all X, and it's my responsibility to ensure it has no bugs, then the agent could tell me all X and I could trust it?
This is not helpful at all? I am capable of running scripts myself and using the output directly.
No? It's like saying "why bother doing maths, a calculator can figure out the solution to any question you have if you just put the right equation in" there might be more to maths, it turns out. Most of it lying inside of finding "the right equation"
I can't intuit, it would take me an hour or two to figure out the right "deterministic script". Hence why i'd like a robot to do it, while also being able to ya know, actually do it
> You could potentially ask an LLM to explain it, but I would venture a guess that wouldn't feel like the right source to help you.
I don’t think this works, at least for me. Even on the 5.5 versions of Claude it’s still arduous to read at this point.
> You could use the LLM to help you identify source material written by humans in order to learn anything new. It doesn't have to "do all the work for you".
This probably depends on the shop, but humans aren’t writing docs anymore. That was the first thing to go, sadly.
I hear what you're saying. I often tweak the CLAUDE.md file to improve the quality of its output. It's an ongoing experiment.
As for "humans writing the docs", you raise a good point although it's not quite what I imagined when I wrote that line. I was thinking more about having the LLM/Agent create a list of verified output by humans with reputations as experts in whatever field interests you. This is important when pushing the boundaries of research, although it's overkill for wanting to know whether or not a new technology could be useful in your existing project (or the next one).
What use case did you have in mind when you brought up the docs situation? Sounds like you had some direct experience with that one.
I think a distinction is clearly emerging between docs intended for humans, and docs intended for agents. The latter is where all the Markdown-ridden slop should go; I'm not insulting humans with that.
In many case it is the sheer accidental complexity. I tried to get into Linux codebase and gosh it's so much harder than FreeBSD. And Chromium, LLVM... I don't believe it has to be that way. I use Common Lisp software (besides Emacs) written by human for anything I can and these have always been a joy to use or work with.
It seems the way industry is heading towards wrt LLM will make this even much, much worse.
It's the Mad Hatter's tea party, you sit down, you figure out how it works, and then poof a third of its gets overwritten and your past investment is significantly washed away by someone who absolutely didn't put the same amount of thought into what is replacing it.
Not OP but similar type of brain here. Certain types of abstractions make me uncomfortable, the worst being "trust me bro" type web framework abstractions where there isn't a direct link between A and B, just a contract definition that says "A and B will integrate... somehow".
When an API says it takes something, I need to know the exact form of that "thing". Saying "you can give it this object" doesn't suit me because where are the edge cases, where are the interactions, how can I know for sure I'm giving it what it needs. The docs never tell you where to find the object def, then you spend 3 hours diving into the source code to understand it's just a two item struct or something.
LLMs introduced an explosion of complexity, suddenly code bases sprout out of nowhere, and there's no point in following all the different datapaths through because the guy who loves testing out the latest claude will just change everything in a few days anyway.
With low level programming, nearly everything is just raw data, aligned to meaning. I like to tell juniors confused about file types, that file types are a social construct, they're just a bag of bytes with some structure. Same thing applies with low level programming. You know what a byte is, you know the endianness, the signedness, you can feel comfort in knowing you're not missing anything, and then you build up from there seeing each block build upon the previous.
> No one confessed the Machine was out of hand. Year by year it was served with increased efficiency and decreased intelligence. The better a man knew his own duties upon it, the less he understood the duties of his neighbor, and in all the world there was not one who understood the monster as a whole. Those master brains had perished.
- E.M. Forster, _The Machine Stops_
Abstraction is terrific and terrifying. It is the shoulders of giants we build upon, but when it begins to scale nonlinearly I fear it endangers us as much as enraptures us.
> I like to tell juniors confused about file types, that file types are a social construct, they're just a bag of bytes with some structure.
I always get confused about what the hiring landscape looks like. How is it that people are grinding leetcode or whatever just to get into places, but still have those kinds of gaps in fundamental understanding?
I'm not in a field where leetcode is the deciding factor but there's similar gamified focuses. The idea being you can pick out self-motivated autodidacts with those kinds of challenges/experience in the challenges, then fill in the connective tissue later.
This was just one example of a capable intern who was tripped up by file types.
Because even prior to LLMs, “knows how to program” and “understands computers” could be disjoint. I worked with someone who didn’t understand how to find what was consuming their disk space, nor how to fix it, or most egregiously, that they needed to empty the Trash (MacOS) to actually reclaim the disk space.
> You know what a byte is, you know the endianness, the signedness, you can feel comfort in knowing you're not missing anything, and then you build up from there seeing each block build upon the previous.
IMO you’ve chosen a flight level you’re comfortable with (which is fine!) but there is much going on below the abstraction that lets you view a file as a bag of bytes.
I’m saying this as someone who uses SwiftUI and CoreBluetooth on a daily basis, and I have to trust these foundations just as you seem to trust the C compiler and a filesystem.
A strict, formal, reproducible way to communicate with the machine.
A compiler is a rather predictable piece of software; reproducible builds are a thing. An LLM has approximate knowledge of many things, and approximate and fuzzy ways to do anything even moderately complex. This is great for research, ok for planning, and it sucks for execution. The fact that LLMs can write satisfactorily working code from high-level requests is a miracle, and, as with most miracles, we're likely not noticing something, being dazzled by the slight previously unseen.
> What determines the flight level of understanding you need to function?
The level where r and fl are seen as different letters?
Look, I know this comes off like I'm being an ass, which means I am being an ass, and I'm sorry. I also make many mistakes and need much forgiveness.
I'm also seeing many more innocent typos such as yours on HN. Does it matter? I don't know. Every other response to your question seemed to understand that you meant "right" instead of "flight", but that's also the kind of error, i.e.
`if x=1` instead of `if x==1` which is easy to make but not always easy to catch visually when you are looking at the logic internal to the loop.
This was actually helpful: I thought "flight level" was some new hip expression that I had not seen before. I was pondering what "flight" added there; is it like the hexagonal architecture that is not actually hexagonal?
But sure, "right" makes sense in the circumstances.
To me, bending "right" to "flight" reads like a fun way to clarify that the question is not about proficiency but about depth. Not a typo the way I read it.
Not the GP, but I think I have a similar urge to go deep in understanding details. I definitely find myself different than many peers who are more comfortable operating with ambiguity.
"A clean interface contract" is actually all I need to stop pulling the thread, but I find that is rare in large software systems. More typical is that the provided interface lacks details about edge cases, such that multiple behaviors could meet the interface. That's what makes me want to dive deeper and figure out what those cases are, what I think is the best behavior, and what the system actually does.
Working on platforms of sufficient scale, there’s a lot happening that’s out of your control. You can build the mental model around it, but it will contain a lot of black boxes where an abstraction hides details.
Generated code changes this, probably, but I’m curious by what degree.
My biggest learning after some experiments - a BF16 (unquantized) Qwen beats a Q8 of double its size for decisions. I guess that’s the reason Kev switched to 4B BF16, from the original 8B version. Isn’t it interesting that quantization seems to mess with decision accuracy?
That’s fascinating, but not that surprising to me. We act like quantisation is free “Q8 is basically lossless” is often said in the local LLM community, but it really isn’t. The trade offs are worth it, personally, and the damage to coding ability seems low: decision model approaches are stricter though
FWIW, my tests used only ranking. I used Kev to drive E2E test runs using agent-device, where Kev had to decide the next action - like pressing a button, or scrolling down etc, to reach a certain goal (UI state). It went surprisingly well, but 4B BF16 was significantly better than 8B Q8. The latter made some wrong decisions in every run while the former was almost at Luna precision.
„Sweet solution“ - anyone having tasted pure vanillin in a nontrivial dose, will tell you that it tastes absolutely disgusting. It only works in traces.
This is true for many (most?) flavor compounds. A little salt is nice, a teaspoon isn't.
Some sort of clove extract is used in dentistry as an temporary fix if a full root canal isn't feasible at the time. Got the tiniest whiff of that on my tongue during rinsing and it tasted downright toxic.
While this may be true, their throughput is practically unbounded. this has been a primary challenge to using local models at work, where its normal for me to have 1-6 sessions churning away at once. So while 100t/s on a single session is very fast, theres no way you could get a cumulative ~500t/s from a consumer setup like this, do the bandwitdth bottleneck on a single card/cpu/ram/ssd (tends to be gpu ram bandwidth is the limiter in my setup, but they all have respective limiters to this kind of throughput)
I think Opus 5.5 is now fast enough with the inference stack improvements that they did. Sonnet 5.5 is around 140 tokens/s when I had measured it last time. A $20 subs on claude and using only sonnet 5.5 will last a lot
I was firmly wanting to move towards local llm, and since the Sonnet 5.5 and recent Opus speed increases, I'm putting that on hold until local inference speeds can be improved. Not sure how much juice there is to squeeze there, but I'm hopeful.
I’m not convinced at all that SI will ever be able to produce the correct result with little input. The advancements in capability I’ve seen in models is that they understand my intention better, and follow industry best practices better, so their output is often acceptable. But I absolutely have to make my intentions known, as detailed as possible, or I will get the mediocre, standard solution that falls out of the weights.
I’ve been having trouble with models always steering towards certain solutions - they insert a queue here, a mutex there, a backoff over there. If I remove those, and let the model freely rebase the PR stack, then it will re-insert those mechanisms because it has been specifically trained to do exactly that. It’s essentially boilerplate, smoothly aligning my codebase to match their weights. Which is often good enough! They are training the models well. But individual problems require individual solutions.
My personal take on the human part in all this is: We get to decide, and the models obey. Our decisions are now super leveraged. So let’s make good decisions, and communicate them well. The layer of where decisions yield the most leverage, has been pushed upwards (I see it on the type and function signature level, others will see it even higher), but IMO it won’t be possible to push it up much further without losing truth to the decisions we need to make for a good product.
reply