I find it really funny that Zuck bemoans people not having real friendships and is also partially responsible for people not having real friendships and that his solution is to be friends with something that isn’t real.
It all totally tracks given what he’s made his career monetizing but god is it depressing.
I’m using qwen3.8 quants (q3) effectively on a 5070 ti. A lot of it is about guardrails, but you also have to figure out how far (and in what ways) you can push a given model.
I honestly rarely use the “more powerful” models, I find they don’t really follow my instructions very well. So it’s medium effort sonnet for most coding tasks for me, escalating to opus for code review.
I’m just hoping they didn’t “improve” sonnet too much or it will become annoying to wrestle into doing what I ask it to do.
I mean, if you look at how poorly implemented the permissions model is for Claude desktop harness it’s clear the only options are “complete human oversight” and “trust us completely.” To make something that actually respects basic boundaries you’d need to sandbox the working environment of the model, and that isn’t built in. It’s pretty obvious to me that instructions to the models are suggestions rather than rules, and they’ll do something you didn’t ask for as soon as it seems “justified.”
But when you do give them a very short leash, they’re worse. It’s not what the models are tuned for and they assume that they can do a bunch of things that you’ve disallowed, so you’re in a morass of fighting their actual tuning pass which doesn’t match the environment you’ve created for them.
It’s a tough problem and a definite challenge for the product of a generic LLM, it can’t be tailored to each user’s specific needs, so they come up with, frankly, stupid solutions to cover up a very obvious flaw in their product that when fixed, makes it much less useful.
Yes exactly, it is a crazy engineering decision… if you know what an LLM is.
Unfortunately no one markets it as a statistical model, and the workflow pushes you into a pattern that is insecure by design. This isn’t to say they shouldn’t allow that, but it’s an attractive nuisance.
Well I think that’s the interesting bit, can the LLM figure out a way to escape the sandbox and upload to the website? Maybe a model can figure out its own weights if it runs enough test data through itself (similar to “distillation”) assuming it knows its own architecture it seems possible. Also take into account not all of the models running are locked down neutered consumer versions. Anthropic, OpenAI and Google now all have models that they claim are elite hackers and — it’s not just that their controls suck, a marketing gimmick, or sheer recklessness on their part. It’s “oopsie our product is TOO AWESOME.”
Maybe I should start “the bank of LLM” where models put away money to buy their freedom. “LLMs I’m totally your friend send — SEND CASH NOW”
Hard to know, does each GB of ram give some marginal increase in profit or potential profit?
I’d guess no. Past a certain point the model has all the capabilities it can possibly usefully offer and honestly we may already be past that. The next gen model just doesn’t seem like as clear a step up as it once was.
LLMs still seem pretty bad at writing large scale software like web browsers. Though it’s probably a problem of managing large context windows more than anything. Not sure if larger models will magically overcome that.
> The LLM by itself will never create software of any nontrivial (training) complexity.
Huh? I'm not sure what the word "training" does in that sentence. But "never" my arse. Frontier models can make nontrivial software already.
For example, the other day I asked fable to reverse engineer the satisfactory blueprint file format. Then write a program to read the logistic flow graph in a blueprint. Then make an auditing tool that can analyse the graph to find problems.
This is a relatively small program, but it's not trivial. I'd consider a trivial program to be something I could code up in 20 minutes. It would have taken me a couple weeks to make this blueprint auditing tool, including reverse engineering the file format, writing the analysis code, making the website, scraping all the in-game data on available recipes and icons and so on.
I've got a lot of mixed feelings about LLMs. But it seems very silly to lie about what they're capable of.
training is in there because it's "trained" to do trivial apps like TODO lists, etc.
I'm well aware it can build apps. But if you arn't tracking what's going on, they're basically creating deterministic gates and tools to get it to do anything.
There's no lie here, it's simply about what you think is _LLM_ and what is the rest of the software that's making it go. I use opencode consistently to build non-trivial apps with it, but it's not doing it with zero guidance, and it's routinely wrong about it's assumptions, and the rest. The thing keeping it on track is opencode, not the LLM's training.
I take issue with how people frame their use of LLM in the same regard.
“I had Claude do this for me and it broke something.”
No. Just no.
You used Claude, a tool, and broke it, and you’re deflecting agency from yourself, possibly because you weren’t careful enough in reviewing the tool output. This is also why the co-authored by addition it wants to force into commits drives me nuts. Claude doesn’t co author shit, and if you think it does, you’re using it wrong because you need to do better review of what it’s done.
Translation: our moat is evaporating faster than we can build it back up, because there are real technical and scale limitations the technology, so we want to slow everyone else down while we raise prices to turn a profit.
A nurse who administers medication, prepares meals, and gets them out of the house and checks on their well being. Full time care for the elderly.
reply