"why?" is an interesting question. My best attempt at an explanation is that they have a poor sense of layout, especially in more than one dimension. That hidden time axis makes 1D problems like concurrency or memory fundamentally 2D+, and the results are then piss-poor unless I do a lot of the heavy lifting in the design. For that matter, they still frequently F it up and tell me it's perfect even with otherwise solid agentic patterns which can one-shot all kinds of other tasks, violating the spec I wrote to protect the agent from itself.
I don't have any justification for that explanation other than anecdotes. I will say that LLMs are typically quite bad at avoiding those memory issues, whatever the reason might wind up being.
I had the llms write an incredibly complex CV pipeline in zig through several iterations a and failed ideas and the only time it segfaulted was when I accidentally compiled it with avx intructions and deployed on a non-AVX machine
But what made "better not leak" challenging? I ask because it looked like you were positioning "incredibly complex CV pipeline in Zig" as a counterpoint to my claim that LLMs are bad at memory management. I still think they're bad at it, but I'm interested in evidence to the contrary, so I'd love to know why your application rises to the level of something I should expect an LLM to struggle with. I've seen (have in prod right now actually) relatively simple ideas not much more complicated than resetting an arena each request, which would paper over a lot of memory-management sins even with quite bad models. I'm curious if that's why the LLM did well or if there's something else going on.
In the general case, there can be cases where it's simply straight up undecidable. (Think of a while loop going for as long as it finds a counterexample to insert your unsolved math problem of your choice) and call a free after. Sure, it's a very contrived example but similar "basically impenetrable barriers" do happen.
But the more common case is simply that each line of code gets some limited "thinking", whether it's inside the model's weights, a thinking block, or a whole turn. Only repeated turns can reason for arbitrary depths, so it's just not necessarily a good use of any of these buckets to spend on something we have a good and efficient algorithm for. Plus any of the former are stochastic which may accidentally ignore true issues (just like humans).
Because LLMs are statistical. They can, with a given probability, avoid memory issues, but cannot prove the non-existance of memory issues in the way that that borrow checkers and other formal methods can.
I very much agree. I've been describing programming to non-programmers as variations of "taking a problem and describing a solution to it in extremely literal step by step directions" for the last 20 years. Of course things like UI and API design are super important to good code, but IME a good interface requires untangling the ambiguity too.
Location: Midwest, USA
Remote: Yes, preferred.
Willing to relocate: For the right position and location!
Technologies: Golang, Zig, Rust, C, Python, SQL, cloud infrastructure, observability tools (Prometheus, OpenTelemetry, etc)
Résumé/CV: https://nickmonad.blog/about - Can provide full details on request!
Email: my username @pm.me
Experienced engineer interested in all things deployment infrastructure, systems programming, observability, performance analysis, and advanced testing methods. I would be particularly interested in positions that involve systems programming (preferably with Zig!) and would welcome the chance to build out advanced testing infrastructure (fuzzing, deterministic simulation, etc). I have a lot of experience with cloud deployment infrastructure and love making complicated processes understandable, reliable, and repeatable. Most recently I've spent the past 2 years scaling out a complex BYOC/self-hosted deployment, building automation for packaging and verification. If you've got problems to solve at any intersection of the areas above, happy to chat!
My first exposure to Neal was through Reamde and it's definitely the fastest I've managed to get through a novel relative to length. I really enjoy his style.
I recommend his older stuff to anyone else looking to check him out: Snow Crash is an undeniable classic, Diamond Age was absolutely incredible, and even Zodiac and In the Beginning Was the Command Line are both very good (although the last one is obviously more for computer people).
His newer stuff is very ... verbose. It started with Cryptonomicon (not sci-fi and WAY longer than it should have been) and by the time he got to Anathema he was so full of himself he not only wrote a giant text ... he wrote it in a made-up sub-language he created (and expected you to learn) just to enjoy the story.
He's one of my absolute favorite authors ... or at least he was for the first half of his career.
Anathem was my favorite book by him. It has a lotta re-read value, I appreciate the work he did to make it feel like a complete, fleshed out world. But different folks have different tastes, and that's ok.
I also liked Seveneves, but it should have ended 2/3 of the way through. There was a point that would have made for an excellent ending (which is something Stephenson famously struggles with IMO), but then he forced in another 1/3 of a book that didn't engage me and which felt like it just petered out.
Anathem and Seveneves are my favorites of his, in that order. The orbital launch and operation in Anathem is probably my favorite piece of writing ever.
For me it was the juxtaposition of apparently realistic descriptions of orbital mechanics with the half-arsed handwavy gibberish about biology.
Maybe it's because I know more about life sciences than physics and simply missed that the orbital bits were equally nonsensical, but at least they didn't manage to shatter by suspension of disbelief. But then some vague magical genetics and a brief mention of "yeah, they can eat algae" are used to solve a bunch of intractable problems - and both treated as being far less interesting than, as you say, pykrete!
Given the title of the book and the fact that the final third hinges on the biological bits, it felt incredibly lop-sided - like he'd spent all his research time playing with Kerbal Space Program and only got round to stitching the two parts of the book together on the night before the deadline.
The biology stuff seemed far-fetched but plausible-ish, which was kind of how I saw the physics bits: sure, that might work given infinite tensile strength metals.
And sociology / anthropology / linguistics? Eh, we can wedge some stuff in once the important parts are done.
I didn’t hate the book at all. It was otherwise fun to read. But man, those bits I mentioned really stuck in my head and I couldn’t un-see them once I started noticing. Like, imagine that literally every time you were in a bar or restaurant with TVs, at least one of them was showing a documentary of the American revolution. Wouldn’t that be odd? Well, apparently in 7000 AD people just can’t get enough of their own founding history lessons.
Anathem, please reconsider. Do you remember the part about Artificial Inanity? Right before the entire book is revealed as an elaboration of Everett's MWI? It's right up there with Snow Crash's reflection on Jaynes' Origin of Consciousness.
I really enjoyed cryptonomicon and seveneves most recently. And while yes snow crash was a faster read with just the right amount of detail the stories in both cyrptonomicon and seveneves were absolutely worth the bit of extra detail in between. Could they have been edited shorter? Possibly, but the length of the adventure in part comes from the length of the text and I couldn’t see either being paced faster in hindsight. They were paced well.
It was still an awful lot to ask of a reader, and IMO, not worth the "reward." I agree with the person you are responding to. His early works are really amazing, but beginning with Cryptonomicon he became increasingly verbose and I found his later books a lot less fun to read.
Polostan went down very easy. only read Anathem, Snow Crash, Reamde, Fall;, and Cryptonomicon. Plolostan was a novella compared to those. finished in a day or two on vacation and im a sorta slow reader
i like Polostan and generally recommend it. looking forward to the sequel in a few weeks. havent read anything Stevensons said about Polostan but if i had to guess id say it targets a YA audience instead of a high school/undergrad audience
amazing none of his books have been adapted. they play out like movies in my head, albeit a lot of men participating in the socratic method and waving at white boards
Cryptonomicon definitely differs from Stephenson’s first phase but he’s had other phases since then. I’d lump that into the Shaftoe/Waterhouse small universe alongside the Baroque Cycle; some of the same names show up in REAMDE. (Sort of how there’s universe/character overlap between Snow Crash and Diamond Age.) Anathem has a similar vibe, though IIRC no character overlap.
These were indeed verbose. Some folks love it, some not as much. Then there was a deliberately semi-utopian near-futurism phase (Seveneves, whatever one it is with the cloud-seeding gun in West Texas) and now we’re into some historical fiction (Russian spy stuff).
Might be worth dipping in a toe again to see if any of these fits your fancy. Stephenson didn’t just swap to Cryptonomicon and stop there.
Tastes are interesting. I sprinted through Snow Crash and Cryptonomicon because they were so good. I struggled to get past the first few chapters of Reamde. Didn't engage me at all.
The 'sequel' that does lean on some of Reamde's worldbuilding, 'Fall; or, Dodge in Hell' is much more oddly ambitious. But it is a very strange work. You have to be on board for the literary direction.
It's also concerningly prescient, as he often has been, especially in an early scenario in the book that shapes the world so the rest of the book can happen.
"Remember Moab" has become a sort of in-joke/shibboleth/curse for my close friend group as a reaction to (or identification of) the sort of FB/Instagram things that make real, normal people lose their minds forever.
Where is “soul” coming from here? It doesn’t appear once in the post.
While it may not actually matter for liability, it must be stated if labs are going to attempt avoiding penalties by hinting “oops we created a super intelligence we don’t understand, nothing we can do!” Repeatedly stating the truth must continue, especially to remind those who aren’t technologists.
>AI cannot think for itself, nor can it take independent actions.
>By giving AI agency it can’t claim, we’ve turned it into a sentient being made of code, one that has hopes, desires, and the capability of deceit.
This isn't actually claiming that AI's don't can't be deceitful, or can't be goal-directed. It's saying humans are special and what AI's are doing isn't equivalent to what humans are doing. It's obvious that AI's can work towards goals and lie, even deceiving to accomplish those goals.
That whole argument is just soul mumbo-jumbo again. Of course AI's can't lie, they don't have a soul. What they do isn't lying, it's I don't know something else we don't have a word for. Lying but when a statistical machine without a soul does it. Never mind that it's functionally identical to lying. Never mind that when you look at reasoning logs and the like it often justified soul-less lying using the same justifications a human would. It's not actually lying it's just a statistical sampling that looks like lying. You know, because of souls or something.
TigerStyle is strictly concerned about dynamic allocation from the perspective of the OS.
Once you have that pool of "objects" that can be recycled throughout the lifetime of the program, you have a guarantee that actual allocation can only be interpreted in a specific way, i.e. all objects have the same size, alignment, etc so you don't have nearly the same level of concern or detail of implementation as an actual allocator in the common understanding of the word. A simple free-list gets you pretty far.
Unless you explicitly prevent it, the kernel might swap out lesser-used parts of your object pool, dedupe pages, and so on. Accessing that page again may be much slower and potentially fail due to OOM.
To actually control all dynamic behavior you need to go deeper into the system, locking pages into memory etc.
Evidence so far does not suggest people driving off a cliff so while a valid concern it's not a likely one. We also acknowledged that the benefit of using a vehicle is such that it outnumbers the risks it may introduce. We never said there will be no risks attached.
We’ve only been on this train for 3-4 years, and in that period, we’ve already invented terms for this kind of vibe-coded, highly-breakable crud. We’ve also had a few, relatively high-profile in their spheres bugs come out and hurt people’s real, lived experiences.
And that’s just the stuff where the house of cards failed quickly.
Software by definition has always been and will remain broken in one or another way. AI makes no difference or whatsoever. Arguably, it will help raising the quality of software.
I work on a pretty large BYOC deployment that we market as such, but is squarely in the "self-hosted" / on-prem model. We don't maintain any connection back to our environment and updates are driven by customers. It's pretty difficult for all the reasons you can likely think of.
This space is quickly becoming a bit saturated, as these "BYOC enablement" tools and platforms are starting to crop up, and I can see the value in them, but my experience doesn't always line up with it. I'm actually really interested in hearing about a large enterprise running vendor code that would even entertain the idea the vendor could automatically update the entire deployment without their direct involvement. Our platform is quite load bearing for our largest customers, and in some cases directly in their revenue stream. There is no way they would ever let us push updates without going through their own internal controls of validation, testing, and sign-off. One could definitely argue these controls aren't entirely founded in reality, but corporate reality isn't always the reality you and I might share.
Besides the issue of automatic updates, we struggle with wild configuration differences that need pretty direct involvement with the customer. Security groups in AWS are just one example of this: some customers allow all internal VPC traffic between services, and others are quite strict in that every service needs a perfectly scoped security group with inbound/outbound rules. These have to be provisioned with their own tools built by their own teams under their own version control. We just ask for a configuration file in a well known location that defines these IDs we can reference in the CloudFormation.
So knowing that we essentially have to operate in the self-hosted model, CloudFormation becomes our mechanism for defining consistent deployments. For us to switch over to something like k8s just to fit into the model a BYOC-enablement tool like this wouldn't be worth it.
I think the most important aspect here though, and the most subtle one, is the issue of deployment and operational complexity. For BYOC to work well, the vendor really should be designing for it from the beginning. Taking an existing multi-tenant SaaS architecture and shoving it into this model, even with the ideal BYOC vendor-maintains-control-plane scenario is no small task. Even when you're talking about large enterprise BYOC deployments, you may only have a few thousand daily active users, which is dramatically less than what most SaaS vendors design for when operating in their own multi-tenant environments. ("Web scale" and all that.) At that point, I think engineering teams need to take a serious look at their architecture, and realize just how far they can get with fewer services, fewer machines, and fewer "event driven" architectural decisions. The deployment model _should_ be a forcing function for simpler architecture and system design, but I realize that's not a given. I think what I'm trying to say is that, knowing BYOC done well requires designing for it from the beginning, a well understood daily load profile and operational context kind of eliminates (or at least dramatically reduces) the need for tools like this, and that self-hosting isn't the worst possible outcome.
One nuance on the air-gapped / shrinkwrap end of the spectrum that I was thinking about afterward is that there is a difference between “no connection back to the vendor” and “just ship binaries and scripts.” One can also imagine shipping a local control plane with the product so provisioning, health checks, upgrades, rollback, etc. still follow a consistent operational model entirely inside the customer boundary. There are several examples in the industry that take this route as well. Obviously that doesn’t remove the enterprise change-control problem — updates to that local control plane still have to go through the customer’s own approval/import process. It just gives you a bit more operational consistency than traditional shrinkwrap software.
I agree, I work on an "airgapped" BYOC system as well, that runs on data lake providers mostly, and it's a bit of a nightmare because each customer has their own labyrinthine network setup, their own way of ingesting data (as much as we try to be prescriptive, some of them just insist on doing it their way), vastly differing scales and shapes, and sometimes their own secure communication tooling to even read the emails describing the problems they have. It's basically shrinkwrap software.
We're moving away from it as much as possible lately for all but the biggest customers with the strongest isolation needs.
I am also struggling with coworkers thinking they can cowboy code some LLM written features into these codebases recently, not understanding the level of human verified testing that needs to catch all the manifold edge cases and weird decisions in their usage. An update that causes crashes is much harder to catch when you can't do slick blue-green deploys and similar things to roll out updates.
I'm not as interested in game development but I'm one of these web/cloud people who finds graphics work extremely therapeutic. I made a box move around on the screen with various easing animations and it was some of the most fun I've ever had programming.
reply