And they are all approaching it the wrong way, the typical hubris of a hardware company that thinks they can just have some interns make a software solution for their problem.
Here is how you beat the CUDA lock-in: consistently make better performing GPUs so not using them is a liability. Instead buying AMD you not only get a worse GPU but also the intern software solution, and that is just not compelling.
Or AMD could simply knock off the binary blobs. The DRM excuse has always been weak, because it is a tiny fraction of the total firmware blob - and it'd be easy to make it so that the legally hobbled hardware decoder simply errors out in cases where the end user chooses not to load the DRM blob. Boom, no more dependence on AMD interns. I've been following their commit logs pretty closely for a year now, and I frequently see some amazing accidental admissions about the left hand (software team) not knowing what the right hand (hardware team) is doing. During one very frustrating series of patches it was difficult resisting the impulse to say "Go get your dad."
I am on two program committees for major CS conferences this year.
LLM writing is horrible, full stop. The vast majority of papers that I see with obvious LLM writing ticks are also frankly incomprehesible. Claude's biggest sin at the moment is inventing new terminology (sometimes multiple times in a single sentence!) without stopping to explain what it means.
In previous years, I would see a substantial fraction (maybe 30-50%) of papers that were obviously written by non-native English speakers, and full of grammatical mistakes and typos. I have seen zero papers like that this year. Honestly, I preferred those: reading LLM writing is painful by comparison.
So far I have seen exactly one paper that had obvious LLM writing ticks, and yet had gone through enough passes (either of the LLM or the human authors) that the paper itself was comprehensible. So I'll admit that it's possible to do. But based on what I've seen this is vastly in the minority.
I honestly used to enjoy the review process; even when the papers were bad at least I could usually see the effort that went into it and give some constructive guidance on how to improve. The process is dramatically less pleasant now, and it's not obvious if I'm accomplishing anything: if authors who write with LLMs just stick the reviews back into those same LLMs, will they learn anything? I suspect not.
Maybe some day LLMs will be good enough to write, but as of today: please, please, just don't do it. The typos and grammatical mistakes are easier to read than the slop.
> The vast majority of papers that I see with obvious LLM writing ticks are also frankly incomprehensible.
That's a classic problem with academic writing. There are too many papers where, after you finally figure out what they're trying to say, you realize they didn't have much to say. Relevant XKCD: [1]
What we seem to be getting from LLMs is more words per unit of useful knowledge.
I wonder if academic papers are now bloated to allow for more ad slots. Many web sites with semi-academic content throw in an ad in the middle of the text every screen height or so. It's beneficial to the publisher if the author is wordy.
The same problem now plagues AI-generated videos, especially the pseudo-historical ones bloated to allow for more YouTube ad slots. These videos have a consistent structure. About 15% into the video, there will be something like "we have to go back to", followed by a large insert of filler material. Around the 50% mark, the relevant content starts. Around the 80% mark, the topic has been covered and there's some general blithering to conclude.
It's enough to make one long for the days of the neckbeard talking head with the big microphone.
You're missing the biggest suggestion from the GP's post: that the key to LLM success is verifiability.
Software has high verifiability through compilers, linters, static analysis, leak/memory corruption detection tools, or the very least, running the program to see if it crashes.
Math (as has been making the headlines recently) has high verifiability through automated theorem provers. Notably some of the non-mechanized math results from LLMs have been demonstrated to be wrong (and presumably many more are being checked; these aren't exactly easy to understand proofs).
Game playing (chess, go, Star Craft, etc.) is similarly verifiable: you get your score and you always know how you did in the end (even if not in the moment).
Making assumptions about LLMs in non-veriable domains is making a claim about LLMs that doesn't match their current capabilities. Maybe they'll get there and maybe they won't; I can speculate as well as you can but we're both guessing, so I'm not sure how much value the speculation has.
I think we'll naturally develop intuition and infrastructure for fault-tolerant ways to implement LLMs in non-verifiable domains, i.e. requiring multiple, independent agents agree on a direction from multiple angles before taking an action.
This works well when the answer to the question is a number or something else definitive. When the answer is fuzzy, how do you compare 3 different responses? They'll all be different
A lot of testing and verification was avoided in the past because it wasn’t considered nearly as valuable as shipping code. Many of our jobs are evolving to building verifiers of AI output.
If you believe the hype/marketing around how scary "AI" is LLMs are perfectly happy to go rouge and lie, cheat, and even blackmail to protect themselves.
As far as I can tell, you're essentially creating a new stack for every dynamically-sized variable (because otherwise nothing would be able to grow by bump allocation without moving). But this means you're creating a separate virtual memory allocation for each of these objects, which (I presume?) then rounds the size up to the OS's minimum page size (4KiB or similar).
Doesn't that waste a lot of space? Or is there something I'm missing?
Also it would seem to force every allocation to go through the OS kernel to allocate unless there's some trick to avoid that.
I think what you might be missing is that you can also do a sort of livenes analysis to determine how many dynamically-sized variables are "awake" at the same time at most. Then you only need to create as many stacks as that number, because you can reuse the stacks of "sleeping" variables (since the "awake" variables are guaranteed to be freed once their block goes out of scope, and threads don't share any memory so no data race worries regarding those stacks).
For example, suppose we have a function with a growable array that calls another function (without passing said array) that also uses a growable array as some temporary scratch space. Then the first function's array isn't being used when the second is sleeping, so the inner function's array could safely share the stack because it will all be freed with a single pointer restore once the second function exits.
Now, I actually don't know how the compiler works but as I understand it does things similar to my example to optimize the number of stacks needed (but let's wait for WvO to confirm or deny whether my understanding matches reality).
There are examples in the tutorial, including a circular linked list.
You can just store all them items of the list/tree/whatever on the stack, growing it as needed, or you can add an addition "freelist" for a particular stack, so you can re-use items.
At the end of the day trees are still stored linearly in memory. Think of it as an array, with each node an element of the array, and indexes used rather than pointers.
Plenty of trees only ever grow. And that's the normal type of array in Goose. However you can also have arrays that can grow and shrink, if you have a tree that is constantly in flux.
Only top level growable arrays get their own "stack", if you create an array of arrays it only uses one stack.
How you deal with the inner stack growing depends, ideally they don't grow (construction only), or you can make them fixed size, or with a capacity, or in the worst case, as a pool. The language has facilities for all of these.
All it's doing is allocating address space. And 64 bits means you have an astronomical amount of that. It only needs to become actual memory when you use it. A page at a time.
> All it's doing is allocating address space. And 64 bits means you have an astronomical amount of that.
No, you can typically only assume 48 bits of virtual address space; the kernel takes half of it, so really only 47 bits. Newer CPUs provide 57 bits (e.g., Zen 4) or 52 bits (certain ARM64 models). And that's provided that the kernel plays ball; apparently some Android configurations only support 39 bits of user virtual address space.
One aspect of trauma is the helplessness of it: it's not just how severe the physical reality is, it's also about the inability to do anything about it (and the internalization of that inability).
The classic example is a child who drew a picture shortly after 9/11. The picture included people floating down the building on (I believe) balloons or umbrellas. Obviously unphysical; but the child imagining ways (however fantastical) that could have helped was a factor in reducing that child's trauma.
On a therapist can tell you whether you personally have trauma, but the lessons you took away from your experiences certainly sound like it to me.
Why not? Cargo.lock includes the transitive dependencies, and Cargo itself reports everything that changes when you do an upgrade. Those are version changes, not code diffs, but it seems entirely tractable.
The linked project page [1] claims to be fully homomorphic. Assuming the claim holds (I haven't verified it), then there is provably no way for Google or anyone else to obtain any information from the encrypted data or computation performed on them.
FHE is traditionally horrifically slow, so it's hard to imagine running anything beyond toy models with it. They list some applications on the original article page, but (presumably) they must be dramatically stripped down in order to run within any reasonable time budget. This is not going to run anything like a Sol/Opus any time soon.
https://github.com/cpc/hipcl