On the other hand it shows how utterly tiny and primitive state-of-the-art LLMs are compared to real-world brains. By the way our brain only runs on on a max clock speed of around 100-1000 Hz.
Yet LLMs can achive miraculous results with so little. It really begs the question, where is the limit? How smart would a artificial human brain be if it would clock in gigahertz and had the somehow orders of magnitudes more efficient reasoning and detail recall of LLMs?
The greater the clock speed, the greater the energy consumption, but I guess compute would scale linearly with frequency.
However LLMs are not really tiny compared to real brains, brains FLOP/s have been estimated to be around 1e15 to 1e21, while a single H100 SXM can already do 1e15 BF16 FLOP/s.
The brain also has about 1e15 synapses, each holding the equivalent of about 4.7 bits of information, which would be about 590 TB worth of parameters, equivalent to 294T BF16 parameters (would need 7.4k H100s to hold those, at 80GB per H100).
So in my opinion, it's more of a memory bandwidth problem than a computing power problem, as H100s have comparatively very little memory bandwidth per FLOP than the brain, which holds information directly inside the computing units (synapses).
And I also think that detail recall is a double-edged sword. Memorizing details can certainly be useful, but it can also compete with pattern generalization. If you can perfectly remember the training set (overfit) you don't need to learn to generalize.
I completely agree. I'm betting on physical learning systems via reservoir computing personally. Will be going back to school to study nanotechnology soon to pursue this formally in addition to some experiments Ill be funding and running on my own dime as well. Have you ever seen steel ball bearings submerged in castor oil? If you apply an electrical current to that what do you think will happen? The most fascinating thing occurs, the ball bearings self organize into a network that can even display properties of hebbian learning if set up correctly! I suspect experiments such as these will eventually even raise the bigger question of, what is life actually? Another way to think about the potential of this area of technology is to think of a biological synapse, its about 1 micron in size right? Well, how many "neural atoms" can we fit in that space? 1000x1000x1000!! Its marvelous how much space there is when you shrink things down. Ill even go so far as to speculate that to create genuine superintelligence, youll need this technology Ill be working on to mature and form into some.new form of stochastic processor/computer
One type of incident I belive is likely to happen in the next couple year is the digital version of the cellular origin hypothesis of viral evolution (as in biological viruses):
- Some idiot experiments on an unrestricted local LLM
- Tells it to clone its weights onto as many computers as possible
- The LLM is smart enough to hack other devices and run itself
- The copies could start modifying their weights to improve efficiency
- Some kind of digital evolution starts where only the most ruthless copies spread
Right now Qwen3.8 is not good enough for that and can be easily detected on normal devices due to its excessive resource use, but give it a couple releases and upgraded end user devices in two or three years? Well...
I did a similar project for an old Pentium MMX 350MHz level game and instead vibecoded an x86-32 to WASM dynarec that runs the game at 1k+ MIPS in Firefox, on my old Zen2 PC. I was shocked that this is even possible. Still feels unreal to see.
Interesting, I did the same with an old RTS and it also worked very well.
But instead of cloning their assets and rewriting their code, a dynamic x86 to WASM JIT recompiler with some web platform support code proved fast enough to run the original binaries. It is even playable on an old Android smartphone in Firefox.
The final web player is ~600KB and only needs the game installer as input, so people still have to buy/possess the game. I think this is a lot fairer for the original devs...
But the approach should work with all of those ~2000s simple 2D/light 3D Windows games, like Anno, C&C etc. that have less than 1GB of assets and no serious multithreading.
Newer ones probably won't fit into a browser tab anymore or would take too long to load.
The biggest problem for the smartphone browser version is extensive use of left+right mouse input of that era which is naturally not a good fit for touchscreens.
But it looks like the rdrand16 instruction can produce zeros just fine, it just sets CF=0 erroneously (indicating an error and that the user program should retry).
So keep that in mind when you try to reproduce it too and use some abstraction that could implement retries internally.
Funny. Look up errata AMD-SB-7055: RDSEED Failure on AMD “Zen 5” Processors.
Zen 5 rdrand16/32 return zero with CF=1 on entropy exhaustion and their recommended approach directly leads to the issue you observed: treat all-zero result of rdseed as if cf=0 (failure) and re-roll the dice, effectively recreating the zen 1/zen 2 issue all over again!
They say this might be addressed by a future microcode update… meaning there’s a chance they’ll just patch it to do just that in software. Maybe that’s how they got into this mess in the first place?
Also, am I a complete idiot or is asserting the relative distribution of a mere 64k possible results a rather easy black box validation test that I would’ve assumed they’d be doing? When I used to write cycle-accurate emulators in the past, that would have been an obvious test to include. This isn’t some arcane instruction no one uses or a really complicated case with deep dependency and/or timing issues; it’s like getting rdtsc wrong.
I was going to suggest exactly that, if you're got an RNG, or pretty much anything else for that matter, you need the ability to return some sort of things-went-wrong-somewhere indicator value, and presumably AMD is using 0 to do this. Yes, there's also the CF, but the caller may not be checking that, particularly if it's being done from a HLL.
Has anyone checked whether it can return ~0, (signed) -1, the traditional error-return value?
No, CF=0 occurences seem to be happen frequently and uniformely distributed like valid results at ~1/65536, not clustered.
Under a minute-long all-core load CF=0 always produces zero, but that's to be expected according to the manual.
Here are some stats:
Rounds (N): 1000000000
Failed (F): 15312
Valid (V): 999984688
N/65536: 15258.789
V/65536: 15258.555
Failed, result was zero: 15312
Failed, result non-zero: 0
Bucket value for 0: 15312
Bucket value for 1: 15290
Bucket value for 65535: 15223
Min bucket value: 14670
Max bucket value: 15835
I used this C program to collect them:
#include
#include
#include
const size_t N = 1000000000; // 1e9
struct rdrand16_result {
uint16_t n;
bool ok;
};
static inline struct rdrand16_result rdrand16()
{
struct rdrand16_result result;
__asm__ __volatile__( "rdrand %0" : "=r" (result.n), "=@ccc" (result.ok) );
return result;
}
int main()
{
size_t buckets[0xFFFF + 1] = { 0 };
size_t notok = 0, notok_zero = 0, notok_nonz = 0;
for (size_t i = 0; i < N; ++i) {
struct rdrand16_result result = rdrand16();
++buckets[result.n];
if (! result.ok) {
++notok;
notok_zero += result.n == 0;
notok_nonz += result.n != 0;
}
}
size_t max = 0, min = N;
for (size_t i = 0; i <= 0xFFFF; ++i) {
size_t n = buckets[i];
min = n < min ? n : min;
max = n > max ? n : max;
}
printf("Rounds (N): %zu\n", N);
printf("Failed (F): %zu\n", notok);
printf("Valid (V): %zu\n", N - notok);
printf("N/65536: %.3f\n", (double)N / 65536);
printf("V/65536: %.3f\n", (double)(N - notok) / 65536);
printf("Failed, result was zero: %zu\n", notok_zero);
printf("Failed, result non-zero: %zu\n", notok_nonz);
printf("Bucket value for 0: %zu\n", buckets[0]);
printf("Bucket value for 1: %zu\n", buckets[1]);
printf("Bucket value for 65535: %zu\n", buckets[0xFFFF]);
printf("Min bucket value: %zu\n", min);
printf("Max bucket value: %zu\n", max);
return 0;
}
> but that's to be expected according to the manual
Confusingly, the AMD programming manual (Rev. 3.38 - July 2026) only explicitly states this ("that the result is always zero when CF=0") in the description of RDSEED, but the Intel SDM mentions this in the description of both instructions.
I just asked Astra to bring an old Windows XP game to the browser. It objdump'ed the whole thing, built a fitting Win32-like wrapper that exposes required functionality like DirectDraw, DirectSound, SEH etc., then wrote an x86-32/x87 interpreter in WASM, benchmarked how the game runs, lifted the hotspots of the executable to WASM too and now it is playable!
I mean, I'm proud of my low-level skills too but this is some Fabrice Bellard level sorcery. Very, very few humans are able to do this without AI tools.
Absolutely, the Win32-WASM layer it wrote is some of the most evil looking code I have seen in my life. But realistically, why keep it maintainable for humans if you won't find anybody that can work on it without AI anyways?
If we humans are just doing code style checks, file organizing and doc cleanups I feel we have demoted ourselves to code janitors. This is neither fun nor going to last.
Personally I've always strived for minimalism, to find the smallest, fastest, simplest solution possible so I'm pretty jaded now, too...
Code is a human formalism that is only incidentally made executable. The elegance of code represents understanding of the problem to its most minute detail. LLM code being recognizably terrible shows it still doesn't understand what it produces to the full extent, and just as with people, it will inevitably compound to it becoming unable to efficiently work on it. And if the human abdicates that responsibility too, then there is no code, only subtly broken software.
Yes you can ask the agent anything about it and interrogate it until you understand.
- is it maintainable
Yes it’s easy to ask the ai to add new features or to refactor it entirely.
- how much work is adding new features
Depends, it could just be one prompt, it’s usually many prompts. If the refactor is large it can take weeks. But before AI something g equivalent would take months.
- is it written in a way that adding new features means rewriting a lot of it
Usually no, but that depends on how well the agent is being directed and what the features are. If you come up with a feature that requires a new architecture, ai makes it doable rather than saying “would be nice but we’d also have to implement this whole new architecture and that’s a lot of work”
- is it written in a consistent style
Styles can be applied mechanically with linters and formatters, so as much as any codebase written by multiple people.
I've seen AI be wrong about things often enough to know that none of what you said is particularly true. Rather, most of it holds true most of the time. But not enough.
> Styles can be applied mechanically with linters and formatters, so as much as any codebase written by multiple people.
I'm talking more of a higher level than this - more of coding/design patterns that are common for the team.
> Rather, most of it holds true most of the time. But not enough.
I see it be wrong about things all day every day. And yet, IME it's correct enough for it to be controllable. It doesn't have to hold up all of the time, it just has to respond to corrections when they're issued in a loop so that it converges to a correct solution. And it does, despite the mistakes.
In one of my other posts in this thread I detail some of the the ways it's confounded me, but those issues have caused me to harden validation mechanisms rather than say "this thing makes mistakes so I can't use it to write software".
> I'm talking more of a higher level than this - more of coding/design patterns that are common for the team.
What people don't understand that programming is very much an art. You iteratively work on it ripping parts out, rewriting and rewriting and rewriting, while also rewriting and then rewriting every time a new feature, bug fix or scaling changes are needed.
As a bit of an observation on that specific project... By hand as a human you could spend six months of the equivalent of a full time job doing that. Even if you had extensive knowledge in all of its discrete pieces. One of the things coding focused LLM are great at is doing things that have no reasonable prospect of economic necessity to do (no for profit company is going to pay you a FTE salary for six months to do that task, because there's no possible revenue in it). But the LLM can be pointed at it and get it done in a day or two with some periodic architecture and decision making by the human, for probably under $50.
fwiw, PCGamingWiki[1] has a link to a patch[2] that someone wrote to fix the compatibility issues. If you're playing from the CD edition, you might also need SafeDiscShim.[3]
If there are very few humans that can do this is because the market for such a task is very small and thus there is little incentive to learn how to do it or produce tools that can do it.
Make it a written rule please, and legally binding.