What is striking in the appearance of sea spiders is how small is their body in comparison with their long legs.
The body usually looks like a narrow twig that connects the big and long legs. This is why one of their alternative scientific names was "Pantopoda", which means "all-legs" in Ancient Greek.
Because there is not enough space in their body, many of their internal organs extend into the legs.
The ideas survive in AVX-512 a.k.a. AVX10, not in AVX, which was a project parallel to Larrabee and resulting in an inferior ISA, which was adopted in the mainline Intel CPUs due to internal politics, not due to technical superiority.
When brought to the Intel server CPUs, the Larrabee New Instructions were rebranded as "AVX-512", despite having no relationship with the AVX ISA extension.
AVX was the creation of the Intel A-team, while the Larrabee New Instructions were designed by a C-level or D-level Intel team, but the latter have benefited from the contribution of a few consultants hired from outside Intel, who had experience in programming graphic applications.
AVX, which included only minimal and obvious improvements over SSE, i.e. double width and 3-address instructions, has slowed down considerably the improvement of the computational performance of CPUs in comparison with an alternate time line where Intel Sandy Bridge would have implemented a variant of the Larrabee New Instructions instead of AVX. This could have been done in a manner that would not have required any significant cost increase over the Sandy Bridge with AVX, because in AVX-512 it is not the width that is important but the architecture of the vector instruction set (e.g. with masked operations).
Indeed, there are fossil sea spiders from the Ordovician, which look much the same as the present sea spiders.
While there have been very abundant arthropods already several tens of millions of years before the first preserved sea spiders, almost all of them, for example the trilobites, were very unlike any present arthropods.
Sea spiders seem to be the first among arthropods who have reached a local optimum in body shape and structure, so they have not changed significantly after that, while also surviving various extinction events that have wiped out other groups.
Despite the name, terrestrial spiders are only very distant relatives of the sea spiders, and they have appeared only about one hundred and fifty million years later than the sea spiders. Terrestrial spiders are much more closely related to scorpions, than to sea spiders.
That may very well be true, but none of those has published anything of comparable importance and quality as any of the many publications of von Neumann.
Whether some people have really superior intellects is completely irrelevant for the rest of the humans unless they distribute to the others some useful products of their intellect.
Von Neumann is one of the hundreds of people from the past who have left a priceless intellectual heritage to me and to the remainder of the mankind, so I am indebted and grateful to them.
I have no reason to feel any debt or gratitude to any of the employees of OpenAI or Anthropic.
Jane Street has published a few useful articles, so I have some gratitude towards them, but none of those was even remotely similar in importance to any page written by von Neumann.
It's an interesting discussion topic. If we have people today who are just as smart and industrious as Neumann, why aren't they making a worldwide impact? Or, if they are, why is the attribution going to the companies they work for instead of them?
It raises questions like, is it even possible to work for a company while getting credit for world-changing results?
And it implies that at any point in time, there might be thousands of Neumanns in the world, and that initial conditions matter a lot more than ability.
Individuals as well as companies, that have worldwide impact, do get noticed, whether via the results themselves or peer-nominated recognition such as as Nobel Prizes, or the Fields Medal.
However, intellectual geniuses are better measured as people who revolutionized or significantly advanced the fields they worked in, and may have received recognition for doing so, not necessarily those who had highly visible worldwide impact, which is often more a matter of luck and different character attributes.
The people we recognize and celebrate as geniuses also tend to be people who had many accomplishments (because they were geniuses, not just one-time lucky), sometime seeing success on whatever they focused their attention on.
We wouldn't consider Tim Berners-Lee as a genius even though he certainly had world changing impact. Ditto for Demis Hassabis - certainly very smart, but hardly a Feymann. Someone like William Shockley also changed the world, and got plenty of recognition despite working for a commercial lab, but perhaps better regarded just as a bright engineer, right time right place.
AI is an interesting case, certainly changing the world, but the people who invented the tech, primarily Jacob Uszkoreit and Noam Shazeer, are really more akin to Berners-Lee and Shockley.
People like Von Neumann and Feymann, who shocked other geniuses with their intellect, really are a very rare breed. For people like this I'm not sure that "initial conditions" make much difference - they play by their own rules, and the world comes to them.
If there are thousands of JvN in the world, many of them surely have better initial conditions than the real JvN had. All those supposed Anthropic-geniuses ought to have a nearly perfect environment to reach their potential, but it seems it is not happening. I believe von Neumann was simply a _very_ rare kind of person.
The conclusion is that von Neumann deserves the credit for the "von Neumann architecture", despite any contrary claims of Eckert or Mauchly.
Herman Goldstine, who distributed the document and John von Neumann, who wrote it, are the reason for the explosive growth of the computer industry during the following decades.
The paper written by von Neumann was an exemplary model of clarity and good logical thinking. Nothing ever written by Eckert or Mauchly was at a comparable level.
Everyone who read that paper understood immediately how to design and build an electronic automatic computer, and a great number of teams in many countries all over the world did precisely this, so a decade later there already existed experimental electronic computers in many countries and also commercial electronic computers in the UK and in USA.
Some of the details from the paper must have been learned by von Neumann from discussions with the ENIAC team, but others were obviously von Neumann's own ideas, e.g. the use for the main memory of an iconoscope tube with fast random access, i.e. a DRAM like today, instead of the slow serial delay lines chosen by Eckert and Mauchly for their following computers.
Whatever von Neumann has learned from the ENIAC team, he obviously understood better than the people who taught him, allowing him to formulate general principles for the organization of an automatic computer.
The publication of the von Neumann paper allowed the concurrent development of many computers in many places, and each of those projects found various improvements that were essential in making the electronic computers successful commercial products.
If Eckert and Mauchly had succeeded to block any competitors, then the evolution of computers might have been delayed by up to 2 decades, until any patents would have expired, because Eckert and Mauchly have never done later any significant innovations and they would have never succeeded to develop better computers at the pace that happened due to the von Neumann paper.
Moreover, Eckert and Mauchly had a history themselves of failing to mention their sources of inspiration, so they were not the people entitled to complain about someone "stealing" ideas from them.
Before ENIAC, the first electronic computer in USA was the Atanasoff-Berry computer. That computer was a special-purpose computer, designed for solving systems of linear algebraic equations. However, ENIAC was a special-purpose computer too, unlike the relay-based computers that were older than it, like the Harvard Mark I computer, as even its name implies (Numerical Integrator and Computer).
ENIAC was conceived as a replacement for the older mechanical "differential analyzers", which were used to solve ordinary differential equations, e.g. for computing artillery tables, so its architecture mimicked the architecture of the mechanical differential analyzers, and it was reconfigured for new problems in a similar manner with those, by rewiring.
John Vincent Atanasoff had written in 1940 a very high-quality document about the design of his computer: “Computing Machine for the Solution of Large Systems of Linear Algebraic Equations”. It is unknown if anyone of the ENIAC team had read it, but it is known that one of them had visited the designers of the Atanasoff-Berry computer, inquiring about the electronic circuits used by them to implement arithmetic operations and data storage. Later, during the design of ENIAC and after that, they never mentioned any connection with the earlier electronic computer.
As I point out occasionally, the bottleneck in early computing was memory. There were no good memory devices for quite a while. IBM had electronic arithmetic in test before WWII, and that eventually emerged as the IBM 603 Electronic Multiplier (1946). That was the first mass-produced electronic calculation device. From IBM's perspective, they needed something reliable enough to install at customer sites that would run without repair techs on site. IBM's electromechanical machines already worked that well, and businesses could get work done. So there was a lot of work that had to be done on tubes, sockets, wiring, soldering, and packaging before they could ship.
Eckert and Mauchley came from a background of trying to get compute done with modified IBM hardware. They were painfully familiar with the lack-of-memory problem. Everybody involved with that era of machinery was. (I've used IBM plugboard wired tabulators, like the one at the Computer Museum in Mountain View. My high school had one.) The ENIAC had a workaround for lack of memory, huge rolling cabinets of manually set rotary switches.
If something better had been available, it would have been used. You have to appreciate how much early architecture was restricted by available components.
The first real hardware breakthrough was acoustic delay-line memory. First mercury tanks, then long coils of steel wire. Once that was available, it was clear that was the place to put the program. But it took a while to make that work reliably. Then there was Williams tube electrostatic storage, and high speed drums, which got things going. Sort of. Memory cost was an issue well into the 1980s. In the early 1970s, a megabyte cost a million dollars. Cheap memory didn't appear until the 1990s.
The big lack in Von Neumann's architecture was index registers. He was into storing into the address part of instructions. The Manchester Mark I had the first index register (the "B box"), at which point programs could be read-only and still be able to index arrays. That completed the basic CPU architecture.
The Williams tube electrostatic storage was just the practical implementation of the proposal made by von Neumann in his report, where it was referred as an "iconoscope tube memory".
An iconoscope was a cathode-ray tube used in video cameras, where it allowed the reading of every pixel of an image, and where the value of the pixel consisted of the electric charge stored at that point (the amount of charge depended on the light exposure of a photoconductive plate). Williams did exactly what von Neumann had suggested, combining the reading of the memory like in an iconoscope tube with the writing of the memory like in a CRT display tube.
The first commercial IBM fully-electronic computers, IBM 701 for scientific computing and IBM 702 for business data processing, used Williams tubes, according to the von Neumann proposal. The Williams tube memory of the IBM computers was an advantage over the Univac computers, which were still using the slow delay line memories invented by Eckert and Mauchly (which had been derived
from the delay lines used as analog memories in radars, during the war).
The next generation of IBM computers switched to the magnetic core memory invented at MIT, except for the cheapest computers, like IBM 650, which used magnetic drums for the main memory. IBM also invented soon the magnetic disks, as a memory tier intermediate between magnetic drums and magnetic tapes. The use of magnetic tapes had started in the first commercial electronic computer from USA, UNIVAC I.
By the end of the fifties, some computers had 5 tiers of memory, from the fastest to the one with the greatest capacity: registers made with flip-flops, magnetic cores, magnetic drums, magnetic disks and magnetic tapes.
>The big lack in Von Neumann's architecture was index registers. He was into storing into the address part of instructions. The Manchester Mark I had the first index register (the "B box"), at which point programs could be read-only and still be able to index arrays. That completed the basic CPU architecture.
strikes me as gilding the lily (I earlier even imagined possibly even a claude-esque stock 'one caveat' flourish, but I know you wouldn't do that given your background). Addressable memory, where any given addressed memory register can contain not just conventional data to be operated on but can also contain an address of some other memory register to jump to. is what strikes me as the highly-enriched uranium of computer architecture. indexing an array is a feature bump by comparison.
Being able to store into the program was once considered much more important than it is now. Today we almost always load code, lock it read-only, and run it.
Even JIT compilers don't patch much. They mostly generate code into empty memory.
(There are exceptions. I think PyPy patches existing when a Python program replaces an existing function while running. You can do this in Python, but probably shouldn't.)
Von Neumann considered self-modifying code important for somewhat philosophical reasons related to his self-replicating automata ideas.
Previous pre-computer programmable machines, back to the Jacquard loom, stored programs in a completely different medium than the data. Plugboards, chains of Jacquard cards, cams, etc.
Von Neumann insisted that programs were just data and belonged in the same memory as the data. This was a conceptual advance. The hardware had to get there to make it work.
The next big innovation was the stack, in the modern sense. This came later than one might expect.
Note that there are at least two types of stack computers. There are ones with a return point and context stack, which is the C view of the world. And there are ones where working storage is on the stack, like Forth machines and early Burroughs machines. The first kind seems to have been invented for ALGOL in 1957, because ALGOL allowed recursion. Stacks made a lot of people nervous back then, because how much memory to allocate for the stack was unclear, and there wasn't much memory. FORTRAN disallowed recursion and thus did not need stacks and stack frames.
So storing into the program hung on in some architectures. UNIVAC 1107 and later mainframes had a Store Location Jump (SLJ) instruction. This stored the return address into the beginning of a subroutine. Not into a stack frame; there was no stack by default. Into the first word of the subroutine's code. For non-recursive non-concurrent code, it worked.[1] But it was used less over time, with different constructs used instead.
Program in the same memory as data, yes. Program can create more program, yes. Program modifies itself while running, not so good. Index registers eliminate most needs for the third case.
The early computers that did not have index registers realized the equivalent of indexed addressing by modifying the address fields of the instructions before they were executed, e.g. during a loop.
In the early computers with non-pipelined execution and without cache memories, and also without security concerns, modifying the executable program during execution did not have disadvantages.
Nonetheless, pipelined execution became widely used and also cache memories appeared soon, and such performance-enhancing techniques could not work well with self-modifying programs, so index registers became absolutely necessary in fast CPUs.
This is a genuinely rare experience. Was it like old hardware on a shelf that no one knew what to do with and you tried it out? Or this was how lessons/clubs/assignments got things done, using an antique IBM tabulator?
When I started public high school near Boston in the early 80s we didn't have Apples or TRS 80s or PCs, it was terminals that used printed output ... can't remember the details, maybe DECwriter to PDP 11.
It was obsolete even when I was in high school, but the school had the basic set of unit record equipment - tabulator, sorter, collator, keypunch. Computers hadn't filtered down to high schools yet.
Von Neumann's architecture (i.e. report on the EDVAC) didn't focus on random-access memory, and in fact his whole discussion on memory is rather rambling. The EDVAC itself, which the report is nominally describing, used serial-access mercury delay line memory. What really matters from an architectural POV is memory address-ability, not speed of access (random vs serial).
The whole document is, as the title promises, really more of an engineering one - a report on the EDVAC - rather than some clean abstract treatise on computer architecture.
When the report was written there wasn't any random access memory. The only memory suited to computers in 1946 was the ultrasonic delay line as implemented by Eckert. What makes it not-random access is that all the bits in the store are being constantly shifted and counted in a loop (about 1Mhz?), so you have to wait for the word you want to read/write to fly past and the controller grabs it then.
The document written by Atanasoff in 1940, which I have mentioned in another comment, analyzed all the known devices suitable to be used as memories and it proposed the first kind of DRAM, made with discrete capacitors, as the best combination of low price and high speed.
However, during the war this document was known by few.
Delay lines were already used as analog memories in radars and from there Eckert and Mauchly took them and modified them to be used as digital memories.
Von Neumann learned about the delay line memories from the ENIAC team, but he did not like the idea, because serial access seemed too slow, so he proposed to use instead of delay lines the second kind of DRAM, with a cathode-ray tube instead of discrete capacitors. Von Neumann took the idea of using a CRT as a memory from the tubes used in video cameras for television, which used an analog memory to store the image after exposure to light.
So when the report was written, probably von Neumann was not aware of the earlier RAM used by Atanasoff in his computer, but he also proposed a RAM as being better suited for computers than delay lines. Von Neumann was proven right a few years later, when at Manchester Williams made such DRAMs with cathode-ray tubes, which were later used in many commercial computers, e.g. in the first generation of IBM computers, until the development of magnetic core memories made obsolete both the delay lines and the Williams tubes (until the magnetic cores were made themselves obsolete 20 years later, during the seventies, by the third kind of DRAM, with MOS integrated circuits).
The most important users of delay line memories were the early UNIVAC computers, because they had bought the failed company of Eckert and Mauchly, for their IP.
It's interesting to read Von Neumann's perspective on CRT ("iconoscope") memory - saying how important it was not to lose the serial aspect of it since of course(!) you would not want to specify the address for every access, with most memory accesses being sequential (presumably meaning each bit of a 44-bit word), but it was advantageous to gain the speed of occasionally switching read location, which would be slower and more costly to do with delay lines (more, shorter, delay lines).
Note that Atanasoff's capacitor-based memory was based on rows of capacitors arranged around a mechanical rotating drum (a bit reminiscent of the later EDSAC's "initial orders" module!), so it wasn't true random access - there would be a rotational delay to get access to a given row. In a way it was as much a predecessor of rotating mass storage devices as it was of DRAM.
I just missed the core memory era myself, at least as a user, learning to program (while in highschool) on an IBM mainframe in the mid-late 70's, which was already using semiconductor memory. Still, years ago I bought a core memory module and a (very similar size) etched 6" silicon wafer, intending to frame them side-by-side as the defining story of my era - the switch from physical to solid state integrated electronics.
Perhaps, and thanks for contributing that; and I'm not taking a position currently on this. But I've long been uncomfortable with the term 'random access' for random access memory (RAM) as it implies that the concept of 'randomness' is somehow a required and fundamental property of the memory technology described, when it's not. 'Arbitrary access' is a much more appropriately encapsulating description of the function of the memory at hand. Alas, the incumbency of terminology.
Nonetheless, the programs whose performance is limited by the speed of the main memory can be divided in programs whose performance is limited by the throughput of the memory interface and programs whose performance is limited by the latency of the memory accesses.
The programs from the first class will have the same performance when using a serial memory or a RAM, if they are equally fast for sequential access, but for programs from the second class, it is important to have a RAM as the main memory, otherwise they would be very slow.
In order to measure the performance achievable by programs from the second class on a given computer, it is necessary to run a benchmark where the addresses of the memory accesses are generated by a good random-number generator.
The reason is that modern CPUs have a variety of hardware prefetchers that attempt to predict the next memory address, so for meaningful results that reflect the true performance of the memory you need truly random addresses that are impossible to guess by the hardware.
Thus nowadays the latency of the memory accesses of a RAM cannot be measured otherwise than with really random accesses and a memory chip must be designed to have a low latency even when successive accesses happen at unpredictable random addresses, not only when the accesses are at arbitrary addresses, but those can be predicted in advance of the actual accesses.
So there is an argument in favor of the term RAM, because random access is the worst possible case and the memory must still be able to handle it.
No, von Neumann does not deserve credit for stealing Eckerts and Mauchleys life work.
von Neumann deserves credit for many things, but not this.
He famously brought a lawyer with him to a meeting after they wanted to discuss the report and the rights of their invention. The whole ENIAC saga has made me reconsider von Neumann as a person. He was a talented mathematician, sure, but he seems to have also been a greedy scumbag.
Eckert and Mauchley did not try to conceal their inspiration from Atanasoff, although I believe there was a court case on this topic.
And it is completely ridiculous to claim that von Neumann would have been able to describe the computer better than its inventors.
It's eaually ridiculous to consider that they would not have published themselves. They were collaborating with von Neumann on the report after all.
Goldstine and von Neumann stole the credit from Eckert and Mauchley, that is documented history.
TFA argues exactly this, that SHA-256 should not have replaced SHA-1, but it should have been added as a second hash.
Nonetheless, this solution has its own disadvantages, which were considered as more important by those who have chosen the replacement solution.
In my opinion, SHA-256 should be used for any new commits, and all old commits should be rehashed, but the old SHA-1 hashes should have been preserved and stored in some format that would have allowed the retrieval of the new SHA-256 hash corresponding to an old SHA-1 hash.
It hashes a double amount of data per cycle with much less than a double amount of operations.
SHA-512 is always faster in software than SHA-256, when run on 64-bit CPUs, and it is also faster in the CPUs that support both SHA-256 and SHA-512 in hardware.
Arm-based CPUs have supported SHA-512 already for many years and the latest Intel CPUs also support it, i.e. Lunar Lake, Arrow Lake S (S is for desktops, Arrow Lake H for laptops does not support it), Panther Lake and Clearwater Forest.
I expect that AMD Zen 6 should also support it, because they are the last important vendor without SHA-512 support.
All modern CPUs support SHA-256 in hardware, so it is faster when SHA-512 is not supported in hardware, otherwise SHA-512/256 is preferable, by being both faster and more secure.
Thanks, that makes sense. I was vaguely aware that SHA-256 halved the internal state from SHA-512 but not that it halved the block size, I haven't looked much into it. Although it makes sense in retrospect that they would simply use 32-bit variables to keep the same structure.
When implementing currency with integers, actually with fixed-point numbers, there is no difficulty in having an exchange rate with the same precision as the integer format, i.e. up to 18-19 digits for 64-bit integers.
You just have to implement the conversion function carefully, i.e. the exchange ratio would actually be represented not by a single number, but by a ratio of suitable integers, to ensure no loss of precision during the conversion done by exact multiplication with extended double-word result and then division.
Using 64-bit integers as a count of a hundredth or of a thousandth part of a cent should be enough for most purposes (i.e. up to ten thousand or one hundred thousand billions of $).
If you want to be able to count trillions of trillions of dollars with a resolution of a millionth part of a cent, then you can use 128-bit integers.
Computing exactly with 128-bit integers is many times faster than computing only approximately with decimal floating-point numbers and it requires less memory storage for the same dynamic range.
Actually 128 int uses 2 times the storage for less dynamic range.
I agree however that for financial applications int is always better. You don't want float dynamic range on finance calculations
You are right, I forgot that nowadays standard decimal floating-point numbers do not use BCD encoding any more, but they pack 3 decimal digits into each 10 bits.
This means that the 64-bit decimal64 format still provides an acceptable precision of 16 digits, even if it is lower than for 64-bit integers, which provide over 18 digits.
If the goal is to avoid rounding, 64-bit binary integers are still superior to decimal64, due to the availability of more significant digits, but due to its scale factor decimal64 is superior if you want to express a huge amount of money while allowing rounding, which would probably be needed only for amounts greater than the budgets of all countries together.
You are right that decimal64 provides a greater dynamic range than 128-bit binary integers, but even so the dynamic range of 128-bit integers is many orders of magnitude greater than needed for any imaginable amount of money, while providing exact operations performed at a speed many times greater than for decimal64.
On computers without hardware support for the IEEE standard decimal formats, much faster operations with decimal floating-point numbers can be implemented if they are not represented like in the standard, but as a product between a binary integer and an integer power of ten.
The body usually looks like a narrow twig that connects the big and long legs. This is why one of their alternative scientific names was "Pantopoda", which means "all-legs" in Ancient Greek.
Because there is not enough space in their body, many of their internal organs extend into the legs.
reply