From the looks of it, this seems focused on datacenter/batch inference, and doesn't tune its kernels to the specific hardware and workload where inference is being run like Magnitude does.
Magnitude is optimized for maximum single-session performance and memory efficiency - so we should be more performant for local inference use cases.
"By [...] utilizing a single-threaded execution loop, TigerBeetle aligns its software architecture perfectly with the physical realities of modern hardware."
Can someone explain why single-threaded execution loop is more aligned with the physical realities of modern hardware ?
Tobi here from TB. Great question, and it's important to be nuanced here.
It really depends on the problem space. For example, many OLAP workloads (analytical) contain large amounts of parallelizable work, then multi-core execution is absolutely the way to go. That also aligns well with the direction CPU technology is taking, with core counts continuing to increase.
For the transactional workloads we see at TigerBeetle, and in other transactional systems I've worked with, the picture is quite different. We see a lot of read-modify-write operations combined with a power-law distribution of the data.
Take a simple banking example: some accounts, such as those belonging to large online retailers, see much more activity than the average individual account. You might have 80 - 90% of transfers touching a relatively small number of these hot accounts.
Operations on the same account must be serialized to preserve correctness. That means this part of the workload cannot be meaningfully parallelized. In fact, attempting to parallelize it can make performance worse because of lock contention and coordination overhead, something the "Universal Scalability Law" captures quite well (but is also easy to test out yourself with a simple experiment).
Instead, we focus on batched execution. We carefully structure execution to make effective use of CPU caches and efficient algorithms, so that a single batch can be processed extremely efficiently without any coordination. Batch execution also allows to amortize I/O and replication.
That being said, there are areas where we could use multi-threading (e.g. compaction) that are not on the hot execution path.
Ha, I was explaining this just yesterday to a few people at $WORK (I'm not at TB, though we do use a bunch of Zig in prod).
"Multiprocessing" (multiple CPU cores independently executing and only able to coordinate via some message-passing system -- a definition which encompasses both multi-core CPUs and horizontally scaled distributed systems, contrasting slightly with the normal definition) is challenging for a few reasons.
Firstly, the details are an open math question, but I'll blindly state that some problems aren't amenable to parallelization. I.e., no algorithm can meaningfully improve performance via parallelization no matter the implementation. Think through how you would more quickly compute hash(hash(hash(hash(...)))) for example. The serial dependency makes things challenging. That isn't too dissimilar to the problem TB faces.
Secondly, message passing is expensive. If the only way two CPU cores can coordinate is through a multi-level cache, at best you're incurring ~tens of nanoseconds of latency per message. Contrast that with a base rate of 512 bytes processed per nanosecond with enough attention to detail on typical modern server hardware (4 pipelined AVX512 instructions at 2GHz). Messages are several orders of magnitude worse than your normal work, so if you need very many of them then you're hosed from a performance perspective (worse with longer delays, like networked computers). Even very parallelizable problems at an abstract level can suffer performance losses by trying to add even one extra core. This blog post [0] doesn't perfectly capture the idea, but it's close (and a fun read regardless).
Thirdly, message passing is an insanely complicated programming abstraction to reason about. My first two points were more about what TB was saying -- realities of modern hardware -- but the programmer experience is important too (even if you don't believe that post-2020, the LLM experience doesn't differ much from the human experience; bad code begets more bad code, slowly). The core mechanism for correctness in most software is being able to reason about "this thing is true, therefore that thing is true" and iterating. You rely on invariants like "this is sorted" to build other working theories. The invariants in multiprocessing code are much more nebulous and less amenable to accidental discovery, also less amenable to being able to build or compose them into other stronger invariants as you add code. The main reason for that is that you know almost nothing about the relative order of those messages with respect to each processer's view of which instructions happened when (and for purely multi-core "multiprocessing" the story is even worse; while my description of message-passing being the core primitive is correct, that's not what's exposed to you as a programmer, and different memory models can have even weirder interleavings than your code would naively suggest -- i.e., your code is being decomposed into smaller subunits than even a single assembly instruction, and the message passing happens at that level). The combinatorial explosion (an exponential explosion really, but big numbers either way) of states you might be interacting with makes it very difficult to understand _anything_ about the system you're examining. That's why you see a handful of primitives used over and over -- if you can decompose your problem into a parallel map plus an associative reduce then you can probably figure out some way to make it better through parallelization (not always, especially if the framework is too generic, see the linked blog post [0] if you weren't enticed to read it previously). If you can't decompose it into know primitives then it's an open research problem every time.
The crux of that third point (and we could definitely add more explanation and additional problems) is that there's a huge cost to multiprocessing. You have to be buying something substantial to even want to reach for it, else you have to be in one of the "easy" problem spaces where somebody else has done the hard work (e.g., stateless webservers).
TB isn't that. Their whole raison d'être is state management, and not in a way that's easily amenable to parallelization.
I love how proactive the crypto team is about post quantum. They released https://pkg.go.dev/crypto/mldsa. The lead maintainer Filippo Valsorda wrote a nice piece here[1] to urge the tech world to start deploying good enough versions of post quantum crypto.
While I'm highly sympathetic to competing priorities crowding out movement to pq cryptography. At the same time it's not sudden at all. It's been 10 years since nist first said "move shit over"?
"bold-faced lie" and "bald-faced lie" are both valid expressions. The original expression is "bare-faced lie" but they're all pretty similar to each other.
Yes but only one of them makes most people who hear it think you don’t know the expression you’re trying to use. There’s no objective reason it has to be this way, but it is, and at least personally I appreciate being told when I’ve got something stuck to my back.
I don't really know why I should care what expression I'm trying to use (or what someone else is trying to use) as long as the intent is understood. If anything, people who are needlessly pedantic about the way I express things that they clearly understood are people I'm happy to get signal from so that I can minimize my need to interact with them going forward; it's much nicer to find out from something inane like this than to wait until we have a communication problem about something actually important.
bold-faced lie is just the usual English drift that was actually questioned as incorrect when it first surfaced. If a lie is bold, you don't have to suggest that the user's face is bold when doing it. You can in thirty seconds of google searching find numerous sources explaining that "bold-faced" is a malapropism.
It's the usual English drift perhaps, but "bold faced lie" has been used since the 17th century, which is also apparent from "thirty seconds of Google searching". Three hundred years is enough usage for me to count it as correct.
On a side note, "bold faced" does not mean the persons face is bold, only that it is said boldly, which implies a level of rudeness that "bald-faced" or "bare-faced" does not.
Modern references seem to align (for instance, Garner's): Bald-faced and bare-faced have slightly different connotation than bold-faced. Despite "bold-faced lie" being in somewhat common use, "bald-faced lie" is more common (over 2x according to google stats), and overwhelmingly agreed upon as the "literate" choice, unless an author intends to refer to the sort of lie that would headline an article.
> Generally, something to be avoided by people striving for clearer communication
Their communication seemed pretty clear to me. If anyone actually claims that they didn't understand what they meant but would have understood it by using the other form of the expression, I think that's a bold-faced lie.
Chromium uses BoringSSL, which opens its readme as follows:
> BoringSSL is a fork of OpenSSL that is designed to meet Google's needs.
> Although BoringSSL is an open source project, it is not intended for general use, as OpenSSL is. We don't recommend that third parties depend upon it. Doing so is likely to be frustrating because there are no guarantees of API or ABI stability.
This person was public on the recent nist list against hybrid solutions. I simply don't understand why they would oppose the safer option. Yes I've read the mailing list, it just all seems quite suspicious.
