There will be winners in this new AI age, and they will be the ones that are technically better, rather than the most popular or convenient for software engineers
I use Elixir (BEAM-based), with LLMs daily, on a fast growing site (current pace is 5 billion HTTP queries per year on a single front node).
Elixir is well handled by LLMs at this point, and we keep the “easy to scale and maintain” aspect (operational characteristics of the runtime are very good).
I can safely say that total cost of ownership is excellent.
Fair enough! That’s how we track growth, it’s easier to communicate to state sponsors.
Roughly 14M per day, but growing 2.5 to 3 on a YoY basis.
My point stands though: much more serenity in ops with that growth, and LLMs do work nicely, especially when most of the code is stateless by language design.
The important metrics is peak requests per second. 14M / day sounds impressive, but it’s only a sustained 160 per second which literally any language and framework can handle on the lowest end hardware.
Sure, peaks are much higher. But a good part of it is reverse proxying with in-memory caching and locking, which I've implemented in several stacks over the years, and many struggle with it (I have been paid to do comprehensive benchmarks on the topic). On the BEAM it's a non-event.
Even pre ai you could toss together a decent cache in about a minute using ets or similar, and faster with a solution like cachex (which was basically what you'd build yourself but packaged into a neat little bundle)
Sure, that's why I highlighted peaks as a more important metric than average. Even an order of magnitude higher peaks over the average is still fairly trivial, compared to where BEAM shines. Maintaining hundreds of thousands of concurrent long-lived connections per server is what it excels at, but also not something that's relevant to 99% of web apps out there.
~160 req/s on average, higher peaks. Not trying to impress anyone though, my point was that LLMs work perfectly fine on an Elixir codebase, and that we get future-proof ops on a single node with zero issues. In my experience, it has been easier than with any other stack I've tried.
To be fair, Erlang runtime semantics (cooperative "async threads") never needed changing to tackle C10k problem, they got that right as part of the early designs before C10k was even on the tasble.
Everyone else just moved towards the same kind of cooperative thread runtime semantics (even if the lingo differs, async in C#, callbacks first, then async in NodeJS, virtual threads in Java, async in C++ and so on).
I think an interesting historical tidbit is that Java actually had M:N threading in early versions (1.1, 1.2, 1.3?) that got ripped out because it was deemed complex and error-prone since people back then often did native bindings naively (And perhaps file/socket API's were less complete?) only for them to reintroduce it again in Java 20 I think it was under the virtual threads name?
Preemptive, not cooperative. Cooperative scheduling is based on the programmer releasing control manually (much like manual memory management) or having yield points baked into the language, often at function calls or IO events. Processes on BEAM can be preempted, they're allotted a number of "reductions" and when those are exhausted, the process is preempted (the process may also yield earlier). This guarantees every process is able to get a turn. Cooperative is fine, but it requires much more care. Go prior to 1.14 suffered from cooperative scheduling, a tight loop (not that you would write this one specifically, but as an example) could hog an OS thread:
i := 0
for {
i++
}
The conceptual equivalent in Erlang would only execute the loop so many times before it would be preempted.
> Cooperative scheduling is based on the programmer releasing control manually (much like manual memory management) or having yield points baked into the language, often at function calls or IO events. Processes on BEAM can be preempted, they're allotted a number of "reductions" and when those are exhausted, the process is preempted (the process may also yield earlier).
BEAM is actually cooperative [1]. In userspace, function calls and IO (receive and !) are yield points, and reduction count is simply the number of function calls you're allowed. Something something Computer Sciencey reduction basically just means function call.
It just feels preemptive because as a functional language, you have to call functions all the time. Your loop becomes
loop(I) -> loop(I + 1).
And that will yield once it calls itself enough times.
Native code (NIFs, BIFs, and drivers) has to count reductions and/or yield explicitly.
You can easily hog scheduler threads in native code that does significant computation or does i/o without using the runtime facilities; most people who write NIFs will mess it up at least once, I certainly have. It's much harder to do in userspace code, but if you fully unroll a very large loop that does not call functions or anything else that can yield, you can do it; I don't see how this would happen without specific intent, I've never run into it, but it's possible.
A blocked scheduler can be vaguely fine or terrible depending on how long it stays blocked and what the system is up to. Often it won't be terrible until you do something that needs all schedulers to synchronize, such as operations around code loading, but you can probably wedge the whole BEAM if you try.
[1] there's not really any other sane way to build preemptive userland threading anyway. You could schedule a timer signal, and task switch in the timer signal handler... but no sane person will recommend doing non-trivial work in a signal handler.
I'm not 100%, but i don't think think get(...) is charged any reductions, so it won't yield. gc_bif potentially does garbage collection which does get charged reductions and BEAM can yield during gc.
I'm just adding at the end to do something with the values, so the compiler doesn't ignore them. If you wanted to write BEAM assembly, you can probably make a terrible function there without nearly as much work. smallnum math doesn't usually charge reductions from what I was seeing, so you could probably do something almost useful with that.
Right, my point is that C10k solutions basically revolve around M:N threading models where M being OS threads and N userspace threads, so from the view of the OS/Userspace boundary it's cooperative since the OS does nothing for the N userspace threads, only the M.
That the Erlang runtime hides it from the developer, doesn't mean that it isn't cooperative. So yes, for the Erlang programmer it's preemtive but from an OS/C standpoint it's cooperative (Like toast0 mentions in his sibling comment about NIF's exposing the cooperative nature of the runtime to programmers is telling about why it's considered "complicated" and not recommended practice).
My recollection is that M:N ("green") threads were pulled out because in the absence of a runtime that can guarantee a thread is never blocked, you essentially have to reintroduce all the complexity of an OS scheduler without the benefits of OS-wide context.
"Project Loom" spent much of its time (AFAICT) figuring out how to make every
Java system-level API non-blocking without breaking backward compatibility. The only one they didn't get to for the initial release was Java's "interesting" built-in `synchronized` object monitor construct.
bcantrill has strong opinions on M:N and have always been curious what he thinks about Loom.
Exactly, didn't want to go into the details. I think a key point is that back in the green-thread days they were kinda early on trying to do it and "extra" native threads probably felt like makings things worse without benefit since single-core CPU's with little memory every extra "real" thread would cost.
The C# async system iirc has "blocking workers", ie extra threads that takes care of blocking syscalls, so you're limited by a queue of them "waiting" for their respective "async" workload to finish but the rest of the runtime can chug along concurrently, iirc Project Loom also adopted that, so you have M:N for most code but some extra theads to take care of legacy codepaths (that the Java ecosystem has rewritten a lot of things into pure Java helps since native code calling is probably fairly "obscure" these days).
10k requests/sec is not really the same thing as the C10k problem which is 10k concurrent connections. You might serve much more or less than one request per second per connection, so req/s and conncurrent connections are not really related.
I also wouldn't say OTP was built to overcome the C10k problem. It is a very good solution for lots of concurrent connections (with a suitable OS), but OTP was built to manage concurrency needs in the context of telephone switches. The constraints chosen to meet those needs does work (very) well to meet the challenges of large numbers of concurrent IP connections, but I wouldn't say it was built to overcome those challenges.
> I have done several experiments where, the only difference are a couple of paragraphs describing the technology stack for the AI tools.
The differences emerge as the project goes on. Comparing feature velocity and defect count is difficult unless your experiment is to build the same project over a long timeframe across several languages. Then winners would indeed emerge, but then that's only in the context of whatever domain said project is tackling. For instance, if concurrency or infrastructure isn't a large part of the complexity, BEAM languages might not confer much benefit.
Immutable value languages like BEAM langs are probably useful too, the LLM doesn't have to guess whether a value passed into a function will get mangled by the callee
I can't see the fuel being able to be reused for aviation once it's been outside the stewardship of the fuel company, but how about draining it out and selling it or even giving it away - if for nothing else for CO2 reasons
Aviation fuel is taxed specially and is sometimes dyed to make it difficult to do this. Jet fuel itself is basically just diesel/kerosene, though, so if any is left over it can just be used for the next flight.
Exactly the same here. When I learned that other people actually could visualise things, it made me realise why I've spent my life being a commercial photographer and camera developer. Capture the images - I guess most people get this for free to some extent
I literally could not tell you what my own wife or children's faces look like from memory. The feeling of sandpaper, I can basically relive it 1:1. Same with music, I can all but hear it. So there does seem to be quite the spectrum, even if it's all imaginary.
It stretches credulity that someone can occlude their vision with their imagination. Like, you are staring at a dog, you can't imagine a brick wall in front of it and not see the dog any more. Similarly you can't not see the back of your eyelids. Your imagination can kick in and your brain does $THINGS. But the back of your eyelid is still what you are seeing when you close your eyes.
I don't mean to sound dismissive of differing imaginary intensity. It sounds, to me, like people are confusing _imaginary_ vs _hallucination_. Sure, some people imagine a stop sign as an HDR image with a back story and others as "red octagon."
I think that language is hard and it sounds, again, to me, like there are people who think there are AR/VR like experiences when someone closes their eyes. That makes as much sense to me as not seeing the back of your eyelids. People are not "seeing" things - they are "imagining" things to various fidelities.
Really nice, i'd like to use this, and if the license was right probably would if it was OSS.
I think closed source service stuff like this was long over before LLMs came along. Now it's like, I can take a few screenshots and make this myself. At least with open source YOU the developer get to carry on being in the conversation and co-develop, rather than giving ideas to others and being left out
Edit: now I can't stop thinking about correlation vs causation - you being under the boulder bleeding out at the same time XMPP encryption suddenly stops working.
Physical hardware certificate thinggy which got crushed?
Because it is politically much easier to just spend on medicare, social security and military than it is to raise taxes to cover that spending?
Interest on debt is already a double-digit percentage of spending, so the problem gets worse by itself.
And honestly: How many people in the US would actually change party only because of irresponsible spending? In my view, this is like priority number 4 at best for most people, and many at most moan about it instead of letting it drive their political choices. Political representatives also basically only ever throw it around as accusation against opponents, without ever actually doing better themselves (just consider the Trump-1 promise of getting rid of the debt during his first term: not even an attempt was made).
I'm very anti anti-democracy, so anti China by default, but I'm shocked how much I'm rooting for them right now. They played this so well, capitalism fucked it up soooooo badly
reply