The plan is to bring the celld model to workerd: run a fleet of workerd instances that use a bucket for coordination and storage. Building on workerd, instead of a separate runtime means our effort isn't split and that APIs are exactly the same in both CF and self-hosted. And of course the deno team will help improve the worker runtime itself.
It's a joint decision and I agree with it. I'm most invested in its success and have put the most work into it - and I no longer think it's where I can do the most important work. There are some good ideas in Deno and it's well engineered - but it ultimately is not solving big problems. It has been sucked into the gravity well of node compatibility, which forces it to behave exactly as Node does. Why reimplement Node? It works. Marginal performance or UX or security benefits are not enough.
I'm interested in building powerful new abstractions. celld has been working remarkably well, depending only on object storage for coordination and persistence. It is not just a slightly different API to interact with the file system or network - it's an entirely new model for server development. I wrote a bit about it here: https://x.com/rough__sea/status/2105853283617440032
We'll ship monthly releases for a year, and Deno stays MIT-licensed. If people want to carry it forward, I'd like that.
I don't often post on the orange website. But this news was enough to make me want to say congratulations, friend.
Looking back now after almost 20 years (Nov 2009 to now for me), I can see the arc from what node was then to what celld can be now as your thinking has evolved
A distributed operating system running in userspace is the primitive that can break apart the siloing that has become the default across the walled gardens we all operate in
Thanks for your response, ry. It's hard to put into words how much I appreciate your work, both on Node and Deno. Honestly, you saved me from a career as a Java dev. :) I've been using Node since 2013 and Deno since 2018 (when it was still written in Go)... I used both first for hobby projects and scripting and later, when they matured, also at work.
I disagree that there's little differentiation between Node and Deno these days. Sure, I love Deno's UX and tooling, but the security benefits Deno has over Node are still the number one reason, we're picking Deno over Node at work. I'm honestly sad we may have to reconsider and switch.
I understand you are working on a new model for server development and I'll take a look at celld for sure. But what about scripts? I have a couple hundred repositories all using Deno for a wide range of scripting tasks. I'd love to continue using Deno for scripting for another 8 years or longer.
I'm just feeling a bit sad, you know. It kinda feels like I'm gonna lose a friend. That happy little dinosaur, now standing in the pouring rain again...
Why let it get sucked into the gravity well of node compatibility then? How can the community carry it forward without you? Will CloudFlare release the name/assets, or it will have to be forked?
FWIW I think the project is likely highly maintainable with relatively small economic commitment. I would be willing to contribute. If anyone wants to discuss how we keep it going past the next year, please let me know.
...well, who decided to embrace Node compatibility?!
The DX of Deno far surpasses Node (or at least it did, idk if Node has caught up in that regard).
What are the big problems you see that still need solving that a future Deno (fork) could handle?
EDIT: Talking to people in the Discord and was reminded that the community may have been the inspiration. Adoption was low so boom, add Node compatibility and now adoption grows. MEH.
To be fair, Deno and Bun's enhancements seemed to do a lot to set a fire under Node and have it implement various things it didn't have when both projects launched. This is a good thing.
workerd is missing the scheduler - the thing that allows it to operate across nodes. Durable Object storage are just local sqlite databases.
celld is the full distributed system (albeit single tenant). It distributes DOs (cells) across any number of VMs. The databases for each DO are in object storage with RPO=0 guarantee.
bummer this expects NFS; this is all a brand new implementation vs what Cloudflare runs internally in production for DOs? I can’t imagine all your workerds share a single global NFS… What would a more realistic workerd with DO support look like?
Honestly, you wouldn't want to run our internal implementation -- it's far too complicated. Unless you have 100+ datacenters around the world -- then maybe.
The goal of this new design is to scale to a cluster while being operationally very easy to set up. Ideal for self-hosting.
NFSv4 is an easy first step, convenient because it's broadly understood, has many implementations, and requires no client libraries. I could imagine a follow-up to support LiteFS instead of NFS could make a lot of sense, though it'll get more complicated.
But yes, celld is definitely ahead of us here. No doubt about that.
