Interesting project, towards a decentralized Internet!
The heavy lifting is done in this function, pairing. You link your personal devices together, solid idea [0].
This Flipflop project relies on the Iroh library for NAT/Firewall puncturing, external address discovery and overlay networking. Connecting two 5G devices, without a coordinating central server is possible, but very hard. Iroh simply relies on a server which you could self-host. Cool stuff, we stress tested it a while ago and it somewhat worked.
NAT connectivity is unsolved. My university lab conducted one of the bigger puncturing studies a decade ago [1]. Recently, we also bought 40+ or something 4G and 5G SIM cards in Europe for puncturing [2]. My master student traveled Europe by bus to get these local SIM cards. He worked day-and-night to connect these weird symmetric 5G NAT filter boxes and got it working. Simple sync collision and birthday paradox brute force... It all works sometimes, but insufficiently reliable for mass-adoption. You can't build a decentralized social media ecosystem on top. Like Iroh, best to use a central server. So then you have legal liability for your users. Unless an AI agent buys the server fully autonomously using Bitcoin, next years students got that working[3] :-)
We (iroh) don't do birthday paradox hole punching yet. There is some internal debate whether it is worth it.
So if you are in an extremely weird NAT you will fall back to a relay, either our public relays, your own relays rented from us via iroh services, or a self-hosted relay.
But I would argue that a p2p social application that occasionally uses a relay is very much preferable to a centralised app.
The relay is generic internet infrastructure. It just forwards end to end encrypted connections. It can not learn anything about the content.
A relay that is hosted on a non KYC server rented via ZCash would be interesting though if you are super paranoid.
Yeah, we need "generic Internet infrastructure" for linking devices.
By 2030 we hopefully understand the mathematics-of-trust and can link all the billions of citizens together in a single overlay.
> non KYC server rented via ZCash would be interesting though if you are super paranoid.
What will happen if a decentralized Internet emerged and presented an existential threat to existing profit margins? Company lawyer would start to attack you, correct?
I think you're both missing the forest for the trees. P2P connectivity is
about as solved as it will get, and the last % will stay unsolved.
The final goal for decentralization is something like: no central party
should become the irreplaceable master over data, community, feeds,
moderation, UI and identity.
We already know the minimum pieces needed to avoid that kind of
centralization: hashed, linked, signed records(/packets), so their
validity and causal order don't depend on whatever connection carried them
or whatever physical server received them.
Once we have that, apps 'transcend' the connection layer, and the parties
that control identity, community, feeds, moderation, etc become signing keys
that can operate from anywhere.
The real problem for a decentralized internet, or "local-first apps", is
that writing applications at this layer, or writing the library that lets
you write them, is hard and nobody has solved it.
IIRC Iroh tried this early on, then pivoted to the p2p socket library it is
today.
I'm not sure where the Delft stack sits with its ambitions atm.
You're describing atproto. They solved all the problems and it's not hard to build on top of it. Maybe even easier than a traditional centralized app because your server is just a view over the network so don't need to worry about consistency or durability.
There is one last centralized part they built to avoid using a blockchain, the PLC system, but it's designed to be tamper proof and easy to replace with a new server. The only thing a malicious operator can do is censorship in the p2p sense (ignoring a user's actions) but that would break trust and make the community migrate to a new provider.
Not atproto specifically. I’m describing primitives that exist in Git, Bitcoin, and plenty of other
systems.
What I’m getting at is that we haven’t nailed the right abstraction. A "sockets" or "HTTP/HTML" moment that
collapses the design space, to such an extent we've collectively forgotten there were more complicated alternatives they replaced.
Somebody spend a lot of time and effort to make this as simple as possible. This is the best & simplest version of it.
Yet its not simple enough. Repositories, event logs, view servers, DID, ...Lexicon RPC (XRPC)? What’s the minimal version of this that still works? Not an SDK hiding the concepts; fewer concepts.
It’s hard to explain what I want precisely because I don’t think that abstraction exists yet. Maybe atproto gets us
closer, but the on-ramp feels like endless links to sub-concepts. The average developer's eyes glaze over.
>What will happen if a decentralized Internet emerged and presented an existential threat to existing profit margins? Company lawyer would start to attack you, correct?
I think it's very misplaced concern. Commercially-viable internet will not be decentralized because of legal restrictions. If whatever engineering solution will be found to weaken them, governments will step in to curtail it. FWIW any moderately successful decentralization effort is more likely to be attacked from exactly this side: some countries will hate it for uncontrolled information dissemination, others for fiscal reasons.
I think this is overly bleak. Bittorrent exists and works since 20 years. And this is something that is more of a threat to commercial interests than e.g. a decentralized social network.
Also what kind of attitude is this? Government is too powerful, we might as well give up. Where's the fun in that. It is hard but also very rewarding.
Bittorrent is legally dangerous in quite a lot of countries where copyright is politically important issue, so as I see it the example supports my thesis.
>Government is too powerful, we might as well give up.
That's not what I wrote. My note is about proper understanding of real threats and obstacles.
> You can't build a decentralized social media ecosystem on top. Like Iroh, best to use a central server. So then you have legal liability for your users.
You can, ATProto has the example schematics for how, but it's has a federated rather than distributed architecture, but I don't think "federation" is a hard constraint of the design, more like an implementation detail, and as long as you can do the signalling centrally, to connect the peers, you can do the rest P2P. So sure, legal liability for the connection of peers, but all data transfers would still be directly P2P. AFAIK, Iroh also does "Centralized signaling > P2P data transfers", even when NAT is in the way.
The self-replicating seedbox is a cool idea. Some places offer interest on stable coin deposits. I wonder if there could be a way to set up an account for AI use that would be self-replenishing so long as a budget is maintained.
Using the term AGI and not including any performance analysis. My AI calls it: "massive marketing overreach".
Somebody called this slop in the comments.
As a professor who published on continual learning I'm leaning towards agreement[1]. It lacks any substance. No relation to related work, no description of algorithm, no ablation study, just hand-waving that we're feeding some data and "Chess is not forgotten".
This "how-continual-learning-works" markdown text is not an algorithm [2].
There is no special algorithm, the finding is that slowing down the LR or the trunk, while keeping the LR of the experts is enough to eliminate most of the forgetting in the network. You can see in that experiment where chess data was the only thing the model read for 524K characters, yet it kept almost the same performance (i.e. held-out loss) on all other domains. If you keep LR the same across the whole network the loss in other domains degrades dramatically - this is a clear sign of catastrophic forgetting in action. What I can say for sure is that any traditional network that does pose a sign of catastrophic forgetting would not be able to learn any patterns from a single stream of data.
There are no benchmarks published as the model is heavily undertrained, but it is learning. And you can see this clearly in the loss and samples even though they are still barely coherent.
I am not an academic and am not trying to publish a paper about a “major breakthrough” or something like this. I am just a small person who found a cool thing that clearly works and wants to share it with the world. That’s it.
It is a goalpost that is easy to move. By "works" I mean learning from a continuous single (meaning batch-1) stream of data. The fact that it produces full words and full coherent phrases instead of a random stream of characters that would any typical LM produce if trained under the same training regime.
I would be okay if you shared it as a potential idea and possibly interesting early result, but the language you are actually using to characterize it is misleading or delusional.
Please get a model to the point where it seems like it has some natural language understanding and then share again with reasonable characterization.
For sure. As it will pass through the whole corpus I will share the weights, run it through established benchmarks for small models and share all of this as an update. I am also planning on making a Youtube video explaining in detail how it works on a deeper level and the whole reasoning behind why it is built the way it is. But no promises here.
Nothing to be ashamed of if you end up pushing back the release date of your feature film :)
I had ideas not completely unlike this so long ago, but one big difference can be summed up in one of your parameters.
>Directories are walked, binaries are skipped . . . and each file is read from its beginning to its end because a document has an order.
For me it was binaries being walked because text and anything approaching a language model was so much further out-of-reach having such limited computer power.
Is "AGI" the language that bothers you? Well, one has to keep his eyes on the prize and I see a bright idea which could lead to AGI, so, why not describe it as such? I also see the inspiration and hard work necessary to move that idea further along, so fingers crossed.
Actually I'm mad that I wasted my time looking at it based on the claims. He implies it is trained and uses the term "AGI" and "continuous learning". He never finished a single training run or enough that he considers not "undertrained". It's not trained. And actually there is no evidence that it can actually learn anything useful.
"The model reads 524,000 characters of chess". This is 100KByte of training data in a toy model with rigid parameters and no global learning. Gap with real LLM and trillions of tokens.
This model really addresses the problem of preserving previously learned knowledge, but by restricting the LR of the trunk it stops acquiring new knowledge. Details: "Rethinking the Stability-Plasticity Trade-off in Continual Learning from an Architectural Perspective"
Insightful! We struggle with containing true decentralised protocols like BGP, Bittorrent, and Bitcoin.
Email, RSS, and also Nostr are too simple. The core protocol does not contain any measures against spam, abuse and fraud. No accountability. Just launch a million fake identities and spam these network. Your Internet address got blocked? Try the a16z-funded startup that uses a USA-based phone farm. Decentralised accountability is unsolved.
My university lab has focused on decentralisation for decades, its hard. We need a decentralised trust framework (e.g. our TrustChain protocol), decentralised spam prevention (StarveSpam) and decentralised democracy (our TwoStepDemocracy protocol). Getting this deployed and do iterative improvements needs more resources then universities have. Another coordination problem for The Common Good.
> We struggle with containing true decentralised protocols like BGP, Bittorrent, and Bitcoin.
I find it hard to parse the intent of this. Are you saying this is good or bad? I think it’s good; I want my communications to be “uncontained,” even if there are downsides to this approach.
I do not believe in such a thing as “The Common Good,” to be quite frank, and find your mention of the concept to be alarming. What is “good” is relative across individuals, cultures, and situations. I certainly have my own beliefs about it but I don’t believe they should be enforced at the technological level.
Most people working on decentralization tech understand this innately. It’s not about reinforcing majoritarian views of what society is or should be.
Yes, we solved it by centralising and ceding our power, so that two companies can see when a lot of people get the exact same message, and delete it for them.
the main method against spam in Nostr is probably lightning. You can simply attach sats that are worth a couple cents to the content. Note that lightning has the concept of hold invoices so the receiver may or may not chose to actually claim the money. I.e. your friends and family will always attach 50cents to a message, knowing that you will never claim it anyway. If a spammer attaches 50cents to a message, the recipients will claim it and the spammer looses.
This is where we are headed in the AI age without Nostr/Bitcoin anyway. On some sites like youtube it has already become impossible to separate spam bots fom real comments. Agents that remote control users real browser instances and post AI slop all over the network will bring spam to another level. Attaching some money to whatever you post will actually be usefull to separate spam from real content.
> You can simply attach sats that are worth a couple cents to the content.
That is a great point! You can attach real money to messages to prevent spamming. However, this has been tried many times and failed to get mass adoption. Friction is too high when users need to load money on their fresh wallet. Giving everybody their own karma-credits could be the best of both worlds.
AI slop is a fundamental problem. The signal in Youtube clicks/views has gone to zero. Even adversarial machine learning can't make it a nice place anymore. I think its even doomed if we find a collective system that filters crap.
btw RSS usage is consistently declining, its 1999 server-focused technology.
The heavy lifting is done in this function, pairing. You link your personal devices together, solid idea [0].
This Flipflop project relies on the Iroh library for NAT/Firewall puncturing, external address discovery and overlay networking. Connecting two 5G devices, without a coordinating central server is possible, but very hard. Iroh simply relies on a server which you could self-host. Cool stuff, we stress tested it a while ago and it somewhat worked.
NAT connectivity is unsolved. My university lab conducted one of the bigger puncturing studies a decade ago [1]. Recently, we also bought 40+ or something 4G and 5G SIM cards in Europe for puncturing [2]. My master student traveled Europe by bus to get these local SIM cards. He worked day-and-night to connect these weird symmetric 5G NAT filter boxes and got it working. Simple sync collision and birthday paradox brute force... It all works sometimes, but insufficiently reliable for mass-adoption. You can't build a decentralized social media ecosystem on top. Like Iroh, best to use a central server. So then you have legal liability for your users. Unless an AI agent buys the server fully autonomously using Bitcoin, next years students got that working[3] :-)
[0] https://github.com/andunieee/flipflop/blob/62bb498520aacb60e...
[1] UDP NAT and Firewall Puncturing in the Wild, https://inria.hal.science/hal-01597969v1/document
[2] Mass Adoption of NATs: Survey and experiments on carrier-grade NATs, https://arxiv.org/html/2311.04658v2
[3] Self-replicating seedbox servers using programmable money, https://arxiv.org/abs/2609.05431