"making a new theory that is really useful is out of reach for the moment in my opinion."

Curious how this ages.

Recursive self improvement, self-play and multi-agent RL could make useful new theories, eventually.

Sure, and I hope LLMs will at some point be able to do it. It would simplify greatly my work.

However, at the moment I consider that they stay in the 'convex hull' of their training set + a provided context, and I don't see that much research that made real improvements to the situation.

Bits of information depend on the readers prior knowledge, those bits are not absolute numbers. That makes communication not just information exchange but also syncing of priors. So some amount of redundant information maybe needed/wanted.

Lossy expansion or bloat?

Depends on if you are buying tokens or being paid for tokens

Very cool projects, and amazing a 14 byte nn can do it. Would be fun to analyze its approach, converting it to a Code-as-Policy.

Thank you! Yes the maze solver has been a very fun experiment with LLMs, and code as policy is a good idea. I’m racking my brain trying to find my next thing to experiment with, but usually the ideas come to me in waves

Very cool. Are you familiar with the game Voxile? Its author also created custome languages, and the game uses Lobster.

I'm tangentially familiar .. I played the demo a couple times over the last few years when they did major releases. It's impressive to say the least.

Thanks for the really nice in-depth post! Hoping for a future one about:

"Much recent work on auto-research, self-improving agents, and evolutionary program search can be organized around this question. Other work on model self-play, synthetic data, test-time training and a broader theme of continual learning also matches the RSI vision (e.g. Yuan et al. 2024, Chen et al. 2024), Zhao et al. 2025, Choi et al. 2026)) but they will not be the focus of this post."

Brings back memories of Soundmonitor (Commodore 64, 1986) with vertical scrolling, hex note tracking, and 3-channel column layouts.

Another happy customer, bought two Jamcorder units. Not just the hardware is simple, the software as well: it just works, quietly in the background. I just had to enable daily auto-reset otherwise it eventually had issues. Whenever someone starts playing piano, it records. The iOS app is so nice, playing back on the phone or on the piano, and keeping track of total hours played and organizing favorites. Congrats, if only more projects kept things simple and useable!