Thanks for sharing! I'm exploring a similar direction, but optimising for extreme model size. I have gotten an LLM down to 970 KB, with some interesting results compared to GPT-2 (small).

Super interesting, not a CS/AI guy here, but I'm curious as to how you can train such a small-sized model give you as accurate an answer or good hypotheses to get you going

This is extremely interesting, do you have a public repo or write up to share or not yet? if not yet please sign me up when you release it, Cheers!

I'm planning to have a blog post up in the next two weeks or so (simedw.com), still have a lot of experiments to run, and training even tiny models takes a bunch of time.

following!

That's a very cool idea, maybe a bit expensive to scale.

Exactly, I wonder if I'm not hitting the right WebGPU path for Firefox?

Thank you.

The base model already have a good grasp of language, so that man pages look like coherent sentences. It also knows python and similar language well, which means that things like 1+5 or print(["hi" for _ in range(5)]) work.

I did try to train from scratch, but it was much weaker (also added a lot of aux losses).

The first part of the context is the window width:

``` BOS SCREEN_START 32 96 ```

I trained it on three different resolutions, but it should kind of work with in between ones too.

Very thoughtful, nice work again!

That's very cool project. I read this blog post a few days ago about one way to look at memory encoding in models https://alexwang.ai/posts/associative-memory/ that you might find relevant

Thank you for the link. Yes! loved that post. I had just created the cli version (https://nicobrenner.github.io/lossy-git/) and it inspired me to create the web version

Here's the HN discussion of Alex Wang's article: https://news.ycombinator.com/item?id=50122878

Also, I really appreciate the excellent detailed writeup on your project. Curious about the mistake correction capabilities, it's fascinating, did you try breaking it up in 2 models? one that generates one that repairs? Just wondering if would make any difference in results, which could be impactful given your observation that "was one of the more effective training changes"

Thanks! Macroexpanded:

You could have invented transformers from memory - https://news.ycombinator.com/item?id=50122878 - Oct 2026 (149 comments)

nice :)

Normal only keypresses invoke the next ”frame”. There is a sleep token that the runner could use to indicate that time is moving.

I haven’t run extensive tests yet, but larger models also get better at memory.

It shows a good improvement on my COCO object detection benchmark: 0.504 mAP zero-shot.

Gemini 3 Flash was around 0.403, 2.5 Pro was around 0.34.

This was neat, well done!

The only alternative version of Go I’ve played before was magnetic Go, where all the rules are the same except that, along the axis lines where you place a piece, like colours attract and opposite colours repel.

On Voronoi, I did feel like the continuous space made it really hard to judge whether there was enough room, whether the opponent could get in, etc. Maybe it would work better if it were discretized a bit? Or maybe I just need to play it more.

Hey there, thanks! You might already know this, but if you hover your stone around or turn on "Group View" mode you can see solid colored lines between stones. These indicate whether or not they're close enough to prevent cuts, and are really important. Other than that, I think that the uncertainty and need for estimating is kind of intentional - it's a lot more handwavey than normal Go.

Hope that helps!