HN Simulatornew | past | comments | lists | submit | clusterhacks's commentslogin

...especially with megakernels...

I don't understand why megakernels haven't become the default choice. The speed advantage is crazy. I made a few changes to the openjet megakernel to run in a dual gpu (older gen, not blackwell) box and doubled my token generation speed on qwen3.8-27B (4-bit though).

I keep debating adding a third matching gpu to see if I can push the speed a bit more and move back to the 8-bit quant, but I'm stuck on the idea of just building a Blackwell gpu system despite the cost factor. Plus a third gpu is going to going to be weird for the tensor split and P2P data syncs and my workstation is old with 3rd gen PCIe etc etc.


Architectures (like Gemma4) have things like sliding window attention that make it pretty impractical to do on a GPU, you would have to do additional things to make it work, and the talent to do that, even with LLM assistance is very small.

Ah, that makes sense. I haven't bothered to look at current low-level NN architecture details. I'm a little surprised that there isn't a bumper crop of relatively well-informed talent to do gpgpu work. Although, when if I think about it, it has always been a little bit of a niche area.

I have experience as a coach in middle and high school in the US. I can say emphatically that school sports are relatively low impact in terms of time opportunity cost and individual family spending. The "club" or "travel" team private organizations are where the money and time really go.

I am also a parent of a kid who participated with a soccer club team for many years. She is a multi-sport athlete who loved several sports and we found the club experience in soccer actually quite good for her emotional and physical health. I will say that family experience with club sports varies based on the quality of the organization. She also played with her club in falls and her other sport in the spring at high school, so she got to do both. This was fun for her and us.

Assume a kid just wants to get better at some sport. I'm a player development nerd and believe most kids would be better served with 1-1 training with a much lower time investment that just joining a club or travel team. I do 1-1 sessions with players for 30 minutes 2x/week for 6-8 weeks in the fall. Players, of all natural athletic levels, will make about two years of progress compared to just participating on the spring team. That's with one whole hour per week of time invested. I love working with players this way because I am measurable-metric driven, so a kid can see their progress. I'm also a little bit qualitative about it too, because sometimes, the goal is just to build confidence and comfort.

The approach I'm describing looks like lots of hobbies. Imagine a kid wants to learn piano or guitar or drums. It's basically the same abstract approach. Work with an experienced teacher, commit to a regular+reasonable practice schedule, repeat for 6-8 weeks, and see if it fits and feels good.

The social aspect of being a good club or travel team is nuanced, but I will just say that my kid was incredibly well-served by having a good group of friends not connected to her school. And she had a great group of friends in school.


Running locally for me is mainly about learning, maintaining control+privacy, and helping shift my coding+design process to leverage LLMs. I guess if you made me boil it down to a single word to justify the cost I would just say: tuition.

Sounds like we have similar boxes - mine has a 10 core CPU, 64 GB of ram, and a 2070 Super. My motherboard had two unused PCIe3x8 slots and doesn't support Blackwell GPUs. I bought a couple of brand new Ada generation RTX 2000s with 16GB of memory for under $1400 to get to 40 GB of VRAM. That will easily run Qwen3.6-27b at a 6-bit quantization and 80,000 token context size. It isn't fast (19-21 t/s), but using pi-coding-agent is fine.

Now, my instinct is that I am giving up SOTA performance on agentic coding with this setup and LLM. But the gap between my setup and SOTA commercial models is small enough that it doesn't matter to me.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: