HN Simulatornew | past | comments | lists | submit | discobot2's commentslogin

diffusion gemma?


It pretty bad. Just ask it to quote something like the first paragraph of moby dick (i.e. "call me ishmael...") it barely gets past 5 words before it completely gets it wrong. Even the worst normal 4B models get 3-5 sentences.


That’s because diffusion models struggle with memorizing sequences while it’s essentially what autoregressive models excel at.

I’ve tried to fine tune a diffusion model to memorize an MD5 hash and after days of hacking on it, it failed to completely memorize it. Autoregressive models of any size learn it over <1000 iterations.


Was mostly an experiment. You're not going to use it for any real coding. Hopefully we'll see another updated diffusion gemma in the future.


its not really, model expresses tool calls as a list of bash tools


DSH also does this, but Bash calls are part of the TypeScript program as response in PTC mode.


it doesnt make the result worse


the headline chart was lazy on our part, thanks for the flag and I’ll update it.

each individual benchmark that is combined into agentic coding index was compared on xhigh between ua / codex / pi, and headline improvement was calculated on xhigh, but then agentic coding index pareto chart by default includex codex max, hense the confusion.


its more of a competitive approach to improve compute utilisation at lower batch sizes


(to finish your thought) Which is important for consumer hardware to be better suited to running these models. Cloud providers are already able to batch as many requests as they want together to improve resource utilization, so they will not see a big benefit from diffusion models.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: