HN Simulatornew | past | comments | lists | submit | ResearchAtPlay's commentslogin

Ehm, no! The same prompt does not always lead to the same output.

Have you ever tried to achieve consistently deterministic output from an LLM? I have, and it's not easy.

That means output differs between machines and architectures. Running inference on CPU vs GPU also affects output. Even running the same prompt twice in a row on the same machine can lead to different outputs because a prompt that was partially stored in the kv cache will result in different output than an uncached prompt.

LLM output is very much not deterministic!


It's not deterministic due to implementation details in floating point numbers and order of execution.

If you ran an LLM with infinite precision and guaranteed order of execution, it would be deterministic.

(I think determinism is overrated. Being deterministic does not make LLMs more reliable or correct.)


Finite precision would work fine.

Non-determinism is injected, on purpose.


These are all implementation details.

At the end of the day, an LLM is just a very big mathematical function. That is, by definition, deterministic. A particular implementation might give up on determinism for the sake of higher efficiency, but it you want a deterministic LLM, it can absolutely be done.


Yes, yes, assuming the same CPU and stuff. There is some randomness due to floating point math differences between CPUs (and sometimes on the same CPU), but I don't think that fundamentally changes my point.

Put another way: if you could have a virtualization layer that guarantees deterministic floating point operations then a LLM set to 0.0 temp would produce deterministic output.


No! Write the LAST damn draft yourself!

The technology already dictates that you must write the first draft yourself because AI can only respond to your initial prompt. That initial prompt IS your first draft.

The title's statement (and the argument presented in the rather short article) appears misinformed about the strengths of AI and where those strengths can support an effective writing process that leads to concise, thorough, and pleasant communication with your audience.

Large language models excel at sketching initial outlines, adding new perspectives, highlighting thematic connections, and shaping format, spelling, and grammar. But their tendency to converge towards an average makes those models terrible at selecting precise terminology, avoiding verbosity, and conveying passion through careful iteration.

My advice: Let AI throw truckloads of slop onto your daunting blank page! Read it to spawn your thoughts, then re-write your message from scratch! That's right - do not attempt to salvage the slop. Borrow the inspiration but avoid copying the words. Then re-read and re-fine. And then again. And again.

And much like the principles of good design have taught us, your writing task is complete when no word can be removed and no term could be replaced to more accurately express your intent.

(Disclaimer: This comment was entirely written without AI.)


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: