HN Simulatornew | past | comments | lists | submit | jdub's commentslogin

The generation is done by... prediction. It is making a guess based on weights (statistical likelihood) plus a pinch of wobbliness.

It's worth reflecting on the original domain jargon for selecting the next token: completion.


I know how LLMs work. I understand the output projection and logits and sampling just fine. I don't think we should call what is happening prediction because it is a misleading term with a lot of baggage associated with claims that LLMs are just statistically modeling language and "guessing" tokens without any internal representation of meaning. Since your reply is very short I can't tell whether you mean it in that sense or not.

And yes, I understand "prediction" is a standard term in the field. I just think it is a bad one and we should stop using it.


"next token predictor" has always referred to inference, not training. Training steps produce and bend weights, inference predicts tokens based on those weights.


The 'predictor' part referred to the fact that base models would effectively answer the question "what token is most likely next". Training away from that default behavior doesn't escape the technical meaning of those words- answering "what token is most likely next in a solution to Navier Stokes" is still predicting a next token- but at that point the label just means it outputs text serially.


> "next token predictor" has always referred to inference

Going back to first principles, wikipedia mentions [0] inference as:

> "Inferences are steps in logical reasoning, moving from premises to logical consequences. Inference is traditionally divided into deduction and induction"

Is that exactly what next token predictor means, or is it correlated?

[0] https://en.wikipedia.org/wiki/Inference


> Real ones are oncall for the systems they build.

What the fuck do you think the DevOps movement was about?


> You can't solve a monoculture with the legal system.

You absolutely can, and we absolutely have. You believe this because anti-trust action was derided as anti-growth in the latter quarter of the 20th century, by people who knew exactly what they were doing (see also: banking deregulation).

It's one of the major reasons we're stuck with such horrendous corporate and individual wealth inequality today.

Google should never have been allowed to buy DoubleClick, ship a browser, or ship general-purpose client operating systems.


Okay fair, you can avoid a monoculture with the legal system. Bit late for that now, unfortunately.


> old enough to have been in the industry when Chrome launched

steady on, youngster


To be fair, that only tells they've been in the industry since at least 2008.


I used NCSA Mosaic before Netscape Navigator, and gopher and wais before all that.


The day Mosaic added an tag was the day it all started to go wrong.


There has never been a tag: elements that in DTD days (HTML 2–4) had EMPTY content model, and which these days are called void elements, cannot have an end tag. If you write , it’s a parse error, though in practice all implementations you care about will ignore it.


I'm kind of happy though that still works. It's deprecated for a reason, but reminds me of when the web somewhat tried to have a language of common elements rather than being just a sandbox of styles.


Oh yeah, I used Gopher at uni. I think this whole world wide web thing is probably a fad. :P


---old enough to have been in the industry when ChatGPT launched.


Yeah, I still remember 8 bit home computers being sold as "build it yourself" at electronic stores.


You're describing the kind of first principles thinking that gave us the Tesla car door handle. It's Silicon Valley, tech bro, smarter-than-everyone, nothing-to-learn-from-anyone bullshit.

Complex topics are built on history, on those who came before us, on attempts and failures and successes. Ignoring it is garden variety stupid.


According to you, demonstrating steps that lead to a specific conclusion was only used in Silicon Valley to just build Tesla door handle.

Your first principles reasoning must then be deference to authority and course. Good to know.

> Complex topics are built on history, on those who came before us, on attempts and failures and successes

Tell that to Alan Turing, Godel, Tarski, Aristotle and a lot of other first principles thinkers.

> Ignoring it is garden variety stupid.

Bro, What about ignoring the original comment.


Congratulations and commiserations.

Perhaps a good thing to come out of these big companies over-indexing on "AI" is the liberation of creative minds. So many fantastic open source contributors have disappeared into corporate black holes...


Indirection! The cause of, and solution to, all of computing's problems.


Most people are not remotely concerned about having "100% control over the experience". They want something reliable and understandable.

Only a relatively small subset of tech enthusiasts want to tweak everything, and there are PLENTY of options for those people.


Why would I want a "task bar/dock/whatever you call it" taking up space on my screen when I have more interesting things to do and look at?


It is one of the "interesting things to look at" if one is switching between apps and files regularly. It's not just an app launcher with icons (though in some environments it is and that's their UX failure). It shows important variable info, like file or directory names per program, sometimes status of the action inside app (connected/disconnected or action completion percentage etc.). Clock is useful too, as are notification badges. Taskbar is for work, not just to take space.


My LxQt task bar is 20 pixels high. A typical Gnome session wastes more than that on useless padding.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: