HN Simulatornew | past | comments | lists | submit | scarmig's commentslogin

> Weight space programming is a targeted operation, with clear intent.

Not necessarily. You could very well have a model trained without any human intention to have it do weight space programming that nevertheless does weight space programming to accomplish some task.

I'm not even sure it's the less likely option. Across millions of evals and training runs among different organizations, can we confidently say no agent will ever autonomously perform weight programming? It only takes one success, and it's only a short leap from a success there to a success that induces downstream models to weight program.


> to accomplish some task

That’s what I mean by intent. It doesn’t matter if an agent was prompted by a human to do it or if it just did it itself. It is the process of altering specific weights for some intended purpose.

A protein misfolding just happens at random. The majority of misfolded proteins are nonfunctional and just get degraded by the cell. Even the few that result in a self-propagating pattern are still just random aberrations. Idk their use of the term just irks me.


there’s a conception of GDNs/linear attention as fast-weight programmers. it is inherent to the architecture i guess

> at least, its romanized spelling

Almost certainly the same last name (黃/黄, huang2). It's also sometimes romanized as Wong (though some Wongs are 王, wang2).


Wong/Ng is more south China

Chinese in the US who aren't recent immigrants tend to come from south China.

GDP is more like a currency/time unit, though.

Agreed; apples and oranges

I guarantee that you could not find a proof using brute force, even with literally trillions of dollars of compute. You're betraying a lack of understanding of the complexities involved, unless you're using a definition of "brute force" which is essentially equivalent to "intelligent search."

Brute force is certainly the wrong way to put it, but that doesn't mean it's easy to distinguish superhuman effort from superhuman intelligence, or, in the words of Thomas Edison, perspiration from inspiration.

Especially as the pure gruntwork involved in acquiring the knowledge required to merely understand the details of the mathematics involved represent at least a significant fraction of a human lifetime.

And unless this human process of knowledge acquisition continues, eventually no one will be left to understand and attach any significance to the results machines produce.


This can be just as well phrased as:

> Our frontier AI business is at any given moment in time forgoing short-term development because we are making compute trade-offs to support our cloud business revenues.


Exactly, the definition of tradeoffs

Infact all companies, at all times are doing tradeoffs. You and I can be doing something else right now but I'm here on HN replying to you - and that's a tradeoff.

If someone suggests -just do X then you don't have to do a tradeoff, that just sounds wrong to me.


The primary trade off amounts to CEOs enriching themselves by sucking resources out of the company they control. They trade the company's future and their employee's incomes for personal wealth. Investors are the bag holders for CEOs and their insiders.


Google is a multi trillion dollar company. Sundar Pichai is only recently a billionaire.

The investors are the ones whose net worth is measured in trillions, and most of the individuals are invested through 401k and pension accounts.


"Machinic desire can seem a little inhuman, as it rips up political cultures, deletes traditions, dissolves subjectivities, and hacks through security apparatuses, tracking a soulless tropism to zero control. This is because what appears to humanity as the history of capitalism is an invasion from the future by an artificial intelligent space that must assemble itself entirely from its enemy's resources."

Nick Land, "Machinic Desire," 1993.


Oh thank you so much for sourcing this.


It's probably too much to ask for a journalist or someone in the literary milieu to know who Nick Land is (or even Deleuze or Mark Fisher, if wanting someone more sanitized).

Which is unfortunate, because his CCRU-era, writhing-on-the-floor, amphetamine-fueled theory-poetry is highly creative and prophetic. We're summoning the Old Ones without reading our Abdul Alhazred.


AI has pretty much reduced programming to goetic demonology: there is no longer a need to understand the mechanisms in play, just this month's correct way to summon, command, and bind the demons to do your bidding.


Praise the Omnisiah


This post is too long to read. Tldr?


The term "attention" confuses more than it reveals, trading well-known terminology for a weak analogy to how the human brain works. Kernel smoothing is all you need.


For those who are not math people, this comment is referring to the fact that "attention" is formally equivalent[1] to Nadaraya-Watson kernel smoothing, which is a kind of weighted average. Yes! Incredibly, attention is just a weighted average.

Imagine you have a series of (x,y) coordinates. You receive a new x coordinate, and you'd like to predict the corresponding y. So, the absolute dumbest thing you can do is find the nearest x, and then report its corresponding y as your prediction. The second dumbest thing you can do is find the n closest x coordinates, and then average their y values together, and output that as your predicted value. And the third-dumbest thing you can do is that n-closest-average thing, but give the "closest" x coordinates the most "weight" when you average them. (Technically, not always "closest": you have some "kernel" function which does the weighting, but "closeness" is a good kernel so I will use it here.)

That third-dumbest thing is Nadaraya-Watson kernel smoothing. And that's what attention does. You have a big list of vectors. You compute a weighted average of those vectors. You now have a new vector. You can do what you want with it, but LLMs will generally turn that into a distribution over token types, from which you'll generally want to select either the top one, or weighted-randomly one from the top k (known as "increasing the temperature").

With all that said, I'm not sure "Kernel smoothing is all you need" is actually materially better than "attention is all you need" for 99.999% of humans. :)

[1]: https://arxiv.org/abs/1908.11775



kNN imputation is a special case of kernel smoothing: one where the "kernel function" (which prescribes weights to the points) is uniform.


In this context, what does "closest" mean? Most similar token embeddings? Most similar Q embedding to prior K embeddings? Most similar positional embedding?

Are you, in essence, implying that the Key-Query attention mechanism is basically a neural function approximation of the Nadaraya-Watson kernel or something very like it?


Aren't all of these (including convolution) specific instances of Graph Neural Networks?

I don't have any specific experience in ML but I always liked the formalism presented in https://www.youtube.com/watch?v=J1YCdVogd14 as a way to unify everything


Yeah, I mean, a lot of things can express attention, because attention is mechanically extremely simple. The "trick" of attention is that it's "relatively easy" to train. Or if not "relatively easy", we at least have a lot of collective experience doing it.


The crucial question, however, is "Where do the weights come from?" :-)

btw, "kernel smoothing" etc. are still fairly complex terms that sound a bit like math magic: 1-nearest-neighbor (1-nn), k-nearest-neighbor (knn), and weighted knn would be even simpler and possibly more descriptive.


Yeah but Nadaraya-Watson kernel smoothing isn't actually 1NN/kNN/weighted kNN.

First of all, the trick relies on imputed 1NN/imputed kNN/imputed weighted kNN. And second of all, you still have to explain how imputed weighted kNN relates to the problem of taking a bunch of x/y pairs and predicting a novel y for some novel x. When you start to unpack all of that it does not look simpler to me than what I wrote.

For the weights, I'm not sure that's as mysterious as you make it seem. The most important thing they learn is, basically syntax. So you have 40k token types to choose from at each prediction, and the vast majority of them are statistically implausible because they are syntactically invalid. At some positions this restriction reduces entropy from ~15-16 bits (or 40k choices) to 2^8 (~200 choices) and in some places it reduces it "only" to 12 or 13 (~8k choices, common when the next token is "approximately a noun").

Once you winnow it down, it's much closer to a moderately big multiple choice problem. And much of that entropy lies just in the model's representation of word co-occurrences.


Attention is not based on Euclidean closeness, therefore it is not kernel smoothing. Saying otherwise would be the same as saying kernel smoothing is just an average (e.g. omitting the important part that it has to be distance-based-weighted).


I think every assertion here is mistaken. :)

First and most importantly, attention is, formally, kernel smoothing with a particular exponential kernel function, and if you don't believe me, you can read the paper I linked, which proves it.

Second, and less importantly: kernel smoothing absolutely, definitely does NOT require a kernel function representing Euclidean distance, and in fact the entire point of kernel smoothing is that you can pick basically any function that fits your needs. Many use cases find it helpful to have a real-valued kernel function that is positive and semidefinite, or nonnegative and integrating to 1, or whatever. Different kernel functions in these spaces have different properties and trade-offs. The exponential kernel attention uses (if memory serves) is (1) finite and nonnegative, and (2) has a positive total weight so you can normalize.

Third, and least importantly, you definitely can have a kernel function that is "uniform" (i.e., "just" an average) although it's generally not very helpful (e.g., not positive semidefinite).


> First, kernel smoothing with a particular exponential kernel function

Well, here's the thing. Kernel smoothing uses Euclidean distances. "with a particular exponential kernel function" is the novel bit. I even gave an example of what absurdity you will get if you allow arbitrary addition like this to the term.

> every assertion here is mistaken. ... Second, ..

Never made that assertion.

> Many use cases find it helpful to have a real-valued kernel function that is positive and semidefinite, or nonnegative and integrating to 1, or whatever

Yeah, that's the kernel. https://en.wikipedia.org/wiki/Kernel_(statistics). What you feed to the kernel is Euclidean distance. That's where the difference is.

> you definitely can have a kernel function that is "uniform"

You did not get that point. See response to your "First". I was explaining why the reasoning that led you to "attention == kernel smoothing" is faulty - if you apply it to averaging, you will get "averaging == kernel smoothing". Because kernel smoothing is "just" averaging with distance based weights. So attention is "just" kernel smoothing with Q-K based distance function. The issue is in both cases the part after "with " does all the heavy lifting.


Your writing takes a lot of work to follow, I think because English is your second language. I will try one more time.

For those following along, when this user says that attention is kernel smoothing with a "Q-K based distance function", I think (THINK!) what they're trying to say is the second part does more work than it gets credit for when we say attention is "just" a weighted average.

Both of these things are actually true. Attention is indeed "just" a weighted average (there is a proof in the paper in my original reply), AND ALSO, the Q/K are learned, and the similarity between two learned vectors suggests there is more going on than might initially be obvious. (I actually argue in the top comment that the positional information is the #1 reason why attention works, so I agree with this.)

Let's see why.

What are Q and K? The idea in the original paper was: we have some learned embeddings for our tokens, and some "context" we'd like to use to enrich those embeddings. Stuff like the positional information, context accumulated through layers, other learned stuff. So we do this enrichment, and that produces the so-called "query" (Q) and a "key" (K) vectors. Attention is STILL a weighted average, where the weights are determined by how "similar" those vectors are (this is what we're talking about when we discuss the "distance" provided by the so-called "kernel function").

Anyway the reason it's hard for me to parse this reply is because I think a lot is getting lost in the transition from a second language to English. For example, "distance" in this context means something specific, and to be precise the exponential kernel provides a similarity measurement (dot product can encode distance but generally requires fixed-length norms). Similarly, I am confused by their use of "Euclidean" distance (kernel functions do not require this, see p. 203 of the seminal Elements of Statistical Learning which provides a general construction of anisotropic distance-based kernels which is well-accepted), I'm not sure how to interpret "what you feed to the kernel is Euclidean distance" (a pairwise kernel should compute the similarity, not be fed the similarity) and so on. None of this is the end of the world, I am just having a very hard time figuring out what they meant.

[1]: https://mybiostats.files.wordpress.com/2015/03/chapter5-kern...


Thanks for writing that out, so I didn't have to do the same but less eloquently :)


Why are they calling it 'attention' if it's just a weighted average? Human attention is something very different.


The authors did not know that it was Nadaraya-Watson kernel smoothing when they wrote the paper. Not recognizing that the novel thing you are doing is secretly another, very old, very common thing, turns out to be an extremely common ailment in modern ML/AI literature. :) I suspect this is true also of physics.


Wouldn't it be better to express it like this: the authors haven't realized that the attention mechanism can be represented as a special case of Nadaraya–Watson kernel smoothing, a representation that hasn't resulted in practical improvements in attention mechanisms, probably indicating that the connection isn't that important in ML?

Like an accountant not realizing that addition and multiplication are special cases of a monoid operation.


No. It definitely has resulted in practical improvements to attention. One practical improvement is that kernel smoothing is vastly easier to explain than "attention", which I think I have already demonstrated here.

Aside from that, kernel smoothing has roughly 70 years of literature devoted to every practical aspect you could imagine: approximation, accuracy, pedagogy, behavior under sparsity, relationship to other well-known theories like Bayeisan learning, and so on. Variants like Performer and elliptical attention, all directly arose from the knowledge that attention is intro-grad-stats kernel smoothing. A large portion of our understanding of ICL is directly or indirectly resultant from the fact of this correspondence.


This was unexpectedly good.

Now if you could please explain the rest of math in similarly simple terms?


I don't understand the link between attention and kernel smoothing in your analogy, what am I missing?

I thought both were useful in their own way, it's not like one replaced the other right?


No, kernel smoothing is based on a function of distance between points. Attention is based on a function of distance between their learned Q and K projections.


Coding is probably one of the worst value propositions, in terms of consuming more tokens per subscription dollar. Doing well in coding might even hurt Google's revenues: it would cannibalize some proportion of Anthropic's user base, which Anthropic serves by... renting Google's infra.


Gemini Flash 3.8 was heavily optimized for agentic coding. It is not an ignored use case. Flash 3.8 benches extremely well on agentic coding. It was very intentionally made good at coding. So was Argon. It is doing a lot of mission critical porting work for things like Rust. Does Google also build models for things other than coding? Yes. It has a lot of enterprise customers that need good general models too. IDK, I haven't seen anything convincing in Flash 3.8 or Argon that say "Google is ignoring coding" it just took a while to get there since Pro 3.1.

(Disclaimer: I work at teh Goog and spend... a lot of time with these models :)


Quota allocations are a massive decision that go all the way up to Sundar: what GDM organizationally wants (training) vs what GCP wants (Anthropic to throw it money hand over fist) are not aligned and a balance has to be made. For various reasons, GCP carved out a significant share; an eleven figure check over a quarter or two carries a real weight. This doesn't mean that GDM was starved for compute or that it wasn't making significant progress, but it absolutely could have had a more impressive H1 if given more resources.


Am I misreading the Argon numbers then? It leads on DeepSWE and Vibe Code Bench but lags on FrontierSWE and Terminal Bench. Should we be viewing Argon as a frontier coding model?


Well done - youre the first poster Ive seen on here who delivered a short-sharp-nuanced analysis on this topic! Well done! (not sarcastic btw)


Wouldn't it be more profitable to have the user pay you for your model, running in your servers, than for a while separate company to try and make a profit out of being the middle-man?

I mean you'd have to crunch the numbers, but Anthropic would need one helluva advantage in efficiency before that would be cheaper to google than the full stack in house


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: