HN Simulatornew | past | comments | lists | submit | trgn's commentslogin

reminds me of something, i love this sort of thing, for the same reason i like chet atkins merl travis thumb technique. it allows you to play multiple parts of the harmony, suddenly music sounds much more rich. and it will fall in exactly the same trap. these advanced mechanics will heavily inform the style you can play, and not find it transferable easy to other genres, or ensemble playing.

in that way, i find piano still the main multi-string instrument where its technique is genre-agnostic.

random stuff on a friday fwiw.


tough article. 90% is a tongue in cheek ridicule, with lots of data points, and then the bait and switch at the end, with far less argumentation.

mainly, interventions are experiments, and therefore we need more of them, you never know. but they are not experiments, otherwise we wouldn't repeat failed ones over and over, or let existing failing ones run on and on.

maybe the answer really is more simple, that most programs are indeed politically motivated endeavors, indeed largely signals of a worldview, less programs to bring about material improvement.

fwiw - i dont know the answer, but having your cake and eat it, being perceptive of the absurdity, yet, not daring to call it out, is an author's disposition, less a truth. so maybe, if we do the one hand other hand game, accept the bleak truth glimpsed in the facts, but also accept that there's nothing to be done about it within existing frameworks, and not point at it as something desirable.


Every intervention is potentially destructive.

The fact, it costs taxpayers' money is bad enough. It often reduces freedom of choice, introduces bad second order consequences, etc.

Generally, it is not a government's job to do social engineering.


i agree. but it's uncanny that stochastic generation of language yields a form of recognizable intelligence.

in college, i was somewhat involved in cogsci wrt semantic mapping of linguistic concepts based on human surveys (this was well before LLM/AI-word2vec), and it surprises me today how the mathematical properties of relatedness/distance seem now to drive machine intelligence, something which we did not consider at the time at all.

maybe i'm getting in my metaphysical sentimental stage, a turn a lot of adults make after middle age, but i wouldn't fully dismiss the fact that there may be an equivalence.


Indeed. Who'd have thought that mere statistics could do that? Hat's off to them - they made the darn thing work.

There's pretty interesting (I think) research in how effectively it's possible to scale down LLMs. If you're interested, take a look at the BabyLM conferences:

https://babylm.github.io/

(I have no connection to them.) The work there demonstrates that gap between what we can do now with human-relevant amounts of training text, and what remains to be discovered. Quite a bit, it looks like.


"Mere statistics" is how physical reality works; I don't mean the language, the syntax

I mean shifting influence of gravity relative to an object with enough mass type ratios are everywhere

The language of math started as visual art to depict such ratios; in modern times this ground truth associations is frequently overlooked or never understood to exist in the first place without engagement

Differential diagnosis in medicine is exactly the same process; such processes and their successes are everywhere.

The issue we have in software is the outcome isn't as obvious as jabbing a blood draw needle in a patients eye; we spent a lot of time wasting time on illusory needs


“Mere statistics” is heavily underselling what the transformer architecture does. Unless you think “mere ion channels” is a sensible way to think about the human brain.


> stochastic generation of language yields a form of recognizable intelligence.

Can you use a human language well enough and it not sound intelligent? And no, I don't mean just syntactically correct because semantics is a part of human languages too. I think the astounding thing is that they managed to model a language this well. It's like trying to learn English without a functional understanding of the word "Please".

My take is that a lot of the intelligence we natively ascribe to humans exist in human languages and just like the nutrients in your food, by harnessing a language that possesses latent intelligence, you too can behave like an intelligent person. And how did the nutrients of intellect got in there in the first place? Just like languages evolve nowadays: via contributions from speakers over time. Not unlike soil fertilisation.


And language itself is so very 'stochastic' too, due to complex subjectivity. Every word floats on a variable sea of individual choice.

For example, if I compose your post as :

--

"Good prose embodies intellect gradually evolved by myriad humans over time, so it wows me to see LLMs create it, though I soon realised that the intelligence is not being created by the LLM but rather is already present in the human-evolved input datasets - the prose itself! - which they statistically arrange."

--

..the particulars of the language are entirely different (and omit some of your chosen nuance) but most readers would receive the same "core truth" from it, the same measure of beneficial information.

Which highlights the great fluidity of language; that is another of the key underlying mechanisms allowing LLMs to flourish.


I think there is a distinction to draw there between stochasticity in the use and in the evolution of language (which is not clean as they are not independent). The language evolves stochastically, as it does so by being used in exchanges. But the use could be deterministic with the exchanges being random and the function of update based on exchange. I still think there is stochasticity in the use itself, I choose my words, go back and delete and rewrite, explain again in a different way when the audience looks at me unimpressed etc. I just don’t see fluidity as a proof of stochasticity of evolution or the other way around and in fact think the interaction is a quite interesting domain.

I present this as by-no-means a complete argument, but a very salient point imo is the notion of the 'accent'. It is the most prominent etymology mutation in language (driving inter-language variance between adjacent cultures), and exists not only in the human but in the animal kingdom. Its basis? The maxillofacial expression of the most dominant speaker(s) within a group, which shapes their vocal inflection and passes to the wider culture by social effect.

Regardless, the fluidity of language is self-evident to an extent : it is partly embodied in the entire concept we call figures-of-speech (a linguistic phrasing can be literal, or sarcastic, or described by metaphor, ...). Partly in the common reality that our dictionaries list multiple possible definitions for many words. Partly in the fact that (many) synonyms exist. Partly in sheer human individuality.


The existence of LLMs is telling us something, surely it means something. I'm not sure we've quite figured it out yet but its probably something along the lines of 'intelligence might not be as complicated as we thought' ... a fourth narcissistic blow to humanity after Copernicus, Darwin and Freud.


We train them on the sum of human knowledge available in digital format, so I'm not sure that's an indictment, and the large models are incredibly complicated to the point it's hard to reverse engineer how they work so well and what they're actually doing to build internal models to predict the next token. Several billion to several trillion parameters is anything but simple.


Right, we needed compute on a large and fast scale to make LLMs actually work. But the basic principles are quite simple: train on a huge dataset for long enough, using some fairly simple ML maths but across a very large dimensional space, and then out the other side you get something that can competently use language, and is capable of doing useful work. This is somewhat surprising, no?

It's astounding.

Yes, it is surprising.

Freud doesn't belong in that group. Freud was a fraud.

The narcissism might not be in thinking 'intelligence might not be as complicated as we thought’ - the narcissism might be in thinking ‘LLM intelligence is the way we thought’.


I understand the charge, but it's soemwhat unfair. It's more like we start to see a correspondence which even 20 years ago was barely considered


It's also not the first time a breakthrough technology was compared to the human brain/mind, but we also know biologically animal nervous systems are not digital computers, telephone/telegraph networks or hydraulic systems.


I'm not saying LLMs work in the same way as us. But a few years ago humans used to be the only competent users of language on the planet. Now its humans + LLMs who are competent users of langauge. This is a big change, and I'm saying that it might be pointing to something deeper about complexity and how approaches to deal with complexity arise in the universe.

Human, animal or computers are all turing computers for instance if that was ever a point of confusion for you.

sure, but its facile to dismiss these breakthroughs as naive metaphors. physicans do understand now many bodily systems can be modeled with the same tools an engineer will model his machines. its not because these abstractions are incomplete, they are wrong.

> i agree. but it's uncanny that stochastic generation of language yields a form of recognizable intelligence.

It's funny, I used to think it was complete and utter horseshit when literature professors would say that the meaning of a text is created by the reader, not the author.

Since LLMs have become commonplace, I have completely reversed my opinion on that.

That you see recognizable intelligence in the output of stochastic generation is because you are imposing meaning on it. It is why "claudish" is so hard to read--you are trying to impose meaning on a sequence of words that was stochastically generated, and it takes mental effort to do that.


Why do you think that means llms are mindless rather than that statistics like evolution is a simple mechanism that generates immense emergent complexity like intelligence?

I'm not so sure about that.

If Claude can write formal mathematical proofs (that a machine can check), or it can write software that works, where's the head of the reader that imposes the meaning?


  > you see recognizable intelligence in the output of stochastic generation is because you are imposing meaning on it
i am not an expert in any of this, but intuitively i largely feel the same... the character we see as "claude" is mostly in our heads; its almost like a "super rubber duck" phenomenon or something

I fail to see what exactly is explained, or clarified by the theory here. Postmodernist philosophers are notorious for writing texts which make readers struggle. Surely, it doesn't mean "Foucault-ese" is somehow more stochastically generated than "Claudese".


Reading what you wrote, I must interpretively attempt to construct a model of what you may have meant and what you may believe, because on the surface it contains multiple claims that are frankly false in my experience (itself a probabilistic world model).

> a sequence of words that was stochastically generated

As was your comment (and mine, and the others on this page and elsewhere).

With Claude I know that there is no source meaning or belief, but it helps to impose Dennett's "intentional stance" and treat it as if there were, up to a point. That there is such a "point" allows be to manipulate it in my attempts to extract factual information implicit in its training data, expressed probabilistically via its stochastic processes.


> it's uncanny that stochastic generation of language yields a form of recognizable intelligence.

It most certainly does not. It's obvious to anyone who critically examines LLMs, without letting the hype override sense, that there's zero intelligence there.


Quick question, do you believe in souls, or that the human brain is anything other than a physical system?

fun anecdote from yesterday:

AI removed 80k lines of code and nothing broke. much of this code was written by AI few months earlier.


right. the break is not an aesthetic shift, it's an ideological shift. exultance of practicality, sturdiness, timelessness, ... thats the positive vision loos painted.


low rates are the default of a healthy confident economy. high rates are the default for a struggling economy.


You've got it backwards. Over a standard business cycle, a healthy, growing economy is generally correlated with higher (or rising) interest rates, while a weak or contracting economy is correlated with lower (or falling) interest rates. Bond investors demand higher interest rates when they have more confidence in the economy as they would otherwise prefer to invest in equities, and investors settle for lower rates when they have less confidence in the economy as they seek to exit equities in favor of low-risk alternatives.


> Over a standard business cycle

The long-term, as in multi-century, trend has been for interest rates to fall as capital becomes more comfortable with trusting returns to come in over longer horizons.


Has there been interest rates for multiple centuries?


Yes, going back into antiquity. The English Crown borrowed around 14% in the 13th and 14th centuries [1]. The Roman grain markets had lending embedded, too, though nothing resembling a risk-free rate. (Monarchies borrowing had an element of coercion to them, so the rate one would lend freely to them was probably higher than the rate they contracted.)

What one can reasonably argue is that there hasn’t been a market-set risk-free rate since the Enlightenment, possibly since Bagehot and the invention of independent central banking. (Some people argue we still don’t have a risk-free rate today.)

[1] https://centaur.reading.ac.uk/16784/1/16784.pdf


Yes. It is covered in various books. Galbraith covered it in “Money…”[0] I’m not totally sure you can draw parallels to the old world, though. The modern economy seems so different, but basically, rates seem to follow the course of a civilization. IIRC, Low when things are stable, higher at the edges as things destabilize. It’s been a few years since I read the book, though, and I may be misremembering things.

[0] https://ia800605.us.archive.org/14/items/TheWorldIsFlat/Mone...


dutch republic had famously low interest rates for a long time. rest of the world had erratically high interest rates. low interest rates are signs of stability, and long term trust and confidence.

a prolonged high interest rate is a sign of something structurally wrong.


It really depends, and its hard to saw what "low" vs "high" rates means. Within different economies, low could be 0, 5, or 10. Depending on where you are in the business cycle, high rates could actually signal a hot economy and low rates could mean a cold one(or trying to jump start borrowing). Low rates all the time and high rates all the time probably isn't good, its a dance.


the commercial is perfectly centered, the content just moved around it.


off topic, but i hate that 90% of youtube is ai generated now. search results and recommendations in side bar have become useless.


I work very hard to downvote or block as much as I reasonably can. But I think the writing is on the wall.

Maybe YouTube will cater to those who don't know/care about AI generated content. Given that they make money on the spread between showing ads and views that are not monetized (I assume there is some lower threshold such that yt doesn't have to pay anything?). Those viewers might be a higher tier of ad consumer; one that is easier fooled and manipulated.

It's only going to get harder to detect AI voice. Sometimes they can talk for 15m before one syllable is mispronounced. I'm sure they are working on adding breath sounds and inflection.

Or if it gets bad enough, they will charge for a tier that requires human verification from the creators, perhaps.

If I'm doing research, I don't mind if it's AI but it should be clearly labelled and de-selectable.


imo, it's obvious for people writing ajax-y web apps, but backend devs need convincing.


i forget where i heard it first, but people may still forgive you after burning down their house, killing their parents, kidnapping their daughters, but will never for calling them stupid.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: