You're describing conditions that actually result from the opposite of unfettered capitalism. Housing supply is artificially constrained because of zoning laws, tax incentives, rent freezes, environmental reviews, and general opposition to anything new, allowing landlords to behave like this (which, as an aside, they seldom actually do). The solution is to eliminate restrictions on housing construction so supply can grow with demand.
Surely you're not talking about the AI industry. Astra was released less than 3 weeks ago, and Fable-level models became public only 6 months ago. The rate of change is dizzying.
I get the perspective from which you're making that statement, but the industry keeps moving its own goalposts.
Change is fast and abundant, and at the same time, it is hilariously more mundane than the dangerous-AGI-in-six-months tune we've been reading daily for years.
I would define it as a quick-moving market, but not nearly moving enough for the fantastic claims they make to justify ever-increasing funding.
AI, if not AGI, has certainly become uniquely dangerous in the past 6 months though. We have a lot of evidence to that effect.
The best case scenario is a situation like Y2K: a ton of people coordinate and work hard to produce no perceptible change, because unlike catastrophe, averting catastrophe feels boring.
>Change is fast and abundant, and at the same time, it is hilariously more mundane than the dangerous-AGI-in-six-months tune we've been reading daily for years.
Absolutely none of this points to “stagnant.” Stagnant is a terrible description of the AI industry.
And yet they have only improved marginally in my use cases since around Opus 4.5.
The harnesses have improved somewhat, but the code produced on large or legacy code bases is still very average and I still see similar mistakes made that I saw back a year ago (although less now that harnesses have become better at steering).
For my use cases, we are definitely on the flatter part of the curve at the moment.
Same experience here, anything frontier human knowledge wise, same if not a regression. For human understanding and emotional intelligence, for many tasks regressiin is so bad that many near anchient llama era models now beat frontier anthropic/oai models. Notable exceptions to capability rot seem to be qwen models, and previously deepseek but the latest gen of models has started showing the same rot. General writing quality is down significantly accross the board, often it is outright ass. For example, I didnt mind reading 4.5's outout, but opus 5 makes me goddamn near violent, its fucking insufferable.
I think this is more a meme than anything else, for a couple reasons:
First, after a while it's just as grating as Claudeish.
Second, my hunch is that it constricts the actual thinking of the LLM, like the same way that Newspeak does in 1984. It shrinks the range of thought that can be expressed if used as an input.
I think the real way to do it is to have another Claude entirely deal with the user as a liaison, but to keep the thinking in whatever format it came in.
Latent space reasoning, if you think about it, is exactly this to a crazy degree: why even formulate a thought as words if you can just keep it as matmuls until the user needs it? And then, if the user needs it, have it always specifically formulated for the user by another LLM rather than constrict its range of thought? Anyway, that's my take.
The fact that it might affect quality is something that is on my mind. In fact, I have mainly been using Astra after being primarily on Claude for several years exactly because of this annoyance. Ultimately, the model (at least for coding purposes) should just... not have this writing style.
I do think an infrastructure where another Claude retranslates the output would be better. Oftentimes I forget to put it in the actual prompt and when I receive back 8 paragraphs of Claudeish I ask for it then.
I would have to disagree that it gets as grating as Claudeish though. Its just direct and professional instead of ring-around-the-rosy clickbait.
The Claudeish drove me back to OpenAI and Codex as well. I find the Claudeish can be improved but it’s impossible to fully mitigate. Opus is unusable for anything I’m going to read.
“I would have to disagree that it gets as grating as Claudeish though.”
It’s hard to imagine anything more grating than Claudeish. To quote Rainer Wolfcastle, "My eyes! The goggles do nothing!"
> my hunch is that it constricts the actual thinking of the LLM
I've found that prompting any constraint on output (length, style, vocab, even simple formatting) not only places additional cognitive load on the model, which burns some of whatever cognitive budget is available, it will also often skew the output in other subtle and completely unrelated ways.
Since I found this artifact interesting, I did some pretty extensive experiments a couple months ago. The increased load is real, although it may not be apparent if you're not near any cognitive boundaries. The subtle skew, however, seems nearly ever-present regardless of load.
When I noticed web chat LLMs wouldn't number section headings correctly and consistently, I began experimenting with modifying the user prompt over a period of weeks. My usage at that time was research and learning not coding. In long, detailed sessions with branching sub-topics and deep follow-ups, I found it helpful if the LLM would number each major section and letter each sub-section in replies. Simple markdown formatting that most web chat LLMs do sometimes, but not consistently nor uniformly.
While extensive, my tests were just following my curiousity, not controlled, exhaustive or well-documented. I identified about a dozen prior sessions of varying length and complexity to test and downloaded them with a browser add-on. I then removed all other user prompt instructions except for the formatting instruction. A test would typically involve changing the wording of the formatting instruction ranging from brutally simple to detailed and complete, then starting a new session, seeding one of the test sessions and continuing it. To get a feel for baseline inter-session variation, I also tried running the exact same prompt/session multiple times back-to-back, at different times and on different days of the week.
Once I identified a promising prompt candidate, I'd make it the formatting instruction in my regular, daily-use prompt for a few days. I quickly got a feel for how seemingly minor user prompt variations impact response quality, compliance and tone across fresh sessions as well as those in various states of context rot, drift, decay and cliff (<--my nicknames for the distinct flavors of session degradation, not technical terms).
My overall conclusion was that every instruction, no matter how minor or unrelated it seems, has some, real impact on the model's cog load, attentional focus and/or attentional weight budget. Both how these impacts manifest and what causes more or less impact is often extremely counteriintuitive. To more fully understand this, I eventually, got to the point of testing null case variants, such as the entire user prompt being one sentence completely unrelated to text formatting or the session topic, like: "Don't reference the cartoon character SnagglePuss" (in a deep dive on ancient Sumerian clay tokens). Similarly, a simple one sentence prompt requesting something the model already always does naturally also has a cost (eg "Capitalize proper nouns"). As others have observed, heavy emphasis, absolute prohibitions or emotional weight in prompts also tend to have outsized impact in both skew (impacting unrelated output tone/style) and in accelerating session degradation. "Avoid referencing SnagglePuss when you can" would have equal compliance but fewer downside impacts than "NEVER reference the cartoon character SnagglePuss" in sessions starting to degrade.
There were also surprises, such as when I was scanning transcripts of an older, longer session and noticed the LLM was doing number formatting almost perfectly. On looking at the active user prompt at the time (I keep a log of every user prompt change I make for every model), it didn't even reference formatting at all. More experimentation showed it a result of the LLM gradually mirroring my consistent use of formatting structure in my prompts over a long session (in which I never mentioned anything about formatting). Unfortunately, that mirrored trait doesn't persist to new sessions and reaching that point requires a substantial number of rounds burning quite a bit of context window.
After spending time surfacing the impacts of just changing lightweight user prompts so they could be observed (which are the lowest priority prompts a model gets), I now wonder just how much more 'brilliant' the models we use daily would be if they didn't have dozens of pages high-priority manufacturer prohibition prompts we never even see weighing them down. We've only ever seen these frontier 'racehorses' when they're already pulling a heavy invisible wagon.
This is the same reason why I am a bit skeptical of LLM superintelligence. LLMs in the end have to operate in natural language concepts and the complexity of natural language is bounded by limits of human cognition. I'm sure super advanced AI could use concepts that humans not only have no words for, but might not be able to understand alltogether. As such if my thesis is correct, the only way forward for true superintelligence may be getting rid of natural language COTs.
> LLMs in the end have to operate in natural language concepts and the complexity of natural language is bounded by limits of human cognition.
I don't think this is true.
They have to express themselves as tokens. The meaning of those tokens doesn't have to be text. See any model that can handle images/video. Also, I don't think math, svg, etc, are "natural" language.
And, only the final expression is tokens. The intermediate layers, with the encoded concepts, aren't "natural language".
But, to address your concern (which nobody can disagree with, since even humans can't fully express through text/pictures), potentially: https://news.ycombinator.com/item?id=49758615
> The model isn't limited to concepts that can be expressed in natural language.
How do we know for sure? We don't even know how the emergent properties we see actually emerged?
For humans we know for sure that people sometimes have concepts that they have no word for (the reason the phrase "It's on the tip of my tongue" is a phrase, after all).
We don't know this for LLMs. When it makes new phrases, it's always a mixup of two existing words hyphenated (aside, that also seems to be the limits of SOTA models creativity - join two unrelated words together with a hyphen).
LLMs never respond with "It's on the tip of my tongue" type responses, indicating it has a concept but cannot remember (or does not have) a word for that concept. Every human, pre-speech-age, has managed to express or convey concepts that they had no word for.
So, no. I'd need a citation, preferably multiple, that did the trials and found that a model can generate concepts for which it does not have any words for.
1. Is natural language holding LLMs back by some %?
2. Is natural language serving as a hard gate that will prevent LLM intelligent progressing past some specific point?
The answer to 1 seems like an obvious yes to me.
Your thesis says the answer to 2 is "yes." That doesn't feel right to me. Think about all of the humans who have pushed various fields forward: Einstein, Newtown, Bach, whoever. If natural language doesn't prevent an entity from surpassing humans in one intellectual field, why would it prevent an entity from surpassing humans in all intellectual fields?
(To be clear, I'm not claiming superintelligence will or won't be achieved; I'm considering your specific thesis about whether or not natural language will be a hard gate)
Each letter has it's own token. LLMs can invent new words if they want to, just like humans do. All the fancy words we use today in science did not exist even 100 years ago. And that didn't stop us.
A few weeks ago two "Claude wrapper" projects showed up here within 24 hours of each other. Both seemed to just sand the details away--useful if Claude is too wordy, I suppose, but seemed to also lose real substance.
I agree there is probably some impact on output quality when doing this (Claude seems to embedded key signals all over the writing, which is much or what makes it horrible to read). However this does work well as a "second pass" over technical docs or designs along the way
Ask for output to the user in "simple technical English", let it think how it wants. It is hard to believe all the flowery prose is actually helpful to the model.
I know this is a joke, but I'd be careful about assuming: I had a trained therapist diagnose me with ADHD ... incorrectly.
It turns out, there are other conditions out there that can appear to be be ADHD. If a trained therapist can confuse them, outside observers certainly can too, so it is possible that "very ADHD-acting" person you know might not have it.
(Or, especially for females, the reverse can be true, as not everyone with ADHD manifests with the same trademark hyperactivity.)
It was originally called ADD.
Then they forced the H in there for hyperactivity.
And now they say there is non-hyperactive ADHD, but... they don't call it ADD.
So the first rule of getting attention for your paper is to re-define common terms...
The word you are looking for is Inattentive - when diagnosed they should state whether the diagnosis presents as Predominantly Inattentive, Predominantly Hyperactive/Impulsive, or Combination.
I understand this is your personal medical info, but could you drop any hints as to the nature of the misdiagnosis? I've learned late in life that I am neurospicy and learning from other's experiences in this domain has been quite helpful.
I listen to a lot of psychiatry podcasts, and they generally think that ADHD is being seriously overdiagnosed at this point.
Executive dysfunction, especially that which was not impairing as a child, has a lot of potential causes. Anxiety, depression (yes, ADHD can cause these, but these can also cause executive dysfunction in people without ADHD), OCD, Bipolar (manic people in particular often attribute worsening of hyperactivity and distractability to ADHD instead of mania), Schizophrenia pretty much universally is related to cognitive symptoms and lowered motivation even when not psychotic, and borderline personality disorder can cause changes in interests and identity.
For many of these conditions, stimulants can act as a band aid on something else. It can superficially make people feel better (of course, it's speed), but it can distract from other symptoms that are impairing functioning, and prevent people from seeking long term solutions.
People are way too happy when they find out that the treatment is taking speed every day for their whole life, as if this isn't going to take its toll on the body and that the brain is not going to adapt to this.
I can roll with that. I do have executive dysfunction and other "fun" facets of a funky neurobiology. I also recognize that every body and mind is different and it's a personal journey to find the right tools.
I recognize that non-medical options should always be front line approaches but there can be a chicken and egg effect as to making that happen.
In my youth I adhered to a "better living through chemistry (wink wink)", I do now too (rip Mitch Hedburg) but the wink wink part is very rare and disciplined and now it's all about pharma and supplements to wring what I can out of what I've got left.
On the non-medical options front, I will note that my lifelong hyperactive and neurotic tendencies dramatically improve with a 3-4 day a week distance running training regimen. I enjoy running enough, even though it is very difficult, that I've kept up with it for several years. I'm not as consistent about this, but I also found yoga classes to be a way to make mindfulness fun and easier.
That's not to say I'm not on the medication train (despite moaning about medications a lot). I do personally avoid taking any controlled substances. This is easy because I hate how stimulants make me feel, but in general I worry about the effects of having a direct reward signal from taking a pill.
Jumping on this conversation a bit: I was recently diagnosed and am still very much in the "what helps?" and "what parts of my personality/habits are connected to this and which aren't?" stage. Do either of you by any chance know a healthy online space to discuss this with others?
I found Finally Focused by James Greenblatt to be quite helpful. He makes sure key minerals and vitamins are in a good range before prescribing stimulants.
This sometimes solves the problem by itself, or it reduces the levels of stimulants needed.
I wish I could run. I recognize that exercise is a foundational facet of health all the way around. I'm "blessed" with enough comorbidities that make that a challenge but I am going to get a personal trainer to help me work through that.
Differentials for ADHD include: panic disorder, metabolic issues, anxiety disorder, depression, adrenal disorders including deficiency and excess like pheochromocytoma, bipolar, autism, learning disorders, personality disorders like borderline and antisocial, OCD, trauma including PTSD and CPTSD, sleep disorders including narcolepsy, sleep apnea, restless leg syndrome, insomnia, ironically substance use disorder like cocaine and amphetamine addiction, thyroid issues again ironically including both hypo and hyperthyroid, seizures, hearing and vision issues, and life stressors.
So, uh, ... a whole textbook. And I probably forgot some.
This comment is so funny to me, because I imagine you being distracted to make the comment before making it to the end where the psych did indeed review the new results.
Well I only know that because I skipped to the end. Then, while scrolling up I realised I missed the whole side-quest about promising to not cheat the test again, taking the test again, but then "accidentally" finding an exploit the system, and then cheating the test again, but this time cheating even more to see the results.
Then, scrolling up more I see that this whole thing started with seeing stars and then buying a telescope on amazon (probably at 3am).
And here I am commenting on HN instead of replying to an email...
Brother the test is quick, easy, and painless, and the results can be lifechanging.
The other thing is, ADHD in my mind is really a syndrome that starts with a symptom - inattention - that can be caused by so, so many things.
I have both, because of course I do, I had a medical problem that exacerbated my ADHD to a significant, significant degree (it's amazing I was able to be a software engineer for so long without treatment). So don't just tunnel vision in... as we tend to be prone to do. Get the whole workup with an experienced clinician, it's so lifechanging it's hard to describe.
I want to add that the comment is also funny to me because it and the blog post are very relatable to me. I don't mean to dismiss it at all. It's really important to get a good diagnosis for anything like this and not dismiss it or lock into the wrong diagnosis because of your own tunnel vision.
You wrote "the test is quick, easy, and painless" but I also want to note that it's not always. Depending on your country, parents, specific doctor, etc it can be very difficult to get a proper diagnosis and help. In some places a big criteria is diagnosis during childhood so if you're 30 or 40 you might have no chance. Or a chance but it depends on your parents retroactively answering a questionnaire, or finding an old report card with the right words.
As you said it's life changing to get the right help and I encourage everyone to pursue this.
The thing is, the test is super quick and easy. And then they throw 50 procedurally complex and tedious tasks at someone who can definitionally not do that, and they know that. Kafkaesque.
Btw no criticism implied on my end, I'm just now constitutionally incapable of not telling people they should get screened. Hmmm I wonder why I have difficulty resisting that urge...
I think I get your point. It's cruel to require Kafkaesque testing and that just makes it not worth it try. But too simple of a test can also misdiagnose.
The chance to be diagnosed with an appropriate gate varies greatly based on where you live and which doctor you can see and more. It can be anything from a 5 minute conversation, to 6 months of inconvenience, to no chance of even existing in your lifetime, to suck it up and don't ask because your career will end.
I think you can get 60 fps out of the old Paper S3. There was a gameboy emulator I saw recently that did it with some trickery I haven't reproduced on mine, but super impressive piece of hardware.
The second sentence of the page: "This also marks the first time we will open-source the weights of a Qwen-Max-class model — the open weights will be released next week."
Reddit has become a fairly worthless site because people just ask questions instead of taking anytime at all to figure out the answer. He just needed to wait for the page to load instead of adding noise to the discussion
