385x? Them's rookie numbers. Classical energies learn to reason 1000x faster than looped models. [1] Of course, all for a very generous definition of reasoning, that has almost nothing in common with the thing most people associate with the term nowadays. [2]
The common definition of "reasoning" for all three is "performing a sequence of intermediate steps with multiple passes through the model before outputting an answer." The main difference is that the reasoning models in TFA and [1] are puzzle-solving models, not language models.
I may be misunderstanding what you wrote, but I think that says more about your imagination than about software. Yes, medics do improvise a lot to keep patients alive when the power is out, or when essential tools or medicines are not available. "Well, the power failed, so did the sustainable backup, let's everyone go home, today's ICU is a writeoff".
Yeah, we remember. And thanks to the attitude displayed in your comment, these complaints were not taken seriously. until these specific Italians ended up with control of massive illegitimate industries, gambling, and loan sharking, ran protection rackets that victimised and intimidated thousands of legitimate small businesses, and murdered hundreds of innocents.
There was always the option to just not let them in in the first place. Many people at the time (HP Lovecraft being an example) thought that Ellis-Island migration into the United States was an existential mistake.
>but has been allowed to fester b/c the Mob was (so far) worse.
It's been allowed to fester because the late 20th century was (so far) the high water mark for societal tolerance of that sort of thing and it hasn't been abused enough since to really see much legal scrutiny.
This is a rehash of an age-old complaint: "I shouldn't have to pay taxes for things I don't personally want or need". A lot of people made that argument, and it does not seem to resonate with the voters. You're naively hoping that AI has finally made it compelling. I wouldn't hold my breath.
You may even be right that a lot of academic prose is suddenly cheap. But just think about the hardest cases of self-justification in academia: classics or literature. These don't promise your hypothetical plumber anything. It's not like he'll ever read a classics journal. People have made your exact arguments against them for half a century, and these departments are still funded. Heck, when "I'd rather give my money to Hollywood" couldn't even defund film studies, "the AI can write a passable essay" has little hope of defunding academia in general.
You could hire a PhD student or tutor to write you an essay or answer your questioms way before Fable 5.1, but hypothetical plumbers cared little either way, so the public commitment to university funding is not about whether plumbers can get the outputs somewhere else.
The government funds academia for the institutions themselves, and the voters who depend on the institutions are not fringe. In many US states the university system is among the largest employers, a huge share of voters are alumni or parents of students, or business owners who depend on selling food on campus, etc. Their stake is high, they're willing to vote on it: the hypothetical plumer's stake is a few marginal dollars which he won't even get since the budget will allocate it elsewhere, on some other service he probably doesn't use. And he doesn't read journal content eiter way. So you'll keep paying your fair share for it, because it's up to the voters, and they care little for what you personally want.
If mainstream corporate America thought it would turn increasingly cheap intelligence into higher revenue growth and higher profit margins, then mainstream corporate America needs to go back to school.
If you want to increase your profit margins, you'll have to innovate! Adopting AI tools that somebody else created, and are equally available to your competitors, won't increase your revenue or margins. The ones who will see increased revenue and profit margins are the ones who actually innovated, by creating the AI.
This is exactly what Apollo reports: the Magnificent Seven increased their profit margins, while mainstream corporate America didn't.
> The only way more efficient operations doesn't make margins go up is if prices go down.
But in general, it is not true that more efficient operations + unchanged margins = observable lower prices. Especially when the mere existence of a technology can make non-adopters' processes overall less efficient. For example, if there is a RAM shortage, and storage becomes more expensive, then SaaS subscription prices will increase, even if all vendors adopt AI to make their operations more efficient.
Cheaper per line of code, maybe. But now you’ve got a billion more lines of code, compared to when you previously only had one line of code.
So now you’ve got to maintain and manage all those new lines of code, which becomes a huge O&M budget problem. And it’s the wrong color of money.
And all that money you think you saved now has to get paid out the other end — with oodles of interest.
This is going to be a very expensive lesson for companies to learn. You can’t just vibe code your way into the future in any kind of a sustainable way.
Not really. Like, yes, relative to your competitors. But, if the cost of growing potatoes is ten cents per potato, and then I make something that can produce potatoes for one cent then I'd expect everyone to just have more potatoes. This would drive down the cost of potatoes, because the supply would be saturated eventually, but AI is supposed to just make more of everything. So, if AI works as advertised, then you'd probably expect a brief period of highly increased profit margins (by the consumers of AI) followed by rapid _deflation_ as the value of everyone's dollar increases exponentially. Of course, neither of those things is going to happen, because the AI sales pitch is actually a pyramid scheme, so you get the people at the top raking in cash, while everyone else tries to sucker the next tier of buyers so that they can make their buck.
I think the problem is moreso that making useful things for enterprise with AI right now is actually pretty hard because context and attention management is hard, the token cost for complex tasks scales fast, and model's lack of intuition for fuzzy decision-making.
There are hordes of grifters promising things they can't deliver and reaping in millions in capital from investors who are equally confused about the challenges of designing useful AI.
That said, AI can do useful things when properly designed. The market finding the people who can make this happen won't happen as fast as people expect.
Yea no. Corporate America needs to go to school to learn that their monopoly supplier will dis-intermediate and squeeze them, which Chinese companies will be able to access abundant extremely cheap AI at marginal cost
Well, I don't think we disagree. It might turn into a Bertrand market instead of a Cournot one in China. That just means the end consumers capture the benefits, though.
PySR itself is still alright, mostly because of previous development. But not for long. Quality control has collapsed, and e.g. the examples in the repo are now embarrassing vibeslop.
Pangram reports 100% AI on most comment blurbs in the examples directory. That in itself is not the problem, AI-generated code could still be valid. But in this case, the AI-generated examples have the characteristic shape of slop: plausible sounding technical jargon, combined with zero actual understanding of either the target domain, or even evolutionary search (loss function design). They were vibed by people who had no idea what they were doing.
I suggest taking a look at the "evolved PRNG" example [1].
First, its loss function is basically designed to reward discovering linear feedback shift registers (LFSRs) only. It deliberately gives a massively lower loss to LFSRs than most other functions.
The AI slop phrased this in terms of "certifying the period", but the period certificate used only identifies LFSRs of max period, is unsound on the rest of the search space, and gives a meaningless very high loss to most non-LFSRs. So the regression will almost always just return an LFSR.
Whoever designed the example (Codex I suppose) also knew that most LFSRs are defective, and make poor random number generators. Thus, they included a "shear" penalty to detect such defective LFSRs, but of course they slopped it together.
The 360 correlations they measure should have been added together (having just one strong correlation is bad, and having two strong correlations is twice as bad as havin one). Instead, they take the MEAN of the correlations, so a perfect correlation contributes only 1/360=0.00278 to the loss.
And indeed, the result of their example run, reported in the docs [2] has such a perfect correlation: bit 29 is copied to bit 31 every two iterations.
I would personally hold this opinion of anyone still using genetic/evolutionary inspired algorithms for symbolic regression. There are multiple deterministic algorithms that do not have the same inherent drawbacks and are generally quite good.
Before I wrote my algo, and was trying to scale out GP, I hit the limits of RNG and had to go find some with seriously huge cycle properties. (howany calls before they reached the end and started over producing the same sequence)
And took a stab at vibe coding with a tiny LLM to port it to JAX: https://github.com/verdverm/pge-jax I'll put some more effort (tokens) into this and get it added to SRBench.
Fundamentally, most of GP based SR research fights issues with the algorithm (loss of diversity, equation bloat, premature convergence). I have not seen the field really adopt a new mentality in the 13 years since, not that I have been paying close attention, but I do look at the papers in GECCO GP track from time to time.
> Why have we not seen an improvements in products? [..] Is everyone just running full speed in circles or something?
The simplest explanation is that they don't give a flying flamingo about what you or I consider "improvements to products".
This report is an example.
There are several changes that modify CI behaviour, where the article gives no corresponding quality measurement.
They replaced type aware custom lint rules with AST-only static analysis. They don't say anything about what those new rules detect, didn't do old-vs-new rule comparison. They switched the TypeScript check from tsc to tsgo. Again, they are very proud of the performance improvement, but don't seem to care about diagnostic equivalence. The list goes on. They don't even report pass/fail agreement between the old and the new CI. They have 4x more tests, but no idea whether this big test suite works any better than the smaller old one, or even whether it works at all.
Are we holding it wrong? Mine does the same godawful external library tests. When it is having a particularly stupid day it also tests language features. For example, checking that using a callback executes the code supplied in the callback (it does).
Yeah, that's what not so many are talking about but a few have pointed out; it's the AI writing wayyyyy too many tests causing the new CI load.
Unless you prompt them otherwise, the models tend to write WAY too many useless tests and in the most inefficient ways imaginable. This balloons test counts and lines of code to insane levels and frankly, likely, slowly makes it more and more costly for the AI to make future changes.. To the point it can't wrap its context around the code base and effectively make necessary changes.
Why not? If you let the LLM run amok, you get 4 one line functions calling each other instead of one 4 line function. Run that for 24 hours and you'll get more than 2000 tests.
> imagine yourself living in the 1700s. how would you justify Newton and Leibniz's work on calculus?
They didn't have to, as there were no state grants or public research funds for mathematics during the 17th century.
Newton supported himself from his inheritance throughout the Great Plague, while he invented calculus, and then from flat salaries as teacher, then flat salaries from working at the London Mint. He later became fabulously wealthy after his appointment as Master of the Mint.
Not so relevant to this discussion, since if you are funded by patronage networks, you need to justify your work to your patron. If you are funded by grants or public research funds, you need to justify your work to the general public.
They did that... I take it you don't accept their justification, but that doesn't mean an attempt wasn't made.
I'm another part of the general public, and I accept their justification.
We can all vote, or use whatever levers our political system gives us to show our acceptance/rejection of basic research, but IMO basic research is absolutely necessary.
[1] https://arxiv.org/abs/2611.17309
[2] https://developers.openai.com/api/docs/guides/reasoning?api-...
reply