Right now, I'm having LLMs audit the actual math in claimed arXiv solutions because despite its policy changes, arXiv is still a dumping ground. The audits have already found six faulty proofs that caused status issues for problems that should still clearly be fully open.
This stuff? A spanish speaking critic of AI safety says, "Hey guys, it's really not so dangerous." What did you mean by "this stuff" precisely? Is it like "you people" perhaps?
To me it seems the opposite. There's a few companies in the world that have enough compute to train and serve frontier models.
As the frontier gets smarter and more useful prices will only go up, as they are set to replace jobs being paid six or seven figures a year - the demand for as much inference on these models for as long as possible will be astronomical, but compute starting in 2030 will not be keeping up.
Eventually prices will fall for assistants but the frontier will be the most profitable thing in the world, and the top companies basically already have oligopolies due to their ridiculously expensive compute investments.
> There's a few companies in the world that have enough compute to train and serve frontier models.
Train: yes, for now.
Host: depends on the scale. At a small scale a wealthy individual could easily build a rig in their basement to host one of these things. At larger scale any cloud company could do it, and many already have the compute on site. At large scale this is true... again, for now.
What you say only holds (in the absence of a state oligopoly) if two conditions are met: (1) AI performance does not asymptote any time soon due to running out of training data or other scaling limitations, and (2) these companies are able to stay at the frontier.
There's little to no moat, so staying at the frontier will be a game of investing massively in compute, talent, and R&D, and they can never stop.
Again, there is a moat based on compute. If the thesis is right, cost of compute will only rise... As it is as you say someone will have it be quite wealthy to host something like Astra with trillions of parameters, but that cost will only rise with demand for serving these frontier models.
That hasn’t been true for anything else in computing, ever. The cost falls with scale.
Look at general compute, storage, graphics, networks, anything. Demand increases. Price goes down. There are price spikes, such as right now with RAM, but they’re transient. The trend is more faster cheaper and it won’t end until we hit actual physical limits. We are not close.
Yes, of course Eventually prices will go down... But there will be a period where things are much more expensive, based on simple inability to make much more gpus than the current rollouts demand.
And after that period when costs are going up due to gpus replacing workers... All bets are off on terms of what the economy even looks like
what suggests that we will hit an asymptote any time soon? Agree with you on the second part. The ever elusive frontier will probably always be changing hands after some point.
I don't see how this will survive the attacker/defender gap as ls get increasingly good at cyber security and finding 0 days... but maybe it's an obscure enough is it doesn't matter?
Probably because pop_os and Cosmic are so niche and their market share so insignificant, that they're irrelevant to attackers and bad actors, when those now have much bigger fish to fry to make their time and effort worth it. See the Arch AUR attacks, for perspective.
I think even amongst the HN and Linux userbase, pop_os is still niche, let alone amongst normies who never heard about Linux. So they can afford take the high road and treat it like their personal sandbox, accepting only human written code.
But larger and more important projects like Fedora and Debian are more pragmatic with the fact that they'll have to accept AI written(but human reviewed) code, if they wish to keep up with the real world development and threats, as expressed by Linus Torvalds himself.
The thing is, the cat's out of the bag on this one now, especially in the field of pen-testing and reverse-engineering. AI can brute-force its way into projects in ways that beat even experienced researchers, so your only choice to keep up is to accept the use of AI generated fixes as a counter defense.
You only need one bad actor. For example, someone reading this thread could easily decide to start attacking it just because someone else said it wasn't worth it, as a personal challenge.
If that's your threat model then you shouldn't use any SW in exitance, FOSS or otherwise. In fact you shouldn't even go online, or even outside you own house, since one single bad actors exist everywhere. You can walk down the street and suddenly someone in a car runs you over(witnessed myself). And yet live goes on.
It doesn't have to be niche (or not) to have bad actors. That doesn't make it likely of course but it is possible, and increasingly more so in the age of AI when you can simply point it at multiple projects in parallel.
I never said they should. I think you're reading too much into my comment, the point was something being niche doesn't prevent it from being threatened.
> Probably because pop_os and Cosmic are so niche and their market share so insignificant
Consider that it's packaged for many well-known distros, so pop_os install base alone doesn't tell the whole story: https://system76.com/cosmic/download
So then COSMIC isn't even TOP 5, for this to be a major target by market share as originally claimed.
2) Secondly, keep in mind that distrowatch is not representative of linux userbase. MX-Linux kept showing up at the top spot for many years despite being niche.
3) And thirdly, TOP 5 DE isn't really an achievement when Linux DE market share is overwhelmingly dominated by KDE Plasma and Gnome as the majority shareholders, with XFCE and Cinnamon trailing. So Cosmic DE if it somehow made the no. 5 spot, would still be ignorable sub <1% market share, as I initially claimed, basically invisible to bad actors.
To reflect some sentiment from another comment you posted - we're not your unpaid tone auditors. Learn to have normal discussions.
For funsies, I threw two different DE-by-marketshare inquiries at Gemini 3.8 Pro in separate sessions and it ranked Cosmic 12th in the first and 7th in the second. Very trustworthy stuff.
Please show me what part of what I said before was not "a normal discussions" according to you?
>it ranked Cosmic 12th in the first and 7th in the second. Very trustworthy stuff.
And that disproves me how exactly? I originally showed "Cosmic is NOT a TOP5 DE", and the LLM data you posted also shows that IT IS indeed NOT a top 5 DE.
"The analytics", is the publicly available training dataset of the LLMs that share the same common opinion on that DE market share.
What do you expect exactly? Do you want me to now manually parse through terabytes of information at your whim for your own convenience? Sorry, but I'm not your personal unpaid servant.
If you wish to disprove me in the comment section, then you need to do the manual work and show us that the my quoted LLMs statistics are wrong. I'm not your personal errand boy to do your bidding, 'massa'.
Point at the public dataset then if it's so public. You really have no idea how the LLM is parsing the data and could easily be hallucinating. That you find being called out on this as some sort of insult is quite telling and frankly funny, what a strong reaction to someone asking for a basic source when the burden of proof is on you to prove (or at least show a source that) these stats are right, not on anyone else to disprove AI bullshit.
Sorry, it's not my job to provide for you the things that you demand from me at your whims, because the original comment I replied to with LLM data also did not bring peer-reviewable information as proof that COSMIC is a TOP 5 DE, and yet you did not demand proof from them in that case. Why is that?
Did you just blindly trust the opinions of others on this topic, and yet I'm the one who has to provide peer-reviewable data for you to back-up mine? Sorry, I'm not your unpaid lackey. Try to formulate a better (counter)argument for why their baseless opinion is right, but my LLM backed up opinion is wrong, if you wish for a even-footed good-faith argument.
I ask them for the same source (and looks like they provided it), your comment wasn't special. LLMs however are especially less trustworthy, that's why. It's not my job to educate you on why they are, as you say, and why people aren't trusting your comments.
>I ask them for the same source (and looks like they provided it)
I also saw it now. That blog is not a representative ground truth, but just another opinion piece, which I can respect as an opinion of the blog's user base, but I can't take as an accurate real world statistic, same how aggregate opinions you read on HN are not representative of the actual real world.
I hope you can understand my PoV. You can also disagree if you want, but you'll need to bring something more than "that's wrong because LLMs sometimes hallucinate" as proof that Gemini's data is wrong in this case.
In theory if you brought a better source than that person then I'd agree with you, but,
> you'll need to bring something more than "that's wrong because LLMs sometimes hallucinate" as proof that Gemini's data is wrong in this case.
actually I can say it's wrong or likely to be wrong especially if it doesn't cite the sources it uses. And if it does, then just paste the sources here instead of the LLM output. It is also unknown where it got the info and as someone else said, you ask it two different times and it gave two different answers, thus it is unreliable.
OK, but what's the sample size of that and who's measuring it? I never heard of that blog or took part in that poll. So how is that blog link the yardstick but mine is not?
>Because that blog actually talked to real people and did real wor
How did you verify that those people from the blog are "real"?
I also talked to real people for my own data, case in point, I asked my mom and dad which linux DE is most used, and the results came out different. Which "real people" are the ones representative for the ground truth of Linux DE sahre?
> and yours is just a hallucinated list from one of the me-too LLM vendors
How do you know it's hallucinated? Ask the LLM the population of your country? Is the answer mostly accurate or is it hallucinated in an inaccurate way?
Aren't LLMs just outputting the highest statistical probability from the aggregate of their scraped data, which in this case would be including opinions on Reddit, and every website and blog on the entire internet (including that random one posted by badc0ffee) on the Linux DE uusage topic, making it a more accurate real-world representation than just a single random blog?
You can call it "hallucinated" if you want, but that doesn't mean it's not accurate. I asked for proof that my answer was inaccurate, not that it was "hallucinated", those are two different things, and your argument didn't prove it was inaccurate nor did it prove it was hallucinated. Would you like to try again?
<< Probably because pop_os and Cosmic are so niche and their market share so low, that they're irrelevant to attackers and bad actors, when those now have much bigger fish to fry.
That is such a weird statement that I am not entirely certain where to begin. PopOS is hardly niche. Its base are all fairly common components by linux standards. And, more importantly, attackers and bad actors may other considerations in mind than sheer population size -- just to point out the glaringly obvious.
<< I think even amongst the HN and Linux userbase, pop_os is still niche
I think rather than trying to disprove it, I think I should ask why you think that? If anything, PopOS annoys me because it is just a step before ubuntu ( and ubuntu is just windows at this point ). Maybe I am defensive, because my first real distribution ( that did not share disk with windows was popos )?
It also means security is not held as high and vulnerabilities not as much found. A simple 0-day may survive for years. Not much effort needed to have permanent access.
Its easier for an LLM to find vulnerabilities in a project with less usage, and those vulnerabilities will stay open longer, making them much cheaper to attack to keep the door open.
What's the point of attacking projects that almost nobody uses?
Do you think Netanyahu, Trump or Xi-Jinping are somehow secretly using Cosmic DE at home, to be worthy targets?
Bad actors have limited time, lives of their own and mouths to feed as well, so they concentrate their efforts where "the fish are" if they want to PWN someone for profit.
That's why Windows was the biggest target in the past for so long and why MacOS and Linux were ignored. Because most of the fish were on Windows.
Offensive security employees, tokens, and peoples' time are still a finite resource that get allocated based on target priorities and operational end-goals, even by state actors.
If you assume Mossad and NSA are Token-maxxing every single niche FOSS project out there to cast as large as possible fishnet on hacking all Average Joes on the planet just in case, then maybe using Mozilla and MacOS gets you hacked too, maybe even visiting HN and commenting here gets you hacked by some zero days you don't yet know.
So you took every single line of open source code you could possibly get your hands on (using scrapers so violently dumb that they amount to a permanent low-grade DDoS) and spent billions of dollars to tune trillions of parameters, and the value you can offer is… “let us inundate you with bad code or else we’ll generate exploits for your software”.
Using AI to find vulnerabilities doesn’t mean that you need to use AI to generate the code that fixes them. And you can still ask AI whether it thinks the fix is okay, as a second opinion.
It'd be great if people did that. They don't. Open source projects with limited budgets shouldn't have to spend money from those limited budgets (or personal funds!) filtering through what amounts to spam. That's why there was a slew of open source projects that turned off issues on github and stopped taking outside contributions- they literally couldn't afford to keep up with the nonsense lazy bums were sending their way.
Except that it would be priced based on pass-through prices rather than existing value perception. It used to cost hundreds to send a letter on horseback, it costs pennies to send an email, consumers aren't going to tolerate spending hundreds when they know it take pennies to send.
Models are getting cheaper, and more efficient, but I don't see how they cross the threshold of full autonomous intelligence in the way that an employee needs to be fully accountable for their actions.
eventually it will be pennies, but based on current trends i'd bet value outpaces price decreases for a while once it actually becomes an employee replacement
the demand then will rise and prices will rise in tandem
it is of course possible that 'deep learning hits a wall' and they don't get there, but i wouldn't bet on it
it seems fairly obvious to me that once ai becomes as good as a worker, people will pay a workers wages to use it - 24/7, can't quit, can't complain that badly, no lunch breaks
So we're talking about the complete destruction of the consumer economy.
Hard to see how Google or Facebook keeps their advertising models in that environment. There aren't enough construction workers and data scientists to buy everything.
> If it's any consolation, in the end it will destroy capitalism too.
That is hardly will be of any consolation. AI will probably destroy capitalism, but not because it will replace some useless f.ucks from their useless computer jobs (most of them will perfectly fine find another more useful job, if only way less paid), but because with surveillance and totalitarian control that AI capable of, socialists like Stalin or Hitler will build resilient and durable society where no member will have any way for resisting.
Odd that you mention that, China is afraid of exactly the opposite. Their systems are so centralized for surveillance purposes that an AI seriously breaching their systems could destroy their control.
Of course it is afraid, and because of that AI will be banned for many use cases and severely regulated by the government for other use cases.
That is like one of the first steps to build the ultimate totalitarian regime. In the USSR bolsheviks were grate proponents of civilian weapons right until they have got power. And then civilian weapons were banned.
> There aren't enough construction workers and data scientists to buy everything.
There will be, when two third of the office workers will be replaced by AI.
It is astounding why so many people does not understand how consumer economy works. I think it is more a matter of the copium, because, well, it is generally hard to accept the truth that in five years you will have to haul bricks as a job, and the guy who used to put that bricks on one another and make three times less than you in your tech job now will be your new boss.
And advertising models in that environment? Why should something changed for it?
