I've found it to be useful for prototyping. You can throw a game mechanic up, have it build some AI players, tell you any balance issues it runs into, play against the AI yourself to see if the mechanic feels fun. Then iterate.
A few times I've made a tweak to a mechanic and the LLM balance tests immediately found a major issue. Sometimes the AI players it creates just suck at the game. You still have to think, but just having balance tests and then having them cheap to update when I experiment, it extremely valuable. I tested this on a game I had mocked up with cardboard and forced my friends to play. It found all sorts of stuff I missed in those gameplay tests. It didn't replace those sorts of tests with humans, but it let find issues faster so I got more out of human interaction tests.
Does RSI actually mean anything specific anymore? RSI, AGI, at this point seem like buzzwords. Sure AGI has definition that are measurable, say "better than 95% of humans on 95% of intellectual tasks" but if we used that definition we already have AGI and almost no one thinks we have achieved AGI. We use AIs to train AIs which we use to train AIs, why is that not RSI? How much human intervention means that is not RSI?
So few terms in AI are well defined. We will get ASI via AGI because of RSI but neither of those three things have any definition except pure vibes.
I struggle with the argument that RSI doesn't already exist like you say, it's existed since before the term LLM (hey, one that can be defined!) was common parlance. Though the biggest use for those is not superintelligence, it's to serve you ads and get your kids addicted to TikTok.
Narrow pre-LLM models that spit out content and ad recommendations have never been capable of also suggesting, let alone implementing, self-improvements.
RSI has always being a well defined name, and you can only have RSI if you have an intelligence capable or creating itself.
It has technically existed for a long time (for longer than the name), but only on academical applications for extremely limited intelligences that could only create something like themselves. And that is still the only form that exists today.
It was never powerful enough to optimize ads distribution, and all the claims people are pushing around today are plain bullshit.
> say "better than 95% of humans on 95% of intellectual tasks" but if we used that definition we already have AGI and almost no one thinks we have achieved AGI
What matters isn't 95% of humans, it's 95% of actual professionals. Benchmarking an AI accountant against people with zero accounting experience is worse than worthless.
That's what 95% is intended to capture. That some level of expertise in an area is should be captured by 95% sample of the population, you could push it to 99% of 99.9%, but you want it to be quantified by a number to avoid arguments that something is not AGI because obscure field or expert exists that AI can not do.
95% better at 95% of the population is already approaching ASI. One could even argue that AGI is 50% better than 50% of the population.
All that said, this over rotation on benchmarking misses something critical. What is general intelligence? We assume that humans have it and we assume it is captured by benchmarks on "intellectual tasks", but it is probably the case that general intelligence is based displayed by judgement on uncertain outcomes. Benchmarks by their very nature have certain outcomes, they have a wrong and right answer.
Test AIs on questions we don't have the answers to and there is no clear right answer, but there will be at some point in the future. What will the economy do? Which US senators will be be re-elected that polling correctly suggests will not be re-elected. Which US senators, currently not in office, will actually pass bills representing the wishes of their voting base? What published papers will be seen are groundbreaking in 5, 10 15 years? What approach to unifying physics should be taken?
I believe jrflo was saying that OpenAI watches the chats of everyone from Anthropic because watching what Anthropic employees type into their personal ChatGPT accounts is a critical source of intelligence on is happening inside of Anthropic.
I would be surprised if OpenAI isn't doing that. OpenAI will take any advantage they can get. If an employee at their primary adversary is typing useful intelligence into OpenAIs website, a website that does not promise privacy from OpenAI, the only reason they wouldn't weaponize that information against Anthropic is ethics or fair play.
A careful reading of "we cannot rule out that de-identified data derived from their usage of our products helped improve our models" could be saying that yes they trained on it but they don't know if that training data resulted in an "improvement" to the model. That is, they can't rule out that the only reason the model found this solution was because it had been trained on this approach.
The term ruled out is very open ended and gives them significant flexibility of meaning. They may have the information to determine exactly what happened, but they haven't looked so they can't "rule it out".
I disagree. Google is one of the few parties that can monetize AI because Google has a massive moat in the form of their products: gmail, chrome, photos, search, etc... and Google already has the custom-built chips and datacenters.
Google spending money on competing head to head with your LLMs is a waste of money for Google. If anthropic wins, Google copies their approach, buys anthropic for cheap or both. All the investors throwing money into OpenAI, Anthropic, are just accidentally subsidizing Google's product development. Google shouldn't spend its AI research capital in an arms race with Anthropic but instead should invest in AI approaches that no one else is investigating at scale. That way Google can hedge against LLMs hitting a wall.
There is a real but small danger to Google that Anthropic replaces Google as a search engine, but that is an uphill fight for Anthropic. Google has massive brand recognition, network effects with gmail and chrome, Anthropic can't just copy what works from Google. On the other hand Google can copy what works for Anthropic. Google would have to play poorly to lose that fight.
Probably the worse case for Google is that software becomes so cheap and easy to create and maintain that all of Google's product offerings become commoditized. Even in that world Google has a lock on infrastructure. Perhaps ASI software creation completely removes that as well? If so we are living in a post-singularity world and probably the stockmarket doesn't exist anymore either.
In retrospect this is very similar to the Apple strategy: don’t invest heavily in doing something that isn’t already a core competency, position for novel uses of the tech but don’t build it per se. Apple probably would have benefitted in the last 4 quarters from hyping a custom model stack but it doesn’t seem that this strategy paid off to even close to 20% ROI while Apple got to hold their cash and organizational focus on their main thing
But paying so little attention to it that it no longer is a core competency? As in, the people most representative of that competency leaving in droves? I doubt that was the plan.
Well it all depends on whether you’ve ever thought Apple was any good at search, Siri, etc.
My own personal opinion is that they’re great at design, UX, hardware, and even some power user desktop software (lifetime Logic Pro addict here). I would say it’s a stretch to call anything they have ever done with ML to be anything near cutting edge compared to Google, for whom all of the “hard” problems of recommendation, search, NLP, and other ML have been the absolute focus of their SaaS, white papers, and web offerings for close to 30 years
And Gemini seems about as good as a search engine as any of the other LLMs, if you use them casually. And Google has the brand-name recognition.
Right now AI companies are competing to make LLMs better at graduate-school level tasks. They all can already competently tell you what the weather is going to be like tomorrow or when the first Led Zeppelin album was released.
Gemini seems like a better search engine than the alternatives because it has access to Google's massive crawler feeds. I think Google could charge 30 dollars month for Gemini + all the data Google has locked up (Scholar, Books, crawled webpages, Google Groups, Usenet archives + search of your personalized datasets such as calendars, photos, emails, docs, slides, etc...). I'd pay that easily.
And that might be why Google loses, they are competing with companies that are willing to operate outside the law (or at least pay Trump to have the DoJ look the other way).
Anthropic has no moat, their models just get distilled and resold. They can maybe get good margins when LLM improvement rate is very high but as it slows down they are looking at commodity margins. Google has the products that makes that commodity more than just cost of electricity + 0.5%.
My case is based on the assumption that Anthropic will not hit RSI or if it does RSI rapidly hits a wall. Faster your growth curve, the faster you eat all the low hanging fruit and s-curve. I could be wrong here, maybe RSI will cause a hard takeoff singularity by 2030 and just keep going, but if that happens the world fundamentally changes.
They wouldn't be the only party, but they are in my opinion the most likely party.
What the people that can monetize it? Microsoft, X, IBM, Salesforce, Discord, Google
Microsoft would likely buy OpenAI due to their investment and integration with OpenAI. Making Anthropic not that valuable.
Google would likely buy Anthropic due to their investment and ties to Anthropic.
X, Salesforce maybe. Not sure Discord has the money. No idea if IBM capable of growing.
All this changes if Anthropic figures out a moat/sticky product that doesn't just depend on having the best LLM. If they pull off https://claude.com/solutions/healthcare then all bets are off.
You can use a self-hosted identity provider with OPKSSH. OPKSSH is just the protocol and the software that plugs an identity provider into SSH. Lots of people use it with authentik instances they run themselves.
> Still, using certificate in OpenSSH (instead of public key) is easier for me once it starts to scale beyond 5 or more hosts. Even do dual-cert, by host and by user combo, while using brittle OpenLDAP-OpenSSH for other class of users.
SSH certificates are fantastic. OPKSSH wouldn't work without them.
> And smashed all my notes on certificates and OpenSSH on my ad-free, revenue-free, JS-free Pelican blog. It's personal on both note-wise and server.
A few times I've made a tweak to a mechanic and the LLM balance tests immediately found a major issue. Sometimes the AI players it creates just suck at the game. You still have to think, but just having balance tests and then having them cheap to update when I experiment, it extremely valuable. I tested this on a game I had mocked up with cardboard and forced my friends to play. It found all sorts of stuff I missed in those gameplay tests. It didn't replace those sorts of tests with humans, but it let find issues faster so I got more out of human interaction tests.