I think a lot of discussion around the sucess or failure of Anthropic and OpenAI seems to hinge on their current uses, but people seem to have amnesia regarding the fact that Claude code wasn't even a thing until about 1 year ago. There are far wider markets these companies can tap on if they are well capitalized even as Chinese "actually open AI" commoditizes things.
There are moonshot projects related to superconductors, materials, and biology that these companies can claim progress on to keep valuations stratospheric.
Math is an interesting case. Arguably a lot of the groundwork was already there. There’s a huge body of academic literature that’s historically been difficult for humans to draw on for various reasons. It’s relatively easier for LLMs to mine all of that for relevant prior work that it can recycle into proof segments to try. And we have theorem proving software that it can use to try out these proof segments and validate them. That creates a tight, self-contained, and relatively inexpensive feedback loop that lends itself well to a host of agents grinding away, as long as you have budget to pay for all that LLM inference.
I’m not sure the same conditions exist in other fields that deal with real world phenomena. Experiments to test hypotheses about biology or physics can take years and cost millions of dollars to run. They rarely scale down to the millipennies and milliseconds scale that makes letting a swarm of agents throw a bunch of shit at the wall and see what sticks a feasible option for finding mathematical proofs.
I posted the original comment, but there are multiple reasons why math success won't translate into drug development or materials engineering success.
It is easy to create verifiable rewards environments for math and coding, and use the resulting data for RLVR post-training. For other domains you need machines or robots creating and running experiments. This is simply something that's never been attempted at scale.
Getting IPO level monies should enable the amount of capital needed to sustain such efforts, and if I were a betting man, that is going to be Anthropic's actual endgame given their nascent efforts in automating bio research.
I work in biotech. Honestly, I think they will run into massive hurdles.
Amodei's academic background is relevant for protein engineering. Hence the recent press releases Anthropic has released which all concern prototype proof-of-concepts related to binding efficacy.
For a large number of therapeutic relevant indications, the bottleneck is actually clinical trials where you must contend with the sideeffects of any given therapeutic. Almost every drug I've worked on involves very well known mechanisms where knockdowns with targeted compounds will work in a petri dish, but can have catastrophic consequences when dosed on a live human.
As far as I can see there is no way to accelerate clinical trials in humans.
Even the most fantastically large and detailed perturbation datasets will struggle to capture this dynamic in a full human. This is why tech oligarchs are investing in virtual cell models. With a large automated research lab running perturbations 24/7 and capturing massive amounts of data, perhaps we will get as confident with predicting outcomes in cells as we now are confident with many coding projects. However, this is a scientific question, not a technological one. Public companies rarely give the people running them leeway to try and fail on big scientific question marks as opposed to well established or soon-to-be well established technology.
> This is simply something that's never been attempted at scale.
isn't that the point though? You invest in a company so they can make these attempts. If it works you make money if it doesn't you lose money, that's the risk.
The sort of mathematics that has been published recently is very particular. Which isn't really downplaying it but it is to say that there's a pretty massive gap between mathematics and semiconductors or wetlab work. I don't want to say anything totally deflationary like "AI will never effect research in biology" or something like that, but I think the contribution of AI in those kind of fields will be much more modest.
lol, no it's not, the machine that is built to find connections between concepts found connections, that's it.
What has evolved is that mathematicians are pearl clutching because they never thought it would happen to them and wondering if they can still pour all their ideas / thoughts / research into these tools after all is said and done.
That last point is the one that kills OAI / Ant going forward. Can they be trusted? Anecdotally no.
> every single niche they've tried to launch and failed to get any traction on.
This isn't true. I personally know a company that has trained over 300 companies this year to use Claude Cowork mostly for "business operations" type tasks.
People love offloading boring, repetitive tasks to it. And the spend is spending - most start on a free plan and after 3 months have multiple people on the Teams plans.
> Chinese models are pretty much on par.
There isn't even an open source harness that is close to Cowork. ChatGPT work is closest, but everything else if coding focused and a mess for people who aren't coders.
Google still has to build and run the chips they rent to Anthropic. I think from their last earnings Google makes ~30% on it, excluding R&D costs. It's a nice business for them, but hardly the thing that is going to make or break Anthropic as a business.
Yuji Naka really fell off hard after Sonic Adventure 2 and commercial failures of Sega's hardware endeavors as a whole. Went from being Sega's golden child, to being arrested for securities fraud.
Allegedly he was extremely difficult to work with and would not share the Night's 3d game engine with an American team that was developing a 3d Sonic game for the Sega Saturn, effectively kneecapping a big part of that console's commercial viability
He lead the team for most of development until leaving the company months before the horribly botched release. He mostly wrote most of the assembly code for the mainline sonic games in the 90s.
Naka tried to justify in a tweet reply that, because most of the 3d engine was written in pure assembly, the american team wouldn't understand it eitherway. That was straight up bs.
It's obvious he had a personal bias with the american team working on a sonic title.
Google's research showed some of the speed benefits tended to drop off at larger batch sizes on an H100. Are there published batch-size vs tokens/s figures for Mercury 3?
We've been happy with how performance has scaled with batch size. The model runs in our API at reasonable costs, which is also what allows us to serve the model with competitive prices ($0.10 per 1M input and $0.35 per 1M output at launch).
Their preprint showed it worked well with small batch size, but most prod deployments right now have rather large batch sizes from serving multiple users
Honestly, a lot of those issues could be remedied with more optimized kernels, which LLMs could definitely write quite well now. I wouldn't be surprised if some deployments of Gemini Flash switched to diffusion for cases where there is low utilization, since Google showed you don't need to retrain an autoregressive model to rework it for diffusion inference.
> since Google showed you don't need to retrain an autoregressive model to rework it for diffusion inference
IIRC you do need to retrain the model, just closer to the level of finetuning rather than e.g. pretraining from scratch. Which also means that the model will perform differently with a different capability profile.
Doesn’t this suggest that the results seen in the OP are merely copying strategies it has seen in other bots (code in the training data), and not actually the agent developing its strategy?
Basically yes, but the space of "AI for RTS and videogames" likely goes well beyond Starcraft, so there's probably a lot of unique programming techniques these frontier models can apply for writing AI bots that haven't been done yet.
Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that".
One of the most well known interpretability researches released some potentially very useful research as a blog post, and he asked claude to make a cool little thing on top of it?
Intuitively this makes sense. When you have small models, they need to compress more by leveraging superposition and polysemanticity.
Nick is looking at a 32B model, which certainly has to compress less...
Now we have 10 trillion parameter models... do they need to compress at all?
As far as I can tell, experiments show that Fable has basically memorized every lat/long position on the map of the earth can can tell you at a glance whether your standing in the ocean or not based on your coordinates
The 'Exploring Image Models' section addresses that specifically, particularly the 'What Neurons Do Under Pressure' subsection.
'If it switched to superposition, we should see overlapping mixtures on top of each other, ... But that’s not what we see. The neurons stay interpretable and generalized, but coarser'
> far as I can tell, experiments show that Fable has basically memorized every lat/long position on the map of the earth … your standing in the ocean or not based on your coordinates
Memorising a small shape file of the earth’s continents can allow you to determine if a person is on land or in water.
All functional knowledge is compression and contextual decompression. Sometimes this compression can be lossless or nearly so. Compression doesn't automatically mean loss of information.
Of course this doesn't mean the model has memorized all the lat, long points. Instead it means the model has a representation that allows it to work out if a lat, long is land or water in a analogous way to how a human takes a lat, long and looks it up on a map (a literal "world model" if you like).
It's a great demonstration of how LLMs aren't "Stochastic parrots" but instead have complex representations.
It doesn't mean its memorized all the lat, long points, but it has memorized a map to check said lat long points? Seems a lot more likely a part of the training data is the rough map of lat and long points, specifically for better positional understanding?
There are moonshot projects related to superconductors, materials, and biology that these companies can claim progress on to keep valuations stratospheric.
reply