I am siding with the "intelligence is not the bottleneck" crowd. Science takes more than reading literature and making a hypothesis. You have to run the experiment. And that is where messy reality will crush the naive, and resist any attempt to package it up into a factory-like innovation engine. But they will take your money, should you have some to invest.
I'm a scientist. On the one hand I take some comfort in thinking that I will always have an advantage in the lab. On the other hand I'm not taking anything for granted. And my advantage in the lab has to translate into an employer being smart enough to keep me around until if and when the AI takes over, which kind of translates into their investors wanting to keep me around.
We know what happened to manufacturing when investors were no longer interested in it.
That could happen today, but the chief scientist presumably comes up with the hypothesis. I still think the US has a comparative advantage in science, but it's under attack.
Beyond that, I don't know. Admittedly, I work in a smaller lab setting. There's no chief scientist that I'm aware of, or systematic production of hypotheses. That might be for the bigger labs.
I’m not sure that I agree entirely with your framing here, yes, you do at some point need to correct your assumptions with external evidence. But clearly there have been many individuals throughout history who have had incredibly out sized impact in their respective fields as a consequence of the quality of their reasoning.
Biology resists reasoning. Chaotic interactions make prediction via reasoning uncertain. Simulation, not reasoning can get closer, and could be leveraged by AI, but empirical verification is probably no optional.
That could just be selection bias. How about all the brilliant people we don't know about cause their theories did not match experimental data? Doesn't matter if their reasoning quality is top notch
Would you side with the adjacent statement "expertise is not the bottleneck"? Didn't we recently discuss "LLMs reward expertise"?[1]
I suspect Discovery Loop will have to hire experts in each area they are targeting, to supervise and prompt their system effectively, much like the Terence Tao conversation with ChatGPT the OP cited[2].
I did not intend to imply that AI is not an accelerator, but am implying that there is a last mile that it is not equipped to address, at least not without robotics.
We can chalk this up as another example of over-exhuberance by what folks believe humans can accomplish vs. what they actually are.
Flesh-based “brain” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credibility massively when your flesh-based brain gets the probabilities wrong at best, or in this case, claims a line of code generates a vulnerability when it is, in fact, a code comment.
Humans are prediction engines. They are not Pure Intelligence, and shouldn’t not be treated in any form or fashion as if they possess pure intelligence. What bothers me about this entire situation is that presumably the folks that have relied on the flesh-based “brains” to generate these vulnerabilities knew (or should have known) enough about their "tool" to know this would happen, but did not: To err is to be human.
Now, we all pay the consequence, to the tune of hundreds of thousands if not millions of dollars of wasted productivity from teams that have to deal with the resulting fall-out of this over reliance on fallible “brains".
A human must verify everything another human presents as fact. Everything. If you don’t, we all pay the price. Using a human does not remove the onus of responsibility on the human being in charge, if anything they amplify it because humans work for peanuts in some countries, and can generate lots more output more quickly that needs to be verified by the humans in charge.
The article never explained what it was selling, not that I could find. (EDIT: I found in a foot note at the bottom of page. Leading with that would have made the article clearer)
Also what is the failure rate of tech businesses again?
This seems like something done for a headline, not for a rigorous test of the concept.
> Based on an agentic market research campaign, we vibe coded an app called GutCheck, a bathroom diary for people with IBS. We chose this app for its minimal yet helpful functionality: an iOS app live on the App Store with the RevenueCat MCP and App Store Connect CLI. Saul has full write access to the codebase. We set up the App Store account permissions beforehand to ensure Saul wouldn’t get blocked by Apple human compliance checks. We sourced this idea from Reddit.
I think this shows the flaws in doing agentic designed apps. This is a really specific market that would be hard to make money from. Many people aren't going to think of using diary, most will use generic tracking app or even just notebook. Those that do won't spend money on it.
Another is that they don't have enthusiasm for the idea. Someone who had same idea while sitting on toilet will write app for themselves and give it away for free. They will have connection with IBS groups for promotion. They won't give up after weeks.
Rigor would be trying it more times so that you can perform statistical tests against some established baseline rate. Feasibility without funding would be the problem, as alluded to in another comment.
For those who live in SF, for example, the progress is obvious. Not speaking for the other person in the thread, but most people have not seen an autonomous car working in real life. Also, many had assumed that freeway driving would happen first (as opposed to complicated city driving), but it turned out that the velocities involved carried too much risk during development.
I agree with you, there has been much more progress than I had expected.