The status page should detect the thing they are looking into. Kinda like a Check Engine light or gauge reading notifies you to investigate a problem with your engine.
I was thinking the same thing. I see, and have seen, what the poster you responded to is saying. But, I think there likely is a bit of selection bias amongst folks who would be using an LLM to learn in the first place because folks who would use an LLM to learn will likely associate with others who would do the same, esp in a workplace. Ex: cars or computers. A small subset will tinker with them to learn how they work or improve them, but the vast majority just use them to enable other goals. But, it is still important that the small subset exists and is curious enough to learn how they work. And the folks who would tinker with a car likely associate with other folks who tinker with cars, and the same with computers...
IMO it's a lot more like a new more powerful engine is now available. Sure, faster drivers/teams will run faster lap times, no doubt, and it'd be a bad idea to not use it. But, less mature teams/drivers aren't going to catch those faster teams by just slapping the new engine in and going. They are still largely held back by the myriad of other things that go into being fast on track consistently IE (driver + datalog review, and just good ol seat time), vehicle [other than engine] setup, aero tuning, pitting practice (esp fueling for endurance stuff), etc etc. All those same things that were making them slower than the fastest teams before. And even still, if you give only the slower teams the new engine, they probably still won't be competitive with the faster teams and, quite honestly, the not great drivers will probably just make the same mistakes they were making before that slowed them down, except faster.
But, even if the new engine is the golden egg, now what? You're still competing against every other team that has the golden egg... And it's right back to the driver mod, vehicle setup, etc.
That's not to say that the slower teams have any choice but to run the engine too to not be even less competitive. It's just to say that having the engine isn't gonna fix the underlying problem, no matter how much of the team you lay off to fuel the new, more power, engine.
It's not, because even the best teams aren't shipping 10X faster. Execs keep demanding it and no one - not Anthropic, not Google, not Walmart or whoever - is actually delivering customer value visibly faster at anywhere near that scale.
How many times I have hit my head against a wall trying to figure out how to get some code to work, only to come back with a clear mind and look holistically to see that it didn't need to be solved to begin with.
None of that is to imply that solving any of these problems isn't groundbreaking or helpful along with the plethora of other things AI has undoubtedly advanced, but I think the breakthroughs will be inferred by many to mean "point AI at really hard problems" is always the solution instead of continuing to use your brain on whether or not the problem _is_ the right problem in it's current form to solve.
Agree 100%, especially with your second paragraph. The majority of domain experts I've interfaced with in my own domain, along with things I am not an expert in, are typically experts not because of the solutions they build, but because of the things they _don't_ build. It is the cliche "a clever person solves a problem, a wise person avoids it" paraphrase.
But I think what is lost is the value in (a) person(s) who intuitively spots and avoids these problems. My hypothetical example that I have seen play out: The domain expert spots one of these problems and succinctly explains that the problem doesn't need to be solved. Maybe it's a byproduct of bad design elsewhere, or there is a much better solution that avoids it altogether. Once the rest of the team/the lead catches on, either hindsight bias or outcome bias, or a combo, takes over and they don't recognize that all the value all along was in the person being able to spot this situation. They throw that in claude and claude does it's sycophant thing (after expressing that the previous situation was flawless for however long), and they are off to the races right back to automation bias with zero pause for thought...
To me, "domain knowledge" seems to resemble a primitive form of "slop filter" to use the modern parlance. In any field, the majority of possible ideas are slop. This has always been the case. People somehow learn to navigate through the slop, to find things that are worth working on.
Even before the AI era, we were flooded with a veritable DDOS attack of slop, in our in-boxes, meetings, etc. Managers believed that innovation was held back by quantity of ideas, or by the domain experts being "resistant to change." This led to day-long group brainstorming sessions that yielded nothing. Part of the transition from student to junior to senior is learning to filter your own ideas, and to guide others in the search for ideas that are likely to be fruitful.
The criteria for slop filtering have always been subjective, arbitrary, or driven by institutional culture. That's what we've got. Any means of generating ideas produces slop in the absence of filtering.
What will slop filtering look like when the AI can generate tera-proofs per second? What does it look like already for our e-mail inboxes? We are asymptotically trending towards slop filters that look more and more like "ignore everything." I already read none of the business memoranda that I receive. I don't answer my phone. The number of workers needed to read and assess business memoranda is zero.
The slop filter still functions in math. Problems are pre-filtered for being of interest. Somebody expressed enough interest in the Navier-Stokes problem to give it a name and attach a prize to it. There are still plenty of "prize problems." Here are some:
* Quantum gravity
* Improvement of superconductors that don't need helium
> Part of the transition from student to junior to senior is learning to filter your own ideas, and to guide others in the search for ideas that are likely to be fruitful.
I wonder how that'll change over time now. I think it's pretty important to have a few instances of thinking hard to come up with ideas and being totally convinced they are correct, only to have someone else trounce them with one or two succinct thoughts of their own. Those were very big personal development moments for me. And equitable, I still have folks with a lot less exp/time come out of left field with maybe not as formulated, but amazing ideas.
So, has anyone at the frontier AI labs considered doing _good_ things with these models instead of continuously proving it is capable of doing malicious things?
Instead of burning tokens doing intellectually impressive “hacks”, or sandbox escapes from which I have concerns could contain run of the mill malware, why not focus on showing people what you can fix?
Or, make software people actually use every day better.
Or, donate efforts towards medical research.
I don’t like equating AI to nuclear energy, but, it’s a lot like continually showing people how large of an explosion you can create instead of showing them how many houses/hospitals/schools/etc you can power.
Have they considered using their amazing new models to... improve something? THere'd probably be a whole lot less anti-AI sentiment if they used these things to actually make people's lives better.
It extends well beyond creative stuff too. It makes me think of Gell-Man amnesia [1].
People can tell when something they are experts in is being done poorly, but others can't, and it frustrates the experts. Then, those same people think something completely different is being done well despite what experts in that respective field say. It's like when someone from your family reads an article about your profession and then proceeds to tell you how your job works. lol
It's a really pervasive issue in society IMO. And a few prompts in someone's favorite LLM just reinforces it to people that don't know (any better|what they don't know).
> One thing people go through as they age is an elevation of cynicism and general misanthropy. After decades of being disappointed when naive idealism and speculative optimism fail, one can just get down on it all.
> It is really hard not to succumb to this. It requires active effort and sometimes even a willful ignorance.
I feel this very strongly over the past couple of years. As always, hackernews is full of extremely smart, well read people so I'd really appreciate any good reading material on this so I can better understand it. Tips are appreciated as well (healthy diet and regular exercise, good extended blood labs (think like Function health), good consistent sleeep habits).
I agree. Whether or not it is what they are intending (I lean towards they are), IMO it has been strongly inferred in all the messaging from frontier models that it's impossible to secure software, which is just not true. In reality, it's now possible to find and fix problems faster. If attackers/adversaries can just point LLMs at software and dump zero days as we are warned/told repeatedly by every LLM vendor, then literally anyone can do it now. If that is true, then if you want to be marked safe from AI enabled attackers, test your own software with it.
This highlights a problem with the Anthropic-Mythos-restriction style "we regulate the best models so attackers don't have them": It also ensures that only the big-guy partners of the program (Glasswing in their case) have the capability to secure themselves at the frontier model level.
