The effective altruism/rationalism/AI xrisk people have always had a shockingly poor grasp on subjects outside computer science, despite their attempts to speak on them. I don't blame the actual biologists and chemists working at the frontier labs for wanting to skim a few bucks off all the money flying around, though! I know a couple who've had not-so-kind words to say about their employers' intelligence.
I suspect there's at least some "telling the bosses what they want to hear" going on. A massive financial incentive exists to exaggerate and fearmonger even internally to the company, because it makes you and your job seem more important.
The emptiness of AI companies' waxing poetic about the future of humankind is laid bare by simply looking at what they actually do, and who they do business with. Actions speak louder than words, and they've driven the worth of their words into the dirt many times over.
> Distillation does not allow the CCP to obtain equivalent or superior AI capabilities to the US, but it can bring the Chinese frontier to within a few months of the US frontier
A message to their investors, it would seem. "They caught up just because they distilled! Obviously they couldn't actually be as good as us!" Really funny thing to say right after an OpenAI higher-up stated point-blank that the performance of K3 can't be chalked up to mere distillation of American models.
It's baffling that they thought the mental gymnastics in this blog post would make them look better. I'd rather they simply fall silent on the issue; I would respect them more (or at all) for it. Open models obviously threaten fierce competition, if not outright destruction of their bottom line. But no, they needed to try and argue that they have the moral high ground for attempting to singularly consolidate power over all human labor.
I don't really understand how they can argue the security angle with a straight face. It's not like GLM 5.2 is a slouch. I've seen it do things like exploit an IDOR issue when I was experimenting with a quick-and-dirty web automation task. I simply fixed it, as one does. Open models make the world better to a far greater degree than they set it aflame.
Their position is analogous to trying to, say, ensure digital privacy for everyone not by making encryption freely available (because that would let the bad guys use it!), but by making it so you can't use general purpose communications devices that can listen to transmissions not intended for you. Do they hear how moronic that sounds?
Each passing frontier-level open model release makes Anthropic's patronizing rhetoric a little more insufferable, because it becomes clearer how unmoored from reality they've become in pursuit of profit.
> Open-weights models that don’t have dangerous capabilities are a public good
Note the hedging against 'dangerous capabilities'. Undoubtedly, all the useful ones trigger this condition in Anthropic's eyes. The rest of the post is filled with similar weasel-wording. Make no mistake, this absolutely confirms that Anthropic is against open models in the sense that any reasonable person understands them.
The way the rest of the post unabashedly appeals to the current US administration's China hysteria is hilarious, and not at all subtle.
I guess we'll see about all the doomsaying here, won't we? Kimi K3 is frontier-level, and there's no stopping it now. As far as the world is concerned, anyway. If the US wants to kneecap itself that's another matter.
> I’m not even going to get into how you could provably transform brute force propositional logic into efficient algorithms.
> The entire problem is showing that an efficient/reliable program actually implements those rules.
Reliability is a standard matter of correctness and captured (partly) by specifications. Efficiency tends to be easy to empirically test, but it is also possible to capture at the specification level [1]. Mind that specifications need not be all-consuming.
> But I did spend a grad class with rocq (coq at the time) and a decade working with “systems engineers” and am not convinced that this is a realistic expectation.
Agree! But this stuff just got massively more accessible, and the tooling around it is growing quickly. I think we'll end up growing specification systems specific to various domains which will be palatable to those "systems engineers", but I err on the side of optimism here. There's definitely a lot left to do for practicality.
[1] See the work of https://cs.nyu.edu/~shw8119 for the case of provably-efficient parallelism and garbage collectors
It is true that finding the correct specification is a formidable task; knowing what correctness even means is arguably most of the difficulty of programming. However! "Moving bugs up from programs to types" isn't how this shakes out in practice, at all. Another commenter already noted that it's often much easier to communicate your intent through specifications, because you can essentially always say what a computation should do much more simply than you can say exactly how to do it.
I think it's also important not to miss the forest for the trees: even relatively simple specifications like "the compress and decompress functions must be inverses for all inputs" rules out vast classes of bugs in a compression library. This is not a complete specification; for instance, it does not speak about how the decompressor behaves on malicious input. But in my experience, even partial specifications carry the promise of hitting warp speed with LLMs in a way that I haven't seen anywhere else. After a certain level of specification, you have decent guarantees of being able to whole-heartedly forget about the implementation details of the synthesized program. And you get a better-built, more robust program out of it at the end!
The comment at the end of the article about having LLMs directly generate assembly against specifications and letting them rip with finding custom optimizations is the sort of crazy stuff this enables. I really think we're only seeing the tip of the iceberg here. People keep asking what we can do with LLMs that we couldn't before; this is the answer.
To maximize personal influence and wealth, of course. These days, many with power don't seem to care much about uplifting society so long as they get theirs.
Hopefully Anthropic eats crow here, lest their wish is fulfilled that we all become slaves to the anointed few who work there. Never getting another dollar from me after pulling these stunts.
Wanna plug LibRedirect which allows you to easily set up proxies for any site (e.g. invideous for youtube, nitter for twitter, libreddit/redlib/whatever for reddit, breezewiki for fandom, etc)
I suspect there's at least some "telling the bosses what they want to hear" going on. A massive financial incentive exists to exaggerate and fearmonger even internally to the company, because it makes you and your job seem more important.