When you're not in the lead, your best strategic move is to neutralize trade secrets advantages of the leaders. I think a strategic approach is being misinterpreted as a demonstration of virtuous ethos, which is crazy given (as the other commenter mentioned) the norm of secrecy when it strategically suitable in basically every other context.
I mean how useful are GPT-3 weights today? That's the kind of useful they'll be relative to the frontier.
I feel like the so-called "Deepseek moment" broke too many people's brains and they've lost the ability to reason about why frontier advantages matter. I think we have to learn some new core concepts and rate of improvement over time is going to be a big one, and not understanding that causes people to exaggerate the significance of downstream distillation that's done in response to it. It's cheaper and open and that matters, but it's not the kind of mattering that replaces the relevance or advantage of the frontier, and I feel like I'm not seeing people demonstrate that they know how to really compare the relative merit of these two things so much as zoom in on one to talk past the other.
Even the original DeepSeek is old hat. The capabilities already released are game changers, whether you want to acknowledge that or not is not my affair.
I was using GPT-3 and the original Llama when they came out and realized that it is interesting, but not good enough - yet. But they are good enough now. And just like there are still people driving Renaults and Volkswagens the latest batch of open models will be useful for a long time to come.
So my whole previous comment was a plea to not do exactly this, namely, treat the example of open models like they somehow negate the value or strategic significance of having a model that leads at the frontier. I would submit that it's the core confusion behind the "Deepseek moment" and what it supposedly teaches about the relative strategic positions of different actors in the AI race.
> whether you want to acknowledge that or not is not my affair.
I felt I acknowledged it pretty explicitly. I said the likes of Deepseek are "cheaper and open and that matters", but it's the rest of that sentence that I think went without acknowledgment.
>And just like there are still people driving Renaults and Volkswagens the latest batch of open models will be useful for a long time to come.
Walking is great too! But you're not filling an economic need by selling walking to the masses, and those companies are marginalized and the economic destiny of the world is being set by those who own the infrastructure to build out the world's next generation of electric vehicles (at least in the automotive space).
I think you have to put up some facts to justify the assumption that uber-super-ultra-premium intelligence will continue to be significantly more economically valuable than boring old super-ultra-premium intelligence. Most things in reality have a point of saturation. The AI industry doesn't like to think about that; they prefer to make claims based on promises of infinite exponential growth.
Why does the OP need to bring "facts" to the discussion when none were introduced anywhere else?
I think you're assuming that the options will be: "uber-super-ultra-premium intelligence" and then something like that but only slightly less so and much cheaper. Those aren't the only potential options. One other option is "uber-super-ultra-premium intelligence" and "nothing that really matters".
That seems to be the crux of the "open-weight" models versus "closed" models debate/discussion. I think it's more of a belief system being expressed than it is a reflection of any reality.
Open source software evangelists tend to have a quasi-religious viewpoint on the openness of software and that the model is superior and will always be cheaper and if not better only slightly worse, but because it's cheaper it's better. They will proclaim its universality, and apply it not just to web frameworks but to artificial intelligence, or state secrets, or anything else where there is some secret or knowledge to be held and maintained. If I were to say I think closed-source software is better and more preferable, open-source software evangelists will come out from all corners of the internet to make sure I know how impossible it is for security to be good on closed-source software, or how software yearns to be free or something. Then it turns into "the AI industry ... prefers to make claims about infinite exponential growth".
All of that is just a belief system. It's not scientific or factual. And it clouds judgement.
I don’t know if you’ve actually followed this field over the past several years, but a headline fact is that open weights models started far behind the frontier and continue to get closer and closer rather than the opposite trend of the frontier pulling away.
I don't know what you are comparing or how you are comparing things. You should provide some examples like, cost per token or some other useful metric other than "open-weight models are closer and closer".
Maybe the best way to think about it is like this: Yes a cheap Android phone from Wal-Mart has a camera, you can text on it, and install apps, but Apple makes all the money. Why is that? Because it's not good enough to just be a cheap and "close" alternative. If it were, then we'd see a race to the bottom on cost and commoditization, but we don't. Technology matters, but so does commercialization and it is immature and incorrect to think otherwise. If OpenAI and/or Anthropic go on to capture 90% of the market's profits and leave the bottom 10% for open-weight model companies or distilled models or what have you to fight over the scraps then OpenAI and Anthropic still win regardless of the capabilities of the other models.
Similarly we don't see this race to the bottom with closed-weight models. If open-weight is or was superior we'd just see nothing but American open-weight model companies come in to existence overnight and take over the market just like we do for any other software. But we don't. The fact is that open-weight models while providing some advantages are only one part of the market makeup.
You can think through this a bit too. First, you can see that China is now requiring export licenses for some of its AI companies and closing up models. Why is that?
Second, can you talk through the revenues of open-weight models? And if closed-weight model companies go bankrupt because of the supposed superiority of open-weight models where will these companies get their weights from and who will train the models and on what infrastructure?
I think the vast majority of physicists have been perfectly able to avoid Penrose's error. I would also think the lesson would be simply to strive to avoid making errors or the kind he did, not necessarily to sacrifice realism about physical objects just to avoid the possibility of someone's mistake. Plenty of people, in fact I would say many more people abuse the observer effect in QM to make crackpot claims about consciousness. But it doesn't mean the observer effect is wrong.
I also think you can be a realist without having to make the jump to ontological irreducibility. You can say that our latest theory is a better approximation than the previous one and the previous one was an approximation as well.
When we say electricity is a "natural monopoly" what that means in the history of transmission utilities, states have stuck a bargain to grant monopolies in exchange for being tightly regulated, and so there's no version of reality where the state isn't already immersed in regulating them.
Every state in the country has a public utilities commission and they are constantly immersed in complex policy work from implementing legislation to working through an endless docket of cases that fine tune policy to specific tasked out responsibilities like getting a "standard offer". Sure it's complicated but I can't think of any reason that defaults to "don't regulate". We regulate healthcare too, and Medicare is always cheaper than private alternatives.
The problem with leaving private actors to their own devices when they have monopolies is that every conceivable grey area of judgment calls that impacts profitability will be subject to abuses of discretion, such as building out overpriced infrastructure, and investing in cheaper, unreliable transmission infrastructure instead of building for the long term.
I think the racist part is the pivot from demographic changes to the inference that they herald an extinction event for a particular culture, and especially tying that assumption to unwarranted extrapolations that misread the data, and especially when it's a fixation on populations that don't realistically face any such threat but nevertheless talking about the perceived threat in existential terms.
The racist part is extrapolating from mass-scale migration of different demographics and cultures that it will cause demographic and cultural change? You may disagree it is a problem, but I fail to see how it isn't true.
You think if, say tomorrow, all 1.4 billion citizens in China were teleported into the U.S., that there would now be 1.7 billion Americans, indistinguishable from the Americans that came before? Or, would it simply be 1.4 billion Chinese, alongside ~300 million Americans (if we're being generous...)?
Demographics and culture are obviously intrinsically tied, each individual migrant holds beliefs and values that have been forged and shaped over the span of the lives that they have lived in their country of origin. This is, and has always, been the case, its just more of a hot-button issue recently because:
A: We are currently getting an overwhelming amount of migration to stave off our own demographic issues, so the strain begins to have a more noticeable impact,
and B: Many of the migrants (from abroad) are coming from countries with cultures intrinsically incompatible with values Westerners have come to expect in their own countries (See: Sharia Law as contrasted with Women's Suffrage), so they stick out somewhat like a sore thumb when advocating to enact such policies and practicing such cultures.
Right, the racist part is equivocating between "demographic change" and "extinction event" and thinking in terms of hypotheticals and extrapolations that aren't supported by data.
And there's all kinds of value prejudging that is entirely external to the data about what cultural change means, what it implies, what effects it will have, all of which involve disputable and extremely politicized assumptions that are trying to borrow from the objectivity of population data for rhetorical momentum in favor of specific preferences of racial composition.
Sharia law is a boogeyman on the same level as the satanic panic and the witch trials. It's about someone who wants power over you making one of your neighbors the "other", which only he can save you from.
We saw the same kind of bullshit against the "papists" with Irish immigration, and the Chinese who came to work on building the railroads.
Immigrant children who grow up in the US, are just like any other children growing up in the US. They don't just follow whatever their parents believed in.
I mean insofar as the idea is about contemplating those mechanisms in the abstract, sure. But insofar as they were predictive or an attempt at a descriptive characterization of where populations would actually go, clearly it didn't hold. And if it didn't hold because it's true on it's own terms in some abstract sense, so much the worse for its inability to model changes that would prove to be relevant to actual population dynamics.
It's also not just that we accommodated explosive populations but that eventually birth rates really did simply start slowing down for discernible reasons that future models were capable of integrating. Even if we continue to have technology that would give us the capacity to accommodate future increases in population, those increases are no longer happening.
> And if it didn't hold because it's true on it's own terms in some abstract sense, so much the worse for its inability to model changes that would prove to be relevant to actual population dynamics.
That's a bit unfair to Malthus though; sure it was the enlightenment, but all he could rationally go on was to assume that the last 2000 or so years of civilization would be similar to the next, in terms of food and population growth.
What would be abstract would be to make the assumption that unprecedented technological advancement would be made in the next hundred years, bordering on ( to what Malthus would be) magic. You can assume any problem will be solved if you also assume a technology will be invented to fix it. But that's not a reasonable approach to make if you have no reason to believe it is possible.
I agree he wasn't able to anticipate technological changes or the impact they would have, but the response to that isn't to simply insist his theory is literally still true just because he tried his best. I'm not suggesting he could or should have done that, but I don't have to have that expectation to nevertheless note, correctly, that it's not a model of the population dynamics we actually see.
Great link, thanks. In a pretty remarkable coincidence the post you linked, which itself is from seven years ago, considers a prediction from over 60 years ago of an anticipated doomsday, and that doomsday date just so happens to be a few weeks from today:
>In the 1950s, an Austrian scientist discovered a series of equations that he claimed could model history. They matched past data with startling accuracy. But when extended into the future, they predicted the world would end on November 13, 2026.
Shucks, you are correct, it's not a coincidence at all. I thought it was about the predicted population boom writ large but TFA's hook is about the specific predicted date.
Well I wouldn't both-sides this one. 2020s population forecasting is better than 1960s forecasting, the latter was repudiated and the former is correctly modeling basically everything. Demographic decline really is happening and has reasonably well understood mechanisms.
I suppose I would separate out some of the more extreme predictions about demographic collapse in Russia or China. I'm not sure what to make of those or if they're similarly credible to other kinds of population forecasts.
2020s population forecasting has less than a decade's worth of record to go by, trying to say anything substantial about it risks falling into the same trap the article discusses.
The main thrust of the article (as I understood it) is historical context and how human processes can change given circumstance. So basically: demographic decline will be happening and well-understood until these circumstances no longer persist.
Hard disagree, population forecasting is more rigorous now but the data stretches back to the origin of civilization, and it's more than a binary of having or not having data, it's having stronger and more predictive methodologies. The biggest risk is the false equivalence between predictions from 60 years ago and now.
It's like Isaac Asimov's essay the Relativity of Wrong, which, if boiled down to one sentence says (paraphrasing), "People who said the earth was flat were wrong. People who say the earth is round were wrong. But if you think believing the earth is round is just as wrong as believing the earth is flat, your view is wronger than both of those put together."
Same thing about equating 2020s and 1960s population forecasting.
I'm not suggesting that current data modelling is no better than it was 60 years ago (it surely is), what I'm saying is that we have 60+ years of predictive record for the latter and 6 for the former, so let's not get ahead of ourselves.
To use your analogy: let's rather not get too complacent about figuring out the world isn't flat and risk placing all future bets on it being round.
I think it's terribly unfortunate that the the "in mice" meme ever took off. If overhyping science based on preliminary tests is bad, so too is the blanket dismissal of that research that overcorrects in the wrong direction.
We share enough mechanisms across genetics, proteins and even our immune systems and that research is capable of asking meaningful questions that matter to us, and frequently has paid off. That's not to justify the ethics of it but I think it's beyond question that research iN mIcE has been an important research platform for things like immunology and lead to nobel prizes. Research writ large is a vast cumulative program of small iterations that are, individually, of provisional significance, and the categorical dismissal of it as not producing instant one-shot cures is at least as bad a misunderstanding of science as overhyped articles are.
reply