You could hack critical infrastructure before AI. Any and all of the bulk internet scanners have had lists of exposed critical infrastructure for quite a while now. At first it was shocking that nothing ever got done about it, then it became routine.
All that AI has done is to lower the bar of entry for criminal activity. Which is a concern, but it's not the primary concern. The primary concern remains that so much critical infrastructure is poorly secured.
The bigger problem here is that you can hack everything, all at once, for very cheap.
Don't get me wrong I have general disgust towards these companies that are trying to get regulatory capture on AI when they can't even secure their own systems. I believe if people know that a random AI agent can hack their systems they will put in a lot more effort into making sure it doesn't happen. This is a personal example, but I didn't really care about securing few systems as I knew no human would be ever interested in finding a vulnerability in proprietary software, however, AI has no concept of that and would hack a random rpi server running a completely undocumented unknown API just because it can't distinguish value and it costs nothing.
Ya, quantity is a quality in of itself. In the past hackers may have used something unimportant to get a foothold but almost always tried to get to worthwhile machines. An AI will compromise everything in the network it can quickly simply because it can (assuming the attacker has a large budget, but I'll assume they stole the tokens).
It's like a new form of spam. Only far more dangerous.
An argument about factors of 10^-6 of a renewable resource, is not a serious argument.
We have real environmental problems, like excess CO2 in the air and forever-chemicals in our biosphere. The narrative must stay with those material issues and not be diverted by trivia.
We have additional environmental problems and water consumption is only renewable in a global scale not if you look at the local water resources.
Evaporated water becomes drinkable water again but not necessarily in the place it was consumed.
And don’t forget that consumption comes on top of the already existing consumption, so at some point you could reach the point of too much consumption. The straw that broke the camels back. There is a reason for that saying.
The saying usually refers to emotional load, and not physical load. Which is very apt here: datacenter water usage is an emotive issue, but one which is completely meaningless in actual environmental terms.
If the water consumption of a datacenter leads to more consumption than is provided back the environmental problem is real and climate change makes it more likely if rain periods and water ground levels change an glacier and snow periods disappear.
Agricultural consumption is already a problem but raising the consumption doesn’t help.
Your argument in the post up-thread appears to be that because the Native Americans did not cultivate or make use of the land in any Western sense, they have no claim to ownership. The Kant quote attempts to repudiate this by stating that it makes no difference whether we consider the Native Americans' use of the land to be ineffective: their deprivation of it is still an injustice and, when it comes to injustice, the ends cannot justify the means.
I don't find Kant's position at all convincing, mostly because the two parties' understandings of the concept of "ownership" were not mutually intelligible (possession vs usufruction). However, the quote does have relevance to your post.
The key here is 'use'. Kant, while obviously not some gospel to be held to any form of authority, is making a point about differences in culture and sure, as you said, use. However, my point wasn't disputing usage at all. Living on it. Hunting on it. Working the land.
However no view exists in a vacuum, and I will indelibly view my example of a group of 4 as a valid statement, about the absurdities of 'walked on the land once in my life, to get to my summer/winter home', compared to some claims of a tribe of a few hundred people, somehow owning hundreds of millions of acres of land, just cause they walked on tiny parts of it to get to their seasonal homes.
So no, I don't see the relevance, because I'm not discussing the concepts therein directly.
Cocaine (in crack form) was found to be the third most harmful recreational drug overall, after alcohol and heroin. It also caused the most direct harm to users out of all drugs. Cannabis, not so much, mid-table. Non-crack cocaine was somewhere between the two.
By your argument, it would not be a problem if the RNG never generated 0. So, it must follow that it would also not be a problem if it never generated {1, 2, 3, ..., 253}.
That means that our RNG now only generates the values 254 and 255. Which of the values is generated is unpredictable on any given call. However, 7 of the 8 output bits are now always fixed and so completely predictable. Can you imagine how an attacker could exploit that?
Failing to generate only the number 0 is a weaker version of the same class of flaw.
We're talking about whether a modification of the expected probabilities changes the dynamics of the game. The example I gave was deliberately extreme, because that makes it easier to reason about.
If you want a casino example, then consider a roulette wheel that always lands on 36 but still pays out as usual. I think you'd want to play on it. Now consider one that always lands somewhere between 30 and 36. Still worth it, right? With careful bets and a good starting float you're still coming away from the table up (with a very high probability).
In fact for a roulette wheel you only need two dead pockets for the player to get an edge. Bias is exploitable.
Sure, but you’re pressing on the truth that a small modification taken to an extreme is a large modification.
When the original point was that a tiny fractional loss in an RNG is not going to make a practical difference. Which I believe is also true. And it is also true that a large loss in an RNG is catastrophic.
They can both be true.
And roulette is 2 out of 38, 5.2%. That’s 17 times more than the 1/256 here, which was already a simplification of the (I think) 1/65536 in question.
With respect, I think you're still missing my point, which is simply that bias introduces vulnerability. Without a real scenario to analyze, the scale doesn't really factor into the argument. A vulnerability is a vulnerability until proven mitigated.
> a tiny fractional loss in an RNG is not going to make a practical difference
I'm not so sure this is true. I don't think either of us is in a place to say whether this vulnerability has practical applications or not. A 1/65536 bias might seem like nothing important to you. It seems like potentially something to me, in a world where the attacker might control the volume of data generated.
It is absolutely untrue that a biased RNG has "zero practical impact." Modern cryptography has plenty of examples of relatively small biases leading to breaks. Check out Bleichenbacher's attack, for instance.
You could be correct that the very small bias here is not enough to be exploitable. But, given the history around this, it would be wrong to handwave it away as trivial.
Some cryptographic algorithms can be broken by this sort of bias (after 2^20, 2^35, or 2^70 operations), others are completely unaffected. There's certainly no excuse for not knowing, at least.
Most contemporary stories around AI include the implication that AI did something humans couldn't. This is because the big players have been shilling AGI hard for a while, and their valuations depend on maintaining the sentiment that serious progress in that direction is being made.
Washing machines can’t do anything humans do, they just remove labour. Trucks don’t do anything longboats can’t, it just need less labour and time/effort to build roads rather than canals. Computers can’t calculate anything humans can’t dry run by hand etc.
Everything in reality is about reducing time/effort/material/cost or achieving more with less resource.
Yes. I'd also argue there isn't anything humans couldn't do in theory, other than things strictly prohibited by known laws of physics. It's been pretty conclusively shown in the last 100 years, we're past tipping point of civilizational knowledge and scientific infrastructure.
What stops us from doing any specific thing is always allocation of resources - there's finite amount of time/effort/material/labor available, and past trivial amounts we need more and more people to agree on some allocation. Reduction of time/effort/material/labor costs of any thing is what moves it closer from "infeasible" to "feasible" for us to do. But again, short of violating laws of physics, it was never "impossible".
What I mean is that "can be done by humans" is ill defined. Are you allowed to use pencil and paper? That's not part of your body? Do you have to be strapped into a coffin for it to count as human-done? How about wheels, sticks and stones? Allowed to use those tools? How about an abacus? Or is electricity the line to draw?
No, they don't - that implication is in the heads of people who believe AI is magic, or believe AI is advertised as magic, or believe that humans are dumb.
The actual implication is, and always been, different: AI did something humans theoretically could, given enough time, motivation and budget, but they didn't, because it wasn't the best use of time or money. AI therefore demonstrates its value, by opening up problems that were previously uneconomical to solve.
> that implication is in the heads of people who believe AI is magic, or believe AI is advertised as magic, or believe that humans are dumb.
Yes! I believe that class of people are called "investors."
Your point is basically correct, I think, but the illusion of value that a corporate entity offers, and the reality of the value that the corporate entity is creating, are distinct things.
In this domain, an apparent single unique piece of work is often composed of several breakthroughs. For example, when Andrew Wiles proved Fermat's Last Theorem, he had to develop multiple new pieces of mathematical technology to get there.
The claim here seems to be that the human mathematicians, working with AI, developed technology to go A->B->C. By training on those conversations, OpenAI was then able to encourage the model to go A->B->C->D.
In my opinion that situation should be acceptable, if openly disclosed, because it is in the public interest to make progress on these problems and because AI is clearly an amazing tool for making progress. But the human mathematicians are saying that OpenAI is presenting as if the model got from A->D entirely independently, without acknowledging their background contributions.
Also, wasn't their B+C research private at the time, with them only releasing those details publicly after this blew up?
If they had published B+C, I think that would lean more towards fair game, as that is how research works and is improved on over time. But it seems like unpublished/private B + C may have been used by the model to hint it into working out how to get from A->D.
That link is a helpful contribution to this discussion.
I'm not at all familiar with this area, but my reading is that he appears to call it out as a relatively obvious extension of his own work:
> It is a creative and at the same time elementary construction that uses not just property (T) for an application of my result with Kun, but also for the ambient group
G in order to overcome the problem, that the Γ-components might be of different size. Once this is achieved, the rest of the argument is straightforward.
Creative and at the same time elementary is where LLMs excel, generally speaking. It's why they are so good at writing code.
> On the other side, I was looking myself for such a mechanism ever since we wrote the paper in 2019 and admire the efficiency of this construction.
He seems to admit very clearly he does not see this as his own work. 'I was looking...' well why did he stop? Because the AI figured it out first.
It seems quite odd to me to 'admire the construction' of something, only for your opinion to sour once that something figures it out first.
I think a lot of the emotional reaction here is familiar to us non mathematicians: you spent years developing expertise, and then LLMs began producing competent work in areas that had previously required that expertise. That's understandably uncomfortable, but discomfort by itself isn't evidence of misappropriation.
All that AI has done is to lower the bar of entry for criminal activity. Which is a concern, but it's not the primary concern. The primary concern remains that so much critical infrastructure is poorly secured.
reply