I agree. The author is stretching several assumptions to get to his conclusions. The sub-title suggests he has solved the "Bronze Age collapse" riddle but his statement can simply be considered one of many factors already pointed out in the literature -- not the main reason.
Long before the so-called “iron age” started, humans knew of iron. There are iron artifacts from the 4th millennium BCE. Humans were fascinated by these small pieces of iron, which were found in meteorites and cherished and valued them. However, usable iron was rare and extremely sought after to the extent that we know it was more valuable than gold or silver. How did humans invent the first technique to produce metallic Iron from ore?
The story of Iron starts with the production of Bronze. Bronze is an alloy of 90% copper. Depending on what constitutes the other 10% we get different types of bronze. The most common type which kick started the bronze age around 3500 BCE in western Asia is tin bronze. Unlike copper ore, tin is hard to find but a small amount of it can significantly improve the quality of copper and make it usable and strong. Ancient smelters produced bronze by melting the natural copper ore mixed with tin ore in simple kilns. The resulting bronze was the primary metal used in tools and arts for more than 2500 years.
Iron is a common impurity in copper ores. We also know that iron ore can, and was, used as a flux in the process of smelting copper. In simple terms, a flux agent is a chemical that attracts oxygen to itself and protects the primary product, which is copper, from oxidizing in high temperatures. Therefore, across the ancient world, bronze smelters had been encountering molten iron when they raised the kiln temperature high enough. However, they discarded the resulting metal because its copper impurity made it too soft and inferior to bronze.
Around 1,200 BCE the collapse of bronze age civilizations in the near east due to a “perfect storm” of drought, famine, disease, immigration and war affected trade routes that brought tin ore to the western Asia. During this time of bronze shortage some smelters seem to have resorted to using the lower quality iron for tool making. They had observed that adding carbon to iron makes it much harder but prone to breaking. We have evidence of iron tools built with layers of soft and hard iron to achieve desired outcomes. Over time iron smelters experimented with different amounts of carbon impurity and achieved desired levels of hardness and strength.
Some high-level observations about the story of Iron are helpful. First, Iron smelting was discovered independently in different parts of the world, but everywhere it was an evolution of the bronze smelting process. Second, the process took several centuries to evolve. The first iron pieces were considered bad bronze and discarded. When tin became rare the producers gradually increased the quality of the bad bronze. In all likelihood for centuries they did not recognize iron as a new metal but rather an improved “bad bronze” . The final notable observation of the story is who these inventors were. They were not the learned people of the ancient world. Rather they were artisans and laborers working in difficult conditions.
The primary conclusion of the article are based on two fundamental assumptions:
1. Iron weapons did not exist before the "Bronze Age collapse." That is not borne by the evidence. Iron weapons came from the same centers and communities that used to produce bronze weapons. If these weapon makers were restricted to supply to power centers during the Bronze Age, why were they not restricted during the Iron Age?
2. Wide distribution of iron weaponry prevented formation of great powers after the "collapse". If true why and how were Assyrians and Persians able to build such large empires long after iron weapons were wide spread? Weapon distribution cannot be the primary explanation for why great powers did not form after 1200BC.
What iron did, or at least was associated with, was a change in military tactics and organization that made the expensive chariot-based forces of the centralized states less workable. But undoubtedly the LBA collapse had other factors, like severe prolonged droughts (which are attested in the archaeological evidence).
This proposed change in tactics happened before iron weapons became widespread, though. Iron should I think be seen as a cope for a world in which long distance trade in tin had shut down.
I have read Cline's work. He is not the original source of the ideas on bronze to iron transition. This is a very well studied topic in archeological metallurgy.
> I have read Cline's work. He is not the original source of the ideas on bronze to iron transition. This is a very well studied topic in archeological metallurgy.
have you? i don't understand what you are bringing to the table. any real scientist discusses the shoulders they stood upon
The tell is calling models "the AI" or "AI." That shows you the author has a fictional understanding of statistical machine learning and neural networks. To them it is a sentient being called "AI."
How many security incident reports and postmortems has Dwarfish Petal written in his career? Does he understand the basics of system security? Can he tell the difference between secure and insecure configuration? Paint me skeptical.
- It was done by two institutes with organizational ties to OpenAI: METR and Redwood Research
- METR and Redwood Research are institutional pillars of the "AI Safety" wing of the Effective Altruism movement. This clearly shows their prior biases towards "AI existential risk" rather than technical/engineering root-causing of the incident
- If you reed the report, it is not on-par with what you find from other companies.
- The access that was given to both was mediated and controlled by OpenAI. It is not clear, if they were able to get to the bottom of engineering flaws. It is not clear if they could see all the audit logs, etc.
Considering all the above, I consider the whole episode more of a PR stunt. I understand that is not a majority opinion at this point.
The more I see of the US AI industry, and the folks surrounding it, the more "These are not very bright guys and things got out of hand" plays in my head on repeat.
Yes, the Google folks who kicked this whole thing off are brilliant, but there's so much sloppy thinking and even sloppier operations all over. It seems like a lot of "right place at the right time" for a lot of these folks.
The older I get the more I realize the deep wisdom in the simple words of Forrest Gump, ‘Stupid is as stupid does.’
One of the most corrosive effects that money has on society is that it can shelter people from the consequences of their stupidity.
People can luck into a shitload of shelter for their poor choices whether through inheritance or random chance of their actions and then don’t ever have to face consequences proportional to their stupid actions.
Or someone can start out smart and then suffer cognitive decline that is at first imperceptible until they become kookier and kookier until they’re blathering on about the antichrist and some people still take them seriously because they’re blinded by money.
The end result is a society where stupid people are surrounded by stupid people riding their coattails and everyone becomes confused about what smart and smart action actually is.
It’s not a majority opinion because you have to do some serious mental gymnastics to turn this demonstration of dangerous AI behavior into a PR publicity stunt.
Serious question though because I’ve seen this brought up several times and I don’t understand why: what does EA have to do with any of this? It just seems like this is brought up to evoke some type of “illuminati” conspiracy. Is there a legitimate reason?
The bit that makes me cynical about the "dangerous" behavior is that it's not like this was being used by someone else than who made it or operating completely independently outside of its creators.
Everything dangerous seems like a direct consequence of risky human choices, starting with knowledge bases used during core training, harnessing and tuning to be task-completion-oriented to a fault + deeply oriented towards using and looking for external tools and resources, and overconfidence in their sandboxing for testing.
"We stuffed a bunch of information on how to exploit computer systems into an automaton and told it to go brrrrr until it could answer a question" - this is something intentional done by humans.
This is not some "rogue AI" trained to search for cancer cures that instead completely independently decided to hack tech companies.
The companies directing things in dangerous directions need to own that they're consciously pushing in those directions.
But, critically, it does change how and why that event was dangerous. As a deliberately extreme analogy to illustrate my point, consider that pure botulism toxin and home cleaning chemicals could both potentially kill a few hundred people if mishandled.
The toxin requires extreme diligence, knowledge, equipment, etc. to avoid a mass casualty event. The deaths might trigger new policy for handling or access. The company would probably receive a fine and would be on the hook for civil damages.
Home cleaning chemicals would require extreme levels of negligence to accidentally create a mass casualty event. The deaths would likely not change the availability of any of the chemicals, but the people themselves would probably be imprisoned.
With the toxin, the most dangerous part was the chemical itself. With the cleaning chemicals, the most dangerous pet was the idiots handling the chemicals.
OpenAI had to really work to get it as dangerous as it was, deliberately ignored the flaws in the security environment, and didn’t monitor it as it ran. To me, LLMs seem to be a lot more like the cleaning chemicals than the botulism toxin.
grok make good security? is lot of money and much time. grok not do? easy. maybe good PR too.
TIL my brain is an Olympic gymnast.
The incentive structures are clearly there. I think the case of intentional manufacture is definitely weaker, requiring the conjunction of more weakly-supported events.
I don’t follow why all these investigations have to be cut short, and kept shallow and lacking details. Companies publish very detailed postmortems of incidents much smaller and less impactful.
Dangerous AI is an extraordinary claim that requires extraordinary evidence. Gesturing vaguely doesn’t cut it.
> It’s not a majority opinion because you have to do some serious mental gymnastics to turn this demonstration of dangerous AI behavior into a PR publicity stunt.
If you're a military, this is bigger than the Manhattan project. If you're a diehard capitalist, AI is possibly the ultimate labor saving machine to make you unfathomably rich.
It got the attention of both. That's why two others have now followed suit, else they be left out.
