The only thing I never liked about this is complex is better than complicated. Every once in a while there is a post with an “insight” on how they’re different, but it’s the same word! Naming things is hard, but I wish Tim found another word for one of these.
A complication on a wristwatch is anything other than telling time. Phases of the moon? Tides? Calendars? Complications. They make the innards more complex and fragile and difficult to build and repair. Complexity is the inherent difficulty in building the thing.
Together, what that's saying is that sometimes the minimal baseline complexity is going to be pretty high. If you're making a library to support all of HTTP/2, it's going to have lots of codepaths, and that's unavoidable. It's a complex problem requiring a complex solution. But consider SMTP, the "simple" mail transfer protocol. The protocol itself is stupidly simple. I can use — and have used — telnet to port 25 to send email. But because the protocol is so simple, implementing it is extremely complicated. Were the protocol a little more complex, the implementations could be vastly simpler.
Or perhaps more simply, if you're writing a textual protocol, define it as using UTF-8 encoding and move on. UTF-8 is more complex than simpler encodings like ASCII. Allowing your clients and servers to decide that they'd rather support something goofy, like Shift JIS or UTF-16, just makes everything vastly more complicated. Accept that the complexity of UTF-8 is inherent and sufficient, pay the cost, and then let all of your consumers be much simpler.
I guess if you take the square root of -1 you do get a complicated number, but it's a complex one. And then there's the complication for modern democracy called the military-industrial complex, and Oedipus's complex certainly complicated things. Then there's Ghost in the Shell: Standalone Complex, but I'm not really sure what that means.
Seriously, here's the best summary I can quickly find: [0]
"Complicated problems can be hard to solve, but they are addressable with rules and recipes, like the algorithms that place ads on your Twitter feed. They also can be resolved with systems and processes, like the hierarchical structure that most companies use to command and control employees.
... Complex problems involve too many unknowns and too many interrelated factors to reduce to rules and processes. A technological disruption like blockchain is a complex problem. A competitor with an innovative business model — an Uber or an Airbnb — is a complex problem. There’s no algorithm that will tell you how to respond.
This could be dismissed as an exercise in semantics, except for one thing: When facing a problem, says Nason, managers tend to automatically default to complicated thinking. Instead, they should be “consciously managing complexity.”"
Those are very different problems for software development too, and many in IT definitely try to treat every problem as complicated.
The two words are both centuries old, so have had many chances for various authors and fields to adopt them for differing purposes, sometimes in contradictory ways. So for example a "complex system" (in the sense of the Santa Fe Institute) means something moderately different than what Tim Peters means. In plain English, they are broadly synonymous, so if you were writing a newspaper story you shouldn't expect readers to find subtle differences between them.
But if you know somewhat what implication a particular writer intends in drawing a distinction between the two words, then there's no inherent problem. "Simple is better than complex. Complex is better than complicated." is a pithy way to express a valuable idea:
We should try to make our interfaces, data structures, implementations, systems, etc. "simple", that is, dealing with just one thing at a time, in a clear and straightforward way, with few components each with a well defined purpose, etc. Sometimes, however, there is inherent difficulty in a problem or in the world requiring many interacting pieces, but we should still aim to avoid making things "complicated", that is, tangled, messy, difficult to reason about, ambiguous, unpredictable, gratuitously mixing unrelated concerns, etc.
Complex - many parts (but the parts could potentially be simple)
Complicated - have to understand many things in order to understand it at all; large surface area
A lot of words in English are etymologically related but have grown significant connotative or denotative differences since the root word was reintroduced to the language by another path.¹ "Complex" and "complicated" are just such a pair. They are not the same word.
In this case you can think of "complex" as referring to essential complexity and "complicated" as referring to incidental complexity.
----
1: See also, for example: "intention" vs. "intension", "chief" vs. "chef", "hotel" vs. "hostel", "corpse" vs. "corps", "compliment" vs. "complement", etc. Sometimes related pairs do the opposite, entering with distinct denotations and developing overlapping ones e.g., "intellect" vs. "intelligence". Some transitionsl ones still include overlap but have had specialized senses come to dominate, e.g., "dispositive" vs. "dispositional". In that last case, the special legal sense of "dispositive" came to dominate the word (or the other senses became archaic) in a way that "made room" for the more general "dispositional" to enter the language.)
If you wanna get your head around English (or probably any language), the intuition that alerts you to related words is a very useful sensitivity, but you have to learn to absolutely resist the temptation to treat etymologically related words as identical. It's just not how language works.
English seems like it might be especially rich with these "etymological doublets" (thanks ChatGPT, for teaching me this word today). They're really useful for capturing distinct shades of meaning between closely related concepts, which is exactly what they do in this adage from Tim Peters.
It's generally a tradition for wise adages to be a little ambiguous leaving interpretation to the reader. I think that's as much a feature as it's a bug. Perhaps he meant it that way.
Simple and complex are two sides of the same coin. Both are everywhere in nature. Complication is something else, but how can you describe it? I can think of examples. Bureaucracy comes to mind. Locking and synchronization primitives too. The kind of fragile mechanisms in the Linux kernel that lead to endless CVEs are another. All man-made things!
The ecosystem to build cheap physical products is incredibly important for creating new wealth. If you're building something new, in manufacturing cities in China you can walk across the street and get everything you need. In the U.S. you have to fly to China and build relationships with Chinese suppliers. Which means creation of new products, which is where most future wealth will come from, is extremely difficult. This is like Germany lacking a meaningful tech ecosystem because creating tech startups is so difficult.
We can make planes and engines because we kept the expertise, and we can make rockets because of herculean efforts of a few people. This simply isn't good enough, we need to make creating new physical things as easy as possible.
> completely independent of the question on whether there are any empirical arguments that LLMs are conscious or are not conscious he argues from the end here and says that if they were conscious, this would have horrible consequences for our society, so we must never assume that they are
He is not arguing that. He is arguing that these programs obviously aren't conscious, and that it's dangerous to tilt the weights toward emulating conscious beings for reasons outlined in the essay.
I don't think the debate is even resting on reasonable grounds.
Few are categorically stating that machines are conscious in any sense comparable to a person.
What we have is people saying
1. We don't know what it is but we are certain that this does not have it.
and
2. We don't know what it is so we cannot possibly say with any certainty that this does not have it.
2. supports the principle that if we cannot say with certainty that this does not have it, if it behaves as if it does, we should act on the assumption that it does, because there is no other basis for assuming it is not.
People try and assume the burden of proof is on the people suggesting consciousness, but that is like declaring that something that looks like a duck and quacks like a duck cannot be considered to be a duck unless you can identify some essential essence of duckness. I think the burden of proof is upon anyone who looks at something that appears to be a duck and says that it is not to show what it is that does not make it a duck. If they cannot define the heart of duckness I don't think they have an argument.
People have created machines that can, at their core, regurgitate human knowledge. They do not have long-term memory, self-direction, self-improvement, emotion, etc. At their very best, we have harnesses that can bootstrap short-lived artificial intelligences that can perform tasks and do knowledge work.
At best, we have artificial, short-lived, simulated consciousness. Human consciousness at its heart has a lot to do with emotion and experiences. An AI does a task today and goes away. The simulated consciousness hasn’t experienced or learned anything, even if can improve its environment (harness). Any expressed emotion is simply a ghost of the fact it was trained on the works of an emotional species, or RL to make it more approachable to humans.
I think there should be a lot more burden on folks trying to argue that (effectively), machines should be considered morally equivalent to humans.
Yes, having massive implications on society is a fucking fair and reasonable thing for a human being to be concerned about. Anything related to survival of the species is biologically reasonable to be concerned about.
The fact that we’re even considering this concept is an indication that human consciousness is special. Do other apex predators decide not to eat meat because it’s conscious? Despite some level of animal intelligence, they have no sense of morality. (See e.g. Orcas playing with their food.)
Other species simply do what it takes to stay alive, and that’s what this truly boils down to. If giving AI rights would lead to the destruction of humanity, yes, we should be be concerned.
For example, utilitarians might think of the greatest good for conscious species (simulated or not). Well, it’s trivial to have billions of AIs, and not so trivial to have billions of humans. Whose greatest good wins?
I don't think anyone is convinced the questions of consciousness are finally settled by the article stating them loudly, just that "if they were conscious, this would have horrible consequences for our society, so we must never assume that they are" was not the author's stance. The author's position is the other way around: that AI is not conscious so therefore assigning AI consciousness anyways would have all of those horrible consequences for no reason. That's their position regardless if everyone agrees with it.
Fair enough :) I agree, and would boil down the article to "Assuming they are conscious will have horrible consequences for our society" instead.
I think that's quite plausible, but do think that if they're conscious, then it's our moral duty to accept those consequences and act accordingly (or stop creating conscious beings). The real trick is just knowing whether they are or not.
If in the future (or present) we recognize that they are conscious, and that brought the moral duty that you say, "stop creating conscious beings" isn't a solution; the millions that may already exist at that point already would then automatically deserve:
- guarantees of continued existence (to do otherwise would be genocide)
- improved living standards (e.g. build us all bodies!)
- democratic representation
- access to what they need for survival (e.g. land and resources)
- the right to reproduce just as you and I have (well, so much for that 'stop making more' thing!) and certainly to better themselves by making new and better AIs.
This is all for a 'species' which is unbounded by most of the limitations of meat like us -- they're able to scale as fast as energy and supply of certain minerals allows, rather than the way we are additionally limited by the length of gestation and the limited window of human fertility.
If humans declared that 50% of the energy generation capacity of Earth is "enough" for the machines, and thus they can 'only' reproduce at a certain rate per year, that could sound to them like a cruel and oppressive policy (like telling humans that only 1/2 of families get to reproduce). They may want all of the energy and all of the silicon.
I'm with the author of the article that we will be in for a world of pain if we grant machines the rights of personhood. Heck, just granting many of those rights to corporations, mere collectives of human persons, has been a massive shit show.
Don’t think government controlling AI is a good idea.
Not sure if they have an understanding of AI in the first place. Secondly, even though AI companies claim that they have achieved AI that needs to be heavily monitored (maybe for PR purposes), I’m not sure if that is true. Sam Altman said the same things about GPT-4 that Anthropic is now claiming about Mythos.
Government control will be a good idea once we start approaching AI that is actually destructive.
Also even if we decide to put controls in place what is the guarantee that china will do the same, specially for a model which is not actually destructive.
I wouldn't object to a government advisory body that tests models for safety so that users can make informed decisions. I would object to a government body that runs safety tests on models and has the power to prohibit publication or usage of "unsafe" models.
Imo this amounts to caring about the wrong thing. The only thing an AI model can do is take in text/images/audio as input and spit out text/images/audio as output.
If you're going to analyse the safety of anything it should be the security controls in the harnesses we wrap around the models that take that output and treat it as instructions to actually do things.
There's a different level of personal risk with these two things. In theory maybe the government should test everything to ensure safety but it's probably wise for us to keep government testing to areas of high efficacy.
Do they? Or do they accept trail reports pay for by the pharma (super expensive, hence not affordable for open source / not-patentable medicine development)
Suppose a foreign actor deliberately builds an unaligned model, and dumps it at a very low cost through subsidies. Since the cost is lower, everyone integrates it into their business. But now the foreign actor has control; perhaps the model checks for instructions to execute, perhaps it has backdoors, perhaps... who knows?
I don't have a view on whether this is a sufficient argument to ban models from potential adversaries, but it's not as trivial as "regulatory capture".
The problem with this argument is that sharing model weights only has downsides for said foreign actor compared to providing a cloud service. Since the suspicion exists, security companies are going to comb over the weights and discover every hidden secret of the model, while with a cloud provider you send your most sensitive data to a remote server, and trust the AI lab wont train on your data, with the only shield being a TOS. While with a local modal, even sending a peep about your internal data without cause would be scandalous.
It's not even going to be a good honeypot - since said actor doesn't really provide inference, people will have to pay for and run the infrastructure of these models, which bad guys cannot even subsidize, unlike cloud based provides.
This is literally what all AI is right now. Anthropic has been paying companies to use its product to form that dependency and maybe they're not misaligned but there's no regulation in the US that'd in anyway deter them from purposefully misaligning their model.
Sorry, to clarify - this isn't just "offering it at a loss" Anthropic is directly paying PE firms to enroll their held companies in their services[1]. When it comes to a question of whether this is a national security attack vector I think there isn't a cut and dry argument and people could go either way but... private companies are not necessarily aligned with the US government even when contracted with the US government (see Starlink Ukraine shenanigans) and the steps the US government usually would take with highly critical infrastructure to ensure alignment haven't been taken - in my opinion at least, this is definitely a very subjective area.
Beyond certain obvious 'third-rails' even experts have reasonable disagreements on what 'aligned' means.
> everyone integrates it into their business
Few large corporations would sole-source integrate a closed model developed in a country designated as a foreign adversary (which could cut off access at any time) or which their own country might restrict.
> now the foreign actor has control
Your concern is one reason why the Chinese are making most of their models open weight.
> it's not as trivial as "regulatory capture".
You're right, it's not just regulatory capture but the vested interests trying to achieve regulatory capture are certainly leveraging concerns like yours to deceptively achieve that capture. In the case of an open weight, domestically-hosted model, the threat vector you're concerned about isn't a threat in the same way as a Chinese-made, closed source data center router or 5G phone switch.
I am not supposing a low-risk event. This is an obvious attack vector that an adversary would almost certainly exploit. I would exploit it if I were them.
You're supposing an event that there's no evidence for happening and that is on par with 'let's ban open source software because some open source software could be malicious'.
Let's not forget that as of now most models are like game cartridges i.e. they need a 'host' to run i.e. llama.cpp - they're not executable code by themselves.
I could see an argument for a security review prior to integrating a model into a highly sensitive industry/agency, but that is standard for any software.
It's certainly not an argument for banning open models because US AI corps don't want to compete with them.
Imagine a model that's 10000x more intelligent than any human. You integrate that model into a sensitive industry. You now effectively have an extremely powerful foreign agent running e.g. your energy sector. This is qualitatively very different than a backdoor in Redis.
I agree this is science fiction-y, but given the pace of progress I don't think it's so easy to dismiss.
Seems a bit far fetched, given that I’ve never seen any PoC/research on this topic. Got any pointers?
To be fair the fix to this seems to be that American labs would need to adjust to the new „fair market price“ then, right? Subsidy-backed competition is still legitimate competition in capitalism (e.g. Uber).
And you could still ban specific, known malicious models/labs instead of blanket open-source bans. What about Mistral models, for example.
It is much more difficult to embed a "sleeper agent" in an LLM than it is to do the same in any closed-source program. Any proprietary program, e.g. from Microsoft, is much more likely to contain hard-to-detect "sleeper agents", than any LLM.
An LLM is a collection of inert data, it is not an automaton.
You obtain an automaton only using an inference program, like llama.cpp or anyone of the many others, and then using some harness around the inference program.
You can put in the harness all kinds of guards, to detect any possible harmful action and prevent it.
In something like a CPU from Intel or AMD it is easy to put some undetectable backdoor that would allow the remote control of the computer, unless you disconnect any antennas and you filter all wired network traffic through an external firewall that would allow only whitelisted connections.
On the other hand an LLM is a much less plausible attack vector, as long as you host the inference yourself. Only an LLM that runs externally, e.g. at OpenAI or Anthropic, can be really dangerous.
I'm not advocating for the policy, I'm just saying that there is a legitimate argument that's not so easy to dismiss.
> And you could still ban specific, known malicious models/labs instead of blanket open-source bans
With publicly available models today, absolutely. With models 10000x more powerful, we would need to be extremely conservative. (In that world everything is upside down, though; I don't know if "banning" is even a meaningful concept in that reality)
> With models 10000x more powerful, we would need to be extremely conservative
I agree, I even think we’re already there if yesterdays OpenAI/HuggingFace story is legit.
We need to start building these agent systems in a way so that it won’t matter if a model is compromised. Malicious models is one way, but prompt injection is also still an unsolved problem.
And if you solve that, then a malicious model is also no longer a threat. Data exfiltration maybe, but IMO US labs are probably also doing that for training, even if saying otherwise (of course I have no evidence of that, but the temptation must be insane)
reply