I listened to this podcast and it was maddening. A few of his frames :
AI is just software, nothing new to see here.
AI safety is primarily a sandboxing problem.
There is no collective action problem, and every company should just slow down if they think they need to slow down.
There is no need for regulation because the existing incentives in the market keep companies from acting badly, which is why no company has ever done anything bad.
If something bad does happen, then we can regulate after the fact.
We will end up creating more jobs than we destroy, so don't worry about it.
Our kids might forget a whole bunch of stuff or never learn it in the first place. But don't worry, they'll come up with new things to learn instead.
The only way to get safety is to move faster because then we will more quickly arrive at safety.
Recursive self-improvement is just what we've always done.
The real danger is alarmism that might scare the public and the young people.
It’s feeling less and less like human progress and more and more like progress for a small group at the top of our economic structure who are hoovering up everything for themselves.
Who gives a fuck if some techno barons can deploy novel software at the drop of a hat if it means I lose my career in an environment without social safety nets.
It’s not progress from my perspective when my life quality regresses.
Answer the question: who's coming to save you? Bernie? Donald? AOC? J.D.?
Better if you don't need saving at all, because you've been empowered to help yourself at your job. Ultimately the goal is to free you entirely from dependence on a day-to-day occupation.
You may have noticed that no one else is even trying to help.
lol, no. It’s currently being deployed in feverish attempt to remove jobs.
I see company doing layoffs left and right because they can do the same or more with less employees. I’m not seeing any companies feverishly hiring engineers because the productivity per hour of labor is higher.
There is also no guarantee that AI will lead to any social safety net, they are orthogonal concepts.
The only thing that would lead to a social safety net is political will and currently all the political will behind the technology firms building AI appears to be firmly behind dismantling democracy and becoming oligarchical leaders of their little fiefdoms while the rest of us can get fucked.
Two questions: 1. Have the "fud-spreaders" actually held anything back so far? Seems like we're barreling forward, even though some of the FUD spreaders appear to be in charge of the main companies. 2. Do you, like Huang, see absolutely no risk in our current approach? Is everyone who disagrees with your perspective on this subject just plain wrong? Misguided, or actively malicious/in bad faith?
Have the "fud-spreaders" actually held anything back so far?
I don't know. Astroturf campaigns to subvert data center construction seem to have had an impact, for example, but ultimately I can only fall back on vague vibes picked up from the news and social media. I don't know the specific success rate.
Some of those campaigns seem justified to me (why in the world do these things have to be built in populated areas?) while other arguments made by the anti-data center movement are certainly propagated in bad faith (Water usage? Really?)
Seems like we're barreling forward, even though some of the FUD spreaders appear to be in charge of the main companies.
That's just so much kayfabe. Don't listen to what they say, watch what they do.
Do you, like Huang, see absolutely no risk in our current approach? Is everyone who disagrees with your perspective on this subject just plain wrong? Misguided, or actively malicious/in bad faith?
I can't speak for Huang as I don't know him personally, but he has to walk a tightrope between an infantilized POTUS with essentially unlimited power to wreck his business, and his own stockholders who have elevated his company's market cap to a level that probably doesn't make sense even to him. As far as I can tell he's just going the obvious, predictable, and safe route of trying to sell as many shovels and tents as he can while the selling is good.
I see a bigger downside in the precautionary principle than in accelerationism. We are going to face immense societal disruption due to AI over the next couple of decades. That is as unavoidable as an earthquake. It is going to happen regardless of who prevails in the political and social arguments going on today. There is no possible way that every actor in every country on Earth is going to be convinced or compelled to "pace the frontier." Consequently, all that matters is how we react, adapt, and take advantage of the new ground rules of reality.
As usual, there will be those who thrive and those who get steamrolled. But I see no reason why we won't all end up better off in the end, as with the rest of the history of technology since the invention of fire.
I think this is overly optimistic; it assumes universities reacted instantly and pivoted to teaching coding with AI as soon as it was feasible. In actuality, university curriculums cannot change quickly, and AI capabilities are changing much faster. In my experience, some students are using AI to write code, but then have no idea what's going on. The "skill" of typing "Write a function to do [basic thing]" will not make them AI-native or help them in the workplace.
For CS students I agree. For non-CS students, who need programming as a way to use computers as a tool, I think they do need a solid understanding of what a computer does and what it's limitations are (and what it's capable of), but for their day jobs (once they graduate), they will most likely be getting AI to do everything for them. As long as they can reason about what's happening, to understand the results and to improve them, this seems likely to be the path in the future.
This is an introductory programming course, designed for non-CS students e.g. engineers and scientists.
> Learning outcomes
> 2. Explain key concepts in AI-assisted programming, including Large Language Models (LLMs), prompting, problem decomposition, and top-down design.
> 3. Apply the workflow of AI-assisted programming and prompt-engineering techniques to guide and improve code generated by AI assistants.
This course used to be non-AI (last year), and they rewrote recently to incorporate AI tools, as they realised the writing's on the wall for non-programmers.
It must be quite challenging to write curriculum when the underlying technology (AI) is changing so quickly.
It also is naive to think, if AI is shrinking head count, that companies will seek anything other than seniors.
2008 triggered a glut of cheap experienced workers that slowly reengaged juniors but this time that the entire ladder is going to be more valuable than any grad.
In my experience, some students are using AI to write code, but then have no idea what's going on. The "skill" of typing "Write a function to do [basic thing]" will not make them AI-native or help them in the workplace.
How many of them can read x64 or ARM assembly emitted by their compilers?
It’s how I deeply understand what a RAM lookup vs having it already in a register means for optimization.
Me, too, but we should both understand what we're talking about: a hobby.
It's like teaching cursive to schoolkids. All well and good, but don't you dare complain about limited classroom time for instruction in other, more important subjects.
Understanding how computer architecture works is a hobby? Is unimportant?
I built and deployed embedded systems robotics that protect water and oil pipeline infrastructure that costs billions of dollars if there is a failure.
That is not a hobby, that’s a career.
The tools I built just helped fix the fresh water access for my entire city of 2 million people. Explain to me what’s more important than people having access to water.
That's great, you sound proud of your work and you should be. But what about all the human pipeline-inspection crews you've put out of work with your robots?
AI is not an abstraction layer. If you work entirely on the level of prompting without any knowledge or understanding of the underlying code, you are not actually an engineer, but more of a half-assed technical manager. (And your job will be first on the chopping block.)
for a guy who is in the middle of it all, the opinion seems sophomoric at best. LLMs are based on heuristics and they will always make mistakes no matter how better they get. You'd need someone to 'fine-tune' the conversation with the LLM.
I'd love to see Huang's definition of abstraction in which AI is somehow "a new abstraction level" but having some contractors build your product is not.
love love this theoretically but in practice, at least in this “phase 1 of AI craze, unknown how long it’ll last” that someone coming out of Uni with deep knowledge of ABCDs of SWE will be unemployable unless she/he is AI native with harnesses coming out of their ears
With the massive layoffs there have been massive spikes of suicides and od deaths that will only get worse. Articles like this are simply trying to cover it up.
>"The first chip Huang worked on had 200 transistors, each of which he said he knew by name, while today’s engineers assemble systems from chips containing hundreds of trillions of them without ever working at that level. “Some of the lower-level knowledge is gone,” he acknowledged, and he later described AI as “clearly” a new abstraction level in the same progression."
AI is just software, nothing new to see here.
AI safety is primarily a sandboxing problem.
There is no collective action problem, and every company should just slow down if they think they need to slow down.
There is no need for regulation because the existing incentives in the market keep companies from acting badly, which is why no company has ever done anything bad.
If something bad does happen, then we can regulate after the fact.
We will end up creating more jobs than we destroy, so don't worry about it.
Our kids might forget a whole bunch of stuff or never learn it in the first place. But don't worry, they'll come up with new things to learn instead.
The only way to get safety is to move faster because then we will more quickly arrive at safety.
Recursive self-improvement is just what we've always done.
The real danger is alarmism that might scare the public and the young people.
Nothing bad can happen, it can only good happen.
reply