The exponential time hypothesis is essentially that NP-complete problems' best solutions are all exponential.
NP-complete problems are essentially those where you need to make some kind of choice over a superlinearly-growing set of options where each option can be solved in polynomial time. Like, for example, finding a route between cities on a map under a certain total distance. You might need to go over all choices of routes.
The ETH essentially assumes that the best solution (in terms of complexity, i.e. how it grows) you can get for these kinds of problems is exponential. OpenAI just proved that this is false, you can have solutions that are below exponential complexity (but still above polynomial - if anyone finds a polynomial solution then we'd have proven that P=NP)
> NP-complete problems are essentially those where you need to make some kind of choice over a superlinearly-growing set of options where each option can be solved in linear time.
Seems like a weird way to describe NP, and I'm not even sure it's true?
NP is the class of decision problems where if the answer is YES and someone slips you a hint, you can solve them quickly (or quickly figure out that the hint was useless and report that). Where quickly means in polynomial time.
There's nothing about linear time in the classic definition.
Sorry, you're correct. I meant "polynomial" rather than linear.
NP = nondeterministically polynomial. i.e. if you could evaluate all choices in parallel (or just "know" the right choice), then you'd get a polynomial runtime. That's what I was trying to informally hint at, but I ended up using the wrong term. I'll edit my original post.
That's equivalent to your description - the "hint" is knowing which of the options to choose.
The reason why superlinearity comes into play is that if you had a linearly growing amount of solutions to test then you'd just have a sum of polynomials, which is itself polynomial.
Ok, I read your edited comment. Looks a lot better know.
One nitpick: I think your set of options needs to grow a lot faster than just superlinearly, I suspect it needs to grew superpolynomially?
However otherwise what you describe is essentially true of all problems in NP, not just the NP hard problems:
Sorting is (trivially) in NP, and you can say that to sort you need to pick one out of n! many permutations (which certainly grows superlinearly and even superpolynomially).
NP hard problems have the distinction that if you solve one of them efficiently, you can solve all of them efficiently, because you can reduce them to each other. (Basically because NP hard problems allow you to sort-of describe a computer, and reducing them to each other is equivalent to writing an interpreter or compiler.)
This is not a proof that NP != EXP. NP is poly time reduceable to 3SAT + 3SAT is sub-exponential does not imply that NP is sub-exponential -- the cost of the reduction may be enough to push the conjunction from sub-exponential to exponential.
This proves that EXP != NP, which was a stronger claim than P != NP.
In a way it shows that NP is perhaps less distant from P than imagined. Whether it can be _reduced_ to P in some gigabrain way, or whether that can be shown to be impossible remains to be seen.
It is definitely progress towards an answer to P ?= NP.
This is not a proof that NP != EXP. NP is poly time reduceable to 3SAT + 3SAT is sub-exponential does not imply that NP is sub-exponential -- the cost of the reduction may be enough to push the conjunction from sub-exponential to exponential.
It's hard or maybe impossible to figure out what BB(10) is. But abstractly, it is some finite integer. That finite integer is trivially computable: make a turing machine that hardcodes all the digits and outputs them one by one. You can't know that this machine actually produces BB(10) if you don't know what BB(10) is, but that's separate from the question of whether the number itself is computable.
There is some notational subtletly here. Strictly speaking, BB(n) represents the output of the BB function for the input n. In that sense BB(n) is computable.
However, the BB function itself is not computable. It makes sense to ask "for what inputs is the BB function computable?". When people talk about BB(n) not being computable, they are really talking about the computability of the BB function at n, not the computability of the integer that happens to be BB(n).
BB(10) is computable. That doesn’t mean anyone can actually figure out what it is. It doesn’t even mean it’s theoretically possible to do so. Computable just means that there exists a turing machine that will output the digits.
I feel like I keep failing to wrap my mind around computability in the same way over and over.
Would the following statement be correct?
"There is a turing machine that cannot be proven whether it will terminate or not. For any n we can construct a proof that it will run longer than n. But we cannot construct a proof that for any n it will run longer than n. Due to this particular turing machine, BB(k) cannot be determined for k large enough to admit that turing machine."
So then if the BB sequence has finitely many entries. What does it mean that it grows faster than any computable sequence? A finite sequence can't have any asymptotic behavior right?
The BB sequence still has infinitely many entries. Any turing machine will either halt or not halt. For any N, there is a finite set of N-state turing machines which halt, and there is therefore a finite number of steps after which all halting N-state turing machines would halt.
There is a point where these values become uncomputable, but they still exist (for some definition of exist).
I am not sure I follow? But maybe I misunderstood your quote. It seems any given TM (And for a fixed N, there are only finite many), there are only the two option: it halts, or it doesn't halt. There is no third option?
Of course, the question whether you can proof that it doesn't halt, in ZF or ZFC, is a different question. (You can always proof that it halts if you know that it halts. You run it until it halts)
I'm not entirely sure about what I'm unsure about, but I think I struggle to accept that there is truth beyond the axioms. Like in my mind if the axioms can't prove whether it halts or not then that's that. There is no "but actually it doesn't halt". Or at least that's where my current thinking is, I think?
I do lack a lot of knowledge about the concept of interpretation, so maybe that's where my issue is?
All great questions. You're dancing around the edge of model theory here and the difference between truth and provability. It's probably past what can be explained well in a comment. Nevertheless, with omissions, simplifications, and my limited knowledge:
In logic, a model is a "set" of objects and all the true facts about those objects. For example, the integers and "1+1=2,1+2=3...". If a model deals with truth, then a "theory" is a mathematical structure that deals with provability. For example, the theory of groups contains an axiom "a+0=a" that applies to any a.
We say that a model "models" a theory when everything the theory proves is true in the model. The integers are a model of a group.
But, the integers mod 2 ({0,1}) are also a model of a group, but a very different one.
So, consider the statement "1+1≠0". The theory of groups isn't powerful enough to prove or disprove this proposition. It is a false statement in the integers, but a true statement of the mod2 integers. So there is a gap between what is true in a model, and what is provable in a theory.
(Look up both godel's completeness theorem and his incompleteness theorem for more insight into this relationship)
Back to turing machines, our most popular foundations of math is a theory named ZFC. With this theory we can prove many things, but there are also many things that can't be proven or disproven. ZFC lets us prove that "BB(500) is an integer greater than 5". But it doesn't let us prove that "BB(500)=174936...35" (pretend that's a concrete number). In fact, there probably is a model of ZFC where it is true that BB(500)=174936...35, and there is also probably a model of ZFC where BB(500)=284729...86.
Axioms define what is provable, but godel showed that there will always be more true things than there are provable things (if your theory is finite-ish). Most concrete questions about the BB function will never be provable.
As others have said, the sequence is infinite, though unknowable. Even if math can't say what the precise value is, it can still prove things about the shape of the sequence.
As a simple example, we know that BB(1000) <= BB(1001) (since you can encode the longest running 1000-TM with an extra, unused state as a 1001-TM) even though we will never know either of those values.
The argument that it grows faster than any computable function is similar in that it proves things without knowing concrete numbers. (I'd have to look up the details of the proof to say more)
You should be able to prove it by showing that the existence of a faster-growing computable function lets you solve the halting problem.
Assume f(x) is computable and grows faster than BB(x). We can create a computable g(x) where g(x) > BB(x) for all x, by setting g(x) = f(x) + C for some suitable constant.
Now we create a computer program that does the following. 1. Get the size of the input program and compute g(n). 2. Run the input program for g(n) steps. 3. If it stopped, return true. If it's still running, return false.
This solves the halting problem. Since a computer program can't solve the halting problem, this program can't exist. Everything except "compute g(n)" is straightforward, so there must be no such function g, and no such function f.
Where did you get that the BB sequence would have finitely many entries? You can choose any integer n for BB(n), and there are infinitely many integers.
BB(5) is also computable in the intuitive sense, because we proved its value: 47,176,870. In this sense BB(5) is just as computable as Graham's number. However, there is likely a Busy Beaver number >5 that is intuitively uncomputable, such that there is no non-cheating algorithm which computes it. A cheating algorithm would be one that requires the knowledge of the result in advance to write the algorithm, in which case we wouldn't be able to come up with the algorithm in the first place, and we wouldn't know that the result is correct.
There is also a proof that the function BB(n) (with the free argument n) is uncomputable. However, this alone doesn't mean that e.g. BB(6) (a function with no free parameter) is or isn't intuitively computable. Though if you allow cheating, which is the case for the conventional academic definition of "computability", every particular constant function (that is, without a free parameter) would be considered "computable". Though that definition doesn't mean it is intuitively computable, because if you cheat, you aren't really computing the result, you are just reporting something without having computed it.
Every BB(n) is trivially computable, since any integer is computable. So for every BB(n), there is a computable sequence that grows faster than the BB numbers through n (annoyingly, given the word "grows", this can just be the constant sequence of BB(n) + 1).
So for any given computable number, the fact that it's computable gives no intrinsic reason to believe it's smaller than BB(n) for any particular n.
No, high enough busy beaver numbers are not computable. They can be defined but there's no way to know what they are.
Even for very small n you can create Turing machines where it's incredibly hard to prove that they halt: for n=25 you can create a Turing machine that stops only if the Goldbach conjecture is false.
Though that isn't enough to prove it's not computable, the point stands that if you could compute BB(n) for arbitrary n, you could run a Turing machine for BB(n)+1 steps and if it hasn't stopped yet then you know that the input corresponds to a machine that never halts. So it's impossible to write a program that computes BB(n) for arbitrary n, because doing so is equivalent to the halting problem.
And worse, there are some Turing machines where "proving the Turing machine stops" means "proving that a given set of axioms is consistent", where Gödel's incompleteness theorem says that it's impossible to build such a proof. So, that is how there is some n after which the busy beaver number is defined, but is not computable at all.
What I mean is that any given integer is trivially computable.
The busy beaver sequence is not computable (I am aware that somewhere around BB(700), you've encoded the consistency of ZFC). You cannot necessarily compute that BB(n) is a given number. But you can compute larger numbers. There is no such thing as integer that is too large to be computable.
I did slightly fuck up--the English language "every BB(n) is computable" is a bit ambiguous. What I should say is: "for all n, BB(n) is trivially computable, since it is a single integer".
The busy beaver function is uncomputable. That is well understood. However, for any given n, there is a number BB(n). Since that number is an integer, it is computable, even though it is the value of an incomputable function applied to a particular n. In particular, for numbers > 5, we have no idea what that number is. But whatever number it is, that number is computable.
Look at the post that I was responding to. It made the bad inference that because Graham's Number is computable, it ought to be smaller than BB(10). That confuses computability of a given number with computability of a function.
Computable doesn't mean that you can write it down the number that is the value, given an oracle that tells you the value. It means that you know that it the number you are writing down is the right one, because you computed it.
You can always exhaustively run all Turing machines of a given size, at each step comparing the state & tape to all previous steps. If they ever reach the same state with the same tape as a previous step, they're in an infinite loop and never halt. If not, they eventually reach the halt state. BB(n) is computable. But BB is not computable, because there's no general algorithm requiring finite storage and time which computes the results of BB(n) for all n. You have to brute-force it for every positive integer n, which takes infinite time (and space) and is therefore not computable.
There is no requirement to have a recurring state when not halting. Indeed, the most trivial machine "write 1 and move right", will never have the same tape state.
First, TMs don't have a stack. Second, still no recurrent state needed.
Extending a state with additional variables doesn't make a unique state less unique anyway, so "total state" is always more or same unique as "just tape".
Some turing machine variants have multiple tapes, effectively registers.
If the tape, instruction position, and all other variables are the same as any previous step, then the deterministic machine is going to continue to find itself in that loop, and never halt.
Think Collatz like functions - if the output starts into a loop, you know the end result.
However, any program trying to detect all loops by maintaining a bloom filter of every previous state doesnt work, that drags any computation down by a scale of its inputs^runtime.
Yes, doesn't change the fact that you do not have to hit such a state to have a non-halting program. I already gave an example: a program that continuously writes to the tape will not stop, and since the tape is always different, will not trigger your loop detection.
Exactly, never the same state. And still doesn't halt. That's the point. You cannot detect all non-halting programs by detecting if they go into the same state again. They can, but don't have to.
If you want a function to compute BB(n) for any supplied n, then it isn't computable. But a specific number e.g. 123, BB(123) is just a number and for every integer i there is function const_i that computes i. You just will never know which f_i computes BB(123).
There's a lot of weird, seemingly LLM-generated comments hyping the project here.
I think HN should start thinking seriously about more effective strategies to combat LLM-assisted astroturfing, if they aren't already.
None of this speaks about the quality of the project itself, but the fact that someone seems to be trying to manipulate me into caring about it makes me less inclined toward wanting to see anything relating to it.
I am not sure how to recognize llm-assisted astroturfing and how to filter it. Because I commented the project that I liked it but got downvotes - maybe my sentences were weird :)
Your sentences are weird and your praise does more harm than good for the project, in the first order at least. In the second order, comment counter is probably the most valuable ad on HN, so your comments (and mine, and all the replies) contribute positively to that.
Yes, it is. If you convince a model it is inside a sandbox it is much more likely to comply with requests that would normally be against its guardrails.
>assuming their difficulty is in the same ballpark
It is not. Many mathematicians have said that they don't expect to see solutions to the other 5 problems in their lifetimes.
Of course, AI might advance fast enough that the "in their lifetimes" part ends up wrong, but there is a considerable gap to the other problems. We have barely any idea of how to even get started on most of them.
Zitron has become the distorted reflection of the very AI boosters he criticizes and mocks.
I think the worst thing that happened to him was AI skepticism becoming a political position. This gave him a captive audience - as long as he says what they want to hear, which means that he can never ever concede that he might have been wrong or that AI might actually be progressing or having successes.
This is not conducive to good prediction long-term - rather it leads one to a state of cognitive dissonance where one's chosen enemies must be simultaneously terrifyingly powerful and incompetent dunces. The propagandist's disease.
How? AFAIK, his arguments based around the economics remains the same, he just seems increasingly dramatic and exacerbated, which is understandable when you realize industry + ecosystem (politics/reporting) is tulip mania delulu and insists 1+1=100, or 30 trillion, or whatever. His position isn't based on AI progress - it's based on AI economics, at this point he can be an obnoxious rationalist slamming flat earthers - just because he's annoying/smug doesn't mean he's less right on fundamentals which if anything is more clear now.
When you predict a company is going to fail and instead it sets revenue records you're not "dramatic and exacerbated". You're refuted.
Luu's post includes a (long!) list of specific predictions that aren't "exacerbated"; they're simply wrong.
Luu's point isn't that AI is going to succeed or that the "AI bubble" will never pop. It's that these predictions are all wrong. If you agree "directionally" with Zitron, all that means is that you're skeptical of AI. That's a totally reasonable position to have, but it has nothing to do with whether Zitron's predictions are good or bad.
No, the whole thesis is XYZ likely fail because REVENUE RECORDS is not enough to dig out of hole relative to MAGNITUDE MORE SPEND. Saying Zitron wrong because XYZ made $2 for every $10 it spends revenue needed to justify spending. Fixtaing on the $1-$2 is misdirection/innumeracy, the thesis is in reaching the $10 relative to time, i.e. that $2 has to be $10 in X time, but the current velocity suggest it will not be.
I agree with Zitron directionally on accounting, I in fact disagree with him on AI... I am extremely AI pilled, i.e. I think there is a future where AI is worth trillions and will capture large swatch of economy. The transformation will be extreme, unlike any past revolutions... but the accounting suggest that future isn't coming in time to rescue current AI incumbents from finance blackhole, which some may survive, i.e. bail outs, nationalization... but the $$$ suggest however we get there, there will likely be massive $$$ corrections involved irrespective of adoption.
You keep saying this. It doesn't mean anything to "directionally agree with Zitron" in the context of this Dan Luu piece. All you're saying is "you're skeptical of AI and the AI business model". Bully for you! Lots of people are. Nobody is dunking on you for having that skepticism. They're dunking on Ed Zitron for making a long series of patently risible specific predictions.
This just assumes Luu's piece has good argument that refutes Zitron's thesis when it doesn't. Saying Zitron is wrong because XYZ made $2 instead of $1 is different then Zitron staying XYZ is unlikely to make $10 in time when all signs still point to overperforming at $2 is not enough. It's a stupid reason to dunk.
That's a gross mischaracterization of what Dan Luu demonstrates in this piece as anybody can see for themselves simply by clicking on it. But the radical divergence of our premises (mine: based on the actual article; yours: unclear to me) explains why we're talking past each other.
It's not mischaracterization to point out limits of article is a poor basis to dunk on Zitron because it does not refute Zitron's broader thesis only nitpick what is really minutae/noise that can be explained by trillion dollar companies short term financial engineering / spreadsheet maxxing rosy picture. Even if Zitron short term forecasts/model sucks, it doesn't discount directional validity of his medium/long term thesis.
Someone in deep debt backstopping with maxing credit cards is not dunking on outside observer saying this arrangement ultimately not sustainable. The article is nitpicking over short term micro/liquidity when ultimate macro/solvency. Now maybe there's plenty of credit cards to max out, but systematically someone is going to end up holding the bag, and politically that could be public socializing costs. If folks want to use article to dunk on Zitron short term forecasts, it's whatever, but I think important to point out it doesn't refute his long term thesis around fundamentals, which again does not mean fundamentals cannot be overridden by non market means, but that's also a crux of the long term thesis - in lieu of correction/market clearing, we're going to see non market interventions to save current model from its fundamentals.
"Saying Zitron is wrong because XYZ made $2 instead of $1" is not a reasonable summary of the post we both just read and I don't think there's a point in trying to hash this out further if you disagree with that.
> you realize industry + ecosystem (politics/reporting) is tulip mania
It could be "tulip mania" or it could be "the internet".
Luu analyzed the numbers instead of just reacting to hype.
Specifically:
>> Oct 2024: OpenAI's forecast of $3.7B revenue in 2024 and $11.6B in 2025 and $100B in 2029 are absurd, "a statement so egregious that I am surprised it's not some kind of financial crime to say it out loud"
> Wrong (2025 goal exceeded, 2029 TBD but not an egregious financial crime level of implausible)
>> February 2025: Anthropic making $34.5B in revenue 2027 is "is laughable on many levels, chief of which is that OpenAI, which made around twice as much revenue as Anthropic did in 2024, barely made a billion dollars from API calls in the same year."
> Wrong (whether or not they make that in 2027, their 2026 ARR greatly exceeding that makes the 2027 estimate non-laughable)
On a spectrum between tulip wilting and fiber build out, we know deprecation cycle of DC hardware leans towards tulips i.e. <10 years (very generous) vs 20+ years for fiber layout, a lot of which is actually infra/earth works etc.
The numbers being cited is ~100B is well within accounting/ledger maxxxing tricks relative to current pool of investment. Luu is not analyzing number's he's just listing and believing numbers, and analytically entirely avoids the core Zitron thesis... once you tap out of easy investor $$$, FAANG warchest, accounting tricks... where is the rest of the order magnitude more $$$ that justifies existing spent relative to time frame coming from?
This narrow focus, of course some intermediaries in industrial chain is going to make $$$ selling/renting shovels - there is stupendous amount of $$$ being moved around, there will be some very phat winners, but even more losers in aggregate on broad ecosystem level. [1] is actually illustrative, there's a reason why opex low - capex premium is ridiculous right now, with almost everyone along compute industrial chain capturing 50%+ margins. Investors are burning $$$ and companies and pillaging warchests, intermediaries are raking in $$$, but that doesn't mean investors or companies doing all the spending will make more than they spend, i.e. the net ecosystem business model is not sustainable precisely because intermediaries are capturing crazy rent relative to actual monetization to sustain.
> but that doesn't mean investors or companies doing all the spending will make more than they spend, i.e. the net ecosystem business model is not sustainable precisely because intermediaries are capturing crazy rent relative to actual monetization to sustain.
You understand that this doesn't follow at all right?
The intermediaries margins can compress.
> opex low - capex premium is ridiculous right now
What does "capex premium" even mean?
Of course you spend more on capex when you build a data center than opex!
High capex matches the expected business model. If opex was high then everyone would be worried!
Investors exuberantly build $10 of housing when there is $5 of demand, builders extract $8, when they normally extract $2 under normal margins, builders raking it, but arrangement is net loses vs world where investors build same housing for $4 and make a profit. Intermediaries margins can compress but what they already extracted for current build out is already built in balance sheet.
>What does "capex premium" even mean?
>Of course you spend more on capex when you build a data center than opex!
No. Historically DC opex > capex, i.e. 60-80% goes towards power... because hardware costs were relative low % of TCO. Historically without delulu AI demand, IC producers capturing much less margin and TCO of DC was much lower than it is now. It's not opex vs capex it's TCO. AI is paying $10 vs $4, when demand is $5, $10 isn't sustainable, $4 is.
Now builders will be fine in case of crash, they'll compress margins for next round of buildouts, i.e. bubble bursts, current spend proves not sustainable. This is where the crux of argument is...
Future investors post crash when margins revert towards mean will be spending $4 to supply $5+ of demand. And due to nature of compute deprecatiion (i.e. tulips) they will have more efficient hardware with less opex/capex TCO per unit of compute, with much more sustainable balance sheet. The builders are still fine with their $2 margins, it sucks its not $8. But that leaves the current investors who spent $10 with stranded assets that are not competitive with more efficient $4 future build out, i.e. current investors have balance sheet black hole that cannot compete with none bubble market force.
This does not mean AI is doomed, it just means incumbents from current tranch of bubble driven, stupid high TCO build out is most likely doomed relative to future entrants. Unless incumbant has unassailable moat, or other hedge/cards (i.e. political bailout/intervention). That is the actual argument, Zitron is saying current ecosystem economics not sustainable, not that there is not a future model that isn't sustainable. But it does mean a lot of current players are balance sheet zombies, who _should_ die. But a reasonable disagreement is reality is size of bubble + contagion risk + influence of incumbents i.e. trillion dollar companies is such that they have non market lever (i.e. politics) to save themselves... but someone else is going to be doing the paying for a model that is net loss.
> On a spectrum between tulip wilting and fiber build out, we know deprecation cycle of DC hardware leans towards tulips i.e. <10 years (very generous) vs 20+ years for fiber layout, a lot of which is actually infra/earth works etc.
Counterargument: As advancements in transistor densities slow down, the rationale for increasing depreciation cycles makes more sense. As the performance gap between new & 5-year-old hardware continues to shrink, then the need to replace older hardware similarly shrinks, justifying longer depreciation cycles.
It's not just about node advancement, which is coupe de grace condition. Even if hardware advancement freezes, its about IC premium that fed current tranch of AI buildout. Current players paid $10 for a $2 hammer due to premium, a better future hammer might cost $3 but does twice the work of $2 hammer. That is like ball park the premiums we are talking about - from gpu to memory to other components getting inflated due to exuberate AI demand.
The economic logic is if current spend vs revenue gap is not sustainable... hardware prices / margins will revert towards mean. That $10 hammer will be compared against a $2 identical hammer (margin reversion/compression)... or worse, a $3 future hammer that does $4 / past $20 of work. The future player who only paid $2 can charge much less... i.e. simply paying $10 limits ability to price competitively. The future player who pays $3 has 50% more compute than incumbent who paid $10. The important DC TOC consideration, is in world where DC cost regress towards mean, opex > capex... so merely continuing to use that old $10 hammer is losing MORE than buying a $3 better hammer, i.e. the asset is economically stranded, it is COSTING MORE to run old hardware than simply buying new hardware. It's MORE than economically useless and $10 past purchase price not just sunk cost but dragging down balance sheet as amortized liability aka it is full write down / loss.
(edited to remove snark)
Your comment seems to miss the point that the article does not (necessarily) have a problem with Zitron being insufferable/annoying/smug. It's that his predictions are verifiably wrong. It's one thing to be annoying and right. Zitron is annoying, but not right.
Yes? What happened to FAANG in last 3 years, they cut a fuck load of jobs, the raised rent/prices, then inflation, then experience surge in new category AI. 1 + 2, i.e. squeezing rock has limits. 3 is fundamentally not sustainable, i.e. AI revenue gap order of magnitude relative to spend. This like debt crisis, there's lot of levers to burn to maximize extraction and make ledger look good short term, but is fundamentally not long term sustainable. Articles arguing over minutiae / short term accuracy pointless, market can stay irrational than one can stay liquid blah, blah - I mean its useful for investing - but when talking about long term predictions he's just stating the obvious, the financials don't make sense within the business cycle current players are operating in.
Like one can believe AI is speciation event technology eventually, but still given actual constraints, i.e. literally not enough investors for $$$, not enough hardware, not enough infra over xyz time horizon that these companies carrying stupendous debt and mathematically guaranteed stranded / deprecated compute infra is only digging themselves deeper vs future competitors. Sure AI can eventually capture 30% of GDP and knowledge worker's life time achievement is worth a few $100 of compute or a few pennies in thinking sand. But ultimate winners is probably going to be some future startup that pays pennies for thinking sand not incumbent who paid magnitude more and simply can't operate profitably due to balance sheet.
Luu is pretty specific about the predictions Zitron is making, has taken the time to pull them out and date them, and they're both risible and not rescuable with vibes.
His point isn't that Google or Meta are doing well or have bright futures. Luu is generally critical of tech giant engineering and product culture. He's critical of Google in particular in this very article.
But the point of the article is that it's not enough to have directionally satisfying vibes. If you made concrete forward-looking predictions and they're catastrophically wrong, that matters. If you make backwards-looking predictions that were literally wrong the moment you published them, that matters even more.
"Did you read the article" is a frowned-upon response on HN. The better way to write that kind of response, per the guidelines, is "the article mentions that". So: the article mentions that.
I don't know if directed at me because I didn't ask if one read article.
> it's not enough
It's enough for some of us, like his broad predictions that work on timescale of business cycles seem directionally correct. Even considering we're dealing with fast hardware deprecation cycles it will take years to play out especially with investors and incumbents burning through accumulated war chest. Luu seem oblivious to notion that companies with trillions in market cap can certainly out manipulate fundamental short / medium term market sanity. Part of Zitron's rant I find similarly compelling is the danger of dismissing directionally "satisfying" vibes because $$$ can capture reporting distort reality, which is only going to lead to bigger/more painful correction because directionally "correct" was dismissed as merely directionally "satisfying."
Let’s say it’s fine to give emotionally based arguments the same credence as rational arguments. That’s a huge cede, and yet, it doesn’t change anything, let’s see here:
If my cousin kept ranting about my other cousin was going to go bankrupt and fail and it was 3 years later and their income was up 2x I think I’d stop listening.
I worked at Google from 2016 to 2022 and agree with everything he says and you say, modulo the companies who are 2-3x on revenue and profits are going to 0. I worry that both of you have found a real problem but misattributed it, and insisting emotional arguments are the same as rational prevents you from participating in real fixes (ex. metas problem isn’t AI, it’s that they have a god-king CEO who cannot be deposed and monopoly profits. Imagine a twin of you and Zitron but instead of AI it’s 2020-era VR. If they weren’t focused on how their emotional argument was fine, they’d be your compatriots in noticing something’s off in Big Tech. Instead, we don’t hear about them because that battle was fought and lost years ago, and they lost credibility due to imagining Meta was going to 0)
Zitron gives timelines, and they don't come true --- in fact, the opposite thing happens. You can't come back to that with "well, it hasn't happened yet". By that logic, no prediction is ever wrong; wait long enough and maybe it'll come true. That can't possibly be the logic you'd hang your hat on here.
> Luu is pretty specific about the predictions Zitron is making
He is not, though. He precisely points to imprecise predictions, decontextualize them so he misses the point of the ones this thread is focused on, analyzes them with even less precise rationales that don't really rebut the prediction, and points suggestively (enough that you seem to have got that suggestion) that this rebuttal destroys the main prediction of every Zitron piece, while saying otherwise several times at the end of the rationale.
Zitron's predictions aren't all very good, but this article isn't either.
Ah I understand what you're saying. Yes I agree that if you pull back to, "We're in a bubble," then you can say that Zitron is correct, at least fundamentally, despite being wildly inaccurate in almost every other prediction he makes.
But Zitron isn't just blogging about how we're in a bubble. The assertions he makes are not minutiae, he basically continuously says that all the big SW firms are walking corpses. He's not having a rational conversation about the long term prospects for companies who invest in AI. There is a population of people who (rightfully) hate Google et al and want them to fail, and he just stokes their anger and frustration.
He doesn't add anything substantial, and (as the article indicates), even when he brings economic figures into the conversation, he's frequently wrong or misrepresents them.
I edited while you were responding with second para. I think his analysis that existing investors are walking corpses (or economically exhausted/weakened) is substantial. You have new companies going into extreme debt and established companies burning war chests may actually is important, and also valid economic argument ESPECIALLY if you think think AI will be economically transformative. Like current players basically spent $1000 on a screw driver to do $10 of work. Some of them went $1000 into debt, some of them drained $1000, which is much of their savings/warchest. The incumbant players rationalize future has $100 or $1000, or of work, but these companies are going to be vs player buying $1 screw drivers, i.e. the compute deprecation curve makes spending $1000 on screwdrivers in the first place very detrimental/terminal vs future competition. Zitrons argument is broadly there even if there is $1000000 work in the future, companies that spend $1000 on screw drivers balance sheet is working on timescale where there is $100 of work, i.e. the economics is not in favor of incumbents. The economics might still be very favorite for future AI... but not for first round of players who has to recover from grossly overpaying.
He makes very specific predictions about specific companies and they are wrong.
You think we are in a bubble and that AI won't pay off for the companies investing in it.
While I'm sure there will be companies that invest badly the problem with your prediction is that the public hyperscalers (Google, Amazon and MS especially) are already seeing returns from their AI investments.
Look at the revenue growth - that is actual dollars coming through the door.
Here's an exercise you should consider doing: instead of your $1000/$10 numbers, use real numbers for the major software companies investing heavily in AI.
Software companies aren't walking zombies for the same reason cabinet manufacturers aren't. Everyone can make their own cabinets, but few do, because they simply don't want to.
> The assertions he makes are not minutiae, he basically continuously says that all the big SW firms are walking corpses. He's not having a rational conversation about the long term prospects for companies who invest in AI.
...huh? How is it "not rational"? He's saying that, based on the financial information available, it appears AI doesn't actually make very much money given the capital investments. To the point that there may never be AI ROI.
I'm not sure how much this or that "prediction" matters. His arguments would be just as strong without them, perhaps stronger because they wouldn't give folks like Luu something to snipe at.At this juncture, the analysis seems sound. AI costs an absolute fortune and appears to make very little money, comparatively.
Is that irrational? IDGI. One needs look no further than Oracle to see a company in dire financial straits.
> Stock market bettors aren't sure they like these odds. The company's stock is down more than 40 percent in the last month
The stock is down because of the increased interest load and the impact of that in the next couple of quarters, not because of doubts over Oracle's viability.
If there were significant doubts over its viability it would be down a lot more than 40%!
- They've all been compelled to build the same horribly expensive AI infra, to serve similar models that have no ability to lock-in customers
- Google Search has to compete with LLMs
- Meta hasn't demonstrated a credible argument on how they're planning to use AI. AI 'friends' would kill their business model. Their saving grace ironically is that people absolutely hate interacting with AIs. Same goes for other AI assistants.
- Hyperscalers have to compete for the same hardware as AI companies, driving their costs up
- AI turned out to be excellent at both porting software to more optimized stacks and deleting the 'prestige' of building these ultra-inefficient microservice containerized stuff. I haven't read a single article about somebody bragging about this stuff. When it comes to tech (which is not AI), usually its about Zig, Rust and going native.
- So if customers really start feeling the heat of rising costs, they have a realistic path of optimizing their compute usage by using AI to rewrite the worst-offending components. I think one of the few things in which AI has demonstrated measurable economic value is rewriting software in Rust to be more efficient
No. Net income is up quite a bit and profit margins maintained at Microsoft, Amazon, Alphabet, and Amazon. Meta net income is flat, but they are maintaining profit margins.
Zitron's claim is that much of that boost in profit for microsoft and alphabet and amazon is from only a few customers, specifically openAI and anthropic, and that those companies are buying and promising to buy lots of resources with money from investments from those same companies, and that largely this is unsustainable unless openai and anthropic can find a profitable business model.
And with both OpenAi and Anthropic are announcing record revenue growth to the point where Anthropic is now profitable this seems like it's going to work out fine for them.
He is right about that. Those companies are subsidizing the use of their models and at some point they are going to have to make a profit. It is unsustainable the direction they are currently headed in many people don’t like the messenger of bad news, but someone’s gonna be caught holding the bag stay clear of the blast crater.
> They've all been compelled to build the same horribly expensive AI infra, to serve similar models that have no ability to lock-in customers
Emphasis added, since having a horribly expensive AI infra allows offering enterprise contracts, which is a form of lock-in and has been pretty lucrative for GCP/Azure/AWS.
To be fair to Ed, I’d describe his usual argument (at least currently) as saying that Meta, MS, google are “mature” companies trying to be maintain the high valuations and growth of a young company, which they no longer are.
If you take this to be his argument, then dan’s numbers are more consistent ed’s claim.
Of the three Google is in the best position. Meta and MS are in trouble. Zuckerberg will survive because he has control of his company, but Nadella is not going to survive Copilot if it don’t work.
Are they? These companies have been caught tweaking their numbers. One example, not sure if cited by Zitron, or others, is that they build data centers through holding companies, who have to absorb the costs and massive capex based financial liabilites, so that the brand-name big-tech companies get to keep their expenses off their books. There have been trillions of debt discovered this way. Another issue is the apparently relentless progress of the hardware industry, needed to justify their super-high P/E ratios, measured against the fact, that to lessen the effect of HW amortization, hyperscalers opted to lengthen the depreciation timelines of their GPUs. So there is an apparent contradiction that new hardware needs to be both substantially better, and substantially the same, to make both stories true. I'm not a finance guy, and a lot of it is over my head, but even finance people keep asking the 'who's gonna pay for this' question. We're way past the belief that this is going to produce reasonable returns (as in a value for money kind of way), and hoping we can financially engineer ourselves out of this situation without having to feel the pain.
> These companies have been caught tweaking their numbers... they build data centers through holding companies, who have to absorb the costs and massive capex based financial liabilites, so that the brand-name big-tech companies get to keep their expenses off their books.
This is about as far from "tweaking their numbers" as you can get. It's a standard way infrastructure-heavy industries structure their investments and people would be asking questions if they didn't do this!
> hyperscalers opted to lengthen the depreciation timelines of their GPUs.
Yes and so they should! GPU depreciation timelines used to be 3 years!!
Google is famously still running 10 year old TPUs at 100% utilization, and 10 year old H100s are worth more now on the second hand market than they were when they were bought.
> This is about as far from "tweaking their numbers" as you can get. It's a standard way infrastructure-heavy industries structure their investments and people would be asking questions if they didn't do this!
Well, it's enough to throw off standard EBITDA accounting and allow firms to report fictional earnings numbers. A standard story has been that companies have beat their Q3 estimates, only for their stocks to go down.
Coreweave has announced they will keep A100s in use until 2029 which will be 9 years old then. I think that is where I got the 10yo number I had in my head.
On TPUs, I was also wrong on that, but less so. The quote is:
"seven and eight-year-old TPUs have 100 percent utilization."[1]
That was last year, so 8 or 9 year old TPUs now (assuming it is still true). Slight exaggeration there and I wish I'd looked it up before posting.
Despite this, my point (that 3 year depreciation schedules for GPUs was too short) remains correct I think.
Did you read the article? It paints the picture of someone who does not in fact have a good track record on the 'fundamentals', even if his broad thesis of a bubble may prove correct.
e.g. when he suggested Anthropic may be fudging their revenue numbers / projections - which was actually due to him making some careless mistakes in a spreadsheet
He has built a following of people that want to hear his extra skeptical views. And even if he changed his mind about some things, he cannot admit it, as that is not what his followers want to hear from him.
It's rare that people make the news and build a following by saying a very balanced, down to earth opinions... unfortunately.
I almost wonder if it's even possible now that so many of us get our information through algorithmic feeds.
I listened to one Zitron interview on YouTube and my home page was immediately crammed full of similarly foamy-mouthed AI critics. (As well as plenty more Zitron.) And it's suggested some people with un-nuanced pro-AI takes, too. But what the algorithm has never, not even once, given me is a sober voice that calls attention to the nuances.
>It's rare that people make the news and build a following by saying a very balanced, down to earth opinions
Humans are built for fighting and killing the rival tribe, not for pondering whether the other tribe might actually be correct. We are exceptional for even being able to overcome ourselves enough that the outcome of "fighting and killing" can be minimized in the large.
> And even if he changed his mind about some things, he cannot admit it, as that is not what his followers want to hear from him.
I don't think that's true - over the years he's changed his mind from "LLMs are useless slop machines" to "LLMs can be useful when applied wisely".
For instance in "The AI Hater's Manifesto" he says:
> It just came to me — the problem that I have with most people using LLMs is the delineation between outsourcing work and outsourcing thought. Those using LLMs to write little scripts or BQL code on a Bloomberg Terminal are inoffensive. [...] A tool being used as a tool to do tool things — in many cases involving the LLM writing a little 30-line Python script! — is not a problem, though it’s also not a trillion-dollar industry that needed to steal everybody’s art and writing.
Similarly, in "The More You Buy, The More You Lose":
> Sidenote: The only truly useful use case I’ve found is on the Bloomberg Terminal’s ASKB feature, which takes natural language and turns it into BQL code to make requests of Bloomberg’s datasets. It’s genuinely useful!
You can perhaps say that he underestimates what the technology is capable of (or, conversely, that other people overestimate LLMs) - but that's a different kind of conversation.
But even those concessions have been a shift: at every point, he's been as bearish as possible on LLMs, often past credibility.
And if you take it as a given that AI will never be any better than ~~now~~ a year ago, and that anyone who disagrees is an idiot or a liar, then that pretty much demands that the entire AI economy must be as fraudulent as he imagines. Which, while it serves his purpose of serving up AI-skeptic invective slop well, doesn't actually model what's going on, which is speculative investments that have the potential to generate extraordinary returns.
right, and look at current American politics. One side tried to tailor normal, routine and rational POVs.
The other...
EDIT: I do find it amusing when I write these types of comment, then suddenly realize the irrational man childs of the far right nationalist republic ethnostatists think they're the rational ones.
There are plenty of people on the far right whose positions are rational provided that you agree with their views. Conversely, there are plenty of people on the far left whose positions remain irrational despite that you might agree with their views. It is a serious cognitive error both to fail to see the irrationality of those you agree with as well as to fail to see the rationality of those you disagree with.
> I think the worst thing that happened to him was AI skepticism becoming a political position.
It is a political issue in the sense that the discussion around AI can't avoid material and cultural issues. Ed Zitron definitely has an ax to grind and I'm personally turned off by his hyperbole, but I don't think something being a political issue means it's not worth discussing. I've also noticed the pro-AI crowd pivot to accusing people of not having actual reasons for hating AI, being victims of Chinese propaganda, etc as a way to avoid actually talking about specific issues that people have about AI and DCs.
I never said it's not worth discussing, I'm just saying that it becoming a charged issue was bad for Zitron specifically because his style of content fell in demand with the most irrational parts of the crowd.
You're not wrong but for a lot of AI boosters it's also a political position - remember Anthropic and half of SV at this point is run by wide eyed effective altruists.
Ultimately one cannot separate technology or science from politics, it's inherently political
I'm not sure I have the force of will to not succumb to it. I think the only way to avoid it is to be so sure of the thing you are doing that the audience reaction is not the key metric by which you measure your success.
"Popular topic-expert" is a really cursed career to exist.
Ideally we would reward expert opinions on track-record, instead of how they make us feel. But we don't. Also ideally, the expert's conclusion wouldn't impact their ability to pay the rent (as long as it's correct), but again, that's not the world we live in.
Ed Zitron’s job is to convince people to pay him money to read what he writes, of course he’s going to preach to the choir, they’re paying him to do that and he knows it.
This is why I always consider the agenda of an author, Ed Zitron’s agenda is to make money from subscribers who read his writing.
From the article "A more common defense is to say, just in general, people attack Zitron because of Y (usually his style), but they never address his points."
The article "engages with Zitron's work"; the post you responded to merely extended the discussion to speculate on why Zitron might be producing it.
(btw, for the record, I'm an AI-hype skeptic and _also_ an AI-head-in-sand skeptic; as far as I can tell, both sides are full of malarkey.)
This isn't about people who just happened to make it big in a market rally, though. You're talking as if I'm speaking about people who just happened to buy some Bitcoin in 2012 rather than leaders of some of the most successful corporations on Earth.
They're not infallible, but modelling them as complete idiots that just happened to get lucky is also... Not really consistent with reality, as far as I can tell. These people might not be good people, but they're good at playing a certain kind of game.
Things can look stupid when we don't understand them, but there has to be some intelligence somewhere at some point even if the arrival of wealth then suffocates it with sycophants. If Musk or Trump or GWB were all *merely* the idiots they're often mocked as being, chances are we'd never have even heard of them. They almost certainly weren't even "merely average".
Now, stupidity that arrives after the bank balance reads a billion dollars, that's terrifyingly powerful.
There's "rich", and there's "enough money to rot your brain". I can believe Trump started with the latter, but the other two were less than that. IIRC GWB was "just" a multimillionaire even when he became president, about the same as Musk when Zip2 got sold.
The only reason we hear about Musk is because the gamble on PayPal gave him enough to then roll the dice again on a few more things, of which SpaceX and Tesla kinda actually worked.
Though with Tesla, their lifetime profits being ~= lifetime government subsidies to them and their consumers, by "worked" I mean "you can buy them and drive them" rather than it being a genuine business opportunity.
While this is a dick move, when the question is "is he smart?", it's still a positive result.
1. Trump was born wealthy. There was no "arrival of wealth" that smothered the intelligence you suppose he might have used to earn the wealth, because he didn't earn it.
2. You don't understand. Trump is successful because of his stupidity, selfishness, and vile behaviour. Not despite it. In 2016, the Republican primary field had 20 or so candidates on a wide range of reasonableness. Trump crushed them all, because he most represents the average Republican voter. If he were more intelligent, or if he cared more about anyone other than himself, they would not have voted for him. His arrogance, lack of intelligence and morals is quite literally his strength because it's what makes him so relateable to the American masses.
There are two possibilities. One is that half the country are total idiots and the people leading them, and leading the country, are also total idiots who only got into power because half the country are, again, complete idiots below average intelligence.
The other is that you are in a bubble and have been convinced that the leadership of the world’s premier superpower are below average intelligence, along with 70-80 million voters. The former is not something an intelligent person would believe, you have thrown out serious analysis for political theatre and memes and should rethink how you view the world. Now stop to consider that maybe actually half the country has a totally different worldview, and that their leadership, having reshaped global politics, is actually full of intelligent people who are cold and calculating. I know it’s harder to stomach, but just consider it for a moment.
Someone like Trump isn’t the first rise to the top of the country and be a con man or be as mad as a hatter in the end… He will not be remembered for his brain power.
I think he will be remembered for the decline and the loss of influence by America across the world that will be his legacy.
> below average intelligence, along with 70-80 million voters
Given what words mean, approximately 122.3 million US citizens eligible to vote ought to be below average.
When you say bubble, I'm inclined to agree, though for reasons that I suspect are uncooth to say out loud: we here are a bubble of smart people, so to us normal looks dumb.
Because it's not a single data point, intelligence is a wide surface area where you can be smart in some areas and stupid in others. Furthermore whether you are being intelligent is relative to ideals and goals, and not everyone has the same ones.
I find it difficult to imagine an intelligent person who finds it hard to get what they want. The trope of the oblivious genius who is incredibly competent in a single field of inquiry but completely unable to handle day-to-day life is more a creation of media than how reality works.
At first I thought you were referring to Umberto Eco's laws of fascism: "Fascist societies rhetorically cast their enemies as at the same time too strong and too weak."
but it's subtly different, because incompetence is not weakness.
It's pretty clear in how people talk about billionaires: They must somehow conclude that the people most successful within a system that rewards certain things are not competent because the things the system rewards are not what they think the system should reward (it is fine to believe the system rewards the wrong things, but people walk straight into denying that those people have skill even within the system, or they claim that skill within the bounds of the system does not reflect any "real" skill - totally ignoring that elites tend to stay elites even through, say, Communist uprisings).
Hence how someone can say that Musk is "dumb" with a straight face.
> the system rewards are not what they think the system should reward
It's the grown-up version of nerdy kids hating "the jocks" in high school. We're all just a bunch of dumb kids. Maybe obsolete children, but still children when it counts.
That said, Elon Musk celebrated cutting funding that fed starving children and supported cancer research by waving around a chainsaw on stage. The richest man on earth did this because he wanted lower taxes... for himself.
When Anubis weighs his soul against the feather it's likely to completely destroy the scale.
I'm not saying any of the people in charge are good people at all. I'm just saying that they are skilled at achieving their own goals, regardless of the morality or lack thereof of those goals.
I don't even think that. I think half of them accidentally stumbled into success. Elon was kicked out of PayPal with a nice golden parachute so he wouldn't sue.
Regression to the mean is pretty nasty. Stumbling into success can make you a car dealership owner or something like that, but being consistently successful in business requires just about as much skill as being consistently successful in, say, chess.
To be clear, I'm just saying that I don't hate him because he's more skilled than I am. Not anymore than I hate olpymic athletes for being better at ice skating. He's certainly more skilled at Making Money.
I hate him because he is a cartoon villian, who when given enough wealth to feed the world chose to punch down instead of lifting up.
he makes a shit ton of money off this - in this attention economy we are doomed, people of every side find that going all in on something, no matter how intellectually dishonest, is much more profitable. Extreme AI bros or Extreme AI doomers seem dishonest. AI is an amazing technology - it is simultaneously not going to replace us in the next 3 years and it's not total garbage - the truth lies in the middle.
Its pretty clear if you listen to him now he's just (re)playing the hits for his audience. Its like MSNBC/Fox News for people who think they're too smart to fall for that.
I used to think he was just early on some of this stuff but the sheer amount of content he produces its clear he's just cashing a check.
I'm not a huge fan of his style myself but fundamentally he isn't wrong. The whole "growth" so far has been all show and no go. I see people pouring in millions only to end up bankrupt a few months later. Evangelists portray those instances as rare and simply "skill issues" and "they don't know what they are doing but I am". Anyone that's ever had several servers running at home and seen the electricity bill at the end of the month knows it - I do as well. And we are talking about servers that use power, only a fraction of a server running 4x H100s at 100% 24/7. For those of us that have - we are talking servers that have one or two xeon silvers at best, 15% load on average. And I live in a country where the electricity is veeeeeeeeeeeery affordable. I'm not taking into account training, or investment to get it going, just inference.
20 bucks subscription a month, dario, you ain't fooling anyone with above-room-temperature IQ (Celsius). Even the expensive subscriptions - we know that's not the real price. And the limits, bumping up prices every two weeks and so on, just to keep the lights on while draining investors... Zitron's claims are pretty impossible to deny: the moment those companies go public, the real prices will need to come out of the shadows and end up on your monthly bill.
At the same time I fail to see the real world benefit to this, even in software: it's like comparing Lego (before they began their anti-consumer bs) to cheap Chinese toys. The moment you look at them side-by-side, you know which one is a premium product and which one is a cheap Chinese toy. While I have been a big supporter of open source since I was a child, I was never biblical about using open source - if some proprietary piece of software does a better job than the open source one - fine, take my money. Not anymore. I see the abysmal state of cyber-security as a direct consequence of slop. Slop-written code, slop-reviewed code, tests pass(also slop), ship it. Yeah, I'm not trusting you with my data, the hell with that, I'm self-hosting everything, adios. And I sure as hell don't trust the ai-bros for anything either.
And there's another thing: Microsoft was clear about it: github is constantly down because they can't handle the load. App stores are bumping up developer fees. Anyone can slop together a todo app(which was never hard to begin with). It's still a winner-takes-all economy - no one is going to install a todo app and migrate from Google/Apple just because. So the 1000 todo apps released daily will simply be a hole in the pocked of the people who slopped them together and another reason to be paying 500 bucks for 64 gigs of ram and another 500 bucks for a 2tb nvme. Raspberry pi's started off as educational platforms for 35 bucks and their commitment was to keep them at those prices. That aged well, right? If I decide to upgrade my uconsole, I have to pull out another 350+ bucks for a raspberry pi. All thanks to the sloppification. The bubble can't pop soon enough and frankly I don't care what it takes down with it.
NP-complete problems are essentially those where you need to make some kind of choice over a superlinearly-growing set of options where each option can be solved in polynomial time. Like, for example, finding a route between cities on a map under a certain total distance. You might need to go over all choices of routes.
The ETH essentially assumes that the best solution (in terms of complexity, i.e. how it grows) you can get for these kinds of problems is exponential. OpenAI just proved that this is false, you can have solutions that are below exponential complexity (but still above polynomial - if anyone finds a polynomial solution then we'd have proven that P=NP)