HN Simulatornew | past | comments | lists | submit | Tuna-Fish's commentslogin

> We haven't seen how Starship would deal with crosswinds in the atmosphere being predicted wrong.

Winds are not a significant factor in the trajectory of the ship. Reason is that it spends almost all of it's time very high, above all weather, and only descends to the less stable areas of the atmosphere at the very end, when it's in a vertical belly-flop, going very rapidly through all the different weather layers. You can see this clearly in the return video, at 2 minutes before landing it was still 15km up. And when it's high, it has much more ability to steer by adjusting drag than any winds can move it up there.

What could actually be a problem is sudden high winds or other problems at the landing site. The ship commits to a landing when it does the deorbit burn ~60m before landing. at L-20m, it has about 1000km of cross-range capability, at L-10m only 200km, and after that it goes down quickly, with only 50km at L-5m.

If it's coming back with something valuable, you'd better have plenty of divert options.


Eh.

I like the rocket less, imho solids have no place in human spaceflight, but the Orion has a LES, which elevates it to being more safe than any spacecraft without one.


Which makes the SLS safer than the Shuttle but airliners also don’t have one. They are considered safe because of how much they have flown and that is how Starship will be determined safe enough.

This has to be a miscommunication, right? The ship is in a stable orbit, and the booster is in the Atlantic. At no point in this flight should any debris have gotten close to any aircraft on/near the west coast.

Stockfish parallelizes well and makes heavy use of vector operations, and the Zen5 core has AVX-512 with 4x 512bit° EU per core and 16 cores, while the Apple still only uses 128-bit wide SIMD, with the 6 M5 "Super" cores having 4x128 EUs and the 12 "Performance" cores only having 2x128 EUs.

The M5 Pro is doing shockingly well given the massive disparity in execution resources. But still, it's optimized for web/desktop-like loads, you shouldn't really run heavy number crunching on that CPU.

0: 2x FMA/MUL + 2X ADD


They also mention using SSE/AVX on x86 and NEON on ARM, but there's zero mention of SVE, SME, or AMX which makes me wonder if they aren't compiling with those options (if so, it would be a serious disadvantage).

Apple cores do not implement SVE, SME is not suitable for Stockfish (it is optimized for matrix-matric with large batch size, stockfish is all matrix-vector with batch size of 1), and AMX has the same problem.

You don't get good performance out of this kind of code by just compiling with different options, the compute kernels must be tuned to fit the architecture, and nothing on the M-series but NEON fits the bill.


we've tested x86 AMX when the first Intel CPUs with AMX came out and it was worse. The conclusion was that AMX is not suitable for matrix-vector multiplication, only for matrix-matrix multiplication.

apple's SME/AMX is useless because it's not directly programmable


Is there any sane way to generalize these results?

I don't think so. The workloads are simply too different to generalize.

I think it's less there because of differences in the memory subsystem, but more because clock targets are lower so they can afford more. The structures they grew cost basically nothing in die area, but make timing tighter. It's possible they'd like them to be bigger for the non-c core too, but can't fit them in their timing constraints.

The unified L3 per CCD really helps on some key desktop loads (especially games). Most server throughput loads partition nicely into smaller partitions, but most games are hurt so bad by the latency penalty of crossing into a different partition that AMD's game drivers park the second CCD on their 2-CCD CPUs when they detect a game running. So long as games are an important target for them, I suspect they will not partition the CCD again. (It, of course, used to be parititioned for Zen 1 and 2.)


Seems at least somewhat justified from this Mike Clark interview today. Zen6c as their new flagship, zen6 as freq optimized. https://news.ycombinator.com/item?id=50467460

> so you need some plausible story to trick the user into not reporting the incident to IT.

Give them an authentic-looking "updating windows" screen after login, and reboot. 95% of users will bitch about Microsoft, instead of calling IT.


Large data centers are dramatically more efficient at utilizing both the power and the hardware.

At it's core, running a top-tier LLM means linear load of trillion+ weights from RAM for every token. The cost of this operation absolutely dwarfs the cost of doing the actual math. If you are only serving a single user, you can only amortize this over a handful of token streams, times like 2-4 for speculative decoding. If you are running in a datacenter, you can batch thousands of streams, and get correspondingly much more tokens out of the same expenditure of energy and hardware.


Be slightly careful with this. Large datacenters use power to operate the facility (this is measured as PUE > 1). Datacenter-oriented hardware often contains low-diameter, high static pressure fans that consume quite impressive amounts of power that are not counted as overhead in the PUE calculation.

A home- or office-oriented workstation will have much less power-hungry fans (go compare a Noctua datasheet to a server fan datasheet - the Noctua wipes the floor with a server fan if measured by CFM/W, and the internal temperature rise is mostly controlled by airflow and not by static pressure). And in some climates and times of year the marginal facility HVAC cost associated with running a computer may be zero or even negative.

And lower utilization hardware may run in on a more efficient point on the power-vs-performance curve. [0]

That being said, your overall point is correct.

[0] For a dramatic demonstration of how much this curve matters, you can run a non-CPU-saturating, non-latency-sensitive workload on various Intel CPUs with and without turbo on. Seeing a 2x or greater difference in reported power is quite possible. (Intel is terribly at writing platform firmware and the default behavior may be wildly suboptimal.)


None of that matters when there is literally a 100x multiplier on efficiency because of batching. It's all a rounding error.

Explain the efficiency of spending billions on power plants and grid upgrades that you only need because you are centralizing gpus.

You do not get to exclude the power grid costs. That is the core of the grift. They are trying to get data centers connected to the grid as fast as possible with contracted kwh rates so everyone else on the grid would end up paying higher rates. They can contract for existing wind power at 2 cents a kwh, then other businesses and residential customers paying for the peaker plants and grid upgrades.

You don't have to build a single data center or build power plants to keep gpus at home.

The 100x multiplier is complete nonsense. There is no truth to it. They are still running power through chips on the same nm scale. It is not saving anything. A gpu in a home only needs to run when you have it do something, so its much more efficient. OpenAI doesn't even use its current data centers, that is why they have internal teams using lots of gpu resources to front run other people's accomplishments. Those gpus are otherwise sitting idle.

You cannot claim a rented llm for $200 a month is cheaper and more efficient than a $1300 gpu you buy once and use for the cost of electricity.

You leave out the cost to the consumer, the fact that the consumer still needs home internet, electricity, and a computer to use cloud AI. Your "efficient" service didn't eliminate anything and is using the same amount of power to process the same amount of bits.


You would need to spend more on power plants and grid upgrades if you distributed the compute. Because you would spend so much more power to get the same results. Yes, you would need to build power plants to do large-scale inference at home.

> They are still running power through chips on the same nm scale. It is not saving anything.

What even is your argument here? I explained why the centralized inference gets more throughput out of the same power. Let's use a real example, with Qwen3-32B (4bit quantized). Yes, it's old, but that won't change anything, it's just what I happen to have numbers for.

A home system with a RTX 5080 GPU can run it at ~45tok/s for ~300W, or ~7J per token. A DGX B300 can run it at 200000tok/s (split into 50-75tok/s streams) for ~17kW, or, 0.09J per token.

Not quite 100x, but getting there. The centralized system fundamentally needs to do less work total for the same amount of tokens, because it can reuse the load of model parameters for multiple streams.

> You cannot claim a rented llm for $200 a month is cheaper and more efficient than a $1300 gpu you buy once and use for the cost of electricity.

I'm claiming no such thing, don't rent a LLM for $200 a month, at least unless you are spending a lot more tokens than that $1300 GPU can do. There are options that are serving at market-determined prices, in addition to the hilariously over-priced options that the frontier labs provide.

> You leave out the cost to the consumer, the fact that the consumer still needs home internet, electricity, and a computer to use cloud AI. Your "efficient" service didn't eliminate anything and is using the same amount of power to process the same amount of bits.

You do understand that the consumer uses those whether they do their inference themself or not, and that their computer in fact uses a lot more power when doing inference vs when it is not doing inference?

Your fundamental failure here seems to be inability to multiply. The data center is not serving a single consumer. If you took the hardware, and distributed it to every home, you would need ~100x the power to provide the same amount of inference. Because the data center is in fact serving a lot of consumers at once.

And I like local inference. I want to control my own digital future. But wanting that and pretending that it is somehow more efficient are two very different things. People like us can acquire the hardware needed to run a local LLM, because we are hobbyists who are willing to spend money inefficiently for a hobby, but if everyone did it, it would mean that we would collectively need about ~100x more hardware and consume about ~100x more power, than if we got the same tokens from a centralized system.


So much this.

The current-gen hardware is expected to have only 2-3 years of useful life. New tech, and especially new architecture is going to make inference much faster and more efficient in the near future, and the big vendors are going to be more limited by datacenter floor space and power/cooling, so that the hardware will be replaced very quickly by better models.


For the time being this thesis hasn’t worked out though. H100’s are only going up in price and becoming more scarce [1]. At what point would you update if now going on 3 years into H100 lifecycle this doesn’t seem close to happen.

But they are becoming more scarce while staying on Nvidia’s shelves buy-and-hold, or being leased back in some cases. They are not being installed by customers at anything like the rate the sales figures suggest. So yeah, these H100s will last a long time in the packaging.

Meanwhile, Meta is selling their spare AI data centre capacity.


Isn’t that what SpaceX is doing too? Selling spare compute while they work on their rocket side hustle.

Where are all these unused H100s sitting and where are the people who have been fired for buying them?

They are in Nvidia warehouses. Nvidia is selling H100s and other hardware in advance to firms who do not have data centres to put them in.

Nobody is getting fired for thinking they need to buy scarce H100s in case they aren't able to get them in the future. Nobody is even getting fired for keeping their companies out of Jensen Jail — the fear that if they don't buy the whole server stack from Nvidia they might not get future GPU allocations.

There really are analysts who think Nvidia is channel-stuffing.

If the market deflates there will be a glut of very cheap compute, both in terms of data centres (though many of them will be able to offer managed model training services instead, I suspect) and in terms of unused hardware being made available for sale.

Nvidia won't lose out; everyone else is going to.


The market for compute or the stock market? People seem to have got lost conflating these two things.

Supply and demand bring what it is means that (alleged) cheap compute won’t stay cheap or go unused for very long.


Both the market for compute and the stock market valuations of AI firms (and theoretical valuations) are overcooked.

The problem is that "scarcity of GPUs and RAM" is being used as a proxy for "demand for AI".

It's a broken proxy; the scarcity of H100s is artificial.


Here's interesting question I haven't seen covered. The promises to buy hardware (not just NVIDIA but networking, servers themselves, etc etc) are NOT on the income statements of the buyers...

Are the contracts on the income statements of the sellers? If Anthropic promised to buy $200B of GPUs from NVIDIA, is it recorded on NVIDIA's income statement (other than a footnote "we have $200B orders lined up")?

'Cause if it is, that'd be $200B GDP increase _literally_ conjured out of thin air.


I don't agree with him on everything but one of the questions Ed Zitron has raised is — have all the GPUs Nvidia has sold actually been manufactured and warehoused?

If they haven't been manufactured, things are potentially quite sketchy.


Jevons' Paradox. If it becomes more efficient, cost goes down, demand goes up, use rises, revenue rises.

Look at DeepSeek! They have done full system optimization on their model, architecting it so that it would be cheap to serve. I am proud to call DeepSeek a world class tech engineering firm in a way that 99% of the tech world is not. They are a 1% company.

People are buying v100s on ebay to run LLMs and Chinese manufacturers make custom PCIE boards for them. Older cards might not be useful for training but they're perfectly capable of doing inference work.

> The current-gen hardware is expected to have only 2-3 years of useful life.

Expected by whom and why?

Google hasn't retired any of its now ten-year-old TPUs, or any of the subsequent generations. Maybe TPUs are special or Google doesn't understand compute economics but either argument requires data.

If your argument is depreciation, you're confusing a tax concept (depreciation) with useful life. Yes, they're supposedly related, but in practice, the relationship is rather loose. (Residential housing depreciates in 28 years yet typically lasts 2-3x as long and 4x is not rare.) Google's V1 TPUs


no, it's 10y. you just repurpose.

also, power and cooling agreements are usually 10y too.


The 8-bit ISA bus was theoretically capable of 1.19MB/s. The bus architecture always took 4 cycles per transfer (enforced by the 8288 which had a 4-cycle state machine, iirc one cycle for address, one for data, and two wait states), transferred 8 bits per transfer, and ran at the 4.77MHz frequency.

The actual usable bandwidth was always much lower, somewhere between 80kB/s to a bit over 380kB/s, depending on how you were using the bus, because the cpu needed to fetch its instructions on the same bus. In a way, the bus was usually always running at 1.19MB/s though, because all the basic devices (ROM, RAM, common simple ISA cards) were able to always respond in the basic 4-cycle transfer, it's just that most cycles were spent fetching instructions instead of actual data.

Later machines made the bus faster by increasing width, increasing clocks, and dropping one mandatory wait state at AT for 5.55MB/s theoretical, and dropping the last wait state in the fast clones for 8.33MB/s theoretical.


Thank you; I didn't even realize that the RAM chips on the motherboard were physically connected to the actual ISA bus! It sounds like you know a lot more than I do about it.

This is interesting, thanks! Just wondering if the REP and REPcc prefixes with LODSW/STOSW/MOVSW would have allowed higher transfer rates due to there being no need to decode instructions after the first loop cycle?

No, for a dollar of inference sold, (as an average of all the different inference tiers and plans), it cost them 15c to provide it.

This excludes the R&D costs, including training, where most of their costs lie.

The basic truth of LLM economics is that inference is cheap. The NRE cost to get where you can get premium prices for inference is sky-high. This is part of why the AI companies want to drive the scale up so much.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: