Hmm... I kind of disagree with that. Technological breakthroughs are not "natural". We shouldn't have to bet on someone discovering something that is able to make NVIDIA obsolete.
Two statements can be true and he stated one of them. Sam is not wrong.
1. Local inference can cause a large under-utilization of resources from users that do not run local inference hardware at capacity over the hardwares lifespan. This is under-utilization and under-utilization is not optimal from an economics perspective.
2. The concentration of inference hardware to singular providers can aggregate power to single systems or organizations, and their incentives may be at odds with individual users.
The same power dynamic and economics struggle has existed nearly everywhere for all of history. For example, if you want to decide to own or rent a piece of heavy construction machinery for business, you will find a long list of do-not's on the rental contractual terms, but you are more than welcome to buy your own and operate your own machinery at several times the price for the hours used and use it however you want.
Few people will argue with the logic of whether it is technically accurate ( assuming actual deployment will be optimal, because that is hardly a given ). It is just amusingly self-serving. It is almost as if McD got to decide you can only eat there.
Weirdly, underutilization of hardware was never a problem before. In fact, wastefulness is so ingrained in programming culture that the previous hype - blockchain - was about an explicit race over who could waste the most computing power on nonsensical calculations.
> Two statements can be true
As I general point, because I keep seeing this line on HN: Of course they can in the general case, but more often than not, the reason why this line is written is because there is some tension or contradiction between the statements. In that case, understanding where the tension lies is IMO more useful than shrugging and deciding to "hold two contradictory thoughts in your head" like in 1984.
Sometimes it feels like a desperate rhetoric device to "save" an opinion when the opposite opinion is supported by obvious evidence.
I wonder if it’s a bubble thing. People carry on and do what they always did, or just began thinking about what to do with LLMs. Big tech companies and startups may have to move fast and adapt, but I guess many other companies and organisations do not have to rush or fear immediate extinction, just because the wind blows in the other direction elsewhere in the world. I see that Hacker News and X are very "noisy" about LLMs (naturally), but that doesn’t mean the whole tech industry must react in the same way.
>"Reading stuff like this continue to make advances and new releases is like a fever dream when everything else is all AI."
Do the advances mean anything though? Alan Kay is a personal hero of mine, I love the aesthetic of Smalltalk, and the dream-that-was once upon a time of such a system was a real missed opportunity for the evolution of programming environments. But is it anything more than a neat hobby at this point?
> But is it anything more than a neat hobby at this point?
The opposite. With LLMs you can now pick up any one of these obscure language and write a full blown AAA game engine in them. Or a database. Or a web framework. Or... anything you set your mind to. The language no longer matters. btw LLMs are exceptionally good at Smalltalk/Haskell/Perl/FORTRAN and other old/uncommon languages.
A lot of misconception around the utility of foundational time series models on HN.
Sure, you could train a XGboost model on a time series with holiday or event covariates, but what if you have 10k different series to forecast for?
You could train 10k different models, one for each series, but then they have very little history to learn from. Alternatively, you could train a single generic model to cover all 10k, but then you have to deal with scale mismatch between series in addition to the feature dev overhead between series. A TSFM saves a ton of work in modeling each of these series and could potentially generalize better to produce more accurate forecasts.
But why is the question, I run about 400k models every 30 minutes at work. I have tried all other approaches, but xgboost model per timeseries has never been beaten before. Even if I wasn't targetting accuracy, Xgboost loading a pretrained model and predicting with it and still faster.
Alternatively, this is the exact ideal application for hierarchical Bayesian modeling. It’s the middle ground between the traditional stratification/aggregation decision.
What's the input like? Does it incorporate language understanding, so that each time series has a natural language explanation of the series and all the data fields? If it takes in really descriptive text like that I can see the opportunity for generalization. If it's just a time series of non-descript numbers I see much much less opportunity for cross-series learning.
All the businesses with product catalogs for example, every product can have several time series associated with it.
If a business has 10k employees, you have, at minimum, 10k time series to perform employee analyst operations over. There are several hundred companies in the US with 10k+ employees.
There are many more than 10k postal zipcodes in the US to perform forecasts over.
The difference is the same, whether you have 10k series representing the same variables or 10k series representing different variables decomposes into the same forecasting problem that you can either over-generalize across all time-series, producing worse accuracy at the tails, or train a model for each individual forecast, producing worse accuracy in general due to limited data input.
What kind of infrastructure do you use for that scale? Are these computationally light ARIMA models?
We're running 150k models per day and while I'm sure I could scale this to running them every 30 minutes, it would mostly be me throwing money at the problem to parallelize the compute.
Some HPC Clusters, xgboost, read pretrained model from NFSRDMA Server, read data, parallelise over MPI with one process per model. And the latest Zen5 Epycs. I could do it all on just one of the 192 cores EPYCs but we like to parallelise it over multiple machines to get the extra speed and reliablity.
It wasn't that expensive, the main problem was the storage bottleneck which got fixed with just some LVM cache and gen5 ssds.
Tabasco is much more of a spice than a sauce, and I reckon thats one of the reasons its so popular. I can put Tabasco on many more foods of different types than I could with Cholula, Franks, Tapatio, Valentina, or Sriracha
That's interesting. I find most of those that you listed, aside from Valentina (I am NOT a fan), as something I can put on basically everything--and actively do.
Out of the hot sauces, Tabasco is my least used these days. 30y ago, it was really all that was available; but, now, we have such a wide and varied option set at just about every restaurant that I find myself almost never using it.
If it were made illegal then news journalism, documentary filmmaking, sports broadcasting (crowds), street photography, and police-body camera footage would become extinct almost overnight.
LARPing would imply pretending to do something, but not actually doing it. Here they appear to be actually doing it (open source product). Is there something Im missing?
You only addressed half of it. I'm with you that the code is open source, but it's an open source client for Meta's proprietary service. The one original piece of hardware on this "Open source hardware for your Muse" page, the Home Link, only works with an active subscription. And the "grassroots" part is clearly larpy: a "by hackers, for hackers" voice in a Meta incubator repo.
https://muse.ai/deviceterms
"If your Muse subscription terminates or expires, your Muse Device will cease to function as described in these Device Terms and the device documentation. The device's capabilities will be unavailable until an active subscription is restored."
Genuine question: why are there so many people up-in-arms on orbital data centers? Like they have a personally vested interest in ensuring their opinions on failure are correct? There are many people on HN that despise it and say it will not and will never work, but here we are, they are flying them up to space and presumably, will start working soon.
Because when things are getting built that obviously should not work, in one's view of the world, it threatens that view of the world. Finding out that your common sense and understating of basic physics and economics has a sizable hole is pretty unpleasant for most people.
But if the project fails, then the view of the world is proved correct again, which feels moderately pleasant.
Actually, some of us predicted exactly what’s about to happen: these companies will launch a handful of GPUs into space, declare that it’s “working”, and then continue to build out terrestrial data centers at orders of magnitude faster rate.
No one is saying it’s impossible to put operating GPUs in space. People are saying that the economics of this make zero sense, and there seem to be hard physical limits for how much you can change parts of the reasons why. This test does absolutely nothing to disprove that.
This little satellite has four TPUs, draws about a kilowatt, and can only operate for 15 minutes at a time. Like's no-one's saying _that's_ impossible. Clearly. Pop a few gaming laptops on the ISS and you're in the same ballpark, except they'll operate continuously.
Are you moving the goalpost? There are other people in this thread right now that are stating exactly that running orbital GPUs in space will not physically be possible (heat limits, etc).
>... but here we are, they are flying them up to space and presumably, will start working soon.
No, this isn't "here we are". This is not an orbital data center, this is just something that will test TPU cooling functionality in orbit for only 15 minutes at a time.
This argument sounds reminiscent of "this is not AGI/ASI". Arguing over definitions is arbitrary. This is an operational data-center that is processing data on TPUs. That alone qualifies as an orbital data center.
Whether we like AI or not, LLMs did open up a horde of use cases that were impossible a few years ago. Orbital data centers are not like that: their value proposition is that they will provide the exact same service already offered by land-based data centers. So the only way for it to make economical sense is if it's cheaper. It's not.
It's more like blockchain than LLM: there's no clear value proposition and you have to squint really hard to see any perceived benefit.
Anyone managing a terrestrial data center that had this level of performance and called it a “data-center” much less “operational” would be fired for cause.
And ah, I see you’ve been dancing around with definitions and goomba fallacies around this thread. You definitely asked your question in good faith.
The hardest thing about AI datacenters is power. The second hardest thing is cooling. All of these are physical challenges and exponentially harder in space.
So why do it? Because datacenter regulations on earth are becoming a referendum by proxy on AI. "Texas has no jurisdiction in low earth orbit!" is the general idea. Critics point out that this is a transparent move to moot the argument over AI, and that it doesn't even work in principle, due to above mentioned physics.
It’s simply a stupid idea that indeed will not work, and this little tech demo is mostly a proof of that. It’s awful to many people (especially engineers, which you'll find more than elsewhere on here), to witness incompetence, waste, and inefficiency. Especially when we’re talking about investments of billions of dollars and orbital launches.
Everything about technological progress is like this. It props up their ego, and they risk nothing. The man in the arena is always dangerously close to being the villain.
> Like they have a personally vested interest in ensuring their opinions on failure are correct?
I haven't commented on this topic before (here or elsewhere) but I don't understand your surprise at people being critical. Everyone has a vested interest in keeping the planet usable. We could try lots of things in space, like agriculture or power production. It takes a lot of fuel to push a usable kilogram of stuff into a low orbit (laser propulsion and spin launches aren't quite there yet), so moving operations into orbit has various major downsides. So long as nobody states an upside, yeah, my mind is quite blown about wasting investor money on this. Even when not considering the opportunity cost of that used money (alternative positive uses), it seems like burning the equivalent amount of cash in a bonfire or otherwise disappearing the money would be a lesser evil
It's like watching flat earthers launch yet another experiment to prove the earth is flat. We already know the outcome, including the subsequent press release that has the words "very promising" and "flawed assumptions" in the same sentence. But this "flat earther" has billions to burn and enough clout to keep investors and the media in the carousel for the better part of the next few years.
Because it’s so fucking stupid for the stated benefits that you get
- people getting nerd sniped and explaining why it’s wrong
- people who assume it isn’t wrong given the level of competence these groups have demonstrated already and are incensed by what they feel are bald faced lies to pump up stock prices like has been done before
- people who think this is some conspiracy(rightly or wrongly) to surreptitiously get weapon platforms or other support for weapon platforms into space.
The majority of tradeoffs for data centers make them always better on the ground. Just due to maintenance and heat alone.
> There are many people on HN that despise it and say it will not and will never work, but here we are, they are flying them up to space and presumably, will start working soon.
This is the one Google says they can run in 15 minute bursts and I’ve yet to find where they document what the cooldown period is before they can run again.
Also 99% of people saying it’ll never work are talking economically, not that theoretically you could get something up that works in the laws of physics.
Why are you so mad about orbital data centers? Your language is full of hate and vitriol. Let people build things that other people say are impossible or infeasible, find out if they work, turn out they dont work, then make improvements. If someone proposed the combustion engine to you in 1700 Im sure you would reply the same negative way.
Quit with the bait. It’s obvious with your follow up questions that when you said “Genuine question:”, it was disingenuous.
In the extremely unlikely situation that you weren’t, and so the mods can see I engaged with the question.
I personally am against for economic reasons and feel like point 2 laid out by me is the most likely, although there is another option I could entertain where the people running these companies are so high on their own supply with the TESCREAL beliefs, or literally high like we’ve seen guys like Musk be in public, that they actually believe in this being economically plausible despite all contrary evidence.
> but here we are, they are flying them up to space and presumably, will start working soon.
From the article:
> Google will run Gemini models on the TPUs for testing, but they won’t be able to run continuously. The cooling system will only be able to operate in brief spurts of about 15 minutes. After that, the TPUs need to shut down to allow the radiators to catch up.
Soon the musical chairs game will come to a screeching halt. Interest rates are skyrocketing and not even a Presidential decree will make them come down.
To me it's clear the U.S. is heading for a default. Instead of toning down the deficit Trump made it much worse with his "Big Beautiful Bill" and is bankrupting the country.
What happens next is anyone's guess, but it won't be pretty.
Oh I'm just against it because I love the night sky and I don't want it littered with even more trash from obscenely rich American oligarchs. This tech doesn't help me, or anyone that is not in power. Instead it (and light pollution, but that at least not everywhere on the planet) destroys one of the few things that is there, commonly available for everyone to enjoy: the majestic view of the night sky.
Debatable. It's not economically viable under current economics. There's an engineering problem that's solvable in the short term, e.g. the cooling issue.
The economics of mass production of satellites has been demonstrated to work out just fine (see Starlink) and what's missing is the same shift when it comes to space launches. Now I'm not saying that this is going to happen, but that's what the whole purpose of Musk's Starship/Superheavy rocket is.
If - and that's a big if! - the whole Starship concept works out, it'd drastically change the economics of putting satellites in LEO and the whole idea instantly becomes much less stupid.
So basically it's a bet on Starship working as advertised in the very near future. If it does, data centres in space (e.g. in the form of laser-linked satellite swarms) become a compelling alternative to terrestrial data centres. No regulatory issues, no infrastructure problems (power, water, ...), no risk of sabotage or drone strikes (see AWS in Middle East), and no land leases required.
Once you add up all these risks and costs and put them next to the launch cost that SpaceX envisions for Starship, the concept suddenly sounds quite plausible.
I'm sceptical myself - though for very different reasons - but I wouldn't call it an outright scam. It all hinges on one thing: Starship has to work. Cheaply, reliably, and within the next couple of years. If it turns out to be less capable, significantly more expensive, less reliable than F9 and F9 Heavy, or not ready until the mid-2030s, the concept is dead.
The engineering problem is called physic and thermodynamics. And unless we learn how to change those rules the problem will be unsolvable. It's like discussing perpetual motion...
I hear this a lot without actual calculations backing up the claim.
First of all, orbital data centre doesn't mean a monolithic 500MW monstrosity orbiting Earth. That's not what the companies actually investigating the idea (Google, Amazon, SpaceX, ...) have in mind.
The real-world implementation would use constellations of Starlink-sized satellites, so we're talking about a ~10kW power envelope. Under ideal orbital conditions (sun synchronous near polar orbit, ~250-450km), you can get away with as little as 40m² of radiator area depending on engineering margins, provided you allow your radiator/coolant temps to be 70°C and above (70°C is what I used for my estimation plus an ideal 400km SSO and a reasonably efficient radiator similar to the improved ETCS on the ISS - no ammonia for the coolant, though see below as to why).
People often cite the ISS and its ~100m² radiators for comparison. The problem with that is that for one, the design operating temperature of the ISS' EATCS is
4°C on the low temp loop and 17°C on the moderate temp loop. That little known fact aside, the current improved ATCS is capable of rejecting 70kW of waste heat - way more than the thermal output a realistic satellite would produce. The active temperature control system (ATCS) of the ISS is designed around humans and science experiments in a 400km 52°N to 52°S orbit, not silicon chips that are allowed to run much hotter than 25°C and sit in SSO.
In short the *actual* thermodynamics work out just fine if you consider real satellite designs instead of sci-fi monstrosities, use realistic operating temperatures for the coolant loop designed around silicon chips, and the desired target orbit.
So when you say "unless we learn how to change those rules the problem will be unsolvable" I'd honestly like to know why my 25+ year old literature on satellite mission and hardware design, Boeing's numbers from "Active Thermal Control System (ATCS) Overview" document, the formulas from NASAs "Guidelines for thermal analysis of spacecraft hardware", this gem from DLR https://doi.org/10.1016/j.asr.2024.11.024 and my calculator are all wrong at the same time.
Sure, I only did some napkin math (still took nearly 45min) using some simplified formulas and crude estimates for the required parameters (I was curious, but not THAT curious), but I always rounded up and used a healthy 2x margin on top of the estimated result.
Have you ever heard of U.S. Steel, IBM, Kodak, and Western Union?