I'm convinced that this is just guerilla marketing from Apple. When this started spreading a day or two ago, it was all from no-name spam media sites that are paid to publish articles. They all claimed "a source" is where they got the intel, without specifying the source. It was spreading like wildfire on socials.
The same thing happened with Mac Mini's and OpenClaw. Nobody cared about or was using Mac Mini's for OpenClaw, but there were all of these very suspicious posts from accounts that were clearly Apple marketing bots (you could tell by looking at their post history, where they would drop "Mac Mini" into every conversation they could across all different subreddits and unrelated topics). Then it became fairly common.
So Apple's marketing department is seemingly using the same strategy again. Because still, it's impossible to find a reputable source for this claim.
People really have a twisted idea about how corporate America works. Apple does not have a psyops division cooking up ever more elaborate ways to seed demand for low-end, low-margin Macs via bot armies. They honestly don’t need to, because they’re making an avalanche of cash from the iPhone.
People really were talking about and buying Minis due to OpenClaw, because they wanted something that was on all the time and had access to all their stuff in macOS. Occam’s Razor.
Having worked at multiple large companies more often than not the truth is there are 5 levels of approval for every tweet that goes out and writing a single blog post can take months.
The smart companies out source their gorilla marketing to a third party, having realized their own inadequacies.
I remember talking to a person who worked in marketing Kingsford, which at the time had like over 90% of the marketshare for USA charcoal. They had a marketing budget that was willing to spend money on any ol' stupid idea because they had already won & just wanted to stay on top by getting their name out there.
I worked in community marketing for few rather large software companies. The struggle is real. Almost every CMO I worked for seemed to make it their mission to squeeze any life out of social conversation with actual users and replace it with bland, boring "on message" communication about synergies for leveraging pivots enabling disruption at enterprise scale.
For every Tweet that goes out on the company account and every blog on the company blog. But that's not what we're talking about here - we're talking about vague sources like low-impact social media snippets getting cited on low-reputation blogs until someone more reputable comments on the apparent trend. There are absolutely marketing people who do that "unofficially".
Different ballpark. Original comment is about shadier methods of marketing not official stuff landing on the primary domain of Apple. I have 1st knowledge of a travel tech company laying off their entire marketing team during COVID that then got hired by a brand new company to find different and riskier approaches on how to keep the funnel alive despite the travel restrictions without direct risk of exposure for the brand.
Tbf comment you replied to never said they were direct hires. Outsourced workers sitting around is possible. You seem to have customized their more general statement
Can confirm though, being acquainted with PR people in the field for >20 years; everything on TV is scripted these days.
Chatted with some of my PR friends coworkers over the years at Christmas parties and the like. They claimed stunts during MTV specials, other "candid" moments from the last 20-25 well known to pop culture were actually organized by a PR company they worked at (never that they worked on that project; such work was reserved for whatever PR rockstar PR firm hired)
But all this too could be PR sales pitch to impress; society is full of hearsay to titillate
Yeah and why they outsource that? First is effectiveness of reach as you mention. Second is the ability to have a clean official record.
Wake me up when an extremely capitalistic company, in the past caught with dirty hands on numerous topics, will reject increasing massively a legal revenue stream with no strings attached. Right, exactly how to become a trillion dollar company and keep it.
I grok that specifically for apple their echo chambers do majority of certain work themselves, but claiming with 100% confidence thats all is a bit of a stretch. If I was a personal close friend with their head of marketing, I would maybe believe that. In any other case, hard logic prevails over anonymous online claims.
That only applies to tweets that are published on behalf of the company, not tweets from sockpuppet accounts that have no direct connection to the company and are made to look like a regular consumer.
Or they hire an agency and the agency does all the gray-hat stuff, and apart from the marketing person who hired the agency most of the company probably has no idea what the agency is up to.
It is multiple layers of approval, but there is no reason to think they wouldn't approve this. At least not from the limited information I have. The approval is about covering themselves, so if they won't get sued (either because it is legal or they can cover their tracks) they will approve it. There is more than lawsuits they track, but the exact set of things is different for each company and I have no idea what Apple works on.
Thus this could be Apple marketing, or it could be someone independent.
“I have worked over 100 of these kinds of campaigns and never had it come back on the client. I’ve been doing viral marketing and reputation management since 2005. =In the past year I’ve worked for a major entertainment network to magnify a rumor within sports entertainment, as well as damage control on a rumor that came out of an actor being hired on a film before the production company was ready to announce that casting.”
If apple indeed use shit posting, there has to be several layers of subcontractors in between for plausible deniability. Apple's own comms guidelines strictly prohibits pay-per-review, manipulative, "shit posting" style practices.
Companies that ignore everything besides their legacy cash cow tend to go downhill. I'm sure there are at least a few people at Apple who's main concern is marketing the Mac mini, and those people are highly definitely trying to cook up new marketing strategies all the time
I'm sure there are people dedicated to marketing the Mac Mini, but seeing as Apple has had trouble keeping it in stock from demand, I doubt they spend too much on it. And considering Apple is supply constrained, there is no benefit to trying to increase demand above supply levels.
I think the Mac Mini is legit popular and is actually being used for local AI use cases, especially Open Claw.
Beyond that, Apple is at capacity for SoCs and ram, and the Mac Mini is going to have a ceiling of how much of Apple's allotment they are willing to give it.
Sure. But thinking that means every article is a complicated psyop is classic correlation / causation.
If you’ve hot any occam’s razor left in you, consider:
1. Apple is a masterful media manipulator, carefully planting stories like this by gradually seeding fringe media first, then more mainstream, with a clever and subtle goal of making themselves look dumb in order to trick people into, uh, ok, I’m lost here, but something. Maybe drum up demand for products they can’t keep in stock anyway?
2. Media outlets chase clicks, and when fringe media finds a hit, mainstream media follows because you can always write a story about the fact that other people are writing stories. Low cost, proven clicks, nobody yells at you for missing a trend.
Which is more likely? I’ll cheerfully stipulate Apple is pure evil, serves cooked babies in their cafeterias, whatever it takes to discuss based on reason rather than big-evil-corpos.
To be fair, the very reasonable take is that if there was guerilla marketing campaigns, they are most likely being handled by external agencies and not Apple directly.
And, this isn't making them look particularly dumb. It is making them look dumb the way that most people make themselves look "bad" when they list their biggest weakness in a job interview. Silly Apple, not seeing how fantastically great their product is and how much people are dying to have it.
The most serious point in favor of this being organic nonsense is just the sheer lack of any actual evidence that it isn't, but the case is being wildly overstated, in part because we're using the non-standard term "psyop" to describe what I'd consider to be mundane marketing practices.
At this point I feel like there is no reasonable way to draw a conclusion. However, viral marketing like this is common and in fact, incredibly easy. This is done through PR services (and even shadier sources) all the time... I'm a bit surprised people on HN are unfamiliar with how utterly common the practice is, it wasn't new when I first came into contact with the industry over a decade ago. The hallmark of this is very similar articles appearing suddenly through the Internet and in the news. It's often not just random plagiarism. This is why the news is constantly running stories about how suits are back in vogue and other vain crap like that.
do people on here not know that things like forbes 30 under 30 are a pay to play game? do they not know that 90% of car influencer reviews are carefully curated by mrketing agencies? and yes apple is notoriously controlling about it too which is the reason to why it is unlikely to be apple driving this.
Complicated psyop? It's called "Personas" and a few of my coworkers wrote a few of them just this morning.
You simply promote or finance marketing materials (blog posts, podcasts, youtube videos) with a clear idea of the kind of people you want to reach, and if your "Persona" (that is, ideal customer or evangelist) model is accurate enough, they will talk endlessly about your product for free without having to directly connect with them.
That's very different from faking a bunch of excited chatter and making up happy customers, which is what the original comment was about:
> Nobody cared about or was using Mac Mini's for OpenClaw, but there were all of these very suspicious posts from accounts that were clearly Apple [...]
There's nothing masterful or careful about it. If it were, no one would notice it. This is as obvious and sloppy as it gets. No different to startup founders paying social media marketing companies $10-20k to promote their slop products all over X and Reddit with their army of AI generated blogs and news sites and social media accounts. This happens on the daily, and it is not a secret - every founder knows it, and most founders use it at some point. So to think this doesn't happen at larger scales with bigger companies is pure naivety.
It has nothing to do with "big-evil-corpo", it happens at all levels. For some reason you're trying to claim that "big-evil-corpo" are the rare exception, which is ridiculous.
Tbh, I won't be surprised that 1 is true; it does not look that far fetched. Organizing scarcity is already a well-established strategy in the marketing playbook; there are examples of other big organization playing a way longer game to conquer a market (see for instance how Japan became a coffee nation https://www.thetakeout.com/how-nestle-brought-coffee-to-japa...)
Because caring about factual accuracy isn’t “defending a corporation.” A claim doesn’t become true (or false!) just because it concerns a "billion dollar corporation".
Besides, Apple isn't a "billion dollar corporation", it's a multi-trillion dollar corporation.
> People don’t like nitpickers. “He literally did the WELL AKTUALLY!” If you say Joe Criminal committed ten murders and five rapes, and I object that it was actually only six murders and two rapes, then why am I “defending” Joe Criminal?
> Because if it’s worth your time to lie, it’s worth my time to correct it.
> If one side lies to make all of their arguments sound 5% stronger, then over long enough it adds up. Unless they want to be left behind, the other side has to make all of their arguments 5% stronger too. Then there’s a new baseline - why not 10%? Why not 20%? This mechanism might sound theoretical when I describe it this way, but go to any space where corrections are discouraged, and you will see exactly this.
I have no proof, only personal experience interacting with commenters on HN. No other tech company gets as much positive support when exerting its soft power under the guise of (bullshit) technicalities, e.g. all the issues with EU digital markets, API restrictions, 30% tax. Specifically, people were defending Apple vehemently when they feature gated remote phone control for EU citing concerns about EUs privacy laws (???).
I never stated that I know for a fact that Apple deploys bots, but I would not be surprised. I don’t even need to know if anyone is paid or not - if I cannot distinguish paid posts from fanatic apologism, then I need not care. The resulting posts are worrying all the same.
Similarly, I do not know if Apple was caught off guard by the demand - I wouldn’t trust this article. But Apple does plan production well ahead of time and forecast demand rather precisely. They could’ve been wrong. Them being unable to produce enough 1/2tb mac studios seems like a loss to me.
Why would anyone attack a billion dollar corporation so vehemently and not earn a single dime? Billion dollar companies also hire hundreds of thousands of employees, paying them a lot. They create products that millions enjoy.
I am surprised this take is controversial. It really seems almost certain whenever enough money is involved in controlling narratives and selling things, regardless of the details.
Certainly employees at these companies are earning more than a single dime, so even if innocent there's still an obvious conflict of interest. That doesn't mean the posters are wrong or their opinions invalid, on the contrary their direct experience is valuable to hear.
But they are certainly paid, and this is fair to remember when reading opinions.
edit: oh, I see, they're focusing on taking the last little bit super literally. great.
Some of us are motivated by interest in facts. I get that others are financially motivated, or focused on sports-style love/hate for organizations.
I can’t imagine coloring my perceptions or actions to ensure they only attack big companies.
But then I also can’t imagine accusing anyone who disagrees with of doing so in bad faith, or of being a paid shill. To me that looks like poisoning your own mind out of the mistaken belief it will somehow effect change.
BTW it’s not even a defense of Apple, it’s an explanation of how the press works. I worked in the tech press decades ago, and it’s only gotten worse since. It is not organized enough for the conspiracy theory in question to be plausible.
There are usual Apple sheeps ... and then it's you.
WOW, this is some remarkable (would rather use different word starting with R) desperate attempt to somehow turn the reality upside-down,of Apple being overpriced unrepairable unupgrade-able junk sold through pure marketing of exploiting insecurities of losers by selling them something that makes them feel "elite".
Unsolicitated "send via iPhone" SMS footer is perfect example.
>
Which is more likely? I’ll cheerfully stipulate Apple is pure evil, serves cooked babies in their cafeterias, whatever it takes to discuss based on reason rather than big-evil-corpos
No, but they have marketing people, and marketing people in my experience (even in "big" corporate America) have personal (anonymized, but just created themselves) accounts they do stuff like this on all the time.
This is exactly the kind of paranoid conspiracy thinking I was talking about. If you see red yarn between everything, you're impervious to reality.
Honestly, I'm an Apple user since 1988 and shareholder. I use my full-ass name on hn. I wish they'd give me money to promote the brand, but in reality they have something even more valuable than paid shitposting: authentic word of mouth and devoted users. That comes out of the product, not the marketing department, and it takes years.
Flawless logic! I can’t believe I got busted again… The other lizard people are going to be so disappointed. I’m already on probation for leaking the fact that the Earth is flat and bungling the JFK thing. Now they’ll probably pull me off the microwave mind reading project too.
I was buying my own Mac specifically for LLMs around this same time, didn't care about OpenClaw, but people I knew IRL were immediately asking me about buying a Mac for OpenClaw, and whether they should get a Mac Mini or what, and these were not programmers or tech nerds. There was genuine hype. Macs are lower cost alternatives to Nvidia GPUs.
I think it might have been a mix of the two to be fair, but probably a lower portion of devs. For example, I have a Macbook Pro with an M4 Pro but in about two more years once I've put aside money for a newer laptop I'll wait for the next "updated" models and buy another Mac just for the inference, the speed improvements and the unified RAM.
Apple has a history of feeding rumors sites fake information to weed out leaks. They take it very seriously.
Wouldn’t put it past them to pay strategic influencers to feed social media fake information to generate hype. As a way to make the leak ecosystem work in benefit of the marketing department (or psyops division if you want to call it that).
> They honestly don’t need to, because they’re making an avalanche of cash from the iPhone.
No, Apple is not content with however much it makes from the iPhone. That’s why it stuffed cringeworthy ads in Apple News, filled the App Store screen with ads and is now doing the same with Apple Maps. It’s a company focused on making money through every avenue possible, even if it means being shortsighted and losing the goodwill of developers and users.
and there's a goddam ad at the start of every apple TV show, and macOS forces ads on you via the SETTINGS app! (both such ads are for other apple products ... still, it's an ad)
True, but so what? Is the idea that they sponsored the creation of OpenClaw and Claude Cowork, CIA-style? Or is the idea that this (presumably lower-level, given that they're working on the mini) marketing employee had a large discretionary budget and was able to identify the trend and immediately capitalize on it with no corporate bottlenecks, mobilizing a series of commenters to underscore the narrative that you need a Mac mini to get into AI?
The narrative — ie being associated and seen as an AI-winner — is worth billions of market capitalisation at Apple scale.
I don’t think there’s a direct guerilla marketing division, but I would absolutely bet money that Apple’s marketing/comms team is encouraging creators, influencers, etc to promote Mac Mini and Mac Studio for AI.
And when those teams brief and speak to people, even on confidential terms, they almost expect some things to get leaked; or will keep sharing it with more people until a third party leaks what they want leaked.
So they may seed things like “super confidential, we were internally really surprised by the demand for AI”. Intentionally.
Might not be in the particular case, but what are your thoughts on the following (just to name a few recent examples): Cambridge Analytica, Net Neutrality, DMCA?
I have a friend at Apple, working in retail front lines. He was tasked by his stores' management to put his personal Mac Mini to use with OpenClaw in order to familiarize himself with what can be done with it, and then use that experience to inform his direct recommendations to customers. He even had an internal checklist to fill out once he'd done his own installation.
I think you're radically misunderestimating the power of the retail front lines. Folks leave the Guru Bar with what they see as 'bleeding edge ways to use Apple tech', and Apple know it ..
Obviously they do not, the same way Amazon isn't treating their drivers poorly, it is just third party contractors "surprisingly" not doing their job well.
It's just hilarious the degree to which people see a thing happen sometimes and begin to believe, despite not just evidence but plenty of counterfactuals, that it's happening every time.
Also many startups and companies are now building racks of Minis and Studios to automate build pipelines, testing, do local inference, run agents etc. I've personally seen this in at least 3 companies out of the half a dozen I've interacted with. The demand is certainly there.
Occam's Razor would be that the tens of thousands of marketers that Apple hires (contractors or otherwise) utilize the most obvious and effective strategy for making sales: Spamming the internet and manipulating perceptions. Their marketing personnel don't sit there thinking "well I'm only hired to market the low-margin Macs, so I guess I'll do nothing today."
Your argument is basically: Why would they try and make more money when they can just sit back and get rich off iPhone's.
You're suggesting there is no chance that Apple is engaging in guerilla marketing, which you discredit by referring to as a "psyop" when it is in fact a well-documented and pretty mundane practice, because they sell a lot of iPhones?
I mean, it's pretty much just blind conjecture that they may be marketing in stealth in the first place, so really who knows on that front. But you gotta be smoking something to think Apple doesn't want their low margin products to sell like hot cakes. I mean sure, they'd love even more for their high margin products to sell like hot cakes (like the Mac Minis that are useful for AI like in this story), but the low margin ones literally make up for it in volume so you'd sure hope they sell well.
They are a trillion dollar company, that wasn't by accident. They are marketing and sales monsters. Arguably of the big companies, they are the ones that have mastered consumer perception the best of any company on Earth. They would like all of their products to like crazy always.
You're right, Apple has a long history with guerrilla marketing, especially gray-area tactics like bots and paid commenters. Even though there is no evidence of this, it "feels" true, so it must be true, right? Why wouldn't they do it, after all? They don't really care that much about protecting the brand, and these types of tactics have zero chance of blowing up in their face.
Even though they can and do pay for the highest-profile placements, they also do shady guerrilla marketing "just in case," because they have a whole lab that just analyzes channel efficiency, and they've discovered that anonymous comments on Reddit in random channels have 8.7x the effectiveness of YouTube ads, especially among the coveted Pale Incel demo.
This is how they got to be a trillion dollar company after all! It was marketing, not product.
As you say, Apple would not be a trillion dollar company without a great product, but you can build something great and also aggressively market it. Nobody is saying they got where they are just with marketing.
Why do you care so much that this is a totally realistic argument? You don't personally have to like it, it's not about how it makes you feel as a fan of Apple.
It absolutely isn't. It is only ludicrous if you hold Apple to some weird standard of ethics that they absolutely do not deserve. If you think this practice isn't common, that's purely a matter of ignorance on your part.
I'm convinced that this is just guerilla marketing from Apple.
I'm not. I have an M4 Mac Mini running 24/7 so I can use it as my Claude Code machine when I'm on the go.
Why would they do a guerilla marketing campaign now if their Macs are generally sold out despite nearly doubling the Mac Mini price in the last few months?
Either they're running Claude Code with a local LLM (technically possible via dev tools albeit unsupported), or they genuinely have no idea why the Mac is good for AI and just bought it because they were told it was good for AI.
The former honestly sounds like a nightmare because the Mac Mini can run some big models but not that big, and Claude Code will guaranteed stuff your context with 35k tokens, and all the Anthropic proprietary tool-deferral and tool-search stuff is meant to be handled inside the inference engine in a way that I don't think any open-source engine like vLLM can support right now (much as I wish it did). So you have a huge context on startup, broken functionality, and an underpowered LLM struggling through it all. Not to mention I have no idea how any of the sidecar LLM stuff is meant to work in that context (e.g. auto-mode classifier and even things like dynamic session naming). Am I missing something?
I don't want to assume it's the latter, I want to be charitable to other people. But.
Either they're running Claude Code with a local LLM (technically possible via dev tools albeit unsupported), or they genuinely have no idea why the Mac is good for AI and just bought it because they were told it was good for AI.
Neither. I use Claude Code subscription through my Mac Mini when I’m on the go.
I develop using a Macbook primarily. Having a Mac Mini keeps the environment the same when I'm on the go. Plus, any home server would not be as efficient or fast as a Mini with an M4.
So your argument against this being guerilla marketing is that you bought a Mac Mini after seeing the guerilla marketing, to do the thing that you didn't need a Mac Mini to do.
I am using Mac studios for local inference, but not for openclaw. It was simply the best bang for the buck at the time, and the software side of things has gotten even better since then. I haven’t run the pricing comparison but I have a feeling with the direction they’re heading that it’s better to hold off for m6 as it may assuage various bottlenecks if you are latency sensitive.
The Pro, Max, and Ultra chips boil down to different combinations of their "Super", performance, and efficiency cores along with the number of GPU cores. The Max is two Pros bolted together. Thus far they've skipped generations of Ultra chips but Pro and Max versions of all the M-series chips have been released.
I can see them not releasing an M6 Ultra but no reason not to see Pro and Max versions of the M6.
>It was simply the best bang for the buck at the time,
I wouldn't have a problem with Apple fanboys if they just stated that they like Macs, but for some reason you have no problem with just straight up LYING. Or at least being so deluded that you state things that are so easy to prove false as fact.
I.e, 600 gb/sec is dogshit slow compared to vram speeds.
You could not get 500GB of VRAM for less than $100k at the time. It was the absolute best bang for the buck as far as local inference and fine tuning. I am not sure why you are so hostile when I specifically called out “latency sensitivity” as far as prompt preprocessing and various other bottlenecks. I run Linux.
The reason Im so hostile is because it personally irks me when a technical forum like HN is filled with Apple-slop being passed off as technical discussion. Its fine to like Apple for what it is, the ecosystem, the industrial design, the advantages of dedicated hardware with specialized software to extend battery life. But instead, there is Apple-slop, which is basically starting with the assumption that whatever you can do on Mac is the standard, and that Macs are great because they fit that standard.
Being able to run frontier models at <10 tok/sec is just an excersize in showing that you can afford a Mac. Its useless for any real tasks. Gemma4 is on par with a lot of the older frontier models like Gemini Flash, and you can run that at 100+ tok/sec on a 2x3090 rig which costs way less than even top of the line Mac Studio.
I'm not so sure. You could literally see configuration options start disappearing from the Mac Mini ordering flow over time as demand increased and supply dried up.
> Nobody cared about or was using Mac Mini's for OpenClaw
This is rather revisionist. A lot of the early OpenClaw publicity was about connecting it to iMessage (which was really only possible on Mac hardware at the time)
It's not. This article is weirdly light on details. The original source article from The Information apparently mentions that companies like OpenAI are buying them up to run RL. So it's yet another case of AI companies buying all the compute.
* I say apparently because The Information wants you to sign up for access to the article, but I found this mentioned in multiple summaries of the article like this one [1].
The easiest one to think about is that the larger the company (or division), the more likely there are leaks. Think about how much as leaked in Apple's hardware or software division. If they were doing incredibly underground marketing, someone would've bragged on LinkedIn or to a reporter.
Not exactly guerilla marketing, but decades of traditional marketing bearing fruit in people who fetish-ize Apple products, need to socially signal how cutting edge thy are, and have too much disposable income.
I follow a lot of people on Twitter who were one-upping each other about how much hardware they had bought (now gathering dust0 to run their Claw assistants. Those same people are now buying some toy robot duck?
The Internet has magnified the "my church leader said that gay people are actually demons" effect. Or "if you throw salt over your shoulder you're protected" or "wolves have alphas of the pack".
People don't fact check in the first place, they don't check that facts came from a trusted source, they don't check if information has changed the last time they got it from a trusted source, etc.
Is it just me or is it scary that it seems to me as if the average person just takes everyone's word for most things, as long as the person telling them is a "trusted source" based only on tribal lines?
There are also all the companies building iOS apps that need mac mini compute clusters for CI/CD because of dumb Apple requirements (ask me how I know..).
The openclaw crowd were just former cryptobros onto their next grift: performative posts about AI that would generate enough views/engagement/ad revenue to pay for the hardware… assuming they were even telling the truth about having it at all.
None shipped anything of value. Same with anyone claiming to be running dozens or hundreds of agents 24/4.
I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have.
I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me.
Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardware to get there.
> it seems really far off from the kind of experience even a basic $20/month subscription gets me.
The $20/month subs are much stronger than the local models you can run, even with how far local models have advanced lately.
The appeal of local models is that the data never leaves your network so you can feel safer putting sensitive content into it. It also feels “free” to use when you’ve already paid for the hardware.
But it doesn’t perform better and if you do the math you’re probably not saving money either. It’s helpful for things that you can’t or don’t want to outsource to a 3rd party.
There are a few use cases that are (somewhat) surprisingly unsuited for cloud providers:
- translations: cloud providers can bowdlerize (censor) bad words/content; also, if you want to do a translation for personal use of copyrighted materials, cloud providers may block it
- image generation: generating drawings with a style that even just resembles a copyrighted one (ie. Disney) may be blocked by cloud providers - for example, generating old cartoons style with GPT may not be possible.
What about a light but bulky AI job, like batch processing 50GB of files? I'm currently doing it on my used Macbook M1 Max 64gb, and it's chugging through it for the cost of electricity (free with my solar).
Cloud does not mean censored. You can rent gpu time and run whatever model you want, with your data kept as private as any other cloud instance you personally run. Cloud is location, with (for some work) wayyyy cheaper access.
I don't think it is really about "sensitive", but basically about any content you put in. Why would you give corporations your reasoning (data on how you interact with AI, how you "talk" etc.).
All of this is private, but not necessarily sensitive. You never know what is happening with this data. They might say they don't log it or don't sell it, then few years later you'll find it all online or read a book that has a story eerily similar to what you chatted about with GPT a year ago.
Although in this use case, it's likely because GPT guided you to write the same story as somebody else. Talking with an LLM about an idea is a great way to make it more predictable and homogenized. If you're fixing a bike or writing software, this is usually a good thing.
Gemini not long ago, when you said something "useful" said thank you, I will use it to help other users with similar problem. When asked "why would you do that, I thought our chat is private?" it would respond "Apologies. My mistake, of course this chat is private and your information will not be used." Funny.
> They also have enough power to negotiate contracts with strong privacy provisions.
What privacy provisions would you want to add to AWS? Most of the reasonable strong privacy provisions you'd want are already there and/or available if you want to sign up for it, even including US Govt Top Secret data if you meet some approval.
If they decide to ignore the contract and violate the privacy provisions, the US government can punish them, can the company from future contracts, throw executive in jail.
If they violate their privacy agreement with me there's effectively no punishment I can get that they will actually feel. There would have to be a class action law suit and they always just settle those for some small amount and admit to no wrong doing
> If they decide to ignore the contract and violate the privacy provisions, the US government can punish them, can the company from future contracts, throw executive in jail.
To be honest, they'll just send a few million and some gold plated trinkets to the right people and it'll go away. You know that's how things work now in the US. Even if you're an actual convicted drug dealer you can have your mother say a few nice things and whoop there's your get out of jail free card. This stuff is peanuts compared to that.
If not they'll just stall in court and settle for peanuts after everyone is exhausted battling it out. That's what they've always done before the current government made it even easier. Fines are not a penalty, just a business expense.
Nobody important is going to do any time for this. Not in this America.
Yes, insofar as any "guaranteed" is cloud providers just saying "we pinky swear to not decrypt your resources!"
I know what I'm talking about here - trust me when I say that the difference between a "government" or "large corpo" account and "some dude" is literally a few flags to prevent resources being created in the wrong regions. It's nothing that a random customer can't do themselves.
I like to buy specific brand of soap. I don't want them to know that, it is my right and so is running local LLM "wasting" money on local inference to keep track of my stack of soap.
Those companies also have data sharing/use agreements that they can get from Cloud AI providers due to their size and spend. The secrecy and data protection is largely what they are paying for. Those types of agreements just aren’t available to individual customers. It’s only when you’re spending $$$ that it becomes worth it for the provider.
glm5.3 is matching fable in lots of places and beating it after more than 1 pass in many. now of course when i say this, folks would claim that it's not local, but it can be. if you happen to own a mac studio 512gb, you could run it.
What supply crunch? Tons of RAM available for purchase. It's just expensive. That there is a "supply crunch" is made up to benefit from Trump administration not giving a shit how big corps operate
There's cloud hosts out there with unused compute. Wasted cycles are all over businesses running unused cloud apps and subscribed to services they don't use.
Still need a local computer to access the cloud; so a barely used gadget still exists. And this creates duplication of effort; we built RAM for servers AND the edge devices.
Seems redundant when tech nerds and corporations are really the only people that care.
And all that data in the web is meaningless yet we create a supply crunch storing it in servers.
This an out of touch nickel and dime perspective given the big picture to say nothing of the mess of strip mining and manufacturing pipelines that go into every screw, wire, and such
I have an older M2 Mac mini that does the OCR and visual description of all my screenshots. Screenshots are stored on my NAS.
I like to screenshot things as a quick way to remember. They are things that I would not be comfortable sending a cloud provider (customer data, prototype screenshots, bank dispute details).
It runs Qwen3.5:9b and glm5.2-ocr with Ollama and uses about 10GB of RAM. It automatically releases the models from RAM after 5 minutes of inactivity so it is pretty seamless to leave running in the background.
All the details are stored in a simple webapp with a SQLite db that I can search through.
> I have an older M2 Mac mini that does the OCR and visual description of all my screenshots. Screenshots are stored on my NAS.
Doesn't Apple do this already within it's OS all locally? It certainly does it for OCR and categorization.
EDIT: Also, no reason to use a generic LLM for this. This functionality exists in something like Immich (both OCR and 'context categorization'), and doesn't tie you into the Apple ecosystem either.
I personally use Apple Photos for this. It stores the original, plus makes them nicely searchable, so I have a Hazel action that takes screenshots from the desktop (and from my NAS where mobile devices back them up) and imports them.
Neither openai or xai. Anthropic maybe but not likely. Mistral is the most likely one because they're under the EU laws but I think they focus more on commercial these days.
having a 64GB mac mini m4 pro the last few years with some increasingly capable usefulness has kept me interested in this stuff in a way that using a paid platform wouldn't have. Similar to running K8s in a homelab, something about interacting with the hardware makes it more engaging/interesting, for me at least.
In general, I'm a big believer in doing more with fewer resources, within reason, and think having local setups really helps me be mindful with what's happening under the hood with these systems and managing context efficiently to get high quality results.
I guess it depends on what you're trying to do. I've run a few LLMs on my 64gig Mac but anything image or video related is ridiculously slow compared to even old NVidia on my PC.
Have you been using an MTP setup? I've been having pretty good luck with the Qwen models with the built in MTP heads via https://mtplx.com at around Q4. My main driver rig is an M5 Max MBP work got for me a few weeks ago. Hitting 1000-1200 TPS prefill on Qwen 3.8-Flash-Next there.
A $20/month Gemini subscription is truly all you need, then yeah, sure.... obviously a homelab setup is a ridiculous alternative on a pure cost basis. For most people doing "real" work with LLMs 40+ hours per week, a more apt comparison would be one or multiple $200/month subscriptions. At which point the break-even point of a homelab is much sooner.
However, most people running homelabs are doing it for other reasons. Independence, learning, and/or privacy issues.
I have not spun up my own homelab, but, I have researched it and the electricity costs can be mitigated to a large extent.
One... the GPUs can be massively clocked down during idle state, to the point where the fans can be shut off as well. The machines themselves can be shut down and wait for a magic wake-on-LAN packet if needed.
Two... even under load, the GPU cores can be significantly underclocked with very little performance loss. The GPU and VRAM/HBM clocks are independent, and the GPU is largely bottlenecked on VRAM/HBM so you can just drop the GPU speed. A commonly reported figure I saw was, basically, 350W nominal cards being underclocked to consume "only" 200W under load with ~5-10% perf loss.
This all assumes you're using discrete GPUs and not AIO systems like a Mac Studio which is going to be pretty efficient just by design; they idle at 35W or so and in practice max out at a few hundred watts. I believe DGX Spark and Strix Halo are similar.
I must stress that this is second-hand anecdata here, admittedly, but my understanding is that it can be pretty manageable.
> There is no reason to believe that equivalent level model output will be more expensive in 12 months
It's almost never a drop-in replacement, and having to check and adjust integrations and workflows with new models gets old fast. My task was perfectly solved by the old model, I don't need a newer, "better" one - especially at higher prices ("more cost-effective" my foot). Local models lets one choose a model and freeze the downstream integrations forever, without being forced on the 6/8-month upgrade treadmill by aggressively short, scarcity-driven hosted model-deprecation schedules.
The big providers are losing on average tens of billions a year on these services, so yes prices must go up. Even Moore’s won’t help in the medium-term due to shortages and difficulty/reluctance to vastly increase capacity.
I thought the discussion was about "frontier labs" in the cloud versus open at home.
Others running open models in the cloud is a nice third alternative, but not a solution for those that need frontier models, which I believe continue to need training as well as inference. These are currently heavily subsidized and hardware constrained several years out.
They said "equivalent level model". It's reasonable to assume that open source models in 12 months will be just as good as frontier models today.
And this sub-discussion is more about whether there will continue to be some cloud model that's both more capable and cheaper than local, not so much about what kind of cloud model it is.
I would think it's the opposite: there's increasing chatter about how long the AI labs will subsidize cheap subscriptions. I suspect that at some point one of them will bump up prices, and the rest will follow. That'll signal the end of cheap subscriptions and prices will steadily creep up.
I'm not saying that local AI is necessarily economically justified right now; but it's certainly quite reasonable to think that in 4 years the subscriptions will be significantly more expensive.
Surely you cannot possibly believe there is no reason.
Don't get me wrong: I hope you are right, and I am generally optimistic about the future of AI.
But do you really think, in a world filled with examples of big software companies repeatedly taking away or hamstringing capabilities we've taken for granted, that you can just count on a big tech company hosting cheap inference on incredibly powerful models forever? Surely we have learned by now that these companies do not exist to provide a public service to us, and the government cannot always be counted on to have the best interests of the citizens in mind.
I mean, how many times have we seen this in just the past decade or two?
- Consistent attempts to pass legislation weakening or banning the use of encryption
- Exorbitant Reddit API pricing (still salty about the death of the amazing Apollo app)
- Google fighting against sideloading on android
- US gov't issuing export control directive to suspend access to Fable/Mythos
- US lawmakers considering ways to regulate adoption of open weight models
- Chinese officials considering restricting overseas access to their most advanced models
I can absolutely see a much more restricted, closed down, and expensive future due to a combination of government regulations (regardless of which nation is doing it) and big companies rug-pulling as the check comes due on all the billions of dollars spent to get here.
This is such a tired argument and it seems to be parroted every single time someone talks about local models on hacker news.
Yes, of course the most economical path is to hand over all your data and become fully dependent on a cloud provider who is already operating as scale, hoping that they won't change/remove models, hamstring capabilities, or raise prices.
If this were a thread about hosting your own email or blog or cloud photos, you'd have plenty of people out here telling you how easy it is to do it yourself instead of relying on Gmail for email or WordPress/Medium/Substack for blogging, or iCloud for cloud photos.
And yet, without fail, every single thread about self hosting local models seems to have some copy/paste form of this cost-savings argument.
Where is the appreciation for this cool thing GP built? Where is the appreciation for the desire to figure out how to host your own version of the incredible capabilities that were not available merely a few years ago? And why, on this site of all places, would someone advocate trading all of the knowledge and independence gained from learning how to host something like this ourselves in favor of throwing it all over the wall to Google?
It's quite shocking to me how many experienced, tech-savvy people, who used to care about cookies and ad tracking - are now willingly sending their business strategies, highly confidential contracts, and intimate personal issues to a cloud provider because "it is only $0.0x per million tokens!".
> It's quite shocking to me how many experienced, tech-savvy people, who used to care about cookies and ad tracking - are now willingly sending their business strategies, highly confidential contracts, and intimate personal issues to a cloud provider because "it is only $0.0x per million tokens!".
Because there are more privacy guarantees there, depending on the provider. "But what if they violate their contract!" is some pretty tin-foil hat stuff.
How is this any different than a business running their website out of the cloud, assuming you are using a provider with appropriate contractual terms?
You can care about tracking and ads but still be comfortable storing your backups in the cloud, and many have been for quite awhile, even sometimes without encryption - that is totally different than e.g. Meta actively trying to track you and understand your relationship graph and your purchases etc.
I'm not sure about the tin-foil-hattedness of worrying about them violating their contract. But that's by-the-by. It is definitely not tin-foil-hat to worry about the data being taken in a breach.
Depends on how you use it. You can use AWS in a way that protects your data even if Amazon is compromised. You can even run LLM inference that way, but it doesn't seem at all common.
I would say tech workers have very weak class consciousness and understanding of power structures. Those who care about not ceeding power is a tiny minority compared to say among MDs and lawyers and their guilds/unions.
I don’t understand the willingness to give up privacy so easily, particularly if you are developing something that you plan to monetize somewhere down the road.
I’m pretty sure that all of those disclaimers that all the AI model makers have for you to sign off on to say that they’re not responsible for anything that might go wrong if your work gets copied accidentally and used someplace else wink wink?
You know the lawsuits for that particular aspect are incoming in the future…
Yeah I use it for hobby only. I would use cloud models to develop new stuff, I don't really care if it gets lost because if I were to publish it it would be FOSS anyway. I hate entrepreneurism and monetisation so I'm happiest being a salaried employee with many hobbies :)
But no way whatsoever I'm uploading my personal files, photos, emails, chats into cloud AI. No way.
I think the price is beyond that though. AFAICT, there's not competitively fast image or video generation on Mac. To buy it will cost me $4000-$12000. So I rent.
Yes but I also get a full fledged computer in the deal. I can sell it later. I can use it for all sorts of things like games and browsing and video editing. Paying for Gemini for other tasks is also in the mix but at the end of 4 years I get...nothing.
> They are things that I would not be comfortable sending a cloud provider
It's also an old machine that the commenter already has; it's intellectually dishonest to compare it to the price of a brand new, 4-iteration-newer machine.
>I realize I’m somewhat limited (16GB RTX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me.
I just ordered a new Mac Studio M5 Max 128GB $5899 ($6400 with tax) to be able to run the bigger "consumer size" models in the 70B parameter range (~96 GB). That said, I have no illusions that this expensive setup with a Qwen Flash coding LLM will be comparable to a $20/month subscription. Even upgrading to an even more expensive Mac Ultra 256GB for $10000 to hold a bigger model still won't be comparable. Apple hasn't shipped my Mac yet and I'm still considering cancelling it and downgrading to a smaller 64GB RAM config ($4299) to save $1600.
Why did I initially spend the extra $1600 if I knew ahead of time that it wasn't as good as cloud AI? Because I thought I could use some local LLM for the easy tasks or when I hit cloud rate limits. No issues with privacy so that wasn't part of the motivation at all. I just wanted some local AI capability to augment a subscription. I've not totally convinced myself of the cost/benefit of this.
Based on today's consumer hardware landscape, you're paying very high prices for crippled capability compared to the cloud AI subscriptions. We're also in a transition period where the next iteration of hardware improvements have some compelling features for local AI. Apple's upcoming M7 (2027 or 2028) is anticipated to have better GPU and neural engine to help with prefill TTFT. AMD Strix Halo is about to release 192GB system which is a big upgrade to their current 128GB ai pc. Maybe apply my $1600 savings towards those newer products. Those future products will still be very expensive but maybe the cost/benefit will be better.
> Why did I initially spend the extra $1600 if I knew ahead of time that it wasn't as good as cloud AI? Because I thought I could use some local LLM for the easy tasks or when I hit cloud rate limits.
The maths don't check. With Deepseek Flash one goes a very long way with 1600$ - even 10$/month, for easy jobs, are more than 13 years, and at a higher quality.
Oh no doubt. But one does have the guarantee that no bits left their home and that's a big one for some.
The low hanging fruit stuff for me is more something I use it for because I have the local LLM setup running anyway. It wasn't the reason I bought it, but now that it's there I might just as well use it as much as I can.
> Oh no doubt. But one does have the guarantee that no bits left their home and that's a big one for some.
No, this is a big misconception, and part of the cargo cult.
Use cases like the parent's are essentially about having a local LLM handle the leftover tasks. By that point, a lot of bits (main/big tasks) have already left home anyway.
It not only about it being an expensive setup (or not), and also other considerations:
- There's no guarantee of the $20/month service, and it likely has some limits compared to dedicated hardware token wise.
- Model are becoming more and more efficient, in many cases an M1 Max Mac Studio is still capable with 32 GB. 128 GB ram may not be the necessary baseline.
- Folks may think they want to only have a general model running locally (it's the comparable after all from the cloud providers), but we have to remember if the tasks we're trying to do ultimately are more specific than general and if there's space for the smaller models to do that.
I think the M5U Ultra 96gb is the sweetspot in that price range. It has more compute and bandwidth so you get to run models better sized to its hardware. I believe the Max would be too slow; personally I'm getting this SKU because I think it'd suck to get the 128gb Max and then discover it's too slow, and end up just using cloud providers anyway.
I agree about getting the Ultra if you're interested in AI (LLM) inference speed. I'm a little perplexed as to why there isn't a RAM option in between 96GB and 256GB, though. For instance, I believe Deepseek v4 flash runs a lot faster on (https://github.com/antirez/ds4) with 128GB than 96. I assume the higher SKUs have low enough sales that Apple doesn't want to be manufacturing too many different RAM configs as that would eat into their margins. (I say this as someone interested in AI hardware in general, even the 96GB is out of my price range).
It's the upsell ladder. This is how they get you to buy a more expensive tier or two. At first I was only going to look at 64gb or 128gb Max, but ended up here.
I also think it’s possible that it may be best to continue waiting on the Mac side of the house despite the increasing prices.
I think that some of the hardware design folks have been blindsided by AI demand and we haven’t really gotten that next generation AI hardware yet, to the point where buying M5 isn’t going to make sense in a couple of years.
Rumors seem to be that the M7 is the generation that Apple is looking to push AI performance much further.
I’m not sure that Apple anticipated this specific route that computer hardware has gone and I don’t think M5 and previous iterations were really specifically architected for local AI performance, more like they happened to be pretty good at it.
> Rumors seem to be that the M7 is the generation that Apple is looking to push AI performance much further.
What specifically does that mean? The M5 series has 10 cores per 128 bits of memory bus; are the cores unable to keep up with the RAM? I thought they did and memory bandwidth was usually the bottleneck. But the memory bus is already very highly clocked and goes up to 1024 bits wide so it's hard to picture memory bandwidth having a huge leap.
LLMs token generation is memory bandwidth constrained. If the M7 has double the memory bandwidth as some speculate [1], then it will help with LLM performance. There is also expectation (possibly unfounded) that the GPU will have improved matrix performance to help prompt processing as well.
14GT/s is a goal but it might be a while, I think 11-12 is more likely in that time frame.
50% more pins... we'll see. If they can reasonably make that fit then even more shame upon the traditional desktop CPU makers for sticking with 128 bits for so long.
Keep the memory. You’ll be glad you did when you realize that you’re better off with a solid coding model plus a good voice model and also a lightweight all-rounder all running at once isn’t of loading dynamically (slowly). It also helps if you want to be able to run a browser, IDE, and container environment.
> AMD Strix Halo is about to release 192GB system which is a big upgrade to their current 128GB ai pc
Big upgrade to memory capacity but memory speed is only going up by a few percent, so its still going to be slow with more than a few B active params (I have one)
you can get 4xGB10 for <20K so that gets you about the same tg and pp will be probably better. Power consumption though will be something like 200W idle so that's a bummer. And you get VLLM and SGLANG unlike them mac where one has to use oMLX (nice but not the same concurrency or cache performance)
I have 128GB M5 Max laptop and sill prefer to run things on other boxes in basement because its no fun to have the primary device being overloaded.
I’ve been running DeepSeek 4 Flash, Qwen 27B and Qwen 9B on local hardware. They work well for coding and document review tasks. I think Qwen 9B local on a 5090 might be legitimately helpful for small task agents in omp, since it is ridiculously fast. But my motivation is that I have data that I unfortunately can’t share with 3rd parties.
I have been eyeing a 512 GB Mac 5 Ultra to run full DS4 pro locally, which I expect would be pretty amazing as far as quality/recall. The only downside is that the speed is a lot slower than something like 27B on the 5090.
I have a RTX PRO 6000 96GB when the pricing was way better than now i also have a RTX 5090 too.
What I noticed is that (1) the great local models are optimized run inference (diffusion & LLMs) well on 32GB VRAM <= GPU's because that that's what the target has ...
(2) The quality of local models (esp. in diffusion) is increasing faster than the need for more VRAM - additional reason for the value of these FAST GPUs to increase!
(3) RTX PRO 6000 96GB is really great for fine tunes (ai-toolkit) :) but doesn't outperform my RTX 5090 with inference by anything significant on the good local models.
I have never run an AI job on a Mac, i also have doubts about performance and compatibilities - since the reviews almost never compare directly.
A huge benefit of local setups at our studio is that a lot of our software can't run headless, so when we're having agents work in Blender or in Unity etc with MCP that'd otherwise eat up our normal computer use. (Try to have two people try to work in the same Unity editor at once... then try ten!)
We also built some QA agents that are always playing our games from the same builds a player would and flagging things to fix/improve; that alone needs the game focused and front-and-center so it can properly screen-capture for deciding what inputs to take next (and for screenshots/replays), which also means we can't really do any hands-on work at all on the machine when it's running.
Having a separate (and tiny) machine for all of this has been great. We don't bother with local models because, you're right, the $20/month sub is way better than anything that can run on small consumer hardware atm.
> A huge benefit of local setups at our studio is that a lot of our software can't run headless, so when we're having agents work in Blender or in Unity etc with MCP that'd otherwise eat up our normal computer use
I'm curious about your setup. I've been tinkering with the idea of setting up Blender (cli use) in a container to allow agents to verify the scripts they are generating compile at a minimum. One thing I've found extremely helpful was generating a RAG of the current version of Blender.
For anyone wondering, I'm running Gemma4 26b A4B on a mini PC with 32 GB of DD4 and a Vega 7 iGPU (llama.cpp w/ Vulkan).
I've been experimenting with local models on an M5 Max MBP w/ 128GB of RAM since March of this year. Generally I've had very good results. Where things were lacking initially was with tool calling and the need to rely on tool calling for functionality like web search, which is otherwise well integrated in the cloud models. There is also a lot more work required on the harness side, however at this point (August 2026) there is not only much better tool calling in local models, but community supported projects have built good harnesses. Pi.dev and OpenCode + a SOTA local model is /very/ /very/ capable these days. Using LM Studio's built-in chat with a decent system prompt and proper tuning with local models is /very/ capable these days. Cloud models are still better, but it's the "harness" (not in the desktop app, but in the backend) that makes it so for the most part.
In my limited experience, not quite yet but we are damn close. Qwen 3.8 27b is it. If I could run this as a decent speed, I would no longer need cloud models at all. I'm actually currently trying it out in the cloud to pay for the inference speed but the model is fully runnable at home.
I realistically costs $5-10k to replicate a ChatGPT like agent. And it doesn't scale.
That's still really close. And models and quantization etc keep improving.
I'm absolutely positive that I'll be switching to mostly local AI in the next 5 years.
Good 60-70 tg and 2K pp Qwen 3.8 27B FP8 can be had for about 5-6K (2xR9700 + PC)
Gives about 3-4 concurrent sessions with full 262K
Fast 150+ tg and 2-8K pp Qwen 3.8 27B nvfp4 is about 8K (5090 +PC)
Gives really only one concurrent session that flies because kv caching is not perfect for ninfer https://github.com/Neroued/ninfer
both are very serviceable, I prefer FP8 on 2xR9700
But, yes it doesn't scale that well but in 5 years the same hardware should still be very capable of running some great MoE models, for example Qwen 3.6 35BA3B on 5090 can fly at 600 tg
I could believe that especially with some of the chips playing catch-up.
E.g., the M7 chip is rumored to be the one where Apple has poured really serious effort into local AI performance where previous generations seem to have mostly been coincidentally good at it.
Maybe this is an incorrect opinion but I don’t personally think that the M1-M3 or maybe M4 or even M5 chips were designed with LLM inference in mind at all. These were designed with things like video rendering, image/video ML, and rasterization performance in mind.
Local setups aren't going to make sense purely from a cost perspective, and definitely not when you are buying Apple hardware. AI subscriptions are too highly subsidized right now.
I think your last point is exactly why I'm so interested in local models. The current landscape doesn't feel sustainable. The last few months we've seen the big providers (OpenAI, Anthropic) start to play with usage limits, resets, banked resets, pulling models, etc. I think local models are close to the point where, with a sufficiently well-architected harness, you can get results that are on par with the experience you'd have with cloud inference. It is nice to know that I have hardware under my desk that I control with open weight models that I can interact with on my terms.
It is certainly not sustainable but that is exactly why I want to use $400-$800 of resources a month for $20 while the deal last.
It is really a once in a lifetime deal.
Once the deal is over the local models will be better than what I am using now anyway and the hardware will be all the better than what I can get now for the price.
> local models are close to the point where, with a sufficiently well-architected harness, you can get results that are on par with the experience you'd have with cloud inference
In my opinion, 98% of the work most devs would send to an AI can be capably achieved with a local model and a frontier-level model is overkill.
The goalpost moving feeds right into Anthropic and OpenAI's interests.
> AI subscriptions are too highly subsidized right now
I've been running into annoying limits with Claude recently. It gives me like 5 questions over the course of 15 mins and then tells me to wait 5 hours. When companies can change things up to make the base subscription nearly useless (the last question always gets messed up, too), then you realize the value of owning your own infrastructure.
You're right that $200/mo is much cheaper than comparable infrastructure. OTOH, you don't get to have a computer that can also be used for other applications, or which works when the internet is down. Also, you remain tethered to whatever pricing the AI companies want to charge. If AI pricing goes like Uber/Lyft did when the VC cash ran out, then we'll be paying much more in a few years' time. We could look back and think "I wish I'd bought my own setup back in 2026" if it's going to be inevitable.
And all this before you get into privacy/security/compliance stuff.
Well it's not just text to text. It's text to code, text to images, research. For me 200 is too much but my 20€ sub yeah I do find that worth it.
If I actually made money with it (I don't and I don't want to) the 200 one probably would start making sense. If I were using it all day I'm sure it would save me more than that 200 by simply being able to do more work.
Yep, main reason I ordered my Mac studio was because of privacy and control. I know no one will see the work being done on that machine and also the model will never be silently swapped out for a different lower quality model like anthropic does or a lower quant like openAI does with their models. Also having the inference engine has huge benefits because of speculative re-generation capabilities for voice agents. Anyways people who care about cost are simply bad at math if they think thell be saving any money running locally versus cloud providers.
Image and Video gen is superior locally, because you can tweak more, use LoRAs, use whatever model fine tunes, and generate uncensored content, plus as you're often cherry picking from multiple gens it ends up being cheaper for comparable quality as well.
Local coding is a step down but good enough for a lot of things if you have privacy concerns.
One use case I find cost effective is using it as a voice assistant for Home Assistant. API pricing on models is very weird compared to the normal chats, so I use Qwen/GPT-OSS on my Mac Studio via llama.cpp server.
Wild oversimplification, and benchmarks vary widely, but I've read a lot of benchmarks suggesting that Qwen3.8-27B (xhigh effort) competes with near-frontier models at a lot of coding tasks. To the best of my understanding it's not going to run very feasibly in 16GB of VRAM at usable quants however.
r/LocalLLM and r/LocalLlama are noisy, but valuable sources of anecdata if you have the time (or the tokens, hah) to comb through them. You are going to see a lot of modest setups there, and also guys with $20K+ of hardware.
The two things (besides my bank account) that keep me from investing heavily in local are (1) we are not guaranteed to get a steady release of open models in the future (2) a lot of the "fun" stuff LLM stuff that interests me involves orchestrating lots of parallel agents, which of course multiples the hardware you need to achieve it.
For example, I've been having good results having both Sol and Opus review the same PR, and then I have them cross-review each others' PRs. A next step I'd like to consider is maybe having a swarm of Luna agents review the same PR and have them fight it out... maybe with Sol doing final arbitration? I suspect 5-10 Lunas might outperform a single Opus. Or maybe not. But at any rate, that would be impractical in a homelab without a pretty big hardware (or time) budget.
It's not that far off anymore. On my 7900 XTX 24GB, I can run Qwen3.8 27B with 131K context at Q4_K_M (55 tok/s with MTP). Excluding hardware cost, it's about $0.02 tok/M in and $0.40 tok/M out (cached in $0.0001). On OpenRouter, that would cost more than 10x what it actually costs me.
Of course, 131k context at 4-bit quant is a trade off, but even then, it's VERY capable. It doesn't feel that far behind something like GPT 5.6 Luna.
IMO local models require a substantial amount of prompt+harness engineering to get in the neighborhood of what you'd get from a cloud model. Which isn't a bad thing, you'll learn a whole lot about how these things work.
What you'll learn pretty quickly from said engineering is that there's a lot more to a good LLM than just the weights themselves. You need a good search provider (also self-hostable, but sounds easier than it really is). You need (well, it's debatable) a memory system. You need a good system for up-to-date library references like a Context7 (also self-hostable but the options are surprisingly not that good). You need a good set of specialized subagents that can perform various tasks well -- for the sake of "doing things well" but also managing context efficiently.
When you've got all that, local models can be _extremely_ useful. But there's one other important thing and that's decent hardware, unfortunately. A lot of people try out local models using small consumer GPUs or Macs and are rightfully unimpressed with the performance. And if the performance doesn't get them, usually they have expectations that they'll perform at Claude levels out of the box. Getting in that neighborhood, like I said, definitely requires some work.
What you’re describing is exactly what I’ve experienced in my time testing out this stuff locally, and I had a hard time figuring out what exactly to blame.
I keep hoping that one day some comment is going to paste a link to some kind of idiot-proof guide or piece of software that’s “90% as good as Claude but running local.”
And by 90% I don’t mean that the model is 90% as good or runs 90% as fast, more like all the other stuff you mentioned is set up out of the box.
Local inference can't compete with cloud on speed, intelligence and economics. It's all about freedom, privacy, control, sovereignty.
It's about not having to accept any of the stupid "terms" of the corporations. It's about doing things the big labs don't allow you to do, like cybersecurity stuff, or even just chatting with the AI about some wrongthink.
The article implies the demand is for running locally. I’m not convinced, at least with a mac minis. Most people I know and myself buy the mini as it is always on, easy to setup, and isolated from my main computer which is a laptop. The mini is driving the use of the $20/month subscriptions.
I think you miss the point and value add. The always on computer isn’t to run the client only, it is to run all the things a user would normally. The laptop becomes a terminal for a computer that can do anything your laptop can. If you don’t have the need or desire for that, then yes, use a rpi or whatever.
In my case, I dev on macOS. The env the agent runs in is the same as my dev laptop, configured and in sync. Has access to all the same tools and environment as I would on my laptop.
I think 24gb is the bare minimum for a local qwen3.8 based setup. I've used qwen3.6 and it's not as straightforward as "can it replace "
Local llms don't suffer from cloud availability issues. Anyone that used Google models know that sometimes they just don't have capacity whatsoever, at least that was the state of things some months back when I used them. Just bear in mind if needed, cloud providers will prioritise API and corporate customers over subscriptions if availability degrades more.
Also they don't have the same guardrails as the other models, so for hacking, reverse engineering and black coding (piracy etc...) these local models might be the only options.
16GB VRAM could load a small quantised qwen 27B model but it would be a ways away from a frontier cloud model.
Though keep in mind not being beholden to shenanigans from said cloud companies (and interference from government entities!) is definitely worth something intangible.
For the amount of tokens you get, based on your comment, ALL subscriptions are heavily subsidized, and the most expensive ones are the most subsidized.
For OpenAI and Anthropic, the $100 subscriptions cost 5x the $20 subscriptions and give you 5x the tokens. And the $200 subscriptions are 10x the cost for 20x the tokens. (Tokens cost 50% as much.)
The principle of KISS keeps coming to mind when I consider local computing. I'm looking forward to the day we can just run Opus-level models at 100 tok/sec on consumer hardware.
But currently it's really hard to beat anything offered by the cloud companies. And the cost and complexity of setting it all up, just to barely (if at all) touch on Opus-level intelligence makes it seem like we're not quite there for the common man (enthusiasts are a different story.)
I am very excited for open source local models, and we're nearly there, but it's still too complex and expensive to be my daily driver (yet).
From what I’ve been seeing, the Mac studios do look like they have potential. I was looking to drop $10k-$15k on one until recently. After comparing a Radeon 7900 XTX vs Ryzen Halos 128GB vs M1 MacBook Pro 64Gb, I landed on just getting an external closure setup with Nvidia RTX 5090.
The model I’m specifically targeting to use at high speeds is Qwen 3.8 27b @q4ks. This model actually proved to be good at coding (it sits somewhere between Sonnet 5 and Opus 5 capability). M1 got 10 tok/s, Ryzen Halo 20tok/s, and Radeon 7900 XTX 50tok/s (can only do 128k context window in Radeon card).
The prefill gets extremely slow around 50k tokens in context window (whatever prompt processing stage entails could be wrong about phases here). It takes about 2 hours to fill the context.
Even with a drafter model intended for speed instead of mtp, I can’t get past 70tok/s, still is extremely slow to process prompts as context grows, and drops down to 40-50tok/s anyway making this config still moot for improvement on my Radeon card.
The only thing I can point to slowing me down is bandwidth of the card itself.
I am waiting to actually get my 5090 right now and I am betting that the 1700 Gbps of capacity will fix my prompt processing speeds. I don’t need full PCIe lane bandwidth to serve my house I just need to load the full model into vRAM and let the GPU do its thing.
Additional benefit to the external enclosure route is being able to migrate the inference between devices more easily. I can develop out the infrastructure then migrate the card to be hooked up to a shared node in the house with all the tools necessary for my family to take advantage of the privacy enhancement that comes with local inference.
How are you actually using the local model? I've played with Qwen 3.8 27b on ollama and the coding harnesses (Claude Code and OpenCode) seem to fail way more often then using the cloud models. And by fail, I mean the edits don't apply cleanly, it goes to add python code, but doesn't indent it properly, or the edit doesn't apply and so it tries again and again and eventually wipes out a different function then it "intended". It just gets really frustrating compared to the relative stability of Claude Cloud.
My use-case is only coding, every model sucks at writing good literature and there is no way around that (have had people try to debate me on this, but it's a taste thing, I have extensive English writing skills from my school years).
Prior to two weeks ago, I was just using Pi and Ollama.
I have tried my hand at putting together a few harnesses and I finally landed on what I like. Been working on this small app to handle running llama-server for me from any device that has the llama-cpp stack setup: https://github.com/SamInTheShell/loom
try using pi harness, hae not encountered these sort of problem myself
also yuou can ask codex to look at the transcript and figure out the solutions to tool call failures that way
Just wanted to share, I had a SaaS AI drop me a script to bench ninfer against llama-cpp and it is impressive. The place that it's doing better at than llama-cpp seems to really be late in the context window.
Thanks for sharing, I'm definitely trying it out after I get through my project milestones for the 5090. The README claims 700tok/s for Qwen 3.8 27b, that would be amazing, I'm only expecting an increase from my ~50tok/s on my Radeon to 200tok/s on the 5090.
I have multiple 48GB friends that successfully run smaller quantized models for general assistance + light editing (coding, spreadsheets, etc), that don't require very heavy models.
So yes, they are genuinely very useful, but they are not yet a full replacement unless you have more powerful hardware and or don't need more intelligent ai.
I have a RX 9070 also. I run llama-swap with a fill-in-middle 7B model, local 9B model, and it proxies up to OpenRouter for the bigger stuff. I think that's where the sweet spot is right now.
I've spent $2 in the last 2 weeks on OpenRouter. I've been trying to only use the medium sized models that I would otherwise be able to run on a nice local setup. That nice local setup would cost ~$4k. I don't know what the operating cost would be, but I would be concerned that my home electricity would cost more than at a datacenter. It just doesn't make sense right now except for privacy reasons.
I'm probably going to hoarde open weights models in the ~31B range until memory costs fall in a few years. Then, I'll buy some hardware to run at home just so I feel more sovereign over my stack regardless the cost/token speed.
In my experience so far, separately from privacy concerns there's a specific use case where cheap local shines--when unlimited shots on /goal with a dumber model is better than limited time with a smarter model. This looks less like the assisted-coding scenario that's commonly brought up as a good local scenario, because if you're searching for a fast and accurate solution to a single blocking problem, the bare minimum for a model that can do that is a 24GB dGPU or a 64GB Mac. Instead, this looks more like a Hermes agent on an Raspberry Pi driving OpenCode on an old gaming computer with just enough RAM and VRAM to handle an MoE, churning out something overnight that would quickly exhaust the subscription plans, like a knowledge graph for a large document corpus.
For software development tasks, Qwen 3.8 27B is genuinely excellent, but you need 32+ GB of VRAM to run it well with decent context, and enough memory bandwidth and compute to run it at a decent pace. With an M5 Max Mac Studio, you can do that decently well.
I was looking at $10k Mac Studio with M5 Ultra and 256 GB for local experiments, but then struggled to find what really good modern model I can fit into it. Yes, it can run a good dense 27B at Q8 with plenty of context, but what beyond that? IIUC, some Deepseek flash variants at Q4 are also feasible, but I am not sure if the quality will be good. They also don’t run that fast, like about 30 t/s
So if I stay within 35B, especially MOE, my M5 Pro 64GB MBP can also run them well, and it can do plenty of other stuff too including gaming. While 256 GB with such RAM bandwidth and powerful GPU sounds like fun on paper, it doesn’t seem to be the next level compared to 64 GB
I feel like for localAI t/s is less of an issue. Just make a PRD and run a ralph loop. For big slogging projects like reverse engineering, or converting a codebase to a new language it actually doesn't matter if it takes a day or seven days.
Yeah this is my experience. My 24GB 3090 + 64GB RAM takes a couple hours to crank out some code with largest Gemma 4 and Qwen3.8 models it can run
But in the meantime I get dishes done, vacuum, flip laundry... etc etc
Frontier models also seem in such a rush to emit anything they produce a mess that needs steering all day anyway
While I have not tested it, it feels like my local setup going slower is better at producing code that works the first time as its not trying to look fast for marketing sake
Is Apple still doing the extended return window for items receiving around the holiday? I know sometimes they allow items received near the end of November to be returned in January, so in theory if you time your early order right you can use it, test it, and even have some time to play with it before you return it.
Right now sweet spot is voice transcription. Meeting recording apps are genuinely better locally than in cloud. Can run on an M1 easily. Latency matters. I built https://github.com/scosman/Biscotti and see zero reason to use cloud ever again.
LLMs are harder: not much useful below 12B, and the 700B+ ones are really much better. Models like Qwen 3.8 27b show promise: in a few years pretty good local AI should be in reach for anyone willing to buy a $1000 computer (but who knows what your $20 sub buys you then).
I might be wrong but subscription models don't give you API access. I'm only interested in API access when it comes to personal use, so local models running for free makes sense for me.
That said I have an RTX 5090, not a Mac Mini, so it's not exactly the same level of performance... The latest open models run at 200 tpm at around 30B params.
I was getting semi-useful results from a 128GB M4 Max. That was a few months ago, and the models have improved (quite a bit) since then, but now I'm happy to send my $20/month to get Claude code.
It's still frustrating as hell to come down in the morning, having given it a list of tasks to do overnight, with tests to pass before they're "done" and find that it worked for about 20 minutes after I went to bed, and decided that it would stop at "3am" (it wasn't) and "not do significant work this at this late hour". Like WTF ? You're an LLM. You don't sleep.
Bloody training data full of humans demanding sleep. I tells ya...
> "not do significant work this at this late hour"
Is this Claude code? Or your local? I assume Claude? I'm more than a little staggered by this, like, it makes no sense! It doesn't even serve Anthropic's interests (surely better for them if it burns your token quota so you have to buy more the next morning.) The LLM just... decided? I'd be so mad.
> I went to bed, and decided that it would stop at "3am" (it wasn't) and "not do significant work this at this late hour". Like WTF ? You're an LLM. You don't sleep.
I think that's Anthropic trying to get you to not extract as much value out of that subsidized subscription as possible.
There are many enterprise environments where running modern models is... difficult. Rather than fighting security for months, a user could petition for a mac studio and have rough cost parity with a z.ai subscription.
if they are a heavy user, perhaps they string 4x together.
I’d be curious to use them on larger data sets. Log files, for instance. High volume, might be low value per line but not much cost per token if you already have a gpu to interpret them using a LLM.
Could you elaborate on your experience with local models on your card? I've been thinking of upgrading to 9070 XT, and was thinking the 16GB would be okay-ish to at least run something usable locally, no?
Usable certainly. But my impression is that useful models still need a bit more than 16GB. Something like Qwen 3.8 27B is useful but squeezing it into 16GB requires fairly aggressive quantisation which will make it unreliable (e.g it'll get stuck in loops) and won't leave enough space for a long context (which qwen 3.8 really likes)
I’m the parent of this thread, the person with the with the RX 9070.
My understanding would be that if you’re interested in this sort of card for AI that you should go with the AI PRO R9700, which is basically the professional version of the RX 9070XT but with 32GB of memory.
It’s significantly more money but not crazy like a 5090.
I just happen to have the 9070XT primarily for gaming purposes.
I’m not quite sure how to describe my experience using it other than “rudimentary,” and a lot of that is on me for not really understanding the best way to set it up.
If you have been using cloud hosted models, you will be severely disappointed with what you’d be able to run on 16GB VRAM. You will spend most of your time fighting with the model to fix its mistakes.
Data privacy and "unsafe" models are pretty valid reasons to use local models. If I want to generate violent images/text you cannot do that using cloud models.
I am running GLM 5.3 across 2x DGX Sparks and was doing comparisons and it absolutely can beat Gemini. Yesterday it corrected a poor Fable 5 response even
The same thing happened with Mac Mini's and OpenClaw. Nobody cared about or was using Mac Mini's for OpenClaw, but there were all of these very suspicious posts from accounts that were clearly Apple marketing bots (you could tell by looking at their post history, where they would drop "Mac Mini" into every conversation they could across all different subreddits and unrelated topics). Then it became fairly common.
So Apple's marketing department is seemingly using the same strategy again. Because still, it's impossible to find a reputable source for this claim.