Dude you have a lot to learn from me I am a good guy I love to talk about things I love you so very so very so very so good luck and good night and I’ll talk soon and I’ll see if you have any more questions please call my phone number and I’ll talk about you
I have a limited understanding of local LLMs, but from what I gather 128GB is not enough to match DeepSeek v4 Flash or GPT 6.1 Luna. And these models are so cheap right now. What's the use case besides having your data locally?
"Having your data locally" is definitely the biggest one.
- The general consensus seems to be that the frontier models are kind of (understandably) "benchmaxxed" towards coding benchmarks and have suffered in some other areas like writing, art, etc. Lot of tweaked/specialized models on huggingface
- Getting around the censorship in commercial models
- While local models don't match frontier coding agents, you can run them for multiple 24/7 with a long running goal (The fans in Macs are presumably not built to run 24/7, but this can be mitigated to a considerable extent IME by simply pointing a desktop fan at it, and you can throttle things in software. On the discrete GPU side of things, you can usually get big power savings without much performance impact by throttling the GPU clock; things are usually bottlenecked on memory)
- Even small 27B/35B models are getting darn good, definitely more than powerful enough for classification etc... you don't need 128GB to run them
- Right now, providers like OpenAI are running at huge losses. At some point, they are going to need to turn a profit, at which point they may get significantly more expensive, and "hosting your own" may make more economic sense than it does today -- and be a valuable professional skill, even.
- Even more speculative: perhaps the future for many applications is training and running your own hyper-specialized models. Right now, training your own is nowhere near cost effective on self-hosted hardware vs. renting GPUs
Note: whether any of these use cases are "worth it" is up to the individual. I'm not trying to sell anybody on the idea. I don't personally do much local LLM stuff at the moment, but it's attractive and I've been close to pulling the trigger...
I'm just trying to understand what the particular uses people have for them.
My raw data alone is terabytes, and it takes hundreds of GB sometimes to run very simple analyses. So there's no way I can have a local AI orchestrating the analyses locally.
But I realize some people have different kinds of data. So I'm interested to hear how exactly people use local models.
The point where it'd make sense for me is if someone makes a compact machine with 512 of unified memory in a couple of years and it costs less than a down payment for a house...
It's more "software & gear discussion" than "use cases", admittedly, I think because the people looking for software/gear discussion already have use cases in mind that they're trying to achieve.
While personally I’m not using them for work, just tinkering, 128GB can run qwen 3.8 flash next at q4(~111GB gguf). That model seems to be quite capable from what I’ve read.
It’s quite evident that the api models can and have been nerfed at the provider’s discretion, then there’s the abliteration aspect too(api models refusing what they consider “harmful”) which makes the case for local.
which configs did you wind up going with on the Studio + Air, I'm in a similar boat. Grabbed a 13" 24GB M5 air before the price hike and now wrestling between a M5 Max studio or an overpriced mini.
It works quite well on iOS, I am satisfied.
- ublock lite blocks most of the ads, the rest get filtered through an Adguard Profile(no app install needed).
- Not sure if possible with other browsers but you can install userscript extensions too.
- I have a shortcut that extracts the text of any tab(you can run a simple javascript, document.body.innerText to get) to send to my local llm, this javascript execution is limited to just safari.
On a mac though, I only use it to stream 4k,HDR on Netflix.
Does anyone else have problem scrolling diagonally on a page that's a bit zoomed in(like on HN, you zoom in a little to read the threads)? It just doesn't move smoothly and instead does a staircase shape traversal.
This used to occur on all browsers before upgrading to Tahoe, now it's mostly fixed but still persists on Safari.
macOS seems to have a “lock direction” for trackpad scrolling on zoomed in windows. Whether you like this or not is in my experience largely a function of how much you do normally do a diagonal scroll. On the one hand it means if you scrolled in to fill a column, you don’t need to be perfectly precise with staying up and down to scroll up and not also scroll the column off to one side. On the other hand if you don’t start with a very explicitly diagonal scroll, it won’t let you scroll diagonally without stopping and starting again.
This was jarring when moving from windows where you can literally scroll a page in a circular motion smoothly. Reading text with the locked scroll made me wonder how have people not noticed this or am I in the minority of reading like this lol. I mostly use chrome so this isn't a huge annoyance anymore and hope it behaves the same on macos 27 and beyond too.
> Most people didn't have the same kind of experience, they got the LimeWire version which was the equivalent of wading through a dollar store that’s just been ransacked and shit’s all over the floor.
Heh, I often went a step down and recorded internet radio using RadioSure. This little utility split each track into its own file which was pretty neat and handy to a younger me. Shoutout to Ryan Seacrests' AT40 for the weekly charts on the weekends, it kept my "collection" fresh ;-) Although, it was mostly 128-256kbps mp3 but it didn't matter to me, it was fun.
reply