I'm still waiting for the day a mac air style shell exists where I plug my phone in and just get a real desktop experience. The chips in phones these days are plenty powerful. I always thought it would be where we were going, but alas, no major manufacturer seems to want to make it feel like a seamless, premium experience. Samsung got sorta close, but the peripherals involved never felt great or didn't make sense from a cost perspective.
I wonder what it would look like if Apple offered a slimmed down shell from the Neo for ~$300 that required a connected iPhone. Maybe as iOS and MacOS converge we'll get there.
When they were that cheap, I would grab my one on every trip to Microcenter, just because I'd probably find a use for it, and if I didn't, I knew someone else would want it. Fun times.
To be fair, you're comparing a Pi Zero which I can still walk into Micro Center and buy for $10 today to a 16GB Pi 5, where the insane 3× price hike is almost entirely due to the RAM: I can buy an 8GB Pi 5 for $175, less than half the price of the 16GB model, again quoting in stock prices at my nearest Micro Center.
For reference, the cheapest 16GB LPDDR4 (not 4X as used in the Pi 5) DIMM in stock is more than $150, which is about what I paid for the 64GB of DDR4 in my desktop a couple years ago.
Yup. I think I also bought an 8gb Pi 4 for $50+ or similar a while back.
The price of a pico is what the zero used to be, and the 4gb/8gb 5 is $110 to $185 depending on ram when the 4 used to be I think $55 to $75 at the same ram.
I shouldn't be surprised because it's similar to/worse than the transition from DDR3 to DDR4, but I still was caught off guard to some degree.
Sure, but unless you're integrating the devices into a product to sell, an increase from $4 to $10 hurts a lot less than an increase from $80 to $200. I wouldn't change my mind about buying one or a few Pi Zeroes for hobby projects over $6/unit, but I may well consider a different platform over $120/unit.
Fortunately for me, I already have a stack of criminally underutilized Pi 3, 4, and 5 units on a shelf from a past (non-hobby) project. No 16GB, but multiple 8GB units, and 8GB is enough for most anything I can imagine wanting to do on a Pi.
Huh. I wrote a whole thing here about serving a niche in serving local LLMs, but turns out, any niche it could have served is probably better served (per dollar) by either more RAM or modern SSDs.
There's an argument to be made for the persistent memory being a cheaper substitute for more RAM, but imo the gap isn't big enough to justify it when considered as a % of the rest of the system you're building around it and the performance tradeoff.
Important reminder to check your biases. I was sure Optane had missed its mark by dropping out just before LLMs became a primary use case for high $ systems.
There was a version of optane (optane DC?) that plugged into DIMM slots and cost less than RAM, with possible higher capacities. That would be a useful tradeoff had it kept up the advantage.
Intel had multiple false starts on getting their platform ready for these devices to be used in a sensible manner. I have one of these false starts in a closet somewhere, and I did some kernel dev work for it. It was an incredible mess.
(I didn't actually have an Intel DIMM module -- I had a different vendor's. But fundamentally there needed to be a clean way to get data from software into the durable memory domain, and there wasn't. There also needed to be a clean way to talk to the module to get health information and to configure it to the extent it was configurable, and that didn't exist either. I wrote the kernel driver for the latter and submitted it upstream, and it probably could have gotten merged if Intel had assisted with the gnarly bits involving coordination with platform thermal firmware that shared the same SMBUS lines, but they didn't.)
That's the persistent memory modules; they were roughly half the price per GB compared to DDR4 from my quick research, but substantially higher memory latency, which would significantly affect usage as a KV cache, which would probably be their primary use case since SSDs beat it in raw data throughput. Who knows how it would have developed over the years since it was discontinued, but my gut reaction to the numbers said you'd rather just invest in more RAM based on the numbers I saw.
In my experience it seems like it finds one bug in an area then kind of gives up and goes somewhere else, and I wonder if it has to do with how training works; if they're deliberately setting up single bugs per area, so it acts most "efficiently" when it stops work in that area once it's found "the" bug.
There was some (admittedly flimsy) evidence from a microsoft model description card that it's doing multiple passes through its own weights before token generation to improve output; it might be an architecture change that's making it slow, moreso than raw size
Can I ask why? The modern models deal with compaction remarkably well, to the point where I dropped my autocompact from 1M to 350k. I've seen no change in efficacy and a reduced usage rate.
I have never once had it work for me. Usually it feels like the model got a lobotomy. If I'm working with the model in a chat interface, I feel like we only really start to groove around 500k or so, at which point it knows my quirks and style, and is interpreting my intent correctly. That never, in my experience, survives compaction.
Maybe things are better now. It's been a while since I've compacted a conversation.
For certain workloads compaction is great, for others you're better off manually doing a /handoff (or a similar skill you write yourself to keep what you need, toss what you don't; Claude is not always good at figuring it out).
Read your memory.md to see what Claude thinks is very important.
That was my experience in the Opus 4.5 - 4.7 era, but recently it's been on the nose with what I want it to carry over, or close enough that a full handoff would be more expensive than just letting it find what it needs again.
I'm mostly working on moderately sized rust codebases (600k~ lines) with reasonably long running tasks. The only time I intentionally do a handoff is if the next goal is in an entirely different section of the code, or I let the session cache expire, or I'm working with 6.1 Sol for more than a few hours, which admittedly has some horrendous context drift compared to Opus.
I've been working on a game server in rust, and every once in a while a review will just stop and refuse to explain what happened. So I know there's a security issue in there somewhere, but neither OAI nor Ant models will tell me what it is. Despite the whole thing, top to bottom, having been built in the last two months on the system they're working on.
Planning to hop on openrouter this weekend to chase it down, but man, it's rough knowing I've already burned the tokens to find it probably half a dozen times at this point, but don't have any information that would help point me at it.
That seems like the least charitable reading of what I wrote.
"system they're working on" refers to the actual physical hardware.
Yes, I look at the code regularly, and a significant portion is hand written, especially the performance critical sections.
A security issue I haven't been able to track came up in automated code reviews. Maybe you're such a savant that you can crack any security issue in a codebase you're working on, but the last few months of critical security bugs in infrastructure with the most experienced and capable maintainers in the world should at least humble you a little.
A lot of the work is carried over model to model, so I think the most comparable product is a live service game. Always have to be building on the last thing, and if you stop you'll essentially lose the entirety of the initial investment because you won't keep anyone's attention. Plus a healthy dose of "winner-takes-most".
Can anyone give me the compelling reason for Terra and Sonnet to exist?
Virtually every task I give to either is better served by going up a tier and down a reasoning level, or down a tier and up a reasoning level. I simply don't understand their use case. No testing I've done has shown them to have a niche where there isn't a cheaper-with-equal-results or more-performant-at-same-cost option.
I think for the majority of the audience here who use these models inside of coding agent harnesses w/ subsidized monthly subscriptions, there likely isn't one. Depending on how Terra contributes to the rate limits, it may make sense to use it for some subset of a repeated workflow to get more mileage out of sub.
If you pay API prices for inference in your product stack & have specific use-cases w/ custom evals, these smaller models provide an additional cost/speed lever to adjust. Ultimately, I think the point is to compete on cost for use-cases where Sol on the lowest reasoning is overkill.
If you have a fairly straightforward usecase where Terra performs just as well as Sol, but is faster and cheaper, why not use it?
The real issue with coding harnesses is that you get a cache miss any time you switch models. I might have some easy task (e.g. "fix lints") that I could delegate to Terra, because my session has been using Sol up to this point, I'd pay a full uncached prefill if I routed it to Terra. It ends up being cheaper in practice to just stick with Sol.
Spinning up a fresh session would be a reasonable choice in the above case, but sometimes you still need the retained context.
Yes but you are missing the point. It's the case where Luna is not good enough, and Sol is too much. The problem is that on benchmarks Luna xhigh is typically comparable to Sol low in terms of capability. So where does Terra fit in?
World knowledge, factual recall without grounding, more complex instruction following, etc.
Benchmarks don’t really tell you much. For API best will always be a model+effort sweep on your internal evals (and if you don’t have internal evals, do that first).
For individual coding harnesses, honestly I just go with vibes. The best workflow is the one that works for you.
Terra 6.1 on low reasoning outperforms Luna 6 on max reasoning, while costing half as much per task and completing each task over 10x faster (per AA v5).
Luna 6.1 will likely close the intelligence & cost gaps, but that speed advantage will remain.
On the API we use a lot of Luna (cost efficient) and sometimes Terra level models. I think we never use a larger model. These are models that our app runtimes depend on.
On subscription for coding and stuff I am basically on Astra Ultra all day long.
You should look at model reasoning level curves on the cost/performance chart on artificial analysis. If you compare the curves for haiku/sonnet/opus I would agree with you that sonnet serves no purpose because it is priced too high. If you look at luna/terra/sol you will see that terra is priced such that it successfully fills a gap between luna and sol. And when you add speed into the equation it adds another dimension where terra makes sense. terra low falls between luna xhigh and max on both cost and intelligence, but it is way faster because it uses way fewer reasoning tokens. So I would say there are workloads where terra makes sense. But from my analysis I don't see any workloads where sonnet makes sense.
There is a chasm of performance difference between haiku and sonnet ime. Sonnet is cheaper and faster than Opus, so if you're providing a service via api where cost and speed is important, it's a big difference.
1. In Anthropic's benchmarks, Opus's advantage over Sonnet diminishes at high reasoning levels. Sonnet max actually beats Opus max.
2. There are rote or simple tasks where higher reasoning isn't needed and one model vs the other wouldn't even be distinguishable.
3. How many tokens are used during reasoning depends as much on the task as the reasoning level.
4. Sonnet is faster and half the price per token, so Opus will only be cheaper if it can spend fewer tokens. This won't be the case for all tasks. Opus and Sonnet will also consume the same number of tokens to load context, so the relative price comparison depends a lot on where the task is context-heavy or reasoning-heavy.
Sonnet is way faster than Opus when doing everyday tasks, I think on the subscription plans it's a better option when you need something done faster than Opus' long thinking sessions.
I have some business automation process for some customers that I maintain, where Terra with low reasoning ended up with the best price/performance & latency. With an equivalent Luna with high reasoning, price was usually better, but quality had a higher variance (very much depends on where the thinking takes the model) and latency for high reasoning was worse.
Cheap and affordable. Dispatching proper tasks to these models can save you a huge % of your quota. Especially if you _know_ the task is simple (ie add something here or there, or archive a bunch of files and reformat kinda thing)
Maybe it's my tasks, but in the cases you're describing, I'm usually better off going down to Luna. I specifically don't see the gap where "It's too complex for Luna-high" and "It's not complex enough for Sol-low" exists.
I've personally found that most models completely collapse at sub-medium effort. I generally run most of my stuff at medium/xhigh with heavy core tasks @ max
If you use API in agent or your own integration with software, such models make a lot of sense. We use Terra 5.6 for "question to SQL" translation for example. It's good enough and affordable enough. Luna will probably not handle complex queries well while Sol is too expensive.
i use sonnet to generate answers to help-desk tickets, which it does for significantly cheaper and insignificantly worse than opus.
if you're just talking about use in codex or claude code, then sure, leave it on opus all the time. but for running routine api requests, sonnet is great.
I've been refused to do cybersecurity (read-only investigative access against the recent brotli cve) with Opus 5.5 when I had full SSH access, so I guess you've got to have a fallback?
I cannot figure it out either.
I use Astra Ultra to plan and 6.1Sol High or X-High to build and even on huge projects with 10s of 1000s of pages it is fast and competent
Right?! I think sol has some obsessive tendencies, i literally caught it the other day spawning a sub sub agent doing a final fourth test for a piece of code that isn’t even critical say in embedded or similar, now I have to instruct it to only test once!
I wonder what it would look like if Apple offered a slimmed down shell from the Neo for ~$300 that required a connected iPhone. Maybe as iOS and MacOS converge we'll get there.
reply