No, it's a reference/joke to the old controversy regarding master/slave naming in computer circles. I don't know how far back that controversy goes, I probably heard about it about a decade ago.
I use some rules and a few generated scripts and have a LLM handle the management of the containers. Security through a ssh gateway, allows them to deploy/remove containers and then the rule is to stay at least two weeks behind on releases, unless the latest release is security related, which it figures out by reading the release notes.
I ran watchtowerr before, but I think this is a smarter way to do it.
I use something similar (openspec change specifications rather than a literal ROADMAP.md file) and I will sometimes spend hours iterating on the details of a change file, before letting my agent actually implement it. Yes, the agent is generally dealing with doing the actual editing, but I will read it and use dictation to direct modifications to the file.
Does that work? I'd think if it could plan things out itself it wouldn't need a roadmap, but maybe it is like how LLMs "reason" by arguing with themselves.
It's a concept car. It might have been designed by a number of interns and any of the numbers including price, performance, range are to a large degree guessed or calculated from theoreticals.
First thing I always look for in new car announcement, the dreaded "concept car". Then you can close the page pretty much.
So strata is a little slower, but keep in mind that ninfer-3090 is very optimized for a Qwen 3.8. Standard Qwen 3.8 runs at 20 t/s, this modified version can do 50 t/s (but it's extremely long in it's thinking, it just goes on and on.
This is on a 3090 that will crash unless power capped, with a Zen 2 CPU, 64GB DDR4 with a PCIe that refuses to go higher than 8x (basically pretty crappy all in all).
Yet with some tweaking and optimizing I still manage to get strata to run at 40 to 60 t/s.
That strata has been optimized on my Oh My Pi conversations. So when I'm using it, it's probably faster and closer to ninfer in speed than during those unoptimized benchmark tests.
They don't but you can still format things in an actually readable way and be nice to people.
People also often us _italics wrong_, just coz it works that way elsewhere. Works totally fine with a "*" instead on here. Just go with the program and you can make beautiful posts even on here. Heck you have AI now so you don't even have to remember anything any more. There's even a `help` link next to the input box and just to write this I double checked it and I was even misremembering and using two too many spaces for monospace! I really do wish `these would actually work` but hey, you really can't have everything!
Unrelated... just use your just locally installed qwen 3.8 and instruct it to figure out why you arent at 16x.
For me, I didnt know I was on 4x and I was able to, giving it enough permissions, with a good harness like PI, to figure things out and go 4x -> 8x -> 16x.
It probably will take some opening the case and switching ssds around, but it's amazing how powerful these models have become.
Not as fast for me. I was able to snatch a spark before the horrible price hike, and I have it at 40/50tok/sec. Before I had the great 3090 setup which is what led me to get the spark. That 3090 is now relegated to creating videos of my kids doing dumb things.
I now backordered two more sparks, before the hike... hoping they will honor the agreed price and dont cancel on me. They should arrive in a month,
Its evidently clear the frontier labs won't keep on giving us cheap inference for much longer. I also watch in disbelief how people say a sub is cheaper, which probably is. But you loose on so many other things (privacy, predictability, censorship...)
Ill check for this in my setup. I thought it came with only one thinking level causing the “meandering” as it refers to its own long CoT.
I’m doubting the quality of the model, even comparing with luna 5.6, it is giving such bracingly wrong answers. It claims other tools have changed the code when pressed why it made a mistake. That code or tool didn’t come close to working on the project. It is scary.
Anecdotally, I've found that 27B at 4bit hallucinates a lot more than QFN at 3_xxs, particularly for less technical reasoning.
I've tried to use both for online comparison shopping. QFN not only seemed less delusional but also made useful observations and problem solved ways around many different website access issues.
In my own experience, qwen3.8-27b 4bit can consistently find bugs in software written by sonnet 5 and opus 5. But it does do that at maybe 1/10th the speed. Still a pretty good deal if you're coding without wanting to spend big.
qwen3.8-27b 4bit has a following specifically for being exceptionally gifted for such a small model.
It’s genuinely the first local model that has actually made me keep it around, it’s pretty good at spotting things existing tooling can’t, like comment rot/drift/summarising accurate git changes, it’s also the first local model that pushes back when it thinks it is right (I don’t dislike that even when it is wrong, it still hallucinates things that don’t actually work the way it confidently states they do).
It does clear the useful enough to be worth it bar though.
Zero interest in remote models but local ones if they offer utility, sure.
I have so many projects on my computer where it doesn't matter if I know how it works or not. Probably well over 100. I think closer to 200, depending on how you count.
They're relatively simple, they do the task they need to and then they wait until they're needed again (or not).
In the past I wrote them, then forgot how they worked, until I needed them again, relearned what I did and adapted it.
Now I just don't have to know how exactly they work, I just get an AI to read the documentation anytime I need to reuse the project and I'll query the AI to fill in the details and to make changes and I ask the AI to run the code and debug it.
Perfect use cases for today's AIs. Doesn't even require SOTA, I can run comfortably on a Sonnet 5 or a Qwen 3.8 and it'll do exactly what it needs to do without making too many mistakes.
>At the core, it's just an Ubuntu machine with a web server that hosts a dashboard and routes HTTP(s) requests to containerized apps.
So no fail over/redundancy? I think at minimum it should be two machines, so one can break. That is one of the important features of cloud applications as far as I'm concerned.
You don't have to deal with the problem of a single computer breaking.
This also adds a lot of moving parts and failure modes. For self-hosting I’d rather have downtime on a simple system that’s easy to fix rather than a complex setup that stays up but needs more maintenance when some component fails.
It's containerized. It's an easy enough problem to solve, if you don't want it complicated.
Here's what I did with my own setup: Stop the containers at night, copy the files to a second system that's configured with the same containers and that can become the primary as needed.
I already monitor my systems for health and backup states, so these would be added to that. But all that does take a few hours to set up.
You want your backups regardless and you want to have some certainty they're working/good. It's a problem that needs to be addressed, even you are going to agree with me on that.
https://www.linuxfoundation.org/blog/blog/master-slave-and-t...
reply