It's slower but more power efficient. I think only some form of onyx models are available and I couldnt get anything to run on the first gen NPU's in the 7940HS cpu.
My kids love something like this we see every year at the traveling fair. It's the same idea but with rubber duckies floating around and having to catch them with a small pole with a hook attached. It's €20 for 2 kids to have 2 minutes of fun and the payoff is a €1 Aliexpress/Temu pollution generator disguised as a toy. They play with it for a day before it ends up in the trash.
The first few years I hated this for exactly the reasons you mention, but now I'm just happy they are happy. I think I have a few more years left before they don't want to go rubber ducky fishing with their dad no more and I doubt by then €20 gets me 2 minutes of joy with them.
In fact it's the same thing. I said fish by mistake (probably because my mind made an association with the fishing rod and what it's usually used for) but they're ducks. I have edited the comment now.
Local Qwen3.8 27B pointed to my local Silverbullet wiki coupled with a harness makes it so smart in my environment. It knows my network and homelab better than me. It can find things in an instant, troubleshoot problems on its own. Magic.
How do you deal with security issues? Up until now, I've only run agents on a separate machine on its own physical subnet. (Admittedly this is paranoid and I could probably run it on a VM.)
I would love to give a local-LLM agent full access to my (very modest) homelab, but can't trust it not to delete my files or do something crazy. Maybe giving it its own non-root username, and therefore read access, allowing diagnosis but not fixing. But even a non-root user can do some damage, or exfiltrate system info just by using curl.
I worry about this myself but find some solace and peace in my backup strategy. I treat the harness as a user and set permissive authorizations where needed. I even let it suggest why and which permissions it would need for specific items and note these down in the wiki as well. The idea is that the LLMs takes into consideration why it has the permissions it has and doesn't try and work around them.
