It truly feels like Europe is just collectively shrugging at their lack of preparedness and aggression they face. What’s it going to take for them to take their own security seriously?
Isn't Europe just following US Army's example then? A similar incident in US Army commissaries has been reported since August [1]. It only took them two full months to launch an investigation [2].
This wave of attacks against refrigerators started in USA with many refrigerators serving the personnel at several military bases, a couple of months ago.
Therefore even the US Army has not been better prepared than supermarket chains.
It would take people listening to professionals and getting rid of several decades worth of bad habits amongst users. Same as most other places really.
The problem is the ironic "home grown" or "internal" security. These external attacks come after decades of fertilizing far right white politicians whose sole purpose is to dither any directed anger.
Brexit was the first to burst and despite their support of Ukraine, can at any moment capitulate to the ghost of nationalism and delusional escapism that is "white power". Americas on the verge of sinking further into the same. With those two entirely preoccupied, of course europe is next and already has the same underlying current.
> It truly feels like Europe is just collectively shrugging at their lack of preparedness and aggression they face. What’s it going to take for them to take their own security seriously?
I'm quite sure your average American company is just as willing to plug random shit into the internet.
Hell, there are probably hundreds of reports out there of American companies failing to secure their AWS S3 buckets, so you don't even have to plug anything into the internet.
I'm all for more open models, but talk is cheap and this is a rather pointless announcement without anything backing it up. Publish your weights and HF repo or shut up IMO.
Is that all that a company about to give away the product of 10,000 GPUs running for a month gets to be now? give it away without a single promotional post or shut up? I support open source as much as the person but this is pretty caustic.
I am also starting to take issue with "we've developed a new cutting edge model, and nobody can use it" announcements. Most recently with Google's Argon, at least they were using it internally and they'd be slowly rolling it out. This one is from a company I've never heard of, they're not releasing weights or offering API access at this point, it feels like a fairly worthless announcement.
It says in the very beginning that everything will be opened up by the end of the month. Maybe slightly annoying to you but not worthless. I for one am very excited to see some new American open models since we have called so far behind the Chinese. Looking forward to trying it out.
Yes -- because they're late to the party (high performing open weights models have been a thing for a couple of years now), and therefore will be compared with all the other open-weights model providers they are competing against.
It's not just that they are doing users a favor with weights; they are just as much seeking favors with attention and usage (in a crowded market!).
> give it away without a single promotional post or shut up?
I think the point is that people are happy to see promotional posts when they actually release it, but only then and not before.
Unfortunately pinky promises from corporations to release something at some indeterminate time in the future aren't worth the bytes they're stored in, especially in the AI industry which is full of grifters and charlatans.
The open weight community really is an odd one. Millions of Dollars for pre and post training given away for free and most often with very permissive licenses that allow commercial use (be it US, EU or mostly Chinese)
Yet going by the comments on localLlaMA or HN, those companies are the devil :-D. Colour me surprised.
You know, the other night I trained a model on my secret stash of GPUs, that now outperforms Opus 5.5 on pelican benchmark and Jev on classification speed, while running on a potato.
I don’t understand what is wrong with some people lol. It says in the very beginning everything will be opened up by the end of the month. It’s like a bunch of toddlers throwing their milk cup on the ground and pouting because they can’t have their new toy right now!
In the case of Facebook, no good thing they do will ever undo the evil things they've done, let alone the evil things they're still doing right now. How convenient for the devil that a single act of "charity" should make him immune from all criticism. By all means, praise whatever good Facebook does in the world, but don't kid yourself about what Facebook is.
Those cards should be able to do a lot more. They support int8 and they have very good processing speed. For reference I have a single V100 running around 900t/s prefill and 100 t/s decode during DeepSWE runs on 27B (4 bit quant). Those cards have half the bandwidth so a realistic decode is going to stop around 50 t/s, but those cards can support NVFP4 more easily than the V100. Combined with a TP build you should be able to get 100 t/s. And you should be able to 2x the prefill of a V100. I have had Claude optimizing llama.cpp for about a week and am at around 300 t/s decode so far on two ancient V100s @ 64 GB RAM.
Also for the 3 people that ever read this and are curious about local models still, Qwen 27B 3.8 matched Sonnet 5 in the 17 DeepSWE tasks I have run so far, solving the exact same 7 it has. Caveat: datacurve combined low/medium/high Sonnet 5 data.
I understand this repository must be under siege effectively with how popular it is, but the team needs to come up with a decent way to handle this. The current climate with Nvidia buying Hugging Face who effectively owns the llama.cpp team there’s a lot of discourse around their control of the team that’s negative.
IMO this comes down to your harness. Any frontier model from a huge shop has an inherent benefit in the system you're using it in. Search, memory, skills, integrations you don't realize even exist make them much more powerful. It is some effort but I recommend trying Hermes Agent and setting it up fully, that's the closest you'll get to a more complete experience.
Some of us are stuck on subscriptions and our executives will never give us API access.
But also, everyone says "it's the harness" and almost nobody ever gives good examples, it gets a bit tiring to read everywhere, as if everyone wants to sell a harness to us.
I've been using Qwen3.6 35B A3B, and with reasoning turned on, I'd say 2/3 (give or take) of the tokens for a response are thinking tokens. Which at 70+ tps locally, that isn't that awful. I run an 80k context across 4-10 "agents" for my solo TTRPG, where Qwen is the GM, each NPC at a location, the director, and the narrator.
Each turn is about 45-60 seconds to generate all of the various responses. The GM and director have reasoning on, and the NPCs/Location/Narrator do not.
It's a fairly good "engine" for that. I'm not sure how a denser Qwen would do here regarding speed.
I like the tabletop RPG use case, and wanted to say: If your hardware likes it you should check out Gemma 4 for creative DMing use case. I found it to be much better at holding the plotlines and being creative on gaming turns. My experimental case was an audio-only Zork and Gemma 12B and even E4B were pretty good!
This sound very interesting, do you have any resource I could look at?
Me and my son did a very rudimentary (compared to yours) setup to play Paranoia, but this is at another level.
I'll go over my repo, and see if it is hiding any API keys and maybe make it public. The issue I have is it relies on a nuget package that also isn't live (its in my local nuget feed).
I'm not sure what all is needed to make that work for people.
I'm working on something similar. My biggest annoyance is that the overly-helpful LLM was making every die roll succeed. I ended up building some tooling around rolling dice. Also some tooling around character stats and inventory management, so those don't get lost in context compression.
It is on a single 3090, and that seems to be where it averages out. I'll get 85tps on turn 0, but then it settles down to low 70s within a few turns, but holds steady at that.
My issue currently is KV Cache, because I can't keep enough parallel caches running (4 is where I'm at), so TTFT (is that the initialism?) can be long when I have a particularly large scene (basically more than 2 NPCs).
But my harness does let me offload to any OpenAI compatible endpoint, I just prefer local cuz free.
Back in 2023 I started my own C# LLM library for doing tool calls and structured output, and over the years it has morphed bigger and bigger, and that is the backbone of almost all of my LLM-based projects.
I've never released it, but its easy to understand, and simple to add your own tools:
[AIDescription("Get current weather for a location")]
static string GetWeather(
[AIDescription("The city name")] string city,
[AIDescription("The country name")] string country,
[AIDescription("Temperature unit", ["C", "F"])] string unit = "C")
{
// make some API call to a weather API and return a string to the LLM
return $"The weather in {city}, {country} is 22°{unit} and sunny";
}
var chat = client.StartConversation("You are a helpful assistant with access to weather data.");
var response = await chat.SendAsync("What's the weather in London?", GetWeather);
I'm sure plenty of better libraries exist for this now, but in 2023, I don't think any existed in the dotnet ecosystem. I've never released it though, because I've never "finished" it.
Not parent, but I use Goose for my non-handcrafted Qwen use cases, I’m also working on handcrafting as well. Goose was the only harness that didnt bloat context too much with system prompts (like openclaw) and I could get reasonable web search working with Qwen.
Just to play devil’s advocate: you can’t compare Qwen to a (proprietary/closed source) hosted model and deduce that Qwen is overthinking, as Qwen gives you the full reasoning/thinking trace while all the proprietary models now give you only a summary “to prevent distillation”, making it hard to properly compare apples to apples here.
People say Qwen overthinks because they analyzed the thinking traces, and Qwen finds the answer relatively quickly but then second guesses itself multiple times for another 20,000+ tokens. Regardless of what other models do, that's clearly overthinking.
This is most likely because the vast majority of the information the model absorbed during training was in Chinese. As a native Mandarin speaker, I frequently need to convert the prompt into English and output it in English in order to avoid that the model falls back into Chinese reasoning logic.
PS: Switching the thinking process from Chinese to English can also significantly circumvent certain self-censorship mechanisms built into the model.
You can use any message you want, but the model was tested to react reasonably well to the specific token sequence of "\nConsidering the limited time by the user, I have to give the solution based on the thinking directly now.\n.\n\n" (from a Alibaba paper, struggling to find it now)
Since i have tools to prune context and run subagents, i just tell it to do either since both require summarization which is usually what it needs to avoid the long if...then chains
Can you actually have an agent call the Antigravity CLI by the TOS? Feels like the type of thing that would get your account flagged by doing it programmatically.
It wasn't long ago when calling Claude -p would trigger usage billing. I don't know if they backtracked that but even apps like T3 Code were warned that using Claude like that was not going to use the Claude sub.
I think that’s the really important implied question right now. Mostly everyone already has some form of device to play games on. What happens when they start to die off?
It’s frankly embarrassing at this point. I’ve got free access through buying a Pixel phone and it’s not even worth using as it’s a waste of my time. Here’s my experience so far using it for basic sysadmin Linux type stuff.
Gemini 3.1 Pro just feels a generation behind, from when models would miss easy things and make bad assumptions. Its not actively detrimental in bad way but the opportunity cost vs using something like Opus to be productive is large.
Gemini 3.5 Flash is the most annoying model I have ever used. It loves to respond in ALL CAPS like “LOOK AT THAT” for no apparent reason. I realize it’s a flash model but I will give it a basic list of tasks and the output will simply vomit “Now I will X” “Now I will Y” “Now I will Z” over and over again filling my screen with garbage. It’s also no smarter than 3.1 Pro and consumes just as many tokens as 3.1 Pro, it’s really pointless without a newer Pro model in place.
3.1 Pro: while it's coding performance is mediocre, a lot of coding work requires minimum actual thinking. I use it often for light refactoring, boilerplate generation, testcase skeleton generation, code review, language questions ("is there a better way to write this code block?"). I don't have a corporation behind me so costs matter. Considering that I'm a Pro subscriber, it's quite cost effective.
3.5 Flash: excellent model for general (re)search. I use it for everyday tasks with Thinking instead of Flash-Lite. It's a much better version of Google Search for general queries like gaming tips, cooking, day-to-day first aid, tax and investment questions, etc.
Google is clearly aiming for cost-benefit here and considering that it gets bundled with YouTube Plus and Google One at $20/month, it's a killer deal IMHO.
PS: I don't work for Google and don't even like Google very much. But this is a good product.
It was a decent deal ~6-8 months ago. I had been using 3.1 pro almost since release, but it really is feeling old. After using other models more in the past two months though...I really can't go back to 3.1 pro, as I just have to explain my reasoning so damn much to get it on the right path, where as opus or fable just "get it" from the context of the project much better.
Sonnet is roughly the same level as 3.1 pro for me.
reply