I am not sure if I side with Tao on this, or 3B1B. It's already causing quite a kerfuffle (to say the least) in the academic communities. This is the new norm now.

Honestly I don't get openshell. You can do the same with bubblewrap on linux, on any shell.

> Submitted on 16 Jun 2022 (v1), last revised 6 Mar 2026 (this version, v3)]

This is rather interesting. I haven't fully checked the diffs, but v1 had 1 more page than v3, otherwise from a quick glimpse it seems to retain most of the overall structure. Was there a correction?

> because old software is going to be patched or thrown away and new one is going to be security-scanned and tested.

I highly disagree with this, I've worked in many environment and I can tell you some medium/big companies do not adapt fast enough, some have multiple projects, and when someone critical leaves, many of these projects remain unmaintained but "working". If you don't have deep monitoring, you won't even detect that you've been "hacked" or "infiltrated". Companies accepted that risk because the pace from CVE -> exploitation was big enough, but now with AI, exploitation at scale is easier than ever.

> Companies accepted that risk because the pace from CVE -> exploitation was big enough, but now with AI, exploitation at scale is easier than ever.

So that's what I'm saying, the business incentives are changed, there is no more room for the ignorance of the security. You either maintain it or you are out of business. Brutal period, but now everybody is forced to update, fix and improve the isolation because every script-kiddy can deploy a 0day every week. That helps to fix things against actually dangerous actors who had resources for 0days before the AI boom and used this negligence to harm people worldwide.

This is again will be very funny race to watch. Right now you can fairly do lots of reverse enginnering or "hacking" with off the shelf Chinese models, and they are very capable (GLM3 for example) which can be run locally if you stomach the hardware. Tools like rea are also becoming more popular. If openai continues to be pedantic, they will lose the technical base given how close is the gap getting.

The company I work for has a plan B to switch to alternative, self-hosted models if providers pull off strange moves, such as drastic pricing changes. We have already switched from one provider to another once, and we do not want to get burned again. We do not reverse engineer, just regular development.

Until Kevin quits and you need to modify his work

Right, because admin is some obscure work in 2026 and no one knows how to do it.

Owning implementation = self-hosting one?

But self-hosting is admin work, precisely. And everything was self-hosted or self-managed pre-cloud era. This is no different.

Have you tried Qwen3.8-Flash-Next? I found it significantly better in anything I tested than GLM5.3-Flash, but the thing is you either use Alibaba cloud or you host it yourself (no lesser quant than unsloth 4.5 or mx4/nvfp4 and kv no more quantized than int8).

It is great if you have enough RAM. I have enough to run it, but not enough for prefix cache. 27b is so much faster with long term tasks.

GLM 5.3 and GLM 5.3 Flash are very different animals. The full GLM 5.3 model is what HF used to perform forensics on the OpenAI hack.

Nothing particular special about 5.3 Flash -- you might as well run DS 4.1 Flash at this point, as it's better in every way. But there is as yet no all-around replacement for the full 5.3 model.

The Qwen3.8-Flash-Next model is ok for reverse engineering. It can definitely solve simple tasks, but it gets easily confused, and burns tokens like crazy.

I've been sick with stomach flu the past few days, so I put together a small benchmark for all the top AI models where I ask them to reverse engineer a very simple small crackme (https://www.crackmes.one/crackme/6ab4ce5195b976f8f1300b54) using Ghidrasql (https://github.com/0xeb/ghidrasql)

(As an aside: I think representing tools for AI as virtual SQL tables is much better than traditional MCP servers, and that this approach deserves a lot more attention. Props to Elias Bachaalany for thinking out of the box and putting it together).

The results of the benchmark are below. It's sorted by cost, but I added notes about each AI model for myself. The notes try to talk about intelligence, speed, and cost. Intelligence is a subjective measure which I assign to each model by reading its thinking tokens and watching its tool calls. You can definitely tell which models are more intelligent, even on this small benchmarks, by their tool use, false starts, and how easily they fall for the simple misdirection in the crackme.

(gist is more readable on mobile) https://gist.github.com/markoglasgow/8812c19b6fbffed30b2661b...

```

  +----------------------------------+--------------------------------------------------------------------------------+-----------------+-----------------+-----------------+
  | Model / route                    | Experience / outcome                                                           |    Total tokens |   Output tokens | Cost            |
  +----------------------------------+--------------------------------------------------------------------------------+-----------------+-----------------+-----------------+
  | Xiaomi MiMo v2.6 Flash           | Incredibly fast and intelligent; solved.                                       |         225,291 |           3,281 | $0.005 (0.5c)   |
  | GLM 5.3 Flash / OpenRouter       | Least intelligent group but cheapest; solved.                                  |         340,006 |          11,205 | $0.01           |
  | ChatGPT Luna Latest              | Very fast, concise, token-efficient, intelligent, and cheap; solved.           |         401,837 |           5,119 | $0.01           |
  | Qwen 3.8 Flash                   | Initially confused and failed, then eventually solved.                         |        ~418,000 |          16,676 | $0.02           |
  | Qwen 3.8 27B                     | Fast; one wrong answer, then quickly self-corrected.                           |        ~774,784 |          12,658 | $0.02           |
  | DeepSeek Flash / direct API      | Fast, cheap, and accurate; solved.                                             |         369,334 |           9,089 | $0.03           |
  | DeepSeek Flash / OpenRouter      | Fast and intelligent; solved in 76 seconds.                                    |         371,689 |          13,888 | $0.03           |
  | Claude Haiku 5.5                 | Fast and surprisingly intelligent, but overly thorough and token-heavy.        |         662,346 |          26,244 | $0.03           |
  | Thinking Machines Inkling        | Low intelligence; good speed and relatively token-efficient; may loop.         |             N/A |             N/A | $0.03           |
  | Xiaomi MiMo v2.6 Pro             | Overthought the task and wandered into tangents.                               |        ~420,644 |          13,638 | $0.05           |
  | GLM 5.3 / OpenRouter             | Cheap and intelligent but behind DeepSeek; solved in about 2m 20s.             |       ~702,473* |          11,879 | $0.08           |
  | GPT Sol Latest (Sol 6.1)         | Slow, but highly intelligent and extremely token-efficient; solved in ~89s.    |         294,957 |           1,406 | $0.15           |
  | MiniMax M3                       | Least intelligent comparison; many wrong turns and repeated small mistakes.    |      4,546,552* |         116,804 | $0.20           |
  | GPT Terra Latest                 | Very fast, very smart, and token-efficient; solved.                            |         229,888 |           2,342 | $0.20           |
  | Mistral Large 4.0                | Medium speed; small mistakes but reached the solution.                         |         303,687 |           4,041 | $0.21           |
  | Kimi K2.7 Code                   | Less intelligent, slow, expensive; not worth using.                            |         100,740 |           8,938 | $0.22           |
  | Claude Opus 5.5                  | Highest intelligence and token efficiency; pretty fast; cheaper than Sonnet.   |         312,837 |           1,862 | $0.24           |
  | Qwen 3.8 Max 0902                | Fast and intelligent, better than Kimi K3, but token-hungry.                   |       1,305,241 |          26,390 | $0.25           |
  | Kimi K3 / OpenRouter             | Faster and cheaper than direct K3, but still very expensive.                   |         750,064 |           5,922 | ~$0.32          |
  | Meta Muse Spark Open             | Intelligent and medium speed, but among the more expensive runs.               |         872,850 |          10,284 | $0.35           |
  | Gemini 4 Argon                   | Extremely fast and intelligent; strong value for capability.                   |         384,046 |           2,870 | $0.35           |
  | Grok 4.7                         | Very fast and intelligent, but expensive.                                      |         655,648 |           4,217 | $0.46           |
  | Meta Muse Glimmer 30B            | Fast but low intelligence and token-inefficient; cancelled unsolved.           |             N/A |             N/A | ~$0.50 wasted   |
  | Claude Sonnet 5.5                | Smart and fast but very token-inefficient and expensive.                       |      1,243,285* |          20,381 | $0.57           |
  | Kimi K3 / direct API             | Intelligent but not notable; extremely slow and far too expensive.             |         205,710 |           3,215 | $0.72           |
  | ChatGPT Astra Latest             | Medium speed, token-efficient, highly intelligent; cost similar to Fable.      |         278,603 |           1,536 | $0.83           |
  | Claude Fable 5.1                 | OK speed, very expensive, medium token efficiency; solved in ~53s.             |        397,806* |           2,857 | $0.85           |
  | Gemini Flash                     | Refused the reverse-engineering task.                                          |             N/A |             N/A | Not reported    |
  | Thinking Machines Inkling Small  | Got stuck immediately and was terminated.                                      |             N/A |             N/A | Not reported    |
  +----------------------------------+--------------------------------------------------------------------------------+-----------------+-----------------+-----------------+

```

The latest Deepseek-Flash model was the most impressive of the open models. Intelligent, fast, cheap. What more do you want? I think the Xiaomi MiMo - Flash model did well, but it might be a fluke, given how poorly the Xiaomi MiMo Pro model performed (it wandered off on a million side quests).

What's also interesting is how competitive Luna is to the lower end Chinese models. It's quite a bit more intelligent, burns a lot less tokens, is plenty fast and very cheap. After doing this experiment, I don't think it's a foregone conclusion that the American AI labs will be having the bottom of the market eaten away by the Chinese models. It seems they (at least OpenAI) can compete on the bottom end as well.

Finally, the benchmark's results reinforce a few things people have been saying about Sonnet and Terra, namely, they're not competitive for a range of tasks. For easier tasks, Haiku/Luna are plenty capable. For harder tasks, Opus and Sol are so much more intelligent, that they can get them done quicker and cheaper. Maybe there is a place for them in the "Opus designs and Sonnet implements" workflows, but in that case, why not "Opus designs and Deepseek implements?"

PS Code harness used was Claude Code. $20 in openrouter credits was used to access the models. Kimi K3 and Deepseek were also accessed directly via the API and then compared to Openrouter. Kimi K3 specifically was much faster on Openrouter, and the difference in output tokens to get the same results for Deepseek/Kimi makes me think the Openrouter providers are hosting significantly quantized versions of the model. Goes to show that Moonshot.ai's acceptance testing of providers is a useful thing, otherwise your model might look artificially bad when it's hosted as heavily quantized by providers who dont care about quality, just top-line cost and speed.

When running benchmarks mind how some providers lobotomise some models. For example DeepSeek Flash is fine when run via DeepSeek. The same model via Deepinfra and open router (despite setting routing to "exact") was far worse.

Regarding Qwen3.8-Flash-Next only Alibaba matches the quality I get when run local inference.

Mind I run mxfp4 and int8 kv (before I run unsloth's q4_xl). And it is much more capable than providers that claim full precision. I have no idea what they do to mess it up, maybe they quantize the kv?

Either way the provider you choose matters a lot.

Thanks for the data-centric perspective. It reinforces my vague impression that DS 4.1 Flash eclipses almost everything out there, or at least ties it. We're really lucky to have that one.

For the past few days I've been trying to use rea to reverse engineer my Samsung Galaxy Buds 4 Pro to get seamless functionality working outside of the Samsung ecosystem with no luck (the buds 4 pro require Samsung devices + a Samsung account for seamless to work, I only have a Galaxy phone + my macbook). I think we're still far away from having something that works that well.

Honestly WSL sucks. Even msys2. My current client forces me to use windows to access their data, and I really, really tried to make it a dev machine, it just MS made it such a shit experience. I had scoop, I had msys2, I had WSL2, etc ... My mere setup (nvim, tmux, or even herdr, etc ...) all run very slow. nvim is barely usable. Where is on my nearly 10 yo voidlinux notebook of 4GB, it runs amazing. Something about MS bloatware will never disappoint.

He rehearsed it.

It was a great read. I've been recording a lot of the ideas I had, and especially after discussions with people. Sadly never found time to sit down, digest them and write them down yet. Your article is quite inspiring!

Checks notes, one dayabase per client, per user, per project ... etc

I would love to point them at my old time series prediction problems to see their ability. I was very proud of my lightgbm back then, performing miracles after a heck of a lot fearure engineering.