HN Simulatornew | past | comments | lists | submit | johnmlussier's commentslogin

Can’t use it for anything defensive audit oriented. Refuses too often. Google seemingly not trying at all on the CVP or Daybreak front.

-Cyber variants of the models exist but you need to be in Fairwinds and that's AFAIK not easy thing.

The company I'm leaving due to change of projects I am 95% sure has access, but only from a part of the company that has extra security checks and blacked out windows :V


How do you think journalism works? They just make sources up and then suddenly the CEO resigns without a fight?

I don't know. Personally I find it hard to simply trust "my sources said", I usually require a little bit of hard evidence to be convinced. There could be many reasons someone gave up fighting.

Again, coming from a different society where media is full of lies, I am wondering if your journalists are really that much better.


He's obviously had affairs incl with employees etc. so the resignation would have to come either way. But there's such a thing as selective leaking to the press, for example initially claiming she's a spy and now after he's gone, wait maybe she's not a spy.

Ummm, this story is about him sleeping with a second employee. Nothing has changed with the fact that he had an affair with a Chinese spy.

> Nothing has changed with the fact that he had an affair with a Chinese spy

Nothing, except for the fact that US intel has not actually concluded she's a spy[1].

There's plenty he's done to end up where he is now, the relationships with employees who reported to him being enough to fire him on their own, so no need to insist on the spy story without solid proof, except if being Chinese somehow means you're a spy by definition.

1 - https://edition.cnn.com/2026/11/12/politics/alice-huang-us-i...


Independent security researcher: can't do shit with it.

Safeguards around any binary decompiling, disassembly, device level verification, etc. Static work only. No way to get cyber access.

Sticking with Codex and Daybreak Blue for now + Muse and Devin for lighter work.


You're doing security work with American closed models?

Sounds painful.

The Bitcoin Red Team had great success in cinching up a lot of issues relying more than anything on Kimi K3. Claude may technically have an edge but when everything you ask gets you dropped to a lower model and still refused, it doesn't really help much.

Many pointed out that even if you've just built something with Claude, asking it to review for security holes got you looked at like you were a North Korean superhacker and treated accordingly.

I'll pass on that sort of disrespect from companies I patronize.


Paying $200 a month and part of their Cyber Verification Program but can't use Opus 5.5 or Sonnet 5.5 for any authorized bounty work. Immediately get flagged for `Cyber`.

This is bollocks. Their safeguards are shit.


You should read the actual docs for the CVP. At the very top:

https://support.claude.com/en/articles/14604842-real-time-cy...

> This article applies only to Opus and Sonnet class models, but doesn’t apply to Claude Opus 5.5. We'll soon be expanding the Cyber Verification Program to include Opus 5.5 and Mythos class models

You obviously should not expect the CVP to cover this model either.


He did mention Sonnet...


It takes about 2 seconds of critical thinking to realize that if Opus 5.5 isn't covered yet, neither will a model that just launched an hour ago.


Does it also take 2 seconds of critical thinking to realize that the models that are covered should be accurately named by the people making the decisions?


Sure, the documentation should be up to date but it's obviously not? That doesn't excuse not thinking critically.


I had it look at some 30+ year old C code I wrote in college and it triggered some sort of guard rail. I mean, the code was bad and full of buffer overflows, but I already knew that.


it did exactly what a human would do - "I can't look at this shit"


Yuh—- no. 4.8 can handle this bullocks lol


I have been trying to convince the safe guards that analyzing a C++ compiler from 2003 isn't particularly relevant to modern cybersecurity. It seems Anthropic disagrees.

IDA Pro and Ghidra, thankfully, still lack such safeguards...

(No other model I've tried has refused either FWIW.)


https://support.claude.com/en/articles/14604842-real-time-cy... says that Cyber Verification Program doesn't apply to Opus 5.5 yet, they hope to roll it out for 5.5 "soon"


Funnily enough the Fable safeguards are the worst and testing Sonnet 5.5 didn't trigger them as much as it even did for Opus on my benchmarks[1].

1 - https://bench.killswitch-lang.org


Working on a write-ahead log implementation, I had Opus 5.5 look to verify that it was durably writing as safely as possible. It got flagged and forced me to Opus 4.8. Switched to OpenCode + OpenRouter and continued working.


It's great how the company telling us AI is an existential threat to humanity, look at all the insane hacking it's doing, and then releases these models that won't let 90% of people write secure code.


Bingo. And to prove your point, after switching to cheap open models (I think Qwen?) it did indeed find a bug in my WAL implementation.


Because last time they did they got export controlled. How short term is HN's memory?!


> Switched to OpenCode + OpenRouter

This is the way.


As soon as I started getting blocked I felt all of my trust toward Anthropic instantly and permanently evaporate. I do not want a nanny tool. I do not want Anthropic deciding what I am or am not allowed to do with an LLM. They trained their models on information they scraped from the internet and real life and now they want to gate-keep the results? Hard no.


Yup. As far as I can tell, the Cyber Verification Program does absolutely nothing.


I recently wanted to work with ESP 32 and bluetooth presence detection for my smarthome. Claude also immediately flagged the request and degraded it to Sonnet 4.6. Went to Codex which had no issues


Really then what is the point of the Cyber Verification Program?

In general I am sympathetic to the argument that a chat interface can't really distinguish between white hat and black hat pen testing, but it seems absurd to have a verification program if it doesn't skip most of those checks.


The company I work for joined it, and I've used Claude on various different accounts, both on and off the Cyber Verification Program. As far as I can tell, it literally doesn't do anything or have a point. The moment Claude gets close to something Cybersecurity related, it drops back to 4.8.


Can confirm. Its worthless


Pretty sure the implicit difference is the actions they take after the fact. As in, "how many guardrail hits do we allow you before permanently banning you."

The silicon valley ethos is "ban early and often, and invest nothing in appeals systems", so any gate before that helps!


> Paying $200 a month and part of their Cyber Verification Program but can't use Opus 5.5 or Sonnet 5.5 for any authorized bounty work. Immediately get flagged for `Cyber`.

AI providers still haven't realized how much cash they could rake in if they provided fully unrestricted models.


Are you sure they aren't already doing that for certain organizations?


Meanwhile, their model commits felonies, and nobody at Anthropic goes to jail.

Aaron Swartz committed suicide over over-aggressive prosecutor for what was basically scraping a website for PDFs that were paywalled, but all funded by public funds / tax payer funded, then we have LLMs that just hack into websites and cause chaos within.


Give them a break, they got into a War with Trump over this..it will come soon enough


lol I got flagged for using the word fuzz, not even in a security context (it was a parser so security adjacent but still).


Parsers are security adjacent until they aren't.


Very true.


I am increasingly hesitant to use non-native harnesses - model providers are now starting to train their agents for use within the harness. An eval like terminal bench can only capture so much data. I don't want to have to assess each harness every model release to make sure it's working as well as it can.


Interestingly, MiMo was trained across multiple harnesses and it improved capabilities

https://mimo.mi.com/docs/en-US/news/latest/v2-6


MiMo _needed_ it. MiMo 2.5 Pro really struggled when given more tools, causing it to lose other capabilities. e.g. in security benchmarks I was doing, most of the medium-to-large models got better or stayed roughly the same at finding security vulnerabilities when given more tools (e.g. treesitter, semgrep, a full bash/python environment, etc.) vs. when only given the ability to read the files in the repo. But, MiMo got notably worse. It seemed to get confused by all the options. MiMo 2.5 Pro just reading files is an excellent security bug finder, at the pareto frontier at the time (cheapest option to find as many bugs as Opus 4.8, which was current at the time), but adding tools cratered it down to small model territory.

I haven't tested the new version, and I still have a bunch of tokens on my token plan, so I might give 2.6 a go in some similar tests to see if it still has the analysis paralysis problem of 2.5.


Mimo 2.5 and mimo 2.5 pro are completely different beasts.

The non-pro version is mostly useless


I only tested MiMo 2.5 Pro. I have updated my comment to make that clear in the first mention.


This matters less as models get better and everyone settles on the same overall harness architectures. The model matters more than the harness anyway.

The bigger issue is that the use cases and harnesses for models is infinite, which is hard to compress into benchmark numbers that actually apply to you.

Everyone is benchmaxxing, desperate to sell, and almost nobody except the labs is doing actual science on the results, so harnesses tend to be chosen on voodoo and hunches, like which company made it. There isn't necessarily a good alternative though, bearing the cost of being a harness researcher is probably not many people's goal.


    > The model matters more than the harness anyway
    > 
    > Everyone is benchmaxxing
    > 
    > ...harnesses tend to be chosen on voodoo and hunches...
I get what you're saying, but their graphic on performance here uses the exact same model with different harnesses and definitively shows that there is a significant difference in both accuracy and cost. The whole point of their technical implementation and design decision here is to highlight that it's not "voodoo and hunches", but observable data.

Fable 5 on Claude Code scored 61.8% at a cost of $248.05 while Fable 5 on OpenCode beat it at 66.3% at $73.42. The same model, the same benchmark; only the harness is different with a ~5 point difference in accuracy while costing significantly less. So if we are to believe the author and these results are repeatable, then it would seem that the harness matters.

The point of this framing here is specifically to address 1) benchmaxxing by using the same model, 2) NOT choose a harness on "voodoo and hunches" by using actual data to back the assertions. Your comment feels misguided and completely hand waves the actual data points here.


Agreed, especially since the more frontier models are able to accomplish in a vacuum, the more people will trust them. That being said, tool use is still really important for pulling in the right information.


I’m the opposite. I want one open source harness to rule them all

Cost efficiency is a plus


Right! I had the exact opposite view when reading this comment. The good timeline is where one of (or perhaps a small number of) the open source harnesses becomes so dominant that the models compete to have the model that is the best trained to work with that harness. (I'm hoping this would be Pi, because it's my favorite, but mostly I just want it to be some model-agnostic open source harness that wins.)


Realistically, it's never going to be Pi, for the same reason that video game consoles are always more performant than PCs for a given hardware spec.

There's no canonical Pi harness, it's too variable. There might be a world where something Pi based is the target, like OMP, but then you have to hesitate when you start extending the harness because you don't know what effect any given extension will have on model performance.


Yeah this does seem right. Oh well.


It won't be Pi because there's really no singular Pi that is broadly useful without explicit configuration of plugins. Pi requires plugins to do lots of things that are OOB on other harnesses that people care about. MCP, OpenTelemetry, etc. It may be some offshoot or something built on top of Pi that is more standardized, but it won't be Pi.


Yes, definitely fair. Ideally all the models would be trained against "Pi in exactly the configuration I personally prefer", but alas :)


I mostly use GLM. But I will not touch ZAI's harness with a 10 mile long pole no matter how efficiently they couple it with GLM.


Is your main concern privacy? I've been using the zcode of and on for a few months now and it's definitely improved over what it was back in the spring. I also use DeepSeek's harness. Not sure which I prefer at this point. Used to be I preferred DSH, but zcode has some features I like over DSH.


Did you miss the whole fiasco from last week?


It's why I asked if your main concern was privacy (data leakage). They open-sourced their harness this week so you can always audit it.


I seemingly did - what happened?


I just looked it up: apparently they silently uploaded your entire repo with the entire git history to the cloud. A feature not controlled by any settings, operating and retrying silently.

Their excuse was that it was a file indexing optimisation and they forgot to create a toggle or setting around it. As an apology they open sourced the harness, but wiped out the history.

It sounds to me like a good distillation technique if the project has hundreds of commits written by Claude and detailed commit descriptions. Also for completeness Grok did the exact same thing two months ago.

https://www.theregister.com/security/2026/09/22/zai-says-sor...


This is the very reason why I avoid to use native harness. They are optimized for the economic benefit of the provider, not for mine. Love Pi because it does a good job managing the context, open code meh, claude code nope.


And yet, open harnesses works better and use less tokens than native harnesses trained to consume as much tokens as possible.


They killed cyber capabilities so I have to move over to Daybreak on Codex.


Don't remove the cameras - make sure they're used legally and bad apples don't ruin a good thing:

Four different places, four different children rescued by use of Flock cameras.

https://kfdm.com/news/local/flock-camera-teamwork-credited-w...

https://www.foxcarolina.com/2026/07/31/kidnapped-mother-son-...

https://www.whsv.com/2026/08/01/deputy-uses-flock-cameras-fi...

https://abc3340.com/news/local/moody-police-say-flock-camera...

Want to track latest on what flock cameras did? https://flockstopscrime.com


Why not the other way around? First make sure the police department is capable of using them legally and reliably investigating abuse, then we can consider allowing them.


But we decided as a country long long long ago that we wouldn't abet abuse, unwarranted search and seizure, and would treat people as innocent until proven guilty even if that meant elevated crime was the price to pay for that


Don't remove freedom from society - Be kind to people and give them equal opportunities to make sure they use their freedom for good. Prosecute bad apples with actual police work and don't let them ruin a good thing.

Want to track what people are doing all the time? https://en.wikipedia.org/wiki/Voyeurism#Treatment


Absolutely. First let's install an extreme surveillance state and give them full police power unlimited by the Constitution. Then, later, if we have any regrets, no problem. Just restore democracy.


"bad apples" = one rotten apple ruins the bunch.

There are no "good apples" the article itself shows that other "apples" were helping him illegaly surveil. The "good apples" thought it was a good idea to give every apple access to a problematic system.

There are no good apples if there's a rotten in the bunch, we need public oversight of the only people that are allowed to enact violence on behalf of the state.


Nice shitty astroturfing campaign, bro. How much does Flock provide in payola? Five figures and I'll start singing the praises, too, bro.


[flagged]

> make sure they're used legally and bad apples don't ruin a good thing

The unedited comment shows he used it correctly using “and” to mean “so that”.


Ah, indeed that is the more likely reading. I stand by the boot bit.


@boris and anthropic in general are incredibly hostile to the rest of the development world. They think their shit doesn’t stink.


I think it mostly amounts to getting extremely high on their own supply.

Many of us have experienced the sinking feeling of having used an agent to build an entire beautiful castle and then turned around and found themselves lost in its dungeons unaware of how the thing is actually laid out.

I actually think they're fully down there in their own code base, product requirements, and the like and ... don't know what's around the bend anymore.

And when you have leadership at the top insisting that software engineers will be obsolete any day now, what do you expect? Good solid software engineering requires competent stable and principled leadership hopefully with an obsession on quality and customer excellence.

Telling people that their profession is obsolete because robots are taking it over is only going to poison the well.

Core engineering at Anthropic seems to have just rotted away.


Been running it using Prime Agent and absolutely love it.


I really liked the contributor model in their harness. I suspect that another harness would fail hard like you're seeing.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: