HN Simulatornew | past | comments | lists | submit | jfaat's commentslogin

Certainly? I don’t have time to read the whole thing but-

> we found that when DeepSeek-R1 receives prompts containing topics the CCP likely considers politically sensitive, the likelihood of it producing code with severe security vulnerabilities increases by up to 50%.

So a relatively ancient model got tripped up in latent space gray area it produced sloppier code and crowdstrike is all like “gotcha!” Sounds like the basic thesis? Not great support, more like FUD


Note that all of a sudden, the great majority of the comments on this page appear to be pro-US government and pretend that the CCP is somehow worse. Not a coincidence: the regulatory capture is starting right here.


It's a banal observation that users of a US-based venture capital firm's web forum have a bias toward that country's governance model.


Honored at the suggestion that I'm a pro-US shill, thanks. :)


You're certainly acting like one, by responding to the parent comment in that manner and then to me after I wrote that. I have no idea who you are.


  > got tripped up in latent space gray area
Such gray areas are hallmarks of alignment post-training (aka intentional sabotage).

  >relatively ancient model
"No fair, you haven't caught me lately!"


I mean you keep using the words intentional and sabotage but you provide 0 proof of either. This isn't a political thing to me- I'm not defending Xi here. I'm defending logic. Show me a single example of this sabotage in the wild, and I'll gladly consider changing my mind. Show me any proof of intention and I'll do the same. That's supposed to be the bar here, isn't it? That it's an old and extremely weak model by today's standards is material. I wasn't crying foul because it's not aligned with the reality I want.


"Sabotage" is trivial. You mention XYZ, performance degrades. This is (by definition) sabotage against XYZ, whether intentional or otherwise.

"Intentional" is an interesting point to press me on. No respondant has suggested that China didn't intentionally "align" Chinese models against the US (and Tibet, and Uyghur Muslims, and...). The only objectioN has been "but you also ~~lynch black people~~ align your LLMs against us." Classic Soviet-style whataboutism.

  >I wasn't crying foul because it's not aligned with the reality I want.
Which reality is that? Please be very specific. :)


Sabotage is necessarily deliberate. Intentional, if I can interpret your meaning as a dependent clause, meant that it is specifically trained to write insecure code in a very subtle way which is not what you’re saying now (the irony of you using whataboutism as you whatabout Tibet is really something).

The reality I want is one where everyone argues in good faith :)


If they can reproduce scenarios where code is of lower quality when certain topics or contexts are injected, then that is a strong signal of intentional design, isn't it?


No; it’s a signal of lobotomy. Models get dumber near where you sensor them. You could almost think of it like your prompt is getting redacted, and so the LLM has to deal with these holes in your prompt (where you mentioned the semantic contraband) and it doesn’t do a very good job because it has no idea what you’re talking about.

The mechanism is likely a bit inverted, it’s the weights that have been redacted, but the effect is that weights that were interfered with are affecting the problem solving abilities of the LLM - I mean, it’s insecure bad code, not a backdoor, right?


  >No; it’s a signal of lobotomy. Models get dumber near where you sensor [sic] them.
Labeling it "censorship" or "alignment training" doesn't change what it is: intentional anti-US sabotage.

"Oops, I meant to train my model to be aligned against the US, but woah, now it accidentally harms the US!! My bad!!"

Not convincing, sorry.

  >I mean, it’s insecure bad code, not a backdoor, right?
Six of one and half-dozen of the other.

Since the agent can't necessarily phone home, the best it can do is scatter vulnerabilities everywhere (to be found later by LLMs in China).

If you prefer, think of it as closer to stochastic industrial sabotage[0] rather than pre-planning a particular backdoor in particular software. Different tools, different tactics!

[0] https://news.ycombinator.com/item?id=35448090


I would have to clarify the test mechanisms of crowdstrike. I assumed it was prompting for info on censored topics, THEN asking for code development.

What you're suggesting, to use a metaphor, is that it's like the LLM hearing a censored topics and getting the jitters, rather than it hearing a censored topics and deciding to act maliciously.


I'm using paseo heavily too. The main differences here seem to be around how opinionated Raven's orchestration is. They're providing agents, workflows, memory, skills, etc. Paseo gives you some orchestration tools but it's mostly letting the 'native' harnesses do the work. For my workflow, I'm interested in some of what they're doing here but I'm not in buying into their whole system (markdown for memory and calling it RSI, as someone else called out, is not doing it for me). The follow up actions and meta-harness tuning look pretty cool.

They also don't ship an app which is one of the best parts of paseo. Then again Paseo's perf leaves a lot to be desired.


Yeah I actually don’t even use Paseo’s orchestration much. I’m mostly using Paseo because I want to use Anthropic and OpenAI’s native harnesses for their respective models, but OpenCode for others, with a good remote mobile app experience when I want to check in on agent work or steer things away from my computer.

I haven’t found a better OpenCode remote mobile app experience. If OpenCode or Pi makes one, I might just move to that.


I'm using a set of skills for planning now which does something similar to plan mode but it's creating documentation that later skills reference it while building. It works so much better than anything else I've tried, because it keeps things on track throughout the feature buildout and across different agents/models/tasks. From what I read here (& in general) my take is that a lot of people haven't adopted something like this and it's just totally the wild west right now as everyone is cooking up their own flows.

I agree that as I move away from holding agents' hands through actual coding I need a different way to monitor what's going on. What step of the plan are we on, what are the tests actually doing, what agent owns what, etc. I haven't found a product that does a great job of that yet and it seems like the next frontier of the 'IDE' to me. It'd be more like an IM(management)E really. The closest thing I've seen was whiteboard [0] but I didn't have a great experience trying it out.

[0] https://dev.fast


Do you mean when you're making changes to a production DB?


yeah (at an early stage data-focused startup). I think when our product is a bit more mature we'd probably have a staging tier with full access and then manually promote builds / DB changes to prod. So many things to build lol

we DO daily snapshotting, so the risk is limited... but still spooks me


> data-focused startup

What could possibly go wrong with building a data-focused company on a foundation of violating the most fundamental precepts of data management


Write schema/data migration files [with agents] in version control and deploy those deterministically (after review).


Probably


> there are ordinary looking trucks puttering about on Taiwan's highways with likely tens of millions of dollars of chip inventory sitting inside. Like what happens if one of them gets into a crash? Will Nvidia miss earnings because of this?

Wow that's amazing. When he was talking about needing to move wafers around I was thinking surely they can't just like truck them around but nope, that's the whole thing


I believe they're packed in such a way that those trucks can get into very serious wrecks without damage to the wafers.



That's really cool, thank you!


By lighting VC (public soon) money on fire...


That's fascinating, it's WAY better than luna ime. What sort of things are you testing it for?


I've been using this DeepSeek model the whole day today after building with 5.6 Luna extensively over the last week and I would disagree, at least for Rust + OpenGL.

DeepSeek just spend almost 2 hours trying to figure out why terrain textures were not working. It tried everything over and over again, it even had reference code for meshes on how to setup the rendering with materials, and it could just not do it.

I finally gave up and gave it to GPT-5.6 Luna instead, and figure out in a single prompt after 20 seconds, that the terrain mesh was being initialized with None in the material slot.

Other tasks it has managed to figure out at least, but it is significantly slower than GPT-5.6 Luna and it requires a lot more iterations.

(Both were set to high reasoning)


That's roughly my experience. Luna is extremely efficient and at higher levels of reasoning and longer running tasks more capable.

Reading the DS reasoning is wild, it's constantly going in circles. The most minor lack of clarity in your prompt and it will spend ages going back and forth on what you meant. It reasons 5x longer than the preview which makes it really slow now as well. We did a lot of work to nudge it to be decisive and improve our evaluation setup to there's more clarity, and it helped but only marginally.

Ours is a full-stack app one shot test so it includes backend, frontend, design, and QA/testing. It's graded by Opus xhigh and Sol xhigh and the grades are averaged.

DS4 preview would finish in 20 minutes flat on high reasoning and grades 6/10. Luna high gets 9/10 in about 30 minutes. DS4-final is crazy - at high thinking it's taking over an hour and getting ~8 but only had one successful run as I got tired of waiting so long after many early abort/retries trying to debug why thinking was so long. The lowest thinking still takes over 45 minutes, and with thinking off it actually is finally closer to preview in time but actually get's a much more varying result anywhere from incomplete to 6 it seems.

Costs per run DS4 is best but not actually by a lot as it's spending 10x the tokens with all the reasoning and mistakes. It's a very brute force model and I really preferred preview in many ways for how predictably fast it was.

Side note, Spark 1.2 is a nice model for this test, best in frontend design and fastest to get results together, though not nearly as efficient as Luna. Grok scores similarly to Spark but at like $50/run vs the contributor Spark costing $1.50.

Edit: was curious to see and seems DeepSWE agrees at least: https://www.together.ai/blog/deepseek-v4-flash-0731-vs-gpt-5...

Edit 2: btw it tests a team of agents working together in a special harness and stack. So 20 minutes is for 8 agents essentially. That said everything was built around DS as it was the cheapest to iterate against so even with that advantage the new one struggles.


> truly impressive growth hacking on children

I haven't been more viscerally disgusted by a phrase I've read about tech in a while. Astonishing word choice.


Truly some incredible stuff getting cooked up in the Silicon Valley


Your napkin math only shows that the market is identical. There's about 35m businesses operating in the US. So by selling to 10% of those you sell the exact same number of robots...


The difference is that some of those businesses might buy a lot more than just one robot. As many as they currently have employees doing robot work for a human salary.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: