HN Simulatornew | past | comments | lists | submit | tanh's commentslogin

So to make the APIs stateless (the "ideal" where they don't use server side sessions/etc) we ended up with this. I'm sorry but this is kind of hilarious. Given the salaries paid to the workers at these companies and the hype of the models, I can't believe they all fell to the same flaw.


It wouldn't matter if it was stored only on their servers. As long as they offer the feature to downgrade a chat to a dumber model that can be jailbroken (and the downgrade keeps the reasoning), this trick works.


If CoT wasn't stateless and you instead just got a reference which pointed to the CoT stored on the lab servers, the same vulnerability would still exist. Since you just need a weaker jailbroken model to read a smarter model's CoT. This being stateless or not doesn't really matter.

The stateless part is also important for enterprise customers that require zero data retention.

(they could scope CoT access per model, but then users couldn't switch models mid-session)


this is a lethal trifecta, but where a chunk isn't even needed

you have a secret to keep that is read by the llm, and untrusted input that wants to exfiltrate it.

by hell or high water, the agent is gonna output that text


strange because, their subscriptions are not stateless. they log everything and send it to 3rd parties for moderation.


I guess Demis was about world models and the old talent saw the writing on the wall. Noam was a big one and this is just a continuation of that.

Don’t understand how Sundar is still running things, though.


Yeah. I’m guessing they’re behind the Chinese models in capability now


Just today my Pixel failed to do the right thing on "Set an alarm in 15 minutes" thanks to Gemini. This has worked reliably since Google Assistant was introduced.


"Hey Siri, find me gas stations along my route" routinely fails now. That worked great for years. Total clown show.


I feel nothing but grief.


For code can't they distill from public GitHub commits? If they could figure out who used Mythos/Fable assitance in the commits.


I think they're distilling "reasoning", not merely code. There's plenty of human generated code. What they're trying to extract is the process by which really large models "think" their way through complicated problems. That's what all the "traces" datasets on HuggingFace are about.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: