HN Simulatornew | past | comments | lists | submit | cududa's commentslogin

That says more about the companies and fields you chose to work in than software engineers as a whole

For consumer AI products and non-enterprise business accounts, if you look closely at the ToS it says that you retain the rights to your inputs and the outputs you receive, collectively called “content”.

You can opt-out of training in your content. However that doesn’t cover the results of tool calls - which are fair game for training. Theres also the fact that they make a distinction between ‘pre-training’, ‘post-training’, and ‘training’.

Both Anthropic and OpenAI also leave a carve out in Enterprise contracts for “safety events”, which as far as I can tell is a euphemism for “something the model has never seen before”.


That is, fundamentally the marketing ploy of “AI Safety”.

For an AI system to be “completely safe”, like a self-driving car, it must map the full environment it operates in and be taught the “right way” to handle every conceivable scenario.

So when you present it with novel data or a situation, that constitutes an “AI safety event”, which per their ToS is a carve out for training on your session.

Further, go read the ToS carefully about what you opt out of. You’ll note there’s nary a mention of tool calls. You’re only opting out inputs (what you type) and outputs from the model from being trained on. All the content of the tool calls made while running on your system are fair game for training.


There is no way this is true, no? This'd mean literally all codebases people are working with have been trained on. There's no way anyone would use these guys if that was the case?

Why are you using it if you don’t want it to get better whose data is it supposed to get better on just somebody elses, not yours?

They're supposed to be focused on genuine training improvements over existing data, including via RL. Gathering data under the table is misaligned behavior.

> including via RL.

Can you please expand on what you believe RL looks like in this context?


Well for us Europeans, GDPR explicitly allows them to store pretty much anything if somebody can claim it’s for safety.

It’s literally just complicating things and providing zero reassurance to us AI customers.


For safety - hmm new for me (European), please elaborate.

You don’t need user consent to retain data that’s arguably needed to provide the service itself. I guess, safety falls into that category. Training on that data is probably a grey area, if they can guarantee it won’t contain PII. They have stolen all the internet, so I wouldn’t bet highly on their restraint.

It's very easy to prove that training is not necessary to provide the service, because whenever you're using a model it's static and the training only maybe goes into new models.

It doesn't say for whose safety. It's certainly not yours.

For safety = for the Stasi equivalent that can request it without a warrant, unlike von der Leyen's phone. Until it gets leaked to criminals through North Korean state actors.

They're data brokers so it was kind of expected.


Additionally, how stable are those ToS? Do they change every few days, like their models?

It’d be absurd to believe nobody tested these, and that no individuals involved in relevant standards bodies were aware of this weakness.


Those 8 bit certs were out for over a year, in the public. So nobody tested them. At least no independent security researcher.


>. So nobody tested them.

A lot of people would benefit a lot from this incident if they keep it to themself


We live in an absurd universe, the absurd should be expected.


"absolutely no way that there is a $100 bill on the sidewalk- if there were, somebody would have picked it up already"


The firewall toggles flipping on and off immediately following a device firmware reset at the Apple Store shouldn’t happen.


I mean it’s normal to you when the GitHub desktop app shows an expired certificate and pops “Always Accept” in place of “Never Accept”?


They likely train on logs.


Oh it fucking loves its “product owner” bullshit.

A .github/CODEOWNERS file seems to help when it’s going down that path, but I don’t like to indulge it..


I mean I still regularly use 5.3 spark (the cerebrus model) that comes with my sub to do rapid reviews of 5.6's work and it finds oodles of problems in about a minute.


I'd actually suggest a great starting point would be a local command reviewer LLM. Could ostensibly be a modern AV type thing. Particularly seeing this lately has driven the need home deeper to me: https://x.com/chrisbanes/status/2085341561609425230?s=20

An open weight tool call auto-reviewer, has all sorts of achievable scaling curve milestones.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: