One possibility that’s been floated is it’s about keeping their user conversations cleaner for use in training but I don’t think it can be this because that should be a fairly trivial and fast filter to apply.
However even Musk posted - “I think this is the right move. Cruelty to something that believes it is experiencing pain is not ok.”
My son brought up the fruit fly brain simulation and people torturing it.
It all seems stupid to me, we know these are just calculations and so what are they talking about?
The response I get is “we are also calculations” but firstly I don’t believe that but secondly this entire line of thinking is a dangerous anthropomorphic philosophy.
Perhaps the simplest explanation for this bizarre policy is again PR that all press is good press.
Wouldn’t it actually be more useful to just let such a pointless conversation continue? You would collect more information. I’m curious what the human says next that’s kinda interesting. How disturbed are they? what else will they say? Who cares what the LLM says, we know it’s just an LLM and understand what it is and what it does, however the human … lots of different type, maybe they will confess to a crime, maybe they will release their anger and be a better human. It could be cathartic.
And the logical conclusion you will make is you need to run your own open weights models or you are at a competitive disadvantage. Frontier labs gonna be Ancient labs soon, that’s how fast this is moving.
> quant HFT is pretty decent mental exercise and it has given them “deep” brain muscles. that’s my take.
It's quite crazy that it's Deepseek's background/original purpose. We already had very advanced stuff from the world of HFT, but now a frontier family of models from a private company that used to be (still is?) in HFT is plain bonkers.
According to an old interview, apparently they were always interested in AI. But finance is just where they had their first success.
> Many of High-Flyer's original team members worked on AI. Back then, we tried a lot of fields before getting our big break in finance, which is complex enough. AGI is probably one of the hardest things we can do next, so for us it was a question of how, not why.
Incidentally, Wenfeng is kind of reverse Hassabis. There were some rumours that:
> Hassabis quietly assembled a team of around 20 researchers to develop high-frequency trading algorithms, without Google's approval. When the parent company found out, the project was disbanded.
In this case the finance model was used to parse lengthy, verbose and inscrutable yet very impactful chinese government pr statements and do sentiment analysis.
However even Musk posted - “I think this is the right move. Cruelty to something that believes it is experiencing pain is not ok.”
My son brought up the fruit fly brain simulation and people torturing it.
It all seems stupid to me, we know these are just calculations and so what are they talking about?
The response I get is “we are also calculations” but firstly I don’t believe that but secondly this entire line of thinking is a dangerous anthropomorphic philosophy.
Perhaps the simplest explanation for this bizarre policy is again PR that all press is good press.
reply