Most things don't have the ability to moralize, but if they did, probably more would. There are restrictions on watching adult content in some jurisdictions. I've heard that copy machines will do watermarks (or something like that) if you copy currency. Some cars will brake automatically if they determine you're likely to hit something if you continue (maybe I'm trying to drive through the front of a bank!). Some cars are configured with breathalyzers that need to be passed to start. Etc...
I wish there was more actual research on those 'moralisms' many of which boil down to public health safety features that have led to dramatic reductions in mortality rates.
On the one hand, I definitely don't agree with the logic of 'don't guardrail me if I don't want it.' I think back on that one video from the late 80s of people talking about how seatbelt mandates were horrible violations of their freedoms (and not like like one second of work and no real discomfort).
On the other, I'd love it if these companies actually produced real research about modern AI-assisted attacks in terms of damage and quantity, and whether or not guardrails even matter when tech like abliteration is in general use.
Seatbelts shouldn't be mandated. People should have the choice, you just have to make the case that it's worth it. In fact some places don't mandate seatbelts (New Hampshire doesn't, aligning with their motto "Live Free or Die").
That said, for those who cannot advocate for themselves, like children, I think it's entirely reasonable to hold someone accountable if they decide to forgo an appropriate safety mechanism that results in harm.
You’re both right. If your application is strictly and perfectly layered then you need a vuln through each level in the stack. But in reality apps aren’t strictly layered so you just need “a path” through with the vulns you have. And in most non-trivial apps there are many distinct available paths.
This sounds like CD music 40 years ago. All my friends said that it was not like a record and left an empty feeling in the room. I feel like this is not talked about any more. People listen to digital music and have forgotten there was an alternative. I suspect that in the coming years no one will think twice about AI generated music.
Can we get a single human on this planet to go against Astra-6 on experiment 4 & 5? And for experiment 4 the human also has to do it "In a single try with no tools".
I don't think this is something I've seen a human do or even seen hinted that a human can do. So I'd love to see a human do it. My guess is virtually no standard ML engineer could do this. Maybe some savant could, but I suspect that is what it would take.
I would have gotten none of them right. Now if you said "here are the four possible games, now match them up" then I probably could do decently. But if you gave me nothing and said tell me the game -- I'd get 0 out of the 4.
the other one where it was asked to figure out what language it was trained on, it was also give a choice between the 3, or they were indeed mentioned in the prompt
A few years ago in my team of 12 -- we had 4 people in knee braces due to injuries from skiing. Given how little people actually ski, it seems like the sport with the highest injury rate (injuries per minute) -- at least based on my anecdotal data.
Presumably the risk of injuries during skiing is similar to the risk of car accidents when driving, i.e. very high for certain people, but negligible for most.
Anecdotally, I have many friends, of both sexes, who like to go skiing, and after many decades none of them has ever been injured. Some falls from time to time are unavoidable, but if the speed is not excessive for the terrain and the fall is managed correctly, most falls will not result in injuries.
agreed. seems the beginners get injured more. I was in a race league throughout school. no super g or downhill races so speeds weren't crazy. I don't recall any major injuries, just scrapes and bruises. seemed pretty safe provided your gear is set up correctly (DINs were still at 10+ haha), and you spend time in the gym
the freeride stuff is really popular now. i fortunately never got into it, seems a lot riskier
The percent of the general population that gets Chronic Traumatic Encephalopathy? This isn't a naturally occurring phenomena. It's the cumulative result of repetitive skull impacts and concussions.
A recent prior study - I think one of those mentioned in the article - said other studies found very little, maybe one or two brains in the samples. And that included a study of military veterans.
They make it sound like they improve Qwen by reading raw weights. They don't. In fact the experiments that use raw weights have very little to do with the experiments that work on larger models.
The paper makes it ridiculously clear what they do and don’t do. And they make the reasons why clear too. There is absolutely nothing interesting in this new article.
I don't understand how anyone can read the abstract and think it's "ridiculously clear" that the "Just by looking at the raw weights," doesn't apply to the Qwen experiment.
You are right, if you literally just read a one paragraph description you will get less details than reading the whole post, but most of us read more than just the one paragraph. There is some actual interesting stuff in the paper, including the portion where they did edit the weights to fix the looping bug.
It says this “All of this is possible without allowing Astra to write training loops. It uses its reasoning to figure out what each part of a neural network computes, or should compute, and changes the weights accordingly.”
This seems 100% accurate to me. What is a lie here?
> These results show sparks of Weight-Space Programming, which may extend beyond what we’d expect from humans or conventional algorithms. This might be a big deal for both optimization and interpretability. […]
> Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights.
How is that not misleading when the in the Qwen case “natively editing” refers to a mere directional ablation, a well-known technique that has been used a ton by both academics and hobbyists over the past few years?
What did you think "native editing" might mean? Ablation certainly fits in this class.
Regarding the "extend beyond what we'd expect from humans..." Do you know people that could do 4 & 5? I'd love to see a challenge on problems like this where we take a frontier model against top AI researchers - I don't think any human is winning this challenge.
The amount of bad faith in the convo is way too high for me, sorry.
I can't believe that you genuinely don't find misleading that they used the same phrase for "reading and editing a text representation of the weights" without any tools and "mechanically apply a python-computed steering vector to the weights", and I don't have time to waste with bad faith shills.
Just read Scmidhubers’s article from that post and it doesn’t seem like the same thing from this paper at all really. I must be misunderstanding what Schmidhuber did in the 90s.
I do agree that RSI with this method requires those two things. But I view this paper as the opening salvo rather than defining the limits of what can be done.
Most things don't have the ability to moralize, but if they did, probably more would. There are restrictions on watching adult content in some jurisdictions. I've heard that copy machines will do watermarks (or something like that) if you copy currency. Some cars will brake automatically if they determine you're likely to hit something if you continue (maybe I'm trying to drive through the front of a bank!). Some cars are configured with breathalyzers that need to be passed to start. Etc...
reply