> "The CRL's were signed with the original private keys (which were supposed to be the correct keys with appropriate entropy) but given that this resulted in incorrect signatures, they were resigned with flawed keys."
Ah yes.
"The system says something is wrong, but it's also 4.30PM. Let's just re-sign the thing and go home."
Keep in mind that a _lot_ of software out there uses TLS as its only trust anchor for updates, commands or similar.
This can and will be weaponized for take-overs of systems. They will be of the consumer kind, but they will also be of the industry kind. Let's pray that they will not be of the state or military kind.
From what the internet is telling me, the keys were allegedly found on https://repository.emsign.com/certs/.crt
Which, if true, is definitely a lot of fun.
Hope you people are ready for lots and lots of suddenly hackable IoT.
__
Edit: I might also be misinterpreting what I am seeing and it's actually weak logic that created the keys, in the sense that one could guess the private key?
Someone actually qualified please give credible answers.
I'm just pretending.
I think you’re right. And since the intermediate certs expire around 2039 and the private keys are known, we have plenty of smooth MitM fun against IoT in front of us.
> Edit: I might also be misinterpreting what I am seeing and it's actually weak logic that created the keys, in the sense that one could guess the private key?
As I understand it, the private key is supposed to be a number in the range [0, 2^256), but they actually generated a number in [0, 256) for these 8 intermediate certificates. Once you know there are so few possibilities, you can try them all ~instantly, so effectively everyone has the private keys. (And when I say "you can", I actually mean you, not some hypothetical more skilled version of you: even if you don't know anything about public key infrastructure, if you have access to an LLM, it can do it for you with a single prompt.)
I don't think being hacked by an anonymous entity grants much political advantage? If anything the downside is shifting the overtone window by dumping all the far right takes on the broad public and making them a bit more familiar.
Just claim that the anonymous entity wasn't anonymous. Claim it was your enemy.
"Look, the Antifa hacked us. We need to ban them just like the US did"
Or blow it up bigger. "Look, the system itself is corrupt and 'the left/the altparteien/whoever' stopped using democratic measures to compete against us. That is because they cannot win on their own merit. Meaning that the system and its safeguards are meaningless."
Something like that. Given more time, one could flesh out this back-of-the-napkin thing into a likely working strategy.
___
Fascism fundamentally isn't data-driven but narrative-driven. So data alone is not going to work against it (though it is a valuable pillar).
I think part of the advantage is providing public evidence in favor of a Verbotsverfahren, e.g. the process of banning a political party for being counter to the principles of the German constitution and with that, pressure towards the state organs to stop ignoring the threat and finally do their job.
Yeah that's also what I thought the idea is behind that, but I'm afraid that at this point, a Verbotsverfahren will not work out the way people hope it will.
If you look at the numbers of votes they're getting, you see a group of people that is so large, it will be self-sustaining, instead of fizzling out. Or so is my prediction.
Idk. I just don't see this ending up in the way we want things to end up.
It's not going to affect me personally all that much, because I can be "passing" so to speak, but that's not exactly what one should strive for tbh. Keeping your head down and not caring as long as you're fine, I mean.
I think the ideal moment to ban them was years ago. The next best one is right now.
And I agree, now that we've waited so long (gee, I wonder if the far-right Verfassungsschutz leader had anything to do with that?), banning them will be ugly and lead to civil unrest. I think realistically, we will need some major state repression of these Neonazis and good luck with that given the political leanings of police.
But I don't think there's a way around that at this point, we still have to try.
"Wehret den Anfängen" and "Never Again" and all that.
Maybe, but otoh, action bias alone is a terrible advisor.
"You gotta do something" is correct, but not every something is suited.
But I think at this point we're entering the circular stage, so.
Idk. I don't want to live in this world we see developing, but what I want seems to be not all that relevant to said world.
If "popular ideas" == Neonazis trying to overthrow the German constitutional order, then yes, that is the appropriate response.
There's no awards to be won for coddling fascists. The protections of a free society cannot be extended by those trying to extinguish it.
> When our enemies say: well, we gave you the freedom of opinion back then- yeah, you gave it to us, that's in no way evidence that we should return the favor! Your stupidity shall not be contagious! That you granted it to us is evidence of how dumb you are!
And I understand that this might seem like hyperbole to the less-informed American reading this.
It is for their benefit I have translated a few quotes from leading AfD members so they can paint their own picture what those "ideas I don't like" are: https://news.ycombinator.com/item?id=49536747
You don't need more evidence for a Verbotsverfahren. That's not the reason one wasn't already started. It's the unwillingness of the parliament to initiate one
I see your point. I would argue that care and the protection of quality can take on many forms, but that behaving like Linus Torvalds is not a requirement.
I can see that viewpoint, and I guess it is legit to have it, however, first of all "behave like Linus Torvalds" is a shorthand that could mean a lot or nothing, and thus doesn't lead us anywhere, and, on top, what is correct gets determined by what works, and not by what feels good.
You've introduced a category that could be described as an alias for "bad", and then you threw in random stuff into that, based on unclear and handwavy criteria.
This can only end in a screaming match, so let's not do that.
Not interested in engaging in a screaming match at all. Yes, I've used an extreme to try to make a point, which is not great. :)
I'm curious, though: How would you picture protecting taste while at the same time adopting an inclusive (as in not putting people off) stance? Do you have examples of such a behavior?
> How would you picture protecting taste while at the same time adopting an inclusive (as in not putting people off) stance?
I don't, because I reject the definition.
Being inclusive does not mean "not putting people off". For starters, because measuring there doesn't work. How people react is not a direct reproducible consequence of your actions. Your actions might influence it, but they have this ginormous internal state that has much more control over it.
But that is a side-track.
The actual problem with the definition is that putting people off isn't bad.
Putting people off for arbitrary reasons like race, gender, etc is, but that "arbitrary reasons" is a strong modifier that significantly limits the set, and was dropped in your framing.
True, and cool that someone actually put in the work to prove it, but the takeaway is still kinda wrong?
Because it's not the abliteration that does this. It's the "untrusted artifact" that does.
Even a "clean" model of a lab could be doing this. Which is the whole point of sovereignty there, and why the US is a bit concerned about CN models (and vice-versa, probably).
That seems like a pretty obvious thing to mention, as it is a clear consequence of the research they did.
___
Add to that, that it's not actually the "artifact", but the "in-memory representation", meaning that anyone or anything that control that can _also_ do this. Be it your inference provider, or, a level further down, the firmware of the GPU it runs on.
> Because it's not the abliteration that does this. It's the "untrusted artifact" that does.
> Even a "clean" model of a lab could be doing this. Which is the whole point of sovereignty there, and why the US is a bit concerned about CN models (and vice-versa, probably).
Does it even need to be untrusted? From the article apparently it only takes ~250 poisoned documents in the training data to backdoor a model, and it survives safety training and fine tuning.
Which means any model trained after 2026 is succeptible to slurping up poisoned documents on the internet. We all know how prolific the crawlers are in a desperate attempt to slurp up everything on the internet. And all it takes is the lab not catching ~250 documents.
So these agents that can process anything on the internet through an LLM trained on the internet only have to stumble across the payload. Seems like agents running through LLMs is a terrible idea from a security perspective.
I understand this in principle, but I've been wondering: what guarantee do you have that the sources that are claimed to be the training are the resulting model weights, without interference?
Maybe you trust the creator more, when they release the training materials as open source, but I don't see how you'd get a guarantee.
Just spitballing-- you'd see other firms generate their own versions with slightly modified training sets. If you were a large org that wanted to do that, you might want to also reproduce the original model in order to chase down any quirks you see in your model. Then you'd see the weights don't match up.
Or adversarially-- if e.g. "Deepseek Open" became one of the most widely used commercial models, it might behoove the US government to reproduce it as a check against intrusion, and vice-versa for China and any American model.
The same is true with open source software that distributes binaries: you can recompile (retrain) it yourself, but if you choose the convenience of the binary (final artifact) you can't guarantee one comes from the other.
It's even worse because AI might be able to tell you that the source matches the binary, but it can't do the same with a LLM.
rather than that, what guarantee do you have that the model does what you want under the conditions you care about?
how do you know you are outside of those conditions?
im not convinced that having the training data and process actually answers the most important questions, because we dont have answers to them.
the rest is great for being able to audit what is there, and being able to play with different variations, but it isnt going to be the same as reading code, then building it, then comparing the build hashes to see that they're the same
I train a new model. I publish my training data and my training code/workflow.
I then secretly do additional post-training to poison it. I do not release the data or workflow for this.
Now, I publish the poisoned weights and an md5 hash of the poisoned weights.
To you, the user - you have no way of knowing this happened.
You might ask "why not publish intermediate training checkpoints" but those are nondeterministic (in the sense of being not bitwise-reproducible). Anything with shuffling, random initialization, distributed races, driver versions changing roundoff, compiled vs. eager, and a million other factors introduce slight differences that yield totally different training trajectories. So publishing checkpoints does not yield verifiability.
If the original model is poisoned then yes a checksum etc doesn’t matter. If you as the end user are being diligent then you are downloading the checksum and checking the download against it for precisely the scenario you talk about.
Personally I find it a bit exotic as I’d probably just stop trusting the vendor first but checking against a checksum does stop attacks where others alter an artifact.
If making sure the download is intact is what the above user wanted a checksum for, they're solving entirely the wrong problem. And your followups here are also solving the wrong problem.
The problem we want to solve is making sure the training data matches the model. The only use of a checksum that helps that problem is if multiple entities do the same training and then make a checksum of the result. But that requires training to be completely reproducible, which it's currently not.
In a big LLM we are talking 10^24 operations or more. When you start talking about error rates per trillion flops, and talking about all the potential ways to compute operations in different orders you are talking about one of the more difficult things undertaken by humans.
There's an entire GitHub repo, my man, with documentation and all. The article discusses this at some length.
Honestly, this whole line of commentary comes across very gatekeeper-ish. Isn't half of what makes it to the front page people showing off their cool shit?
Well, yeah, that was kinda the point. But also kinda not?
The motivation you're reading into it is incorrect. (And rather un-HN for that matter. Both assuming bad faith and the whole.. uh.. reddit-ness.)
My point is that the linked blogpost is a poor representation of the work they did. And that is bad, because they've shared it themselves. So it makes sense to tell them that it might be a good idea to next time show more of their actual work and less of the stuff everyone else that just bought the machine could've done.
That is negative feedback, yes, but it is not negative to punch down on them, but, ideally, to lift them up. Anyway.
> it is not negative to punch down on them, but, ideally, to lift them u
I think some thought is needed on phrasing of your initial comment in this light. It very much comes across as punching down - it's not phrased as "this is cool, but it would be better with A and B", but rather it comes off very dismissively as "this post doesn't have any real value"
The spirit of this platform is different. You're wanting it to behave like other social media, where one has to soften everything and constantly glue on all the context one could possibly need just to feel emotionally safe.
Which is why those platforms do not get any useful conversation done, because it is incredibly exhausting.
Please stop trying to terraform this last bastion where capable people can have high-bandwidth interactions down to baseline Internet mediocrity.
You can use literally everything else to get the space that you want. We who don't can only use this.
As someone who was intrigued by these new machines but also has no interest in running Windows, I'm quite happy to learn it works. Though admittedly I was expecting to see DGX OS first.
At this point, "conspiracy theories" (the ones that check out logistically. not the actually crazy ones) are just factual news that made it far too early into the news cycle.
Ah yes.
"The system says something is wrong, but it's also 4.30PM. Let's just re-sign the thing and go home."
Good luck recovering from this.
reply