Read this last night. I was completely thrown off when I read this line:
> We expected the engine to return an error for any prompt it could not process, and our long-context tests returned successful responses, so we did not believe this to be an issue
Surely “we expected and error but got a successful response” merits investigating rather than moving on?
Off topic but doesn't intel do something weird where some i3 chips can use ecc but not i5 or i7 until you get into server chips? I remember seeing that when plotting nas builds a couple years ago. Should have pounced on actually building it then....
Yes, I liked using them for NAS builds. Great for ZFS machines.
Intel is weird with their stuff like that, I wrote about how they introduced and then dropped an SHA instruction that could be used to optimize hashing (useful for things like block checksum hashes on storage servers) before: https://neosmart.net/blog/will-amds-ryzen-finally-bring-sha-...
> Just like Intel not shipping ECC for client SKUs when DDR5 design makes it really necessary
The ECC in DDR5 is on-die ECC and that's not "real" ECC, it's what DDR5 needs to not fail. Side-band ECC is the standard method, employing extra memory chips and 72bit memory. OEMs mix these in the marketing materials to fool the users into thinking they buy the better option.
Yes, here I'm referring to actual ECC, not the on-die ECC. All DDR5 chips have on-die ECC as a prerequisite just to reach error levels comparable to standard non-ECC DDR4, but from a hardware analysis perspective, even with the on-die ECC, DDR5 still needs ECC more than DDR4 (which already needed it).
My interest was piqued and I tried to read on… Holy mother of all that is slop, I couldn’t even get to the text because of all the interactive slop I had to scroll through.
This is incorrect. You cannot identify the presence of text watermarks if you do not have access to the models so that you can spot where expected tokens diverge from actual tokens based off the weights. That does not preclude Pangram from being able to reliably identify LLM-generated text from models whose outputs they’ve trained their own detection model upon.
e.g.: "You must not use OpenAI models or their outputs to generate, prepare, select, or supply training material or signals for models that are intended as alternatives to or substitutes for OpenAI models or services".
Of course they disallow it. In saying they’re not thick enough to not realize how it looks to publicize their anti-distillation efforts. You must not have seen the tirades and screeds Anthropic posts in comparison, then.
People in HN live in a bubble. Plenty of real world use cannot use subscription plans (not even talking about enterprise agentic use) and is absolutely at the mercy of the cost per task metric.
We agonize over cost per task to the extreme for services/functionality built atop of APIs and can absolutely guarantee you that it is the real world metric that counts most right after “is the model good enough for the task at hand,” and furthermore that, yes, in fact, it is quite often that we see certain models/providers easily using 3x or more tokens on average for the same tasks (or even the exact same task) than a supposedly similarly matched model from another provider. Once you become intimately familiar with the thinking traces from specific models, you can preempt this to some extent with manipulations to the system prompt (essentially heading off entire thinking branches) but the nuking of reasoning traces in the name of preventing against distillation attacks has made this harder.
Our own open-source, cross-platform, multi-language, cross-framework SecureStore is an embedded secrets store (sort of like SQLite) that lets you version your secrets alongside your code in git (and deploy updates via CI, accordingly), only securely.
Available for rust, golang, php, python, js/ts (node, bun, or in-browser), C#/.NET, PowerShell, bash, and more. Open Source (MIT), not vibe coded, and lovingly maintained for a decade.
Encrypted secrets in Git with a tool written in Rust under a fully permissive license is the right way to do it. Encrypted copies of secrets in environment variables is okay too, as long as they are encrypted.
The somewhat widely known Mozilla SOPS is a tool in the same broad category.
I hadn't seen securestore-rs before. Thank you for sharing this.
Wow, Voxtral 3 sounds so much more artificial and “AI” as compared to the old Voxtral 1 in the comparison clips on that page! I absolutely hated listening to the new one.
Also, comparisons with Whisper always really tick me off. It’s not a dictation model so much as it is a transcription model. It’s really the only one you can use to capture incidental audio rather than one or more persons deliberately speaking into a microphone, e.g. the only one you can use to transcribe audio from surveillance footage or background audio recorders; it has a higher WER but it actually tries to capture everything (leading quite often to hallucination, unfortunately) whereas the other models they compare it against tune out background noise and dialog altogether, in my fairly extensive testing.
> We expected the engine to return an error for any prompt it could not process, and our long-context tests returned successful responses, so we did not believe this to be an issue
Surely “we expected and error but got a successful response” merits investigating rather than moving on?
reply