Astra is plural. Sol is singular. Checks out to me.
Fable goes beyond the density of the text and makes a mark on the culture/oral tradition. I didn't say it was a great literary scale but it does make some sense.
My understanding of LLMs are that they are giant functions and generate output deterministically. However, the way these providers setup and route traffic means that your traffic lands on different boxes where each box has different inputs that run alongside your prompt -- essentially creating "non-deterministic" behavior. Basically it's BS to make LLMs look sentient. Each model could be burned onto an ASIC if you really wanted to be locked into that model and get incredible token speeds. If I'm wrong I would like to be corrected, thanks.
Is there a model that doesn't need babysitting? Half my day is spent untangling BS that Claude and Codex hallucinate. Particularly around hallucinated product requirements. Its still faster than hand-writing code but only just. I'm seriously considering just going back to the early copilot days where I lead coding and just let it try to auto-correct. That was a great balance of AI enhancing my speed and workflows without removing any confidence in what was being shipped.
give your models a more organized context/harness if they are hallucinating product requirements... and I'm not talking about some hulking context framework. just organize your documentation in a canonical way. This product requirements hallucination is something that would bite me constantly if...they ever actually hallucinated them (and they don't, ever). This is most certainly user error.
All the time. All the frontier models love to take a requirement and bend it into a need to create an obscene amount of overhead. I needed to add a permissions for a user type. Instead of adding the type to the existing list of permitted types it created a bespoke experience for that user type complete with a full testing suite. This is without diving into the absolute insane amount of no signal tests that these things generate. It once decided the privacy policy should have its own testing suite, which basically amounted to 50 different regex tests looking for "key" words and phrases, and snuck that into a PR for something tangentially related. They love to burn tokens on creating useless shit and it happens on the daily.
I've worked in some pretty big global orgs. Everything was decided by committee which meant that everything had to work for everybody, which meant that everything was shit. I spent some of the best years of my life building and rebuilding the same bullshit. I think I shipped a handful of truly novel experiences in that time (probably about one novel feature every 2 years) but the rest was beating a dead horse to make the experiences we already had "better."
Eng was neutered at that large org. Product people had the "vision" and the say. A lot of product people just used our team's product as a stepping stone onto their next role and in the end it showed. I would hope Google has a better culture...but theres a good chance it doesn't.
One could argue that if you hate these megacorps then the best thing you can do (as an engineer)is to work for them and purposefully be a monkey wrench in the cogs. Be competent enough to not get fired, maximize the company resources you take up while minimizing your output. Delay delay delay and if you're really good at it, never deliver. Take as much of their time and money as possible and give as little back. Thats basically what they're doing to the rest of us. And when its gets too evil you can always leave. In the meantime you are taking up headcount that could have actually advanced their goals further, faster, evil-er.
Maybe? But I don’t think most folks working for these companies are trying to get hired as insurgents employing the CIA handbook. Slowing down Facebook resource-wise as an engineer would only expedite the rent-seeking behavior. This also assumes manager incompetence and not getting fired - which I’d buy for a few months but years would require some spy-level dedication.
Not shilling for them but Ollama cloud hosts domestically with ZDR afaik. I run 95% of my open weight inference through them. The rest goes through Opencode Go $10 plan (which is enough to run 3 hermes agents on DSF 4.1 and leave plenty of left to experiment with when new models drop).
Cloudflare doesn't retain request bodies by default, and if you don't trust that then you shouldn't trust the third party AI provider either. Cloudflare does cache responses, but that doesn't typically apply on API endpoints and is trivially disableable.
you shouldn't trust the third party AI provider either
I trust competing American AI labs to not collude with each other, so I can spread my work across them and nobody sees everything.
I trust Chinese AI providers to not work with the NSA.
And of course there's always tinfoil.ai's attested model engines... probably the only non-evil use of Trusted Computing I've ever seen.
But Cloudflare is different. They're the single point of access for everything. Which is precisely why they attract irresistable government pressure. Trusting them is like painting a target on your back.
Anthropic - Literary scale.
Google - random elements with no clear relation to model value/capacity.
reply