Realistically, the entire chain proposed by your safebots - I like the idea - but I cannot see viable ways to get it actually deployed in a useful manner.
Especially with the hardware stuff, this is plainly put unachievable by many IT departments.
The fact that models vastly outpaced their harness and permission systems - I wouldn't dare to doubt this fact.
Claude Code on auto is still rolling a dice with its sonnet classifier - whether that IaC action I told it and explicitly stated multiple times it is permitted and authorized to run - yet it always randomly allows or denies it.
Therefore this is totally still a unsolved, perhaps unsolvable problem.
Realistically; anyone paying for llm access (anthropic, openai, gemini), is getting their access, and a service provided billed by tokens, subscription, whatever.
All the efficiency gains, which publications like deepseek v4.1 flash seriously frontload like it is their most important topic to have accomplished improvements on without diminishing performance too much - now this is a thing anthropic and anyone else also cares about, but for different reasons.
American "providers" with closed models are setting their token pricing somewhat arbitrarily, which is fine: it means more profit, and pretraining and RL experimentation is super important and expensive.
They (closed model providers) have very likely super optimized inference too, just like deepseek, but it's not at all something that any customer really has to care about - they just want the service to be as cheap and great as possible.
I feel like all the closed model providers are milking it as they likely know open models on local hardware will one day eat their lunch. We all know it's not a matter of if but when. The company goes bankrupt, the hardware and property sold off, banks holding the bag.
The only way out is to develop a model vastly more powerful and capable that we have now. The market believes theres a good chance of that, although I've never understood why its truly winner-take-all
Yeah avoiding all mention of the huge financial incentives that may push for “pacing the frontier” makes it seem like the opposite of a credibility boost for these firms.
It seems damaging since most folks (who lack insider knowledge) will naturally wonder if it’s due to plateauing performance per $ or some other non “alignment” reason.
Cloud models will always have massive benefits of scale.
Caching is the simplest one to understand, cloud providers often reach a 90% cache hit rate, so hosting the same request locally on the exact same model on the same hardware is often way less efficient than on the cloud where a group of users generates a healthy cache.
KV cache is per conversation, I'm getting 100% hit rate on my single tenant local set up.
The benefits of scale are on the token generation side, you can batch rounds and generate tokens for multiple conversations per pass instead of just one token per pass.
everybody do be cooking with water.
Chinese Labs provided pretty good, primarily cost-reducing techniques, like the sparse attention patterns recently.
I'd bet OpenAI and Anthropic use their variants of those too, so they can get greater margin on their tokens - not something they'd really want to / need to self-report.
You got to know how to use the model+harness+prompt to achieve the results you want, but honestly for many projects and questions all the models already pump out their same best version of an answer.
Sometimes it is really akin to a git clone, although it was a LLM request.
This rewarding expertise is somewhat wishful thinking.
At the end of a day, it feels and is more like gambling, even with the recommended expertise and a good approach, don't delude yourself you're simply pulling the lever too, as any novice.
The "anti LLM" or "no AI" clause is really more a "covering all ass" clause.
If you want to improve Forgejo and have that AI PR; you better clean it up of all traces - resulting in a PR that is likely neat and going to be merged.
I see no issue with that. It simply enforces a pace and process that is very befitting and in my opinion mandatory for such a central system they're maintaining.
Especially with the hardware stuff, this is plainly put unachievable by many IT departments.
The fact that models vastly outpaced their harness and permission systems - I wouldn't dare to doubt this fact.
Claude Code on auto is still rolling a dice with its sonnet classifier - whether that IaC action I told it and explicitly stated multiple times it is permitted and authorized to run - yet it always randomly allows or denies it.
Therefore this is totally still a unsolved, perhaps unsolvable problem.
reply