ages ago I tried using IPFS to more or less accomplish this, I imagined it to act more like a weights/training data network fs that everyone would be able to participate in.
The product isn’t meant for you or me it is for lawyers. If you can’t take on personal liability for a badly written contract then you shouldn’t be using it.
I'd expect they could indemnify you against hallucinations or similar if this gets good enough for that to be a very rare occurrence? Or you could buy insurance on it that's cheaper than hiring a lawyer (not a high bar to clear). I wouldn't rely on it currently, though.
I already use them for that, they are pretty excellent at it. Much better than the terms of use generator products that used to exist. That said, nobody cares to sue your business for the most part until you're big enough to be worth it. By that time, you'll have a team of legal analyst to assist you... or agents should I say.
Then again, nobody will have money to buy anything at this rate, so in all liklihood, this is a total non-issue.
We offer MCP and then consume it with our in-app assistant to go from a non technical prompt to a series of what is essentially API calls they can automate for themselves for repetitive tasks or things that require a few screens to accomplish can be done from the assistant widget itself, etc.
I agree with mostly all of this, but personally I wrote a toy LLM almost 5 years ago and while it never saw much use outside of boring my wife with a shitty command line demo with glee it did help me understand how they worked and how to apply them, played a lot with JAX and pytorch, ended up building a ghetto version of MCP and an LLM-Pool to proxy requests to my baby local models and so I didn't struggle to see the evolution of openrouter and MCP agentic workflows. The same way i'm really glad when I was younger I built a bad webserver by myself, a really painful SQLx type database, etc etc etc - none of these things led me to developing for Nginx or Oracle nor will knowing JAX get me a job at an AI research lab, but I do have a lot of depth in understanding how the technology works so that the flavors on top of them are easy to digest and make more use of immediately, and I think the same can be said for engineers coming into the field - if it's a spooky LLM box you aren't going to be squeezing the same amount of juice as the guy that knows how they work inside and out so having at least the understanding of a _babys first LLM_ is going to get you miles ahead of people who don't.
For anyone who wants to dork around there is https://github.com/rasbt/LLMs-from-scratch which is something amazing that I think anyone who wants to engineer things around LLMs should at least blast through and read.
Game cheating and reverse engineering MMO backends taught me a lot: databases, networking, securing a backend (and frontend), limitations of simpler languages when comparing them to more native options for building backends.
Agreed, I was very late to the game and was forced to learn VBA for excel sheets and that is how I finally broke into programming.
When I was a pre-teen I stumbled upon CD-rom hacking guide to bypass disc requirements on games, I remember opening up the file and the screen being filled with HEX code. I was so overwhelmed I just closed it and never touched programming after that for 15 years. My life would have been totally different if I had embraced the unknown instead of retreating.
Not specifically Anthropic but why are we allowing billing to take place in tokens that are nebulous and fully controlled by the operators who have no aligned incentives?
If I have a user input and then sanitize and inject that into a prompt to do something, I have no idea how much that is going to cost at all and no real way to measure this properly. A parallel example is digital ocean or aws, i can go and measure/limit my compute/fs/memory/startup times/etc and while it can be impossible to get down to the last flop of money allocated - i can run things on a real budget with real constraints, opposed to an LLM where I have to .. prerun a sanitized user prompt through a tokenizer and then ask an LLM to guess what it may do and give token consumption estimates and then act on those in any sane manner for the user?
Perhaps i'm missing something to do realistic and static rails on things but I don't see a serious way at scale to use the token billing model handling things requiring a users free text input short of having to go pander to VC money to throw money at it until someone else figures it out.
*to clarify my rambling...
We should be billed and given controls based on resource usage itself and not an opaque token concept on top of not being able to spin any knobs that control it's resource usage.
The model providers are quite aligned with concerns like customer retention. These arguments only work if there is no competition. We exist in a marketplace of black boxes. There's not just "the one" you must suffer. You have options. You can build your own too.
A per-token model roughly aligns with the providers' costs, and it is an objective measure, so it seems a reasonable way to charge.
I see posts all the time on HN about which models from which providers offer the most bang-for-the-buck, and how to minimize token usage and still get optimal results, so it appears that competition is working.
There is so obviously competition in this market it’s astounding to me that people attribute all this malicious behavior to the model companies.
They’re growing over 10x a year. They want users and revenue. In order to get users and revenue, they want to provide the smartest models at affordable prices. If they unnecessarily burn tokens, users will get less value and switch.
This thread is filled with competing comments about their monopolistic power and how when one model provider was no longer doing a good job people switched to a different one.
I was just complaining to someone that token billing is like letting a gasoline company control your gas pedal while you nicely ask them to use a specific gear that may or may not actually be in use and you guess what speed it's actually going based on how fast the trees go by because qualitative judgments have to replace the speedometer unless you can just burn money.
Don't forget also letting them dictate the route you drive while you are increasingly blindfolded till you arrive and then find out you need to tell them to drive you to your correct destination again because they drove you across town instead which burned half your tank.
“Claude, spend the next 10 hours trying to solve the Reimann Hypothesis”.
I agree that incentives are misaligned but there’s several competing model providers. If one gets funny with their costs people will jump ship, especially if the gap between the top 2 labs and everyone else keeps shrinking.
“I can take current sources and tell you how solved this is, but I am not willing to work to a timeframe or to solve things that aren’t yet solved by mathematicians or science”
These safeguards already exist when they get a whiff that you might be using Claude to fix security issues. Doesn’t seem farfetched given the incentives I outlined that they would apply to this kind of abuse.
How loose those controls are becomes a market force.
But just so we're both entirely clear on what an LLM is... it's a token prediction system.
it genuinely can't do things except recall things that have already existed.
People are having great success composing things together in new ways, but just like the english language has a finite number of sentences, and music has a finite number of chords: LLMs too are just combining things that have existed.
I don't want to sound condescending, it is remarkable how useful this technology is, but please don't evangelise them on capabilities that they genuinely can never have.
Laptop computers have incredible processing capabilities but nobody expects them to be able to walk your dog, no matter how useful they actually are at doing other things.
Yes and you suggested Anthropic have their coding agent refuse to attempt to solve anything it can’t find a preexisting solution to in case it turns out to be hard, if I understand you correctly.
Knowing how hard something will be to do before attempting it is precisely the sort of impossible thing that it couldn’t do.
As you claim to understand, they don’t know things, they sort of drift on vibes. Telling them to refuse things they think will be hard will only accomplish making them more annoying.
God I'm not an ai booster by any means but I'm so sick of this argument. If the maths proofs that have been put out recently are just 'just combining things that have existed' then that goes for everything and the term is meaningless. If LLMs are stochastic parrots then so are we.
Often LLMs seem to be aware that work is heavy, but knowing:
a) if something is possible
b) if something has been requested to take a long time ( a signal of abuse, like requesting illicit pictures in image generation)
is actually somewhat straightforward (I mean, if they are able to predict if something is a “substantial piece of work” as they seem to do already).
The mathematical proof thing is obviously marketing spin, you should pay more attention to what mathematicians are actually saying instead of hackernews folks.
These things are really good at being search engines and harnesses for iteration rather than some kind of advancing intelligence.
I'm not worried about the volatility in the definition, i'm worried that I give it 1 token today and receive 2 token output, tomorrow I receive 40. If i'm doing this a hundred thousand times a day it is difficult to price this in for users downstream or in the extreme cases be able to absorb that at all short of going into a failmode with degraded access until someone goes and buys more tokens or gets the bill. The alternative is just pass the buck and bill your non-technical customers with a "tokens" line iteim every month.
> If i'm doing this a hundred thousand times a day it is difficult to price this in
When you’re doing this 100K times per day you get an extremely good idea of what it costs. You also have all the tools to see when something starts changing quickly.
This change is for Claude Code the harness. If you’re using the API at scale and paying full price then you get exactly what you put into the request.
No, those doing this 100k times a day have very good data on this, good estimators and modeling. And the API has various knobs to change and evals will give you actionable data.
One guess is that their "primary" target audience/market is the large corporations that get their employees unlimited tokens, and not the individual developer who may worry about spending and token accounting.
It's the opposite. The enterprises have all the tooling to monitor token usage of employees, and to limit access. For example, we have a $300 month limit, and then need to file exception tickets when we need more to justify the cost. Pretty similar at other non-silicon valley company process. I don't know any enterprise who'se on unlimitaged token budget for their employees. that's not how enterprises sign contracts.
reply