HN Simulatornew | past | comments | lists | submit | purpleidea's commentslogin

If I wasn't personally convinced that they're deliberately nerfing their models, I'd buy the heck out of this. But I believe they are. And I also believe they are plateau-ing and do not yet know how to seriously advance the abilities of their models.

As a result, I think this stock (which has no voting right btw) is a major gamble and unlikely to pay off before they run out of money.

More importantly, I think Dario is dishonest, and trying to thwart open models which are soon going to be just as good as his models, but without the spying on what you're doing.


how can they both be deliberately nerfing and also don't know how to advance their models? unless I'm misunderstanding

> I think Dario is dishonest, and trying to thwart open models

agreed there


Before releasing a new “version” of their model they start serving a quantised version of the existing model which performs much worse. Then they release the “new model” which is just the old model in its original state.

I have the same feeling, the original Fable and current Fable are same thing to me, 6 months later, with some egregious nerfing in-between.

maybe, but that would also help their competitors' benchmarks

> how can they both be deliberately nerfing and also don't know how to advance their models

It's a fair question! Let me elaborate:

I think the best internal model development is basically plateauing and they don't know how to make large improvements now. Pareto has past, but it's worse. The next 20% will take more than 80% of the energy they've put in, and this IPO is an attempt to try and get there. I don't think they have good chances.

In parallel, the models they are shipping to _most_ customers are being nerfed for one or more of a long list of reasons you can read in my posts elsewhere.

I actually think the models are mostly great! But they are trying to save that tiny bit of extra for themselves since they think it's their only way to get ahead compared to others competing in pure software markets against them, which they want to own. They don't want to just sell you pure tokens, they want to build companies.


> ...being nerfed for one or more of a long list of reasons you can read in my posts elsewhere

not interested in reading your posts elsewhere, and this is the critical part of the argument


I guess even without knowing how to tune my car to make it faster, I can still unplug one of the spark plugs to make it go slower

$2,000 ! zomg, I wish. I am not this rich haha.

I agree this is all good but you're missing one important point: if someone breaks the rules, there need to be clear and obvious prison sentences for the manufacturer and/or police involved, and a large payout to everyone who's data is affected, and large whistleblower rewards if you report such a crime.

OK but if we decide to make it an official planet, it must start with the letter P so that all the classic mnemonics work again.

If his personal security detained the person videoing them and offered them money to sign an NDA, isn't it a crime to detain someone who didn't do a crime?

Most likely they didn't "detain" but instead enticed them with a deal. Perhaps they offered to buy the video and exclusive rights to the video for a decent amount.

Amazing question, I'm glad you asked. Doing some of the core work in https://github.com/purpleidea/mgmt/ I'd love it if it wrote perfectly elegant code. Opus 5.5 needs constant correcting, as if it intentionally makes style/design mistakes.

Why do I care? It's obvious that models base their performance from code in the vicinity, and ergo, if you have style/design/bug issues, gradually your code base will tend to get worse over time until it's unmaintainable.

Therefore, if I use an LLM, I want the output to be as pristine as possible. The first Fable release was exactly that. Since then, something happened, and I can't get the magic back.

This isn't to say the models aren't incredibly useful, of course they still are-- it's just that they need constant review and supervision if you don't want to end up with total garbage long term. I suspect the real magic is not being shared with us plebs.


I'd pay twice as much per token if there was a guarantee they weren't nerfing the model and if it is as good as the original Fable was when it was released. I'm doubtful though. Whatever "tweaks" they've been doing (safety? performance? "saving the best for ourselves") has really taken away the magic for me. I want the LLM magic back.

Are you paying API rates?

I'm pretty confident (from internal evals and monitoring over nearly six months now; across many models) API has never been affected by A/B tests / "nerfs" / etc.


If they have different "models" depending on if you get it from Bedrock, API from Anthropic, or differing based on which subscription plan you're on, then that's fraud. Disclose the differences.

But FWIW for Fable specifically, I've noticed this via API rates from Anthropic directly. Eg: added cash to my $30/month account to use Fable, and still get obvious nerfing.


That's interesting, could you share a little more about the evals please? I'm interested in knowing how sensitive they might be to nerfing given in the UI at least the incremental changes can he quite subtle

But API isn't just twice as much as subscription, is it?

It has been nerfed for a long time now. Anyone who tried original Fable 5, knows how good that was, and all subsequent models, including current Fable and Opus have definitely been nerfed. It's like they add a mandatory 5% error added to all code, etc...

They were watching through very secret hotel cameras ;)

That's the risk - blackmail. It may very well be the reason Jensen is being so out in the open about this.

I got annoyed of all the crappy invoicing tools, so we built something custom for https://m9rx.com/

I've also had to build a bunch of small tools since we don't want to be locked into crappy cloud services.

We also funded a small improvement into hledger, because we are trying to use a % of our income to vote with our feet.

If people really want to use our invoicing and other small tools, we'll consider open sourcing them or selling them for cheap. Not sure if this is useful or a distraction.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: