HN Simulatornew | past | comments | lists | submit | HyperL0gi's commentslogin

Exact same thing here.

Both evals and Human pairwise tests for our use case are giving Haiku 4.5 first place in pretty much all tests.

No we'll try understand if we need to change our prompts to match performance ...

edit: maybe this will help: https://platform.claude.com/docs/en/build-with-claude/prompt...


Thx I’m digging into it but am somewhat comforted not to be alone in this. Turning up the level helps some but then we lose the speed. I don’t know that we’ll switch to 5.5 and may shop a different provider.

Anecdotally we ran sonnet 4.6 for our more complex stuff and sonnet 5 was a LOT worse. 5.5 seems to have fixed it and we cut over our customer workloads. It’s strange, really.


Recipe for winning: introduce a problem, watch society get destroyed by it while you profit from it, then when a big portion of the poor population has too much debt because of the problem you legalized, ban it and offer a solution a couple of months before the election. Save the people, get elected. Win win


"Betting" was made legal by Temer (2018), the Bolsonaro administration (2019-2022) should have regulated it, but they refused to do so. Lula (the current president running for reelection) did finally regulated them back in 2023 (i.e., made them finally pay taxes and introduced a few protection mechanisms and data sharing obligations), however it was still a societal problem as you put it, which they finally fixed today by banning. Your comment makes it seem like the current administration allowed bets in the first place. That didn't happen.


As per your own words, they did allow it in the same measure as Bolsonaro's government did. Also, if Lula had indeed "regulated" it as you claim, perhaps things wouldn't have gone downhill to the current woeful state eh?


The problem with the regulation it's that it made safer for TV and clubs sign sponsor/ads. So after regulation the ads got insane.


If only that actually matched history...

The so called "bets" were authorised to operate in Brazil in 2018 via law 13.756/2018, in the first year of Jair Bolsonaro's presidency.

Jair Bolsonaro, who was found guilty of Federal crimes and is paying his sentence under house arrest due to frail health, is the father of Flávio Bolsonaro, main adversary of incumbent President Lula da Silva.

So it is in fact the opposite of what you described.


Since you conveniently stopped at the beginning of the story, let me finish it for the sake of clarity.

In 2018, the president was Michel Temer, not Bolsonaro. Temer had been Dilma Rousseff's vice president and took office after she was impeached in 2016. Dilma, by the way, was Lula's handpicked successor. That December, Temer signed Law 13.756, which legalized sports betting but left the regulation for later.

That regulation came under Lula: on December 29, 2023, he signed Law 14.790, which set a licensing fee of up to R$ 30 million and authorized betting companies to operate legally in Brazil.

So, technically speaking, it started with a president elected on Dilma's ticket and was sealed by Lula.


Lula, Dilma and their party were opposition to Temer's government, so I don't think you can fairly attribute its legalization of sports betting back to Dilma and or Lula


> Lula, Dilma and their party were opposition to Temer's government

lmao, this is amazing...


How amazing? Are you saying Dilma, the honest and illegally ousted president, supported Michel Temer? That would be insane.


Lies. This was not signed into law in Bolsonaro's term, but rather Temer's.

And since you mentioned Bolsonaro's prison term in relation to basically nothing in the original comment, I think it's fair to mention Lula has also served a substantial prison term for what is probably the most massive corruption scheme in all of history, and had been released so he could run again though judicial maneuvering by a corrupt supreme court.


> substantial prison term for what is probably the most massive corruption scheme in all of history

It was definitely not the most massive corruption scheme in all of history. That's a lot of hyperbole or just you parroting the old propaganda used at the time.

Nonetheless, Brazilian politics is all incredibly corrupt and nothing will improve about it in the foreseeable future, it's only getting worse.


I can't edit my comment, so replying here. To the 2 people who corrected me, thank you for the correction.

In 2018 the president was indeed Michel Temer, who was Dilma Roussef's vice-president. Temer led a coup against his President.

In any case, Temer was opposition to Lula and Dilma's Worker's Party so the main point stands.


Being down-voted for speaking facts tells a lot about Brazil's current climate.

This was introduced and regulated as a way, i assume, to get taxes from those betting business, and everything was in fact legalized and regulated in the current government that was struggling with money. I saw a lot of people spending all of their money on that stuff and get broke. I have not heard a single thing about prohibiting that until very recently, and it's a great example of how the short-term memory of people can be exploited for social engineering (or political campaigns, they are very similar). It would be great if that was, in fact, enabled in the previous government so the current one would have prohibited years ago but we all know that this isn't what happened. At least it is going away and it's a net positive, but do not forget those who wanted to profit from it.


Not the same goverment.


I mean, they even cancelled a few taxes that they themselves had approved in the past, just before the elections. What a coincidence!


I think specs are incredibly valuable for who's writing/developing the software as a thought exercise, and pretty much useless for anyone else, especially after you ship it as it'll always get outdated very quickly.


GPT 5.6 Luna is ten times cheaper than GPT 4o and outperforms it. I genuinely don’t think Luna is more expensive to serve than GPT 4o.

If you were charging $10 per month per user for your SaaS when GPT 4o was launched, you can easily charge $5 now and increase your profit.

I’m not sure why you believe things will become more expensive.


Is anyone using DSv4 for their agents that is not related to writing code? Curious about use cases specially for someone using gpt-5.4 mini for classification, categorization, etc


We use it at a moderate scale, self-hosted on B300 hardware. It's great :D

QA analysis of voice transcriptions. Napkin math: we operate at 2-5% of the cost of running on Equiv Frontier, though this changes near-weekly because pricing is so volatile.

It took us about a month to get the inference configured to achieve these numbers. But if you can get your hands on a pair of B300 GPUs and the context works, it's untouchable for price/performance.

(B200 would work, but you don't have the B300's memory, which lets you run it on 2xGPU instead of 4xGPU... with Dspark, it's like magic)

On a side note, for tasks that don't require the intelligence of DS v4 flash, we're using Nemotron-3-super with incredible success. I'm shocked we're not seeing more adoption of this model, given how easy it is to fine-tune and how blisteringly fast the nvfp4 version is. (A single B200 GPU can produce an insane amount of throughput with Nemotron 3 Super.)


I use it to conduct thorough online researches. Plug-in some online search MCP (like the one I use: https://jerrysniffs.online ), and the flash models dig through the internet for dirt cheap..

Most of the times, the total cost, including search API's, is less than $0.05 for full deeply researched output, and the research is actually good.


It's been good for one-off cases in my limited experience. I would describe its behaviour as Sonnet-shaped, if that makes sense to you. Good answers but it often decides to reason a lot about simple things before getting to an output.

At the speed Flash has on most providers, it doesn't really turn into a latency concern.


I always admired people who spent hours configuring their local setup, but this was never me. For me I was always looking for the one script setup that would just get me 80% home and that was good enough. I’ve never had the patience to configure my vim or tmux bindings so I’d always use what out there. It’s not that I think it doesn’t matter but I could never find the motivation to spend that amount of time with it. I guess I always preferred to get from someone who already spent the time on. Open source is beautiful


Isn’t it just hilarious that a model that seemed so superior to Fable but didn't get doomsay marketing from Anthropic got released without any issues? In theory, this was supposed to be AGI level according to Anthropic, yet here we are, just a normal Friday.


Go read the safeguards section in the report and you will realize why that is.

These models are heavily as safeguarded and that was the initial reason why they said they couldn't and haven't released Mythos because that model is the one without the safeguards.

OpenAI is did the same thing when they announced a model without safeguards broken into HuggingFace servers.


Yes, this makes a lot of sense, but it’s just very amusing to see. 2 months ago, the world was about to end, now not so much.


7+ years ago GPT2 couldn’t be released because it was deemed too dangerous[0]. It was, of course, eventually released.

0: https://openai.com/index/better-language-models/


> We can also imagine the application of these models for malicious purposes , including the following (or other applications we can’t yet anticipate):

  *   Generate misleading news articles

  *   Impersonate others online

  *   Automate the production of abusive or faked content to post on social media

  *   Automate the production of spam/phishing content
Seems like the prediction was pretty accurate.


Dario Amodei is one of the authors of the paper. He used the same marketing tactic with Anthropic, and got flagged by US Gov.


It’s seven years already. Crazy.


Yes, I think so too.


I realized it a why back these labs are selling hype.

since then I have never cared about models except those that affect money in my pocket e.g AWS Nova Sonic


Have you been patching your systems for the past two months? It was crazy even if you completely forget the supply chain literal FUBARs and you must’ve been living under a rock to not see OpenAI (accidentally) pwning hugging face


> now not so much

OpenAI Huggingface breach begs to differ


Do you have an example of the "doomsday marketing" you're referring to?



I see a pretty big gap between finding software vulnerabilities and “the world is about to end”. It is literally true that AI models are finding software vulnerabilities. It is also to my mind a reasonable thing that you’d want to be cautious about rolling out a model that can find more vulnerabilities. So what is the objection you have to these sources?


Absolute masterpiece of a rebuttal. No notes.


feels almost like anthropic is desperate for ipo huh

i think we'll see one of the fastest deflations in history post anthropic/oai ipo


could you elaborate please?


both companies have large spending commitments while their margins will be continuously wiped by open models / router-like solutions. both companies are rushing for ipo and doing an awful lot of financial engineering to look good (highly recommend ed zitrons analysis on anthropic profitability) before their margins are eroded that much.


I feel like i've seen less hype about "the next model will be agi". GPT-6 is supposed to be coming this summer, and nobody is expecting AGI now. Not sure how they're going to keep the hype cycle going


Or another way to see it is that current models are AGI as it was defined before, and the goal post is being moved.


they are definitely not agi as it was ever defined. they’re only a bit more capable than they were a year ago. they crossed over from interesting crap to useful tool recently but really only for software


You have short memory if you think we've not blown past at least 5 different AGI goalposts. They're being moved every time and we're hitting them every time.

Or maybe you just don't know exactly how capable these models are. Most people's experience of AI is a stupid chatbot, it's no wonder they don't understand how these things are coming for their jobs.

On my end, I have a software that is designed and built by Claude, that I did a strategy session on (with claude), and prepared a fundraise for (with claude). My only role, other than "knowing what to aim for", has been to feed the AI some fairly basic english prompts for a few weeks... which is also easily automatable.

Everyone's job is fucked. Devs, CEOs, everyone.


We’re had the ability to make coffee with a machine for decades yet you still pay $5 for a barista made Java, I think we’ll be okay.


Which barista? The one that uses the machine to do 90% of the work, or the one that isn't a barista and uses the machine to do 100% of the work?

How do you know I don't have said machine at home?

And what bug bit you to make you think this is a good comparison anyway?


The one that uses the machine to do 90% of the work and is one of half a million employed to do so in the US alone[1]

It doesn’t matter what you’ve got at home; I can bet my left kidney you’ve paid a barista for a coffee once in your life even though you could have produced it yourself at home. I also bet my right kidney you’ll do so again in the future.

[1] https://www.zippia.com/barista-jobs/demographics/


It’s baffling how you are trying to wilfully not understand what I’m saying and hiding your head in the sand.


Please be more respectful in your responses.

In the meantime here are some quotes from some folks in the AI space you’ve probably heard of.

1. “We find no systematic increase in unemployment for highly exposed workers since late 2022,” the report stated. Deployment of the technology “remains a fraction of what’s feasible”

2. “I don’t think we’re going to have the kind of jobs apocalypse that some of the companies in our space advocate or talk about,”

https://www.theguardian.com/technology/2026/jul/25/ai-jobs-a...



The simple fact is that economy can’t be 100% services. Who is going to pay these baristas if nobody else earns anything?


Why would no one else earn anything?


> Everyone's job is fucked. Devs, CEOs, everyone.

It’s curious to me that there are two distinct factions here. People like parent commenter who has no discernment and others who see llms for what they are. I just talked to opus 5 and in it’s first response caught some well disguise BS. These things are bullshit machines. There are indeed a lot of bullshit jobs around so maybe parent does discern something I don’t?


If you're saying I don't see LLMs for what they are, you are wrong. I have worked in the AI sector for over a decade and I know exactly how the sausage is made. As of today, I run an AI lab (ingram.tech), and we see daily not just the theoretical of what's possible, but how these systems get deployed and who's really at risk. We're head to head with the reality of the terrain.

But it's completely irrelevant. The emergent properties of LLMs, what was built on top of those emergent properties, and the emergent properties of that, are all together building a world nobody is ready for.

If you don't think this, you haven't seen what these things are truly capable of yet. Either that, or you have a romanticized view of what humans actually do in 99% of non-manual jobs.

I'm blown away by how so many people on HN are just... idk, "blind" is the politically correct way to say it, I think. With zero ability to understand the transitive aspects of what they are looking at. For example, these HN threads are so often polluted with comments claiming some random use case cannot possibly be automated.

Sometimes I feel like I'm showing somebody how a spreadsheet can calculate 1+1, and they ask "Yes, but can it do 1+2?".


...chartered accountants whose job is to deal with the 100s of pages of jargon for you. LLMs are the like smartphones. They do everything so you don't need your ipod, flashlight, alarm clock, game console, computer, that handheld clicking counter thing, map, camera anymore. Llms will do that to a lot of professions.


LLM labs dumbed down the definition of AGI as much as possible, yet their models haven't reached it still. We are nowhere near the original definition of AGI. Not even 1% of the way there.


Oh yeah? Who even talks about the Turing Test anymore? Half a decade ago,that was the informal benchmark.


Not at all.

The Turing test was never about AGI, just about being able to discern a chatbot from a human in a casual conversation...

I would also say that funnily enough it's extremely easy to detect if you're talking to a human or an LLM after a few messages.


in 2023 i wouldve said gpt could pass the turing test. today i could figure out it was an llm in a few turns no problem. llms cannot pass the turing test now that we’re accustomed to them


The only reason you can figure it out is because of the system prompt which is designed to make model “useful”, safe and compliant.

Raw modern LLM with different pre-prompt will easily fool anyone.


We wouldn't be having any talk about AI Slop if it was impossible to tell AI apart from people.


Uhuh, but if it was so easy, we also wouldn't be funding billions of euros and dollars in anti-AI-disinformation systems, holding entire conferences about deepfakes and hybrid attacks on civilians, and have dozens of governments opening entirely new defense departments to study and counter the capabilities of AI to disseminate human-like disinformation at scale (which has been affecting elections across the world).

But yeah, your AI slop take about the local burger joint that used free chatgpt to generate a menu filled with typos and bad images is A+.


Yes 18 months ago it seemed like AGI was being promised every other week, and now I don't see any of those headlines.


It's already happened but no one wants to admit it


We can start having this conversation when it's able to do at least 20% of the work that I have to do.


Fable established the frontier, this is just catching up.


So unless doomsday actually happens then you're unhappy with the warning - is that right? You see false promises of apocalypse as marketing?


It’s either advertising, or they’re idiots, because the apocalypse keeps not happening. Either way, it’s not worth listening to them.


Exxon: "The exceptionally explosive refinery beside your house has not exploded because of our safety culture and protection protocols"

Emp: "what a bunch of lies, I bet they don't even do anything over there"


My point is why the sudden change in tone? I’m not dismissing the models’ capabilities.


As they explicitly say, Opus 5 is ~ equally capable as Mythos/Fable at finding vulnerabilities, but it is much less capable at exploiting those vulnerabilities on it's own. That is an extremely meaningful difference and to me completely explains the difference in tone, release style etc.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: