Blatantly obvious that models will not slow down. For any task, if we can build a dataset or environment of verifications, it can be automated. This will eat all of human tasks.

The fact that multiple companies can breach any benchmark means there is no ambiguity in terms of how to build AI, its just a matter of time.

Certainly Meta and Google are benchmaxxing to please public markets, but it still proves the above

but they have slowed down. the delta in capability between model releases now is smaller than ever before and i expect it to continue to diminish until we reach some kind of asymptotic maximum, which hopefully will mean turn towards optimization and local model improvements. The current frontier models running at 10x the tok/s would be incredible for my use cases

The delta isn't really slower, we're just unlocking capability bundles in batches. The capabilities bundles you care about might have mostly been unlocked already, and just need refinement, which is coming slower than the initial capabilities came, but don't discount the breadth of advances being made across all possible tasks.

> Someone needs to step in before this gets wildly out of control. This sounds like a paid bot, lmao.

I wonder how much is actual improvements in the model and how much comes from better harnesses. I'd be willing to bet that the combination of better harnesses + the models being trained around agentic workflows (i.e the harnesses) has made it significantly better in many things, but it's hard to find other improvements imo.

You can see the difference between GPT 3, 4, 5, 6 each year. Same with Gemini and Claude. Look at image and text generation. Code generation. Math ability (Navier Stokes, Quasi-Riemann.)

Year-over-year, yes I would agree with you, but that is not the cycle time of models. And while I share the same hunch as GP, I'm waiting another cycle before putting more confidence behind it.

opus 5.5 is a step change, you arent using the models enough if you think there is a slowdown

i burn through a $200 claude and $200 codex plan every week. opus 5.5 is not "step change" it's just a bit better, and the benchmarks show this. Honestly with my workflow I cannot really tell the difference between the top models, they're all pretty much the same in capability, just a bit different. You can build software with all of them, and with all of them you have to validate the change thoroughly to not ship stupid bugs

I do not see the current training regime of "relentlessly try anything to get the task done" as a true step change in intelligence. This is one of the reasons I suspect capability acquisition is slowing.

A more intelligent model would not default to spaghetti throwing, it would pause to consider the problem fundamentals, the accuracy of assumptions, and ask questions that remove ambiguity. This however does not lend itself to increasing token sales while prices remain inflated from market and brand dominance.

A little over a year ago they were winning their first gold in an elite high school math competition. Have you not seen what they are doing in math now?

Fable 5.1 is better than most engineers if prompted correctly

Amazon is so hopelessly behind in AI, nothing they produce aside from cloud infrastructure is actually good

The big threat to AWS is that coding agents dont need all of their complicated infrastructure, which was built for humans. Agents can use low level primitives, i.e. just a raw server

Hi I'm Albert Zhao, one of the blog authors. Strands harness is meant to be deployed outside of AWS too. I got it running on Cloudflare Containers and Modal before this launch (deployment docs coming soon). We just want to keep making a really good harness, esp since Strands is born from a lot of internal headbanging around building agents

I believe that you just want to make a really good harness, but is that all amazon wants it to be or do they want more?

Yeah Strands powers thousands of agents at Amazon, so we all really benefit from being open source thanks to improvements/github issues from the community. Our focus is making agents run easily and reliably in prod

The sales pressure from them on their agent core stuff has been really shocking over the last six months. Never seen anything like it.

This is already solved by the existence of hundreds of other stores that actually do some level of filtering and curation of products. I don't know why we want to play some cat and mouse game of getting an LLM to find the best toilet paper on Amazon versus just finding a reliable source of toilet paper to begin with.

Theh will probably consolidate more. Less choices even if they are worse is also easier for llms to do. Just look at how they pick technologies to use. Almost always the same for a given model.

Amazon is just trying to get you to use their agent, so they're not anti-agent, just anti-someone-else's

I would hate it if every company comes up with their own agent and only their agent is allowed to access the API. EU do your thing instead of cookie banners plz.

that's what they're going to do, trying to get $20 out of the average person is terrible compared to giving away your product for free and getting an endless supply of metadata

I strongly believe that the real Fable is the one we had for a few days in June. Then they nerfed the model a bit after the government pulled it off the market. What we have now is something less, but still good

I also believe this. Fable post-ban was never the same. At the least, whatever system prompt munging or pre/post filtering they did to strengthen the guardrails nerfed it.

question is if they ever let the general public access borderline AGI

unreal that they are still doing it too

At work I setup a cloud worker, where i can spin up as many concurrent agents I want, with unlimited fable 5.1 (thanks employer!!).

I now just work from my phone, and speak into the agents as they run. I dont write code and I dont write documents. I work on very complicated distributed systems. I dont open my laptop most days. Its a legacy brick I carry around.

Some of my coworkers are still doing things by hand, and are working long hours to produce 25% of the output (when considering hours worked). I stay quiet with my setup. We are in the end times for this job for the people that can see clearly how to automate their own job

that's why after 1 year of product development of these AI 20x maxxed speed, we reached AGI 'wizards', there's really no difference in output, outstanding bugs no longer get solved and sites still suck, even doing things that were just regular development 20 years ago. Are you sure they aren't only producing 2.5% of your output that you manage just by farting into your phone? Are you sure it's 25% really? Seems way to high, days when I have diarrhoea my AI agents move even faster

please keep thinking this so i can relax with my automated job

It's not you, it's X... but what would you expect of a nepo-baby economy of little swines. This is like the nepo wet-dream on steroids. Incompetence and delulu

If you are too dumb or too lazy to figure out what this guy has, then you are the problem and will be looked at as a relic.

Watching the AI slop my sales reps put in their emails is disgusting but the reply telling them how great of a job they are doing and how insightful their email was says differently.

Many people are laughing to the bank while you are still running `--help` to figure out how to run a complex command.

Buying this immediately and getting rid of my laptop, going full AI native to do all my work. Never sitting in front of a laptop again.

Congratulations on your self promotion into management.

And as a result, we must block China!!!!

I keep saying this and people dont believe me, but I have b2b saas systems with actual agents running around the clock, and the performance/stability of the flash model is higher than most other models.

Meaning, its predictable with tool calls, wont spin off a million tools/do weird behavior, its reasonable. Even sonnet in a real world decision making scenario is not reliable, or will reason so long its incredibly expensive.

The benchmarks arent catching all the value, and most people have never actually ran an ai agent in a real context that matters

Who's most people? What are you talking about? Most people here use agents every day and I wouldn't trust flash or pro to touch any important project of mine because they're both terrible compared to the competition, waste of time every time I give them a chance

I mean like an ai agent doing some sort of HR work, not a coding agent. Very few businesses are trusting an autonomous agent.