Fair points, but you are glossing over important issues:
1. You are using the current point-in-time cost for batteries as compared to a subsidy spent over years while battery costs were dropping.
2. You do not include transmission upgrades and costs to distribute energy from centralized batteries.
3. You ignore the benefits and efficiencies of having generation and batteries distributed and close to the source of demand. It also increases resilience of the grid.
4. You assume the political will was there for government and taxpayers to bear the full cost burden of development of the batteries.
1. The subsidy was fixed per kWh at a level that has always been vastly above utility scale BESS.
2. Power usage is flatlining. There is no reason to need transmission upgrades for BESS. Successful projects are colocated at retiring generators, or at existing major grid terminus.
3. Are those benefits worth 4x the price? Noting again, that's 4x the subsidy, not even the actual installed cost which is closer to 8-12x.
4. It would have been cheaper even at just the subsidy rate.
Where are you getting that? The 2026 AEMO plan says:
"Electricity consumption is expected to almost double over the coming decades, driven by population and economic growth, electrification across homes, businesses and industry and the emergence of new large loads such as data centres."
Are you really holding up Snowy 2 as a success story? We're past 20bn on Snowy 2 now. Run those numbers against grid scale BESS, on both capacity and peak generation and get back to me - 10 years ago it looked good, now it's not favourable.
lots of people confuse a $75-$120 / kWh cell price for grid scale battery installed price, which these days is about $300 / kWh.
1 gigawatt hour of installed BESS therefore at current prices is roughly $300 million.
Snowy 2 is 350 gigawatt hours, which would cost $105 billion in equivalent batteries which is about $150 billion in Australian dollars, or ~7x cheaper per gwh.
If you price it on throughput and not storage size, because Snowy 2 is very constrained on when it can charge and discharge then the realistic 8Twh per year you can get out of Snowy2 loses to batteries by a factor of 3 based on your costs.
But that's assuming the batteries can charge and discharge in full every day.
The truth is probably somewhere in the middle where Snowy can act as seasonal storage with a big capacity that isn't used every day.
I think it would then also lose on price if compared with a portfolio of batteries alongside solar and wind to directly supply electricity and continually recharge the batteries over seasonal scales but that is a more complex comparison.
In general I think even good hydro schemes (and Snowy 2 is not a good one) rely on non-electrical factors to be worthwhile now batteries have plummeted in price, e.g. managing seasonal water flows for agriculture, and the electricity can be seen as an additional benefit.
sigh This reminds me of all of the tedious arguments by people reminding us that solar panels don't work at night, implying that they are therefore useless.
Or the ones that remind you that batteries are useless for seasonal storage.
Or that the wind doesnt always blow.
Yes, individually all of them have crippling flaws and all of them work best when combined with technology that has complementary crippling flaws.
Snowy 2 won't be used instead of batteries it'll be used alongside batteries.
Right now nothing comes close to beating its price / gwh no matter how much the cost overruns or how many people dump on it for being 7x cheaper per gwh than batteries instead of 10x.
More countries should be building one as well as building out grid level batteries. They complement each other.
This is an interesting idea, without giving away your benchmarks specifically what kind of stuff do you test? I might try to assemble something myself, it's so hard to determine model quality from the system card these days.
I have actually been planning to open source the framework. Only benchmarks I care about are the ones that look like real work I do. So it makes it easy to trawl your own repos looking for benchmark task candidates from real bug fixes or features. Then as a bonus I can compare the agent’s solution to my own as a reference.
So most look like that but I did include a few one-shot “build an app that solves this problem” and some qualitative design tasks and a tough algorithmic optimization one.
Also working on a product to build tasks from your own work for testing coding agents. Main thing I would offer is to look carefully at the agent trajectories - they love to figure out ways to cheat. Additionally, consider what "winning" means. If just using test pass rate, consider that tests might not encode what good means in your repo. I have been having success using "equivalence with merged PR" as judged by an LLM as a signal.
Yup, trying to be really strategic about testing. I didn't end up sticking with it, but I tried requiring test cases to cite a matching clause in the task assignment. But also: these tests are only indicative. Only a human can score a run.
Would open sourcing it make you feel like the results might be compromised? I'd rather keep it private so it's specific to my use cases and not included in any training date (no matter now small the signal in the overall data).
1. You are using the current point-in-time cost for batteries as compared to a subsidy spent over years while battery costs were dropping.
2. You do not include transmission upgrades and costs to distribute energy from centralized batteries.
3. You ignore the benefits and efficiencies of having generation and batteries distributed and close to the source of demand. It also increases resilience of the grid.
4. You assume the political will was there for government and taxpayers to bear the full cost burden of development of the batteries.