Now that's a textbook strawman

It's literally the question I presented in the comment they responded to with the answer "no".

did you actually verify that it's output in those scenarios is good ? in my experience opus has been a disappointment and constantly trailing behind actually solving hard problems versus the OpenAI models. I'll say that both have terrible writing style though.

> did you actually verify that it's output in those scenarios is good ?

Yes, in the sense that it reproduced results in the paper or known solutions obtained by other methods. In fact, Opus is very good at checking it's own work in my experience.

But is it byte for byte identical ?

You're right to push back.

Your experience sounds really interesting, can you tell us more ?

I mean sure, you can add a third, and a fourth and a fifth one if ur ok with the added cost, latency and it actually helps. Redundancy is a core concept in software and CS and at the heart of making many systems, complex or otherwise, reliable.

What you want can be accomplished with an extra doc, call it the Log, where the llm appends things tried, lessons learned, failed experiments etc.. while leaving the comments as terse accurate snapshot of the current state. I've been using this log pattern and it works well.

It was communicated fine, can you be more specific ?

No, it wasn't. For example: https://news.ycombinator.com/item?id=48019226

you really went digging with that 3 months old hn comment. What's wrong with that comment btw? Are you saying he was lying and they did in fact commit to the rewrite at that point?

I didn't dig. It was prominent at the time this occurred, because like I said, it wasn't communicated at all until they were already in the middle of it.

And yes, I think he was not genuine there and that they had indeed fully committed to the rewrite prior to that point and comment.

If you're not aware of any of this, how are you arguing it was communicated clearly?

it's not the way to go if you hit the limits very quickly and have to waste immense time migrating.

From my experience the limits are way higher than people think.

from my experience they're actually much lower than people think, especially when using Postgres for things it's not well-suited for like queuing, and the problem is you rarely can just throw more hardware at it to solve the problem and it's really not a good place to be.

Thw bottleneck become compiling and running the code every iteration so it wont be seconds

JIT compilers largely solve that. The real question in my mind is whether fast LLMs bias stuff towards languages that either don't have a compile/typecheck phase at all or have one that's very fast (e.g. Java).

This has not been my experience at all with gpt sol on ultra/max. Some concurrent code is just really really hard to write and even in those cases LLMs can help, the barrier for extensive thorough testing and writing your own custom simulator, liveness and correctness tests is simply so much lower now.