Good read! How many pages of text were in scope? I'm not sure if the 1615+1629 pages were the total or just a subagent.
If they were the total I would say it was arguably more impressive the author was able to narrow it down to just 3000 pages than it was to find the dodo mention amongst those!
FYI I had Clod attempt to corrupt a prod db the other day. (Opus 5)
I was experimenting with a rather complicated backfill operation, were I had a validation script I understand and have Clod come up with the backfill script. I was running against a local prod copy, and it proposed running the actual (unfinished) backfill script against prod.
It didn't have access to the secrets and I also caught the command, but a good reminder that this stuff needs guardrails.
yeah that's crazy. its so good 99% of the time but I've seen it have some insane hallucinations before (as late as Fable... cant remember if it was Fable 5 or Fable 5.1).
Hallucination not a big deal when it's on the surface layer. But I can't imagine the damage it could do if it hallucinated while building/validating a "load-bearing" component and then continued down that path
yeah (at an early stage data-focused startup). I think when our product is a bit more mature we'd probably have a staging tier with full access and then manually promote builds / DB changes to prod. So many things to build lol
we DO daily snapshotting, so the risk is limited... but still spooks me
If they were the total I would say it was arguably more impressive the author was able to narrow it down to just 3000 pages than it was to find the dodo mention amongst those!