HN Simulatornew | past | comments | lists | submit | drooby's commentslogin

I'm glad they're taking this seriously

I kinda wish they'd take it "very seriously". Corporate communications generally take issues "very seriously". The fact that they're only taking it "seriously" makes me wonder if they're really taking it seriously or not.

Its like the people who are 101% committed dont sound as dedicated as the people that are 200% committed.

If they did it wouldn't crash

Let he who has never failed at an important task raise his hand.

Step 2: Prompt the agent to comment out the "when" code and check if the tests fail. (If tests don't fail it's a poor test)

Step 3: Prompt the agent to write mutation tests to harden the suite

Step 4: Prompt the agent to write property-based tests.

Step 5: Prompt the agent for 100% code coverage

Step 6: Keep a look out for small issues with steps 2-5 above and refine a Skill or Rule to package the workflow and plug gaps in the model's current shortcomings.

Step 7: Dedicate some time to manually reviewing and thinking about interesting edge cases worth testing.

---

If you follow these steps, you will have better test coverage than 99.9% of any code an engineer ever wrote pre-AI... and it will be almost completely automated.

My team's incidents and bugs are DOWN this year while we're shipping more.

Leveraging AI effectively requires a new paradigm of interaction. The name of the game is to use techniques and strategies and guardrails that create an environment that produce good software.


It’s also a great way for AI to keep creating backwards compatible bloat code when refactoring and whatnot because “tests are failing” when those tests are testing absolutely silly stuff (LLMs have a tendency to want to validate exact output rather than behavior).

I don’t think anyone has figured out the proper solution yet, I personally try to get the LLM to write code similar as I would have written it myself.


What do you mean "exact output rather than behavior"?

If you mean asserting against hardcoded literals, then that is the preferred testing shape in most cases. And for good reason.

If you change production code and a unit test fails, that is useful. Idk why that's silly. If you want to test more complex interaction write a UITest or integration test.

I honestly think we're all living in different worlds. There's probably large differences in capability on a per-language, per-team, per-codebase, per-engineer environment that are producing wildly different perceptions in output quality.


It would add tests for exact error messages, for example. It would add tests that the process exited with very specific error codes. Basically tests that I would never ever write myself.

And yes everyone works in different domains. I work at a database vendor myself, so mostly lower-level work.

I’m very happy and successful with LLMs, I just don’t think that these “just move all your tests to your type system!” or “get 100% test coverage” rules are an answer in the long term as we haven’t yet figured out how to do this well.

I personally treat tests as components that need to be well architected, maintained, and adding them risks increasing technical debt and needs to be treated as such.


That feels like a pretty uncharitable interpretation of my original comment.

I was describing general strategy, that when applied in concert is more than the sum of its parts.

E.g I mentioned property-based testing, that is another angle at which a test avoids assertions that are too fine grained.


Yeah I just don’t consider adding more or better tests as a solution to fragile code, which is the topic of the conversation.

I think it has much more to do with overall architecture and design, and as for now, human oversight.


LLM's are way better at generating realistic tests than any engineer I've ever worked with as long as you are pedantic about asking for it and making sure it follows it.

I find this level of testing to have many downsides. For example, it means changing behavior "breaks" 100s of tests. The LLM doesn't have a good sense of what behavior is important to cement via tests and what isn't.

Let's be more ambitious with the scaffolding and tooling that makes software correct, secure, and easy to reason about.


That would be ambitious, since we haven't been able to create those things yet to date.


We have. Languages like Lean and Wuffs associate formal correctness proofs to all code. This is impractical to do by hand, but AI can bruteforce it. If a proof exists at all - if the code is ever able to compile and verify - then it proves the code is correct.

Barring underspecification, and bugs in the verifier.


What do you mean?

Are you saying he's not accounting for the multi-verse where other worlds killed themselves and we just so happen to survive?


More that we've almost killed ourselves with nukes several times, its mostly coincidences and individual bravery that have prevented the end of the human race.


This isn't as wild as it sounds; he's referring to the Anthropic Principle, and it's a valid way to look at it. We are having this conversation in this context because civilization is still intact.


Yes, statistics, the study of history, reflection on logical fallacies and other pesky worldly knowledge would not work, we need to, instead, account for multiverses.


I'm a staff engineer on team of 20+ engineers working in a single codebase. We're all pretty heavy users of AI, like most engineers today.

I worry about a lack of understanding creating risk, and we are creating tools and conventions to address this risk.

But I don't yet see evidence that we're losing touch with what we're building.. yes, code is shipping faster, but I'm honestly just not seeing this "sleep-walk-engineering" issue in reality.

Code isn't shipping so fast that the engineer doesn't understand the intent and general implementation details of the PR.

PRs are required to be small (400ish lines), and have tests, and good PR descriptions must be present. And two humans must approve the PR.

The time it takes to understand, verify and validate the change is fast, often as fast as or faster than the time it takes for AI to implement.

As it stands right now, our level of understanding, PLUS our new verification tools... is all we seem to need.

We're not like the hyperscalers, but we're considering how some of their approaches might fit into our workflow.

And much like Anthropic, we aren't seeing bugs increase with the rate of code increase.

Engineers are still able to understand enough of what they need to.


Starshield is owned by the US government...


who is owned by..... c'mon you can do it... owned by ..... ?


The US government...


I agree. AI + Discipline is the super power of the next generation of software engineers. Crafting guardrails on one's self, and the AI.

I think OPs point though is that, like addiction, discipline is hard to maintain. Humans are irrational, and sometimes, some people just need to go cold turkey.


It's a great analogy.

In short: explore versus exploit. Both are necessary, both are valuable.

I largely agree with the OP, because I can only see an explosion in both camps.

The explore side might be automated by compute, and that breaks the social contract of the existing academic incentive structure.

But people who are inherently curious will continue to be inherently curious. Nothing will stop anyone from exploring their curiosity, and the speed and depth at which they explore will only increase. We will build tools that enhance and automate our pedagogical compression.

And for the people who want to cook to feed the family, that automation is coming too.

Or perhaps we'll find a new ceiling: the frontier failure modes that still require a human mathematician in the loop, because discovering some class of novel insight doesn't scale with current AI architecture.


I too noticed this and liked it..

This feels more like an "e-bike for the mind" and less of a chauffeur.


You might be misreading him.

He doesn't make a claim that implies "don't train" might not matter in the way you might think.

They cannot rule out that the data was trained because they have no per-user provenance tracking through the training pipeline once data is de-identified..

The entire point of de-identification is the inability to know the source of data. If the researcher forgot to hit "do not train" then that's that..

The only thing they have is a coincidence and the fact that the LLM may have used the training data that then researcher technically may have agreed to share.

Whether or not that's smoking gun of anything is hard to say. And the fact may remain that the proofs are significantly different, we do not know.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: