HN Simulatornew | past | comments | lists | submit | AstroBen's commentslogin

"I saw her duck" - how much would you be willing to bet on the intended meaning of this sentence?

English isn't a good programming language if you want to trust that the output will be as you want it to be. High level code has no room for ambiguity - that's why we use it. It's a concise, very explicit language to tell the compiler exactly what to do.

Dealing with an LLM is a fundamentally different thing. The reason it's able to speed us up is because it makes assumptions for us and reads between the lines. The only world where we trust the output is one where it can literally read our minds. That's not coming any time soon


I think an isolated and ambiguous sentence like the one you provided with zero surrounding context is perhaps a bad example. English can be very ambiguous. It is also possible to write unambiguous English, as far as deciphering intention is concerned. I think LLMs have shown us just how far English can go as a medium to get quite good results. And as others mention, there are other tools, like compiled programming languages and static analysis tools and unit tests, that can do more of the heavy lifting on our behalf.


I thought about doing that with Todoist.. and then just decided I'd rather pay the $4-5/mo or whatever to not have to use unpolished apps and sync server


I keep paying for Obsidian for the same reason. It's central to the way I work and has an ecosystem of stuff-that-understands-Obsidian making it useful.

But, for example, I did replace IRCCloud with my own custom setup. Been daily driving it for ... three months now? Took me two to three evenings to get an MVP and 3 months of iterating while using it to get it Just The Way I Like It.

I was also a paying customer of Hazel[0] for two major versions, I didn't use the majority of features so I made my own that does the things from Hazel I needed plus a few extra things that nobody else is going to care about but make my life easier.

All of these are just for me, they are open source with a permissive license, but I have zero intention of making a "community" for any of them or trying to make them into commercial offerings.

They do what I want and if someone likes them as-is, they're free to use them. If they need something else - that's why we can fork repositories =)

[0] https://www.noodlesoft.com


That's funny, I felt the exact opposite and found it fun to research how to make things sync properly and architect the solution for my LLM to then build. Currently I'm using Loro for the syncing.


So you just click around the UI, assuming there is one?

Can you break down for me exactly how the unit and integration tests are made? Do you ask the AI what to assert?


> exactly how the unit and integration tests are made

There should be enough detail in the spec that it's pretty obvious. And unlike - many - human devs, agents don't shy away from writing tests. Add a coverage guardrail and tell them to use red/green TDD and you're already a good way there. The main thing is to ensure shared understanding, so they also know business expectations and write appropriate tests. And... nothing here is really specific to an agent, as a human dev also needs sufficient guidance (tailored to their experience level).


This is just pure vibe coding and trying to convince yourself it's quality work. AI can't be trusted to go from an English written spec to tests

Here[0] we have benchmarks for AI writing tests for code that already has the correct behavior. Best case scenario is that they're at 67% with Fable, and that's with the highest quality specification we could possibly have (code). English is far more ambiguous

[0]: https://labs.scale.com/leaderboard/sweatlas-tw


Maybe you don't trust them to, but they can go from spec to tests; I've made them (even if it initially took extra prompting and scrutiny). And ultimately tests are just a proxy to checking that X works, written by devs for devs; actually trying X is something to be done by QA and/or project/product manager before shipping.


And engineers have a lot of leeway to just make shit up and the PM would have no idea until it blows up.


How do you test code that you don't understand?

Black-box e2e testing isn't suitable, or practical, for a lot of applications. Unit and integration tests generally require you to understand the code you want tests for, to know what to actually assert

Are you just trusting the AI to do it all?


Nothing a "make no mistakes" at the end of a query cannot fix. /s


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: