> 2. A one-shotted POC in the span of an hour with $4 of tokens that is clearly wrong in 20 different ways?
Definitely this. I've had plenty of useless meetings with no helpful outcomes that lasted an hour with 6 or more people in them, and a few of them will have had more than $4 spent on snacks, let alone everyone's wages.
That said, poor communication will always get you no matter what: had one day spent going back and forth with a client, they kept saying they wanted a button "wider" each time around, I ended up making it full screen width and they still said "wider", asked them to draw a picture and they finally said "wider vertically". That may or may not have been faster with an LLM, depending on if the LLM ever thought to ask the client to draw what they meant, but fundamentally would have been exactly the same problem.
It goes beyond this, the job is to understand the underlying need. When a clients asks for a button to be bigger, what's in their head is they fear it's not visible enough. Many times the client was happy when I implemented something unrelated but that solved the issue, like changing the colours to increase the contrast between the background and the button, adding space around, etc. so that it pops out more.
> 2. A one-shotted POC in the span of an hour with $4 of tokens that is clearly wrong in 20 different ways?
A meeting between all the stakeholders to agree on requirements easily runs $1,000/hour. Taking your $4 functional PoC into that meeting is going to save so much time...
> A meeting between all the stakeholders to agree on requirements easily runs $1,000/hour. Taking your $4 functional PoC into that meeting is going to save so much time..
It is though! Because previously your have the meeting last 4x as long to end up with a list of notes that you already knew.
Now you can generate the PoC, showcase it during the meeting and leave with actionable changes in project direction.
I had to re-read parent's comment over and over, because I simply was not sure if he was being ironic, facetious, or just had trouble expressing his point.. and, honestly, I am still not sure.
Not everyone in meeting will be the same person you are. Visuals help. Prototypes help. They certainly can save us from vague ideas and unclear language, because at least someone can point to the big button at the bottom and say:
"Can I have this in cornflower blue?"
It is useful, simply because it makes it more concrete.
Will it change? Is the pope catholic?
I was being entirely serious. Even at $400 of tokens it would probably still be worth it to take a functioning prototype into the discussion. At $4 it is a steal
I used to spend an hour throwing a PoC together with PHP (or sometimes in MS Access) because hey you can throw shit together real fast, and because I used either a shitty language or a toy tool they were treated as throwaway.
Now LLMs make that easier to do with a larger scope.
Draw outlines, or use graphics, wire up clickable areas, and interaction flows, and you’ve got a presentable “live” demo that doesn’t run afoul of issues presenting demo sites to non-techies (placeholder text, ugly buttons, test domains, colours, etc).
Obviously "a blank sheet of paper" here is meaning thinking about the problem deeply. I'm surprised how dismissive people are to using brainpower whatsoever. I get that juxtaposing paper to LLMs might have triggered some people though.
As a counterpoint, bringing a one-shotted POC without thought may actually do have a shock and awe sort of effect, dazzling the stakeholders to go down the wrong path since the AI prototype was so easy.
Long ago I have read somewhere a good method to prevent this:
When doing a proof of concept, always use ugly black-and-white buttons and icons. That makes it obvious even to the non-technical people that it is not finished yet, and still requires some work to do.
Sure, the concept of a _Vomit Draft_ is not new, but the ability to come up with one quickly is.
It plays into the generalisation that people aren't good about saying what is the right way to do something, but much more open to saying what something is doing the wrong way.
In a previous job we used to do prototypes in languages like Perl because we couldn't ship them and knocking something non-production ready was easy. We were able to quickly come up with the PoC, iterate on it, and then plan and write the whole thing in $LANGUAGE_OF_CHOICE.
Using the "let run multiple agents run amok" approach you surely get a lot of "i don't like that part"-feature requests, yes.
In my opinion, a good engineer is not good because he understands every single obscure technical detail, but because he can build solutions that the people using his technology actually want. The list of things they don't want on the other hand is endless.
Personally at some points (like frontend design) I just ask the agent for 16 options with a convenient comparison system and just yeet designs until I’m happy, guiding it along in a very uncommitted way. “Quantity has its own quality”
A one-shotted POC can also put you straight into the very wrong direction, with a very high degree of certainty.
I also prefer doing a POC, but it's not without risk, especially if your stakeholders are just a bit too eager. Another problem I've seen is that a POC can hide complexity, so a lot of weight is on you as the author to do the due diligence before and bring that along.
This is an often underrated problem. I used to work at a company where we would hold demos every other friday, with drinks and nibbles. We would put a lot of thoughts into what we were going to present, and were really careful about the messaging, because execs and sales were watching.
The last thing we wanted was for an "Isn't it neat?!" hacked-together demo to be mistaken for something shippable, if workable at all.
What's even more surprising is that it can be given a grafana API key and reason about bugs from the hints of patterns in the logs even with pretty sketchy and inconsistent logging in places.
It can then raise, investigate and resolve those problems while also patching up and rationalising logging to improve future work.
The AI deniers have their heads in the sands at this point and are demonstrating they simply haven't tried a modern agent. ( Or perhaps are forced onto copilot? )
Again, no, "what to build" is the humans input, but it's not just grunt work that modern AI tooling is good at... Architecture and what to avoid are also strengths of AI.
> Architecture and what to avoid are also strengths of AI.
Absolutely not at all. Every single day that I work with Claude, there is abundant evidence that AI sucks at these. Ask me for examples if you need them.
The first links shows a basic failure of reasoning, which would have gone unnoticed if I was just vibe-coding.
The second link shows weak architectural sense and an inability to see the big picture.
The last link in particular is telling: the agents worked with me (over weeks to months) to implement a db abstraction layer without ever bringing up the fact that as it stands, that would have never worked with any indexes at all. I did not constrain them to my choices, but they never stopped me with that design. It took my eagle-eyed attention to their offhand comments to notice this problem, which then required a lot of rework to fix.
Unfortunately, all three comments have apparently been flagged by somebody for some reason and I can't read them. This is my least favorite aspect of hacker news. It's exhausting, honestly.
Like a sibling comment replied, you can set "showdead" to "yes" on your profile and click update" so see flagged comments. Not sure why they were flagged.
I'm writing a unique MUD, and It was surprising to me when I posted some bug reports I'd made and it identified things I didn't complain about. "User had trouble with X" and then suggested mitigations for those problems. I ended up rejecting a lot of them because I knew I had different mitigations coming eventually, or it was working that way for a reason...
But it was pretty eye-opening to see how good it was at watching a user struggle and identifying possible fixes.
> The AI deniers have their heads in the sands at this point and are demonstrating they simply haven't tried a modern agent. ( Or perhaps are forced onto copilot? )
Could be either, but also the free version of ChatGPT seems rather braindead compared to the paid version, or to the free version of Claude.
It's true. It says a bunch about their competitors where Nvidia isn't at their a game and have been offering 8Gb cards for a solid decade and nobody out competed them. But also gamers pride themselves on how cheap they are and how they can get a deal, it's not surprising that companies simply just give Nvidia more cash.
I fully believe the only sustainable future is that hot patching will need to itself become fully automated. Hot patch now with a minimal allowed amount of line changes to head off the AIs, review it in office hours for auditing and the team is actually awake.
My second belief is that environment variables as a development concept will need to be retired. If it's something that causes a security breach if it by its mere existence on a public git repo, it is information that by design cannot ever pass to a filesystem out of a chain of trust, not by mere development conventions people choose to adhere to.
Due to the small nature of hot patching (eg. bump this dependency, rotate these keys, turn off that config, follow the runbook) it will unlikely need to be a frontier model and is likely able to be done LocalLLM/Jev style decision making rather than having to hand all the keys to frontier model operators.
The barbarians are at the gates and they're interested in laundering your compute and network for other attacks, scams, and things like transferring CSAM.
How much time do you need to audit? You might have an hour - for now.
For the record, there was a bunch of non-functional vibe coded fixes flying around during this PHP bug. If you were just running the first thing AI made because you were worried about "not reacting," you would likely end up running a crap script.
AI is quite good at finding vulnerabilities, but it's been rough for patching them. The old adage "defenders have to be right every time, attackers only have to be right once" applies to AI security patches.
Great, but the fact remains that easy hot patching which skips normal review is going to open you up to many of the same liabilities you're trying to minimize by introducing it. The target just shifts, and people will attack that infrastructure.
The actual solution largely involves the same technologies but in reverse: leveraging AI for live security monitoring, better correctness validation and security testing before code goes live, etc. And of course more pressure to use battle-tested widely-adopted libraries and systems, even though of course that comes with its own tradeoffs.
Importantly, as an engineer, when someone tells you something has liabilities or tradeoffs, the response is not "Everything is a liability, existing is a liability"; that is unprofessional. The appropriate response is to engage with them on these tradeoffs and working through the best ones, because that's your job. Otherwise defer to security engineers who are ready to back up their arguments. It could be as simple as following up with, "But do you think those liabilities outweigh the benefits of hot patching without proper review? What are some of the main liabilities you see, and what methods/technologies do you think properly address them?"
With AI, in the long term open source code might actually become more secure than obfuscated closed source, since it's had so many more (AI) eyes looking at it.
Not advocating this, just more addressing the mindset on it, but in the game of office politics people don't mention concerns ahead of time because they don't want to give the opposing view point opportunity to research how to defeat their argument in a meeting, not that they "can't" give an intelligent response. They're choosing not to in their own self interest.
"The enemy must not know where I intend to give battle; for if he does not know where I intend to give battle, he must be prepared in a great many places. And when he prepares in a great many places, those I have to fight in any one-given place will be but few."
You generally also get paid with ~10% of everyone else, regardless if you get 3x more done or 3x less done.
So, if like most people, your working to live instead living to work - your employer is doing a terrible job incentivizing you to do more than the bare minimum.
We all know you are far better off doing almost nothing and then all your time trying to get promoted for it, than trying to get promoted by actually doing a good job and thinking you'll get promoted based on merit.
Anyone who would advocate for this strategy against any SME is a moron anyway.
It's like watching someone who's expecting a knife fight walk in with a concealed handgun only to be immediately confronted with a tank.
The email is usually an attempt to not waste time in the meeting and get shit done. Leadership always favors those who can make it so. Anyone who stalls for time or hides information will always look bad. I don't get why people so desperately try anyway.
Most of these strange gambits in office politics are a losing game for the inept and inexperienced. One can only wonder why we keep pretending like there even is such a "game" of office politics.
id place the reason that its more work to identify and raise those concerns ahead of time, since if they are correct, you did all that work coming up with the happy path and the issues ahead of time, vs if the issues come up later, you can focus on and fix them as they come up.
I mean, doesn't this exact statement also apply to the adoption of AI and all the winters that have come before this wave?
Also as someone doing my own implementation at the moment, I expect that there is a big relational database backing the MELT store, just the graph aspect is functionally a big RAG interface on top of it acting as a search cache, not itself the data store.
My local knowledge graph dropped my token usage spend on reads from being 30% of total spend to 2% and I haven't suffered a regex false positive deleting 300 lines of code randomly since. As of this weekend it also now feeds my QA metrics, replacing what was multiple tool calls taking multiple seconds down to about 25ms because it's a single cached graph traversal now rather than multiple tools traversing the same ground repeatedly.
The AAA industry is broken. But there is nothing to repair. The indie industry will continue telling new stories without it. Every gamer will have no shortage of finding games to play. The only problem is if your goal is to employ hundreds or thousands of people and extract money from your audience. If you're in the business of making games and entertaining people, you're already just fine.
> The indie industry will continue telling new stories without it
The problem with this version of events, is that ever since the floodgates opened on steam, individual indies aren't really making enough money to fund themselves. It's not really sustainable to have an industry where only folks who are already independently wealthy can afford to be game developers-
Most of the indies who do make money these days are in a much more AA-like model where they work with a traditional publisher, who funds development of the next game up-front.
This. We are currently in the new golden era of Indie games or even gaming in general.
You can’t plan success anymore with marketing and market research.
But you can succeed spectacularly with genuinely good games like Balatro or Slay the Spire.
1. A blank sheet of paper
2. A one-shotted POC in the span of an hour with $4 of tokens that is clearly wrong in 20 different ways?
reply