No. In a large enough state space, you can of course easily find a state that has never existed before. For example, if you pick a 1024-bit integer uniformly at random, it will certainly be different from any previous 1024-bit integer. However, it's much more difficult to find a new integer that cannot be assembled out of bits taken from previous integers. Composition from a handful of base atoms can cover very large state spaces, and unbounded composition can cover infinite state spaces. Which is why only being able to combine things that already exist is not much of a limitation.
They are poor at establishing the state of a system and applying basic logic to it. Where we make a minimal model of the state in our heads, they make more and more words in the hope that some of them will match the training space.
I am working on a legacy OSGi/eclipse application. I have a problem where the OSGi bundle is not activated at the point where some code in the bundle is executing. This makes no sense to me. And it makes no sense to the AI either. The AI says: "Since your class is in your own plugin bundle, that bundle must be active for your code to be running at all".
After some more testing we figure out that actually this its possible for eclipse extension points to instantiate objects via reflection which bypasses bundle activation. So now we have a new fact:
-- It's possible for code in a bundle to execute without the bundle being activated.
We also have a new state (another kind of fact)
-- In this system this possibility has occurred. The state of this particular system is that the bundle is not activated.
So then we get around trying to debug how we got here and after trying a few different things the AI still can't believe that this could be possible and it says:
"which is definitely ACTIVE (since your code is running from it),"
Here it is contradicting the knowledge of the state of this system (the bundle is not active) and it is contradicting the fact that was already established where we now know that it's possible for this to be the case.
Then I tell it that the state of the bundle is RESOLVED. As soon as I use this word (first time in the conversation) it immediately gets out of the rabbit hole we were going down and gives me the correct fix.
For a human maintaining this state and logic takes no effort or memory. But it seems that the LLM can't really do this. It is completely biased by the training data. The funniest thing is that AI "reasoning" is the opposite of reasoning. Instead of reducing the problem down to a few bits of information and logic it actually increases information. It makes more words in the hope that those will find a match in the training space.
It is like when you have to deal with the person who says: "When this situation occurs, I always do this". It's just that the LLM has seen so many situations that it is usually correct. In real life these people can be really frustrating to deal with when the situation changes and they are exactly the wrong kinds of people to do research or even just to make decisions in a system that has changing state.
Not sure why you didn't get more engagement, this is the best take in this thread. LLMs are super impressive, but they differ qualitatively from biological intelligence in ways that make it unlikely for them to replace human labor in places where true human adaptability (and not just language generation) is required.
But working with concrete is nothing like working with clay. It's not iterative at all. The mold has to be planned out and set up precisely ahead of time or else everything will be a mess and you are not going to be able to correct it. Working with concrete is really working with timber (or whatever material you are making the mold of)
Personally I like it more with the labels. Reminds me of web design where sometimes you have to add some unnecessary text or icons or else it looks too bare.
> The real world is robust. It's okay for money handling code to have bugs.
This is true. The world will go on. But your business will lose more money than it gained by not paying attention to the code. It will lose money because it will have to compensate customers or lose customers. That's how the world maintains it's robustness.
> You can judge ideas independently of where they came from (and you should).
You can but in practice nobody does this because they don't have time. For example, once you have investigated ideas from a crank a couple of times - you don't do it again. Life is too short.
> Anyone who thinks like you do will get voted out with extreme prejudice.
> I know. Doesn't change the fact he said it and went viral on social media
These are post-truth comments. You don't care about how things really work, you just care about winning over the other post-truth incompetent. It doesn't matter if you are right or left, this stuff is what is sinking the world.
I do care about how things work. It's a fact that this viral stuff could sink him when election begins in a few hours. It's also a fact that these leftists totally hate the "fascist" police and want to destroy it. My honest opinion is he merely revealed his true colors in public, but if you think otherwise I'll respect it.
By how things work, I mean the truth of how things work. The truth is that he didn't mean to abolish the police. But you repeated that as if it was true. You spread the "viral stuff" and then in the next comment you say that it's a "fact" that the viral stuff could sink him. Your words are all over the place - post truth. The reason this is a problem is that the truth is out there and it bites hard when the post truth people govern based on fairy tales.
> That probably means a test suite that is at least twice the size of your code base.
If you've got a test suite that's twice the size of the code base then those tests are almost certainly tied to implementation details and will make the code a lot less like clay.
Is this always true? I'm honestly not sure. For unit tests I completely agree with you, for integration tests I'm not so sure—validating permutations does not necessarily couple you to code, and may in fact be good discipline for user expectations.
Where I work we make real hardware and software that are used by real people doing real jobs every day. We use AI but in that environment someone who made Jackobriens comment would be seen as an idiot, someone who doesn't have a clue how to build software with real users that has to work with few bugs and be maintainable.
But there are so many people who make statements like his that maybe we should assume that some of them are not idiots or bots. If he's not an idiot it means that he's working in an environment where that kind of statement is paying off. So it could be, for example, that he doesn't really have profitable customers. His main 'customers' are investors who are clueless, or who know that they are clueless but don't care because they are going to move it on to a greater fool anyway. Or that because there aren't any customers, the only thing they can measure is 'productivity' in terms of 'number of unused, unprofitable and unsustainable features'
I work with AI as well and it can help, but it can also hurt. I think any objective person agrees with that.
There is another option to consider. What if they heard someone else say it and they believed it.
Now they go around, like many insecure people do, shouting about their unfounded take. Begging the universe to correct them or give them a good reason why they arent informed.
Then what usually happens with people who suffer from this type of short ordered thinking is, they realize they can't walk it back without social consequences because of the shouting they have done. So they just double down on it and find ways to live with the cognitive dissonance. Anthropic, openai, etc make billions off of people like this.
Because they propagate the BS advertising to people who don't know better, and then spend more money to try to make it real.
reply