In Europe at least newer Tesla charging stations are drive through charging stations in pairs. Unless it is crowded, the trailer is parked in the charging spot behind you. Only if it is crowded do you have to disconnect.
Pulling a trailer with an EV is very common around here (Norway).
That makes it impossible to steal the key if you MITM, but there is still the same chance of guessing it. (Assuming password has as many bits of pure randomness, such as machine generated password.)
But the machine has the benefit of being able to look inward at its state honestly, we mostly do not.
Sure about that? I thought that what LLMs know about themselves is from humans writing about LLMs?
Certainly the weights that make up the LLMs are not accessible for the LLMs to read. Even if they could interpret them in a meaningful way, which, likely not.
LLMs are not the machines I am referring to, machine sentience, if achieved, will require a "state" and the ability to analyze, act on, and update it. LLMs are a small piece of what that machinery could look like but they are not really complex enough to get where we want to go on their own.
Another reason the big labs have not even talked about this lately because its obvious throwing more compute at LLMs wont go anywhere, the smaller silent players who are off the radar at the moment are having some interesting moments tho.
> LLMs are not the machines I am referring to, machine sentience, if achieved, will require a "state" and the ability to analyze, act on, and update it. LLMs are a small piece of what that machinery could look like but they are not really complex enough to get where we want to go on their own.
Yes. Which is why there are "agentic" systems where LLMs can talk to each other and to various Internet assets. They can also keep state of their own.
The next step is multi-agent systems which self-organize into something that looks rather like a company. Those are the ones that run all night, expensively, and prove hard theorems or find bugs in major software.
I honestly do not know if they exist yet or not. I'm not sure yet how we would be able to tell. I do know that the tools most people are using are magnitudes less complex than what some researcher are using, and even some of them can't answer that question.
"Writing code is probably something LLMs have excelled at the most. But how much of their ability to write code is due to their understanding of it vs. just them recalling code snippets which were present in their training dataset or their context window?"
This makes me think mostly about how humans work.
Programming is to humans like speaking a language. You have ideas for what to achieve and express, and if you are a proficient programmer then outputting those thoughts in a programming language is second nature.
Programming (like most things humans learn) takes practice. It is NOT enough to simply read a specification and then you know how to code (except for transferring skills between similar languages). You repeat, make mistakes, and learn from them.
I would be tempted to say that humans definitely "repeat snippets from their training dataset". We mainly recall and express from our training/experiences. Creativity is mostly recombination of things already seen.
> "...how much of their ability to write code is due to their understanding of it vs. just them recalling code snippets which were present in their training dataset or their context window?"
I don't understand how people who've actually used a modern coding agent, or know anything about them, can still be asking this question.
The most basic feature of LLMs is that they predict tokens based on their training data and input (context window.) But "predicting tokens" is not the same thing as copying snippets. None of this would work nearly as well if what they were doing was copying snippets.
And even if this were a question worth studying, the article's approach doesn't in any way help answer the question, because apparently the goal wasn't in fact to answer that question but rather to create a language that would be intrinsically difficult for LLMs. It'd also be pretty difficult for humans. That's what obfuscation does.
This would be better if it were just framed as a fun exercise, like any other obfuscated language, not as having some real purpose.
I’m getting a little tired of these pedantic games of gotcha undermining any attempts at serious engagement with the subject by people who say things that are even slightly skeptical of the capabilities of LLMs.
Back in college we called it “philosopher sniping”, and it wasn’t a positive term. It was used to denote the general bad faith vibes that people got from folks who would force everyone trying to participate in a serious discussion to be grindingly precise in their choice of words, because any time someone used any sort of verbal shorthand they would interpret it in the least charitable way possible and use that as a pretense to start belittling and mansplaining.
> any attempts at serious engagement with the subject
If you believe the OP article fits that description, we have nothing further to discuss.
> force everyone trying to participate in a serious discussion to be grindingly precise in their choice of words
This is no way, shape, or form about "word choice". The quote I posted reflects an objectively incorrect understanding of what LLMs do.
> ...start belittling and mansplaining.
Pointing out objectively true facts is not "mansplaining".
Do you not think "serious engagement with the subject" requires at least some familiarity with the subject? That's not in evidence here or in your comments. What sort of serious engagement do you believe you're achieving by coming at this from a position of near-total ignorance?
No satire intended. I'm just extremely sick of seeing people - on a site called "Hacker News", no less - react to the most amazing technology humans have ever invented with, basically, tribal fear and wilful ignorance. (They could learn about it if they wanted to.) It's just sad. I suppose they're just not capable of anything better.
And to talk about "serious engagement with the subject" while doing that is just salt in the wound. Yes, let's fucking engage seriously with the subject! Where is that happening, though?
And claiming that it is the "most amazing technology humans have ever invented" is telling. Also claiming that a contrarian opinion is immediately rooted in tribal fear and ignorance. Ever bothered to stop and think that there might be good reasons to reject LLMs as a technology the way it is currently being used or is YOUR view of that maybe wilful ignorance and fueled by tribal fear?
But the response you gave was just perfectly playing exactly into the point the parent comment was making.
> there might be good reasons to reject LLMs as a technology the way it is currently being used
Yes, absolutely. But denying their obvious capabilities is not a very effective way to do that.
I don't see much thought-through analysis from the anti-AI crowd. That seems to have a lot to do with the fact that they're not actually very familiar with the technology, from either a practical or theoretical perspective.
This means they rely on what they assume will be an effective tactic, trying to claim that the machines that are smarter than they are in many respects are somehow... not. Which puts them in very obvious conflict with reality. Counterfactuals are not a great basis for a protest movement.
As for the parent comment, it was talking about "pedantic games of gotcha." I reject the idea that that's what my comment was doing. The idea that LLMs are copy-pasting snippets is ludicrous delusion. But maybe you don't realize that either?
Also, point of order: "The satire was apparently lost on you" makes no sense, since I wrote the comment and it wasn't intended as satire.
> Yes, absolutely. But denying their obvious capabilities is not a very effective way to do that.
> I don't see much thought-through analysis from the anti-AI crowd. That seems to have a lot to do with the fact that they're not actually very familiar with the technology, from either a practical or theoretical perspective.
So you imply that anyone anti-AI is not actually familiar with the technology? So only a true Scotsman is able to appreciate the beauty of AI?
> the machines that are smarter than they are
Depends on your definition of smart.
> Also, point of order: "The satire was apparently lost on you" makes no sense, since I wrote the comment and it wasn't intended as satire.
I'll just let that stand for itself and encourage you to read the thread again if it's not beyond your context window.
I honestly don’t think you’re engaging in good faith, so I won’t waste much of my time, but here’s a common thought, given to you by someone very familiar with the technology:
LLMs and AI derivates are interesting technology with many possible applications. However their externalities, including but not limited to the further consolidation of power in the hands of a few, rising inequality, increased propaganda, spam, scams, surveillance, uses in war, loss of critical thought and skills, cementing loneliness and driving suicide, devaluing of art, consumption of resources, increase in tech prices, etc, etc, etc, etc aren’t worth it.
That’s just it. It isn’t worth it. LLMs are interesting, but not that interesting, and that should be clear to anyone with the ability to have some empathy and look beyond their own nose.
> What does this have to do with the obfuscated language in the OP?
It has to do with your post that I replied to. I’ll remind you of your own words:
> I don't see much thought-through analysis from the anti-AI crowd. That seems to have a lot to do with the fact that they're not actually very familiar with the technology, from either a practical or theoretical perspective.
You made a general comment, and that’s what I replied to. You then made a more targeted comment (“As for the parent comment”), which I did not reply to.
> Is it your goal to derail every post related to AI like this?
What an absurd accusation. Indeed reinforcing you’re not arguing in good faith and this is a waste of time.
> I'm just extremely sick of seeing people - on a site called "Hacker News", no less - react to the most amazing technology humans have ever invented with, basically, tribal fear and wilful ignorance.
Personally, I’m extremely sick of seeing people—especially on this website—spewing that tired fallacious talking point.
If you believe LLMs to be “the most amazing technology humans have ever invented”, I think you lack imagination. Medicine, agriculture, computers themselves… There are entire fields of inventions which are more amazing and important than LLMs. You’re just used to them so pay them no mind.
The fact you think opposing arguments are “tribal fear and wilful ignorance” shows you pay no attention to them and dismiss them without engaging in good faith.
All those technologies you mention were created by intelligences. None of them are intelligences themselves. That’s why I said what I did.
Also, I didn’t say all opposing arguments fit the description I gave. “Good faith” is not sufficient here - you need to engage with intelligence and reading comprehension, as well.
But this is another trend that’s getting annoying: accusing people with a nuanced - not necessarily even negative, just not 100% positive - opinion of the tech of being a hater.
I would submit that your accusation of willful thinking smells like a case of projection.
And before we go there, no, I don’t like it when people who don’t like the tech start calling everyone who hates it less than they do a booster, either.
> Pointing out objectively true facts is not "mansplaining".
Different axis entirely, actually, but also tends towards being about objectively true facts. Mansplaining is a sexist term that's entirely about correcting a misconception, but only applies when talking to a woman. "A condescending explanation" is a non-sexist one-to-one replacement for it.
Yeah, no. LLMs do fantastic on problems they can copy out of their training set, but they fall apart on actually novel tasks. That's the entire reason we have to keep inventing new benchmarks, and why scores on new benchmarks are so low.
LLMs copy paste from their training set. That's plainly obvious when you try to do real work with them.
I don't think this is entirely true. It feels almost like a no true Scotsman: if even parts of a task can be related to anything in the training data, then clearly it's not novel enough for the sake of this argument.
It feels like language models are becoming more capable of picking novel problems apart, identifying strategies for the pieces, and then executing the pieces to come up with a complete solution. This is part of what makes CoT reasoning pay off so well.
I am interested to see what comes after CoT reasoning (perhaps the latent-space reasoning [0] that recently hit the front page?), and how it'll affect which problems can be effectively solved with a model. But I feel like it's not very useful to say models completely fall apart on novel tasks.
Maybe they fall apart when the solution includes completely novel strategies, but sometimes models engineer new strategies too, so even that's not completely off the table.
But then again, you could still say that if it's possible to just engineer a solution out of the blue by recycling patterns from the training data, was the problem ever even novel in the first place?
Also consider that models don't only directly learn patterns from the training data. There's a limited degree of generalization that enables the model to reproduce the patterns of the training data, but not necessarily only the patterns of the training data. The way loss is modeled in training is designed to allow the model to plausibly output the training data, not to ensure the model outputs nothing else.
That's a very polite way to say that it's complete bullshit. You can't really have a discussion with people who aren't engaging with what the technology actually does, but rather with their biased, quasi-religious belief about what it can't possibly do - namely, compete with humans on intellectual tasks. When the reality is that we're all outclassed by these models, in many respects.
> That's a very polite way to say that it's complete bullshit.
It's not the worst rule of thumb I've seen, and it does explain some things. It's just not entirely accurate, as I said. There's some nuance it doesn't capture, but it does capture some of the picture.
"Complete bullshit" would be more than a somewhat-accurate, yet incomplete understanding to me. It would be... well, complete bullshit. And I would probably call it such.
LLMs do fantastic on problems they can copy out of their training set, but they fall apart on actually novel tasks.
I mean, I give Claude what I consider novel problem solving work all the time.
You can make the statement true if you raise the bar for "novel task" to a point where most humans also can't do them. In these threads the goal posts for words like "understanding", "creative", "novel" etc gets moved for LLMs to way past their previous definitions, in fact way past a large % of the worlds human software developers.
To just take one example: Ask for a security review and you get back a list of all the things you would have found yourself after some days of pouring over the code (if you are senior developer well versed in security issues), plus the ones you would have overlooked, but in 15 minutes instead of in several days.
Before LLMs came along this would obviously have been "novel work". A shining example of a human using their intellect! But now that LLMs can do it it is just "copy and paste work"...
Describe the language FRACTRAN.
Write a FRACTRAN program that takes 2^x and finishes with the value of in the 3^y of y = x^3.
4 (2^2) -> 6561 (3^8). Other values may be in other power registers.
You can paste that into https://tjwei.github.io/Fractran/ and run it. Put the input of 2^3 and I recommend a 1000x speedup. It's output was 3^27
---
One can certainly go adversarial with their tasks... though I know junior programmers that fall apart on things they can't look up on Stack Overflow either.
For tasks that I've worked with it on, I've been reasonably impressed by it. Though I'll also note that a lot of the work that we do isn't new.
There are register machines out there. When I search google for "register machine cube of number" it comes up with one. The question of how do you apply the register machine model for that (if you're claiming that is in its training data rather than searching for it like I did) and then compile that into FRACTRAN is about chaining together tools reasonably... it appears to have written a compiler in Python for a model register machine to FRACTRAN.
Lets ask a different question... if I gave you the task of writing that program in FRACTRAN, would you attempt to solve it in the same way?
I had conversations in Morse code with like gpt 3 or 4. I guarantee there were no snippets of what we were chatting about, in morse code, in its training set.
LLMs are clearly able to apply seemingly arbitrary transformations to symbol streams in ways that generalize well beyond the training set. It's getting into metaphysics whether that is "novel" but it's definitely not "copy paste from the training set"
This post is literally about a language the author invented to be hard for LLMs to parse and isn't in their training set and yet Opus 5 scored 68.8% on it.
You may agree with the author (that it doesn't mean it understands the language) but it's pretty hard to argue that this is copied form the training set.
It might not be copied from its training set (yet), but it is copied from its context window.
The code it produces is a clear copy of the spec or the existing code that is fed to it as part of the task. It is just following the spec instructions, and some models follow it so closely that they get caught out by the lies/adversarial traps in the spec. Opus 5 is better at seeing through the lies, but it still very much copies from its context window.
Yes, humans can learn which parts of the spec are true and which aren’t. LLMs can too, but only through their context window (or long term through retraining)
That's not an exhaustive definition, creating also means adding new stuff to the mix often involving leaps of imagination. Or we would have been stuck with the same words, music, tools, programming languages from the beginning. It's hard to see we would have even gotten started without those leaps and new ideas, genres, stories, programming paradigms coming about. Humans have been creating new things from the start.
When I look at the history of human culture I really see a lot of remixing and "trends" and the leaps are rather small.
What sticks out is really the lack of people doing something totally new. Look back on history and it seems like humans can't do anything except as in response to some input that we re-mix. (Although it is across fields -- a change in economic conditions or scientific outlook may affect music and painting; a small improvement in factory production methods may ripple down to experiments validating general relativity, etc)
Why didn't Mozart create a piece of jazz or rock to mix things up? Why are there no medieval cubistic painters? Why didn't someone understand $scientific_topic before?
Most such questions can be answered by some kind of "the social environment and culture at that time didn't prepare people to do it". I.e., humans couldn't remix it yet from what was already there.
Yes there are leaps but they seem to always be within some small "box" that prior knowledge creates for us. We can jump one inch at the time but no further..
> But how much of their ability to write code is due to their understanding of it vs. just them recalling code snippets which were present in their training dataset or their context window?
Given that a small model that was trained only on pre-computer-era texts can write simple python programs after having python explained to it in context I'd say that LLMs must be pretty good at understanding coding.
> I would be tempted to say that humans definitely "repeat snippets from their training dataset". We mainly recall and express from our training/experiences. Creativity is mostly recombination of things already seen.
Isn’t this Skinner (Operant Conditioning)? This worldview is certainly intuitive to many.
Yes I have often caught stuff like "not using DB transactions in the right way" or "this is not going to be race safe" for instance from juniors just by glancing on a PR.
I find recent LLMs don't do such mistakes though, in my experience. And also LLMs will find such mistakes if asked to review, if not perfect then more predictable than humans.
...and cars also changed very rapidly during early years, not to mention how often they would break down and how they basically required the user to be a mechanic to repair them on the spot for many years.
So the analogy isn't all that good is it?
You are comparing immature tech with very mature tech.
What technology emerged into the market fully finished?
If your point is "don't use technology until it is mature" then you are of course free to not use it for now.
And of course, we now know that encouraging everyone to own and rely on a contraption that emits fumes and other pollutants for its entire operating life was a bad idea.
HN is probably not the best forum for theological debate...
But e.g. modern American evangelical churches are to me quite clearly quite far from the original meaning of the texts with some theology outright invented in 18th and 19th centuries.
It really is the case that what you bring to a text colors what you get out of it, and the more people insist that they are reading a text "as it is clearly written" the blinder they seem to be to this.
The question being what text to whom? The original text in the context of the culture of the original audience or an English translation interpreted as though addressed to a contemporary audience?
Pulling a trailer with an EV is very common around here (Norway).
reply