From my understanding of how LLMs work, I'd argue thinking will become the most valuable attribute of humans (as it has been for some time). LLMs will inevitably surpass humans at functions that we know how to do, and likely through cross-domain knowledge and connecting disparate concepts will surpass us at many functions. But LLMs haven't shown they are capable of truly novel thinking, because it's next token prediction, which is inherently based on existing sources.
I think we'll see a shift towards arts, creativity, research, etc. where true novel creation and an understanding of the human experience exists at a premium.
I have a bit of both. Most of my thoughts aren't really originally because I'm not playing around and go wild. This is what LLMs seem to be lacking a bit.
Most of my thoughts are things that other people have come up with as well. Aka it feels like I'm just predicting the next token based on the data that I already have.
It's only when I go to places such as "what if life at the speed of light is possible, what would that look like?" that I get to places I personally find somewhat novel. It's almost a different dimension for these guys of course since a million years can feel like 10 years to them. Yet, they're moving so fast, but time-wise they move so slow, maybe even too slow for us to understand that anything can be alive at that speed (I'm not claiming there's life at light speed by the way, I just think it's fun to imagine that there is).
More seriously, the only time where you’re jolted out of your prediction flow state is when you’re surprised, which means something happened counter to what your brain was predicting.
You did a quick calculation in your head in which getting up to get a glass of water now instead of 1 hour later just made more sense.
You can do chain of thought without verbalizing it. The LLM can't. Now you might broaden the definition of "token" to include whatever nonverbal pre-verbal impulse your brain produces. But then the definition doesn't mean anything. All state machines of any sort are doing "next token prediction".
Same! I do think a vast majority of tasks are essentially next token prediction, but that's why I'm thinking that long, creative thinking periods will be what's actually valuable/rare.
Open to learning. How could RL yield novel discoveries? What I mean to describe are things reliant on empirical or real-world testing of human reception. How would that be synthetically derived?
We're seeing the cost of code drop dramatically and are facing the same problem in building new companies across most disciplines of, "what's a problem painful enough that someone will pay us to solve it?".
I suppose if we get to a massive enough scale we can effectively model every atom's interaction and human on earth's reaction to any stimulus but I think we're still some time off from that :)
For a cost-prohibitive example, you could put a humanoid robot whose pretraining contained no chemistry, in a chem lab, and put it in a hypothesis loop, where it generates a random action, guesses the result, and then does it. Every time the lab burns down, you just build it again. Reward the model for producing novel actions and reward the model for correct hypotheses.
In other words, you generate more training data by actually experimenting, just like humans do.
Not sure what this is supposed to prove. Yes it can generate new words - that’s the application of established creative patterns to come up with plausible-sounding words.
A different thing is to come up with a novel idea, which (in this case) is about creating a new pattern itself.
if patterns can be put into words, why wouldn't the same process happen at the word level? do you think it's impossible for a LLM to write even a simple python program that will generate millions of plausible random sentences in the general problem space we aim to generate new concepts for, then filter and rank them semantically?
More generally, do you really think humans randomly come up with new ideas, in any act of creation, such as art, science, etc.? Why do you think art studies and stuff like Oblique Strategies exist? How long should one wait for a specific random human without any formal education in physics to have 50% chance of coming up with a new workable theory of cosmology?
As you say, putting a random sequence of plausible-sounding words together is a trivial task - even a non-LLM algorithm can do it. No, I don't think humans are unique - they also absorb the ideas available to them and re-elaborate on them to produce novel ones.
An LLM might come up with a sequence of words that express a novel idea/thinking pattern, but so can I on the spot right now. The question, though, is: is that idea any good? In the case of humans, maybe the first idea is good, the second ok-ish, but inevitably they get duller.
Most people come up with very few novel (good) ideas in their lifetime.
The question, then, is whether a model can, unlike humans, *repeatedly* generate new ideas, continuously.
My guess/assumption is that NO, a model might come up with SOME ideas that are somewhat original [1], mixing the existing sources available to it (similarly to how a human would do it, just with more sources and more efficiently), but it cannot continuously generate novel ideas.
If that's true, I think there will be a space for humans to be thinkers, even if models are super smart, as a different unique perspective and a different reconfiguration of previous ideas will be more unique than raw intelligence.
Please note: I wasn't refusing the notion that LLMs don't produce anything original, just noting some fallacy in your example about word generation. The jury is still out on whether AI models will own thinking in the future - or will simply be partners to humans - and I don't pretend to have the answer.
[1] More should be said about what makes an idea original (even if derived from others), as opposed to a transposition of an existing concept. I admit I'm not a mathematician or anywhere near an expert on the topic, but from what I've read, the latest AI math achievements seem to belong to the latter case.
As Stalin said, quantity has a quality all it's own.
11000 agents explored one of the recent math problems solved by AI (can't remember exactly which one). About 3 of them had crucial insights which moved the problem forward. A 4th one also had one of the key ideas, but it's session got messed up and the idea was lost, and only noticed in retrospect.
Maybe 11000 top mathematicians working on it would have found the same ideas. But certainly not in days.
Even if LLMs generate novel ideas at 0.01 rate of humans, you can deploy vast amounts of them on even trivial problems, that humans would not bother with.
For biological reasons, human intelligence is kind of fixed and only very slowly advances by better compression and synthesis of the school curriculum.
There is no clear present bound on machine intelligence. One might argue that machines won't be able to get smarter than the material they were trained on (human output). I kind of doubt that. And certainly they can be much faster.
Somebody whipped up a website with GPT2 that does that and it was discussed on Hacker News at the time: https://news.ycombinator.com/item?id=23169962 I don't think anyone is going to argue GPT2 is doing thinking of any kind, let alone novel.
It is based on existing sources, but doesn’t mean that we produce the same output. There was a paper here in HN a couple of months back on how LMs are good in the deductive and inductive thinking processes, but not the abductive.
So if that’s (and stays) true, humans will have a place in the job market after all. Time will tell.
This is kind of an aside to the topic, but your comment made me think that one of the advantages of humans is just that there are currently 8 billion different neural nets trained on different experiences with different model configurations running different architectures on different hardware.
Open to sharing what workflow you're using on the lifestyle image creation? I've tried the Higgsfield MCP, and direct Figma integrations - but keep running into hallucinations on product dimensions, etc.
At this point, I mostly ask Claude to use Claude in Chrome to use ChatGPT to create lifestyle images (because I am cheap and I don't want to pay for API usage when I could get them free) based on whatever product shots I have. Sometimes I give it some more specific direction about what I want, but usually for the first pass I just tell it to make 10 or 20 or whatever and see what it gives me.
When it's done, it generates a web page with the images and a simple UI for feedback, then uses the feedback I give it to iterate.
It has always been very good at styling things well (i.e. if I'm making infographic-style images, I give it an example of one and it's good at copying the style/fonts/colors/etc.), but I suspect I get better-than-average results in terms of the content just because it has built up so much context from doing this and taking my feedback across so many different brands and images.
The product dimension thing really stopped being an issue for me with ChatGPT Images 2, and the new 2.5 is better. If you're running into trouble, there are some cases where Gemini is better, but they're rare. But I will say if you're prompting yourself and running into issues, you should get Claude/ChatGPT to help you write the prompt that goes to the image model. They're very good at it.
I think the difference is people being comfortable with giving away their own data for convenience, vs. someone recording them without their permission (who they can immediately confront, unlike Flock and the like).
You will confront, they will record the confrontation and post it online to farm clicks. Few people will sue, there will be a slap on a wrist and voila, its normalized.
Now people will not confront because they would only provide content and see themselves in new viral video.
There's a weird literal click bait sports headlines online website out there that's basically just farming sports teams and athletes because I assume they are easy to game. It was -good- enough that news.google.com was syndicating these articles that had zero substance.
I think we'll see a shift towards arts, creativity, research, etc. where true novel creation and an understanding of the human experience exists at a premium.
reply