But as a professor, enrollments are way down. In general, this was expected but not prepared for just due to the "demographic cliff" due to people having fewer children in the USA. I think university leadership had an aspirational hope that foreign enrollment would fill that gap, but instead foreign applicants have gone way down (and this was starting to happen even before the Trump administration accelerated it).
Universities are very slow moving and most professors, even in computer science, haven't really been using agents. Even as an AI professor, it is _so_ hard to keep up with the pace of change. I'd say my opinions are more dire now, than I express in that article I wrote a year ago.
The university in the sense of the Lyceum survived the Roman Empire, the Middle Ages and even critical theory and will surely survive genAI.
The university as remade in America from the Morrill Act of 1862 through the 1944 G.I. Bill has had large and largely positive influences on agricultural education and the research efforts and training of the American workforce that capitalized on the postwar era and produced the laurels they now rest on..
Generative AI threatens the parts of these training and research endeavors that are most lucrative in terms of tuition and grants and arguably least aligned with the original Lyceum's ideal.
It would be beneficial, from the overall perspective of the history of the university in western society, if this precipitated a restructuring that did away with dubious gegrees and research projects, but as there is no one ready to advocate for the return to the intellectual ideals of the Lyceum, my personal observations at the SEC institutions that I deal is that the overall strategic shifts are currently more focused on moving towards profitability enhancement in collegiate sports teams [1,2] and degrees that appeal to social media influenced high school students [3,4] and similar..
University-corporate partnerships which, while too varied to characterize generally, seem the best hope for American style universities to realign their vocational and research missions, given the uniformly unresponsive nature of their own management structures. The twin threats of demographics and rapidly proliferating generative AI capabilities in the mid and lower levels of most professional and research workforces mean that, absent a coordinated and fluent response from university administrators, the best thing American style universities can do in the professional spheres that their students have traditionally aspired to is to serve as facilitators for on the job training, essentially vetting and supporting students and workplaces (from whom they should draw revenues proportionately).
More importantly, in the era of 1.5C+ climate change, it seems likely that their original role after the American Civil War as agricultural education and research institutions will once again become their most important future role..
I'm told by colleagues in the life sciences and pharma space that it isn't just "safety" but more like artificial market segmentation. If one has to pay extra for guardrail removal, then that's market segmentation. Anthropic makes most money from company whales. Want the guardrails gone? Pay a lot extra over baseline. But Mythos probably had PEFT adapters, so that at least makes it a bit different. But still there is not nearly enough transparency about these things.
And oopsie whoopsie, looks like we accidentally guardrailed out all of the startups that might challenge your pharmaceutical company. Thanks for being such a loyal customer :)
Well, mine was re-enabled, and in the history of past requests I have two past requests to disable training that were granted. And there it was enabled again. This is baffling.
I wonder if it is because I do a lot of AI research.... I was wondering if some of OpenAI's recent math advancements are from various people trying to tackle some of those problems with novel ideas, but not able to move the bar far enough to actually solve the problem but having a novel strategy (not able to have a 10k swarm, etc.) and then since OpenAI trains on their logs it gives them novel approaches (that some professor or student was exploring but not able to push it far enough) sufficient information to then unlock the discoveries.
OAI already retains the data, so I don't think they need to flip any switches. They probably claim that data retention is disabled while still using the data to train their models.
Again, why would people trust a company that stole millions of books and have bad history with all these data retention to keep their private conversations safe?
Why would one assume larger is needed? I am an AI professor doing work on a wide variety of things but Anthropic models have cost me so much time even when not blocked. My old Anthropic agent wrote rules into its system prompt to avoid all machine learning work. My openai ones do the work but tell me they need to correct my opinions (on fields I helped found) so that they are aligned with the consensus....
Mostly using open models from China in the past two months + Argon.
This is actually wrong. A large fraction of people do not have internal monologues. Not at a computer or I would dig up cognitive science and psychology papers, but language evolved as a means of communication not a means of thought. The vast majority of human thoughts are not verbalized. I have a strong internal monologue, too. I was surprised to learn that a non-trivial number of people do not have one.
>How does someone reason about the actions they need to take to tie their shoes in words?
Its not for everything. It's mostly for intellectual tasks or planning. Eg Trying to solve a programming problem, or remember the list of items are needed for dinner while shopping. Personally I reason "out loud" in my head while doing those things, some people don't.
As another post here says, they're probably referring to an internal monologue. I imagine that in reality it's more of a matter of how active that monologue is.
Though, also, I think that the internal monologue is maybe less an aspect of how we think, and more an aspect of how we remember. I haven't looked into it, but my feeling is that a more active internal monologue might be correlated with someone who has an easier time remembering sounds than, say, images.
It doesn't have to mean that every action is accompanied by a monologue, a task like tying laces can be done with no active thought about the specific movements.
> Have you ever talked through a problem with a rubber duck?
I talk to myself to find good ways to express a thought with language, but not to think.
> Or written in a journal?
I do this to remember and sometimes to prompt thinking, but writing isn't the thinking. It's the outcome of it.
Do I just have a narrower definition of reasoning?
The reason I'm interested in this is because I have a very low verbal IQ but a relatively high spatial IQ. I have often wondered if this divergence is because I don't reason with language, or perhaps what I experience is normal but I just disagree about the semantics of reasoning. Unlike some people, I can do internal monologues, I just have no idea what people mean when they describe that as reasoning.
For example, if you're trying to work out why some electrics aren't working, do you say in your head, "I should check the fuse box" or do you just visualise the system and components that need checking? I do the latter, and words would only enter my head if I need to write that down or say it to someone else. Furthermore, I don't even understand how you can monolog in your head about something prior to thinking it. This concept makes no sense to me and would be an alien form of reasoning to me if people can do this.
I really hate this open weight but closed science approach. These companies just take from academics and all the Chinese companies that are doing good science, but without understanding the recipe it makes it hard to know where the failure points will be until your agent accidentally commits a crime.
1) First, you are talking about positive forward transfer in continual learning. I've been giving talks for the past 6-7 years about how that community (I was one of the founders) went astray and wasn't focusing enough on that topic, but continual learning of the kind you are thinking isn't in any of these systems right now. I think some people left the Grok team to make a start-up to focus on that. By forward transfer, what I mean is weights update over time and past learning improves future learning such that we get better sample efficiency.
2) Psychologists distinguish among different kinds of intelligence for Spearman's g (IQ). Crystalized intelligence is using already acquired knowledge (frontier models probably have maxed out that). Fluid intelligence is reasoning and finding solutions in novel situations or without the necessary crystalized knowledge. [Giving colloquial definitions]
3) Now, interestingly, neither of those are correlated with _creativity_ (just they are independent, note some have this threshold theory but it hasn't held up in recent papers). That's what the AI's really are terrible at -- creativity. But I'd argue the vast majority of humans aren't very creative, with truly out-of-the-box ideas. Given that this is HN, and a non-trivial number of us have ADHD, creativity is positively correlated with ADHD.
I did a bunch of research on these topics for my AGI course that I teach each Spring (where I then point out conflicting definitions and start using multiple alternative terms rather than AGI to distinguish among the different definitions).
It should obviously not come as a surprise that having areas of the brains working differently can explain an awfull lot of things... but to answer your question directly, yes of course of course there are!
Simply look up all the many, well
-supported and -researched known correlations with ADHD first. In the second step, you can construct the set of all possible correlations, and subtract the well-researched ones if it. What is left is the set of correlations that are either not explained by ADHD (the big majority I would assume) or explained by ADHD, but as-of-yet unknowingly so.
This might be a bit anticlimatic, but clinical psychology is pretty straightforward study design and statistics, and set theory is not that new either, so... no big surprises I am afraid.
Not sure the thrust of your comment, but what is really going on is just novelty seeking to get a bit of a dopamine rush, which slides into creativity.
It all comes down to dopamine at the end of the day.
> That's what the AI's really are terrible at -- creativity. But I'd argue the vast majority of humans aren't very creative, with truly out-of-the-box ideas. Given that this is HN, and a non-trivial number of us have ADHD, creativity is positively correlated with ADHD.
I think it's hard to define creativity in the context of AI because they seemingly just make up new hyphenated terms for everything. Is that creativity? If not, what about when they do the same thing different ideas in the latent space?
If we say that simply nailing one concept to another isn't creativity, then AIs are incapable of creativity, while the vast majority of humans are incapable of creativity. This is just a long way of saying "0 AIs have creativity, 0.00001% of humans have creativity", and the difference between zero and a very small number is infinity.
It’s not that they’re incapable of creativity, it’s that LLM-driven creativity is terrible, and nothing makes me cringe more than when it uses a word in a “novel” way.
But monkeys-with-typewriters, they sometimes stumble upon something that doesn’t suck. But if you don’t want to spend a fortune retrying the same task until you get a suitable result you have to inject your own taste.
Sometimes I start with a super vague prompt and see how close agents can get to something that doesn’t suck. I inevitably get frustrated about 6-7 prompts in when they’ve created a complete mess because they have no taste. So I restart and inject my taste into the process. Things like linters, test suites, which 3PLs to use, etc.
Taste or vision? They are hill climbing and they can't see the other side. And sometimes that is because they don't live in your head and don't know what you want.
Very insightful actually. Taste and vision are both instances of holding a model that predicts a good result beyond the threshold of validity in theory but reality happens to align. Is it luck then? Perhaps meta-luck where lucky weights produce “accidentally great” results with some predictability.
Steve Jobs had a mental model that brought the iPhone. No one really wanted it but something in his life biased the result.
So creativity is having weights so good you can project way out into latent space beyond what is reasonable.
I hope you’re not actually trying to compare human cognitive processes to an LLM, because that would be incredibly reductive, not to mention lacking in any empirical grounding.
Nice to get some resonance. And if I may, past --> taste; vision --> future. And to compensate for our poor memory, taste<=>value network while vision<=>policy network, borrowing from RL parlance.
> That's what the AI's really are terrible at -- creativity
Good thought piece here "We Are Losing the Ability to Discover What We Didn’t Know to Ask[1]" By Anne-Laure Le Cunff
It keeps playing on my mind as I see people at work follow some predetermined AI workflow to get their jobs done, the art of being curious and exploring around the problem is so important to the really big innovations. Been thinking about how to address this through some of the harnesses we are developing in the knowledge working space.
I met a senior (as in, 4th year of college) recently and she asked me: what advice do you have for someone just graduating in the AI age? (as I had told her that I've been in ML for 20+ years, etc.)
My recommendation to her was: just _play_ with the AI! It's a brand new tool, and none of us knows its capabilities, limitations, boundaries etc. (which are fluid, of course). So just spend as much time as you can tinkering with it, playing with it, making it do things it was not expected to do, etc. and you'll develop an idea of how to make better use of it.
Tangent: Anne-Laure Le Cunff is a neuroscientist with a talent for community-building and writing, and bringing evidence-based approaches to bear on practical solutions to various domains. See eg https://nesslabs.com
Hi thanks for the insight. Do you see a role for Control Systems(i.e. ones analogus to Instrumentation engineering) playing a role to modulate certain parts of continual learning? One very important way we learn are lived experiences, it's like telling memory:this part is more important( for emotional or social utility values), pay attention. Good or bad lived experiences both count. I guess is that a path that practical research is considering?
Appreciate the input! Responses below to your first two items:
1) I'm leaning more into a broader sense, which is that given the priors a system already possesses, how efficient can it acquire competence on a novel task? If I'm reading the point you make, you're focussing on continual learning right? If so, I'm not necessarily restricting my statement above to that.
Here's another reframing: How much of the benchmark improvements come from overwhelmingly large training distributions vs improving the models for adapting to things genuinely outside of it?
2) Excellent points about crystalized and fluid intelligence. Wouldn't the LLM scaling gains be a representation of crystallized capabilities? In regard to Gf, that is exactly what I am asking about. That is what seems to be lacking, Gf like adaptation under genuine novelty.
My concern is that it's increasingly difficult to tell of what looks like Gf like behavior is really coming from better adaption vs. having broad priors from the model's large learned distributions.
> weights update over time and past learning improves future learning such that we get better sample efficiency
Is there an architecture-independent definition of forward transfer?
For the practical experience and implications of AI progress, I think we are increasingly discussing what these LLMs can accomplish inside a stateful harness, the state of which could be described as part of a (very squirrely) parameter space.
"Given that this is HN, and a non-trivial number of us have ADHD, creativity is positively correlated with ADHD." Is this a statement of fact? As someone with ADHD and pretty confident in my creativity, I still believe this is much more cope than fact(which could be more a self-doubt thing than anything else).
Further, if there is a correlation, I'd bet it's not so much an intrinsic "creativity" trait, but more effectively higher creativity because more trials. That is, along the lines of Chollet's paper, a measure of creativity should be based on a fixed budget with fixed knowledge.
Among many other possibilities I haven't considered, perhaps another mechanism could be that because ADHD people spend more time thinking in less goal-oriented ways and mixing thoughts on accident, perhaps we do in fact gain some learned creativity via experience with vagueness[1]? But that might also imply that part of creativity is actually being able to diffuse more freely through thought space and lowering the barrier to attempted connections between ideas. That lower barrier leads to less likelihood of any "collision" being meaningful but maybe it's overcome by higher collision rates? Or maybe effectively higher order (not just pairwise) collisions?
Disclaimer in case it's not obvious: I don't know any of the literature on what creativity even means or how it's quantified.
[1] Which is me injecting an assumption that creativity ~= connecting things with no obvious or well-troden reasoning path between them.
edit -- oops just looked at your profile after seeing someone elses comment. I assume you are stating a fact then, leaving original anyway
I think it's just that many of us don't have the capability to just do things in rote or the conventional ways, instead we must rely more on the creative and unconventional ways of doing things and parts of the brain responsible for that
This comment and the one above from astrobiased feel like coming into a messy codebase, and it’s more work to sort it out than it would have been to write it from scratch.... And since I actually do intelligence testing as a clinical psychologist, I have experience with this in both practice and theory. So now I’m going to waste an hour because I just have to respond to “something is wrong on the internet.”.
Chollet's distinction is useful. High performance on known tasks is not the same thing as efficient adaptation to a novel task. Prior knowledge and training data can buy skill. That is a central point of On the Measure of Intelligence. But it does not follow that current frontier progress is only "coverage-driven competence." That is a hypothesis. It is not a result established by Chollet's framework.
"Overfitting at scale" is also the wrong term. A model that learns broad representations and applies them successfully to unseen examples is generalizing. The relevant concern is whether apparent novelty is actually inside the effective training distribution, not whether the model is "overfit."
There is also an unstated premise here: that adding broad knowledge and skills cannot improve the machinery used for novel problem solving. I do not see a basis for assuming that. Learned representations, abstractions, reasoning patterns, and cross-domain analogies can themselves support transfer to new tasks. Whether this becomes sufficient for general intelligence is an open question with insufficient data. But its a perfectly valid hypothesis right now that, given enough domain knowledge and symbolic reasoning examples, LLM COULD maybe "Grok" AGI at a certain critical threshold.
And ARC-AGI-3 was specifically designed around novel abstract environments that require exploration and adaptation. Astra scores 99.9% with OpenAI's context-preserving Provider Adapter, and ARC reports that Astra constructed compact symbolic models of unfamiliar environments. That does not prove AGI, but it points in that direction more so than the other way around.
Gc roughly maps to acquired knowledge. Gf roughly maps to reasoning in relatively novel situations. Naming those two categories does not tell us whether increasing acquired knowledge and learned abstractions in an AI can improve Gf-like behavior. That causal question is exactly what is disputed.
And "Frontier models probably have maxed out crystallized intelligence" is just obviously wrong, unless you think they have been able to dig up every a scrap of paper with knowledge/information on it in the entire world, AND that there is no more useful knowledge to be generated left in the universe.
And the statement that intelligence and creativity are independent is simply wrong. A meta-analysis of 112 studies and 34k participants found a positive correlation of about r .25 between intelligence and divergent thinking. It also found that using g, Gf, or Gc did not eliminate that relationship. Creative achievement has a smaller but still positive meta-analytic association with intelligence, around r = .16. These are distinct constructs, not independent constructs.
And this is just a bad take: "AIs are terrible at creativity". At best that depends on which creativity, and I think its straight up wrong. On divergent thinking tasks, the operationalization behind every ADHD study you could cite, LLMs score above most humans, with the top humans still ahead. If you means Big-C, paradigm-shifting creativity, that is a different construct and none of the ADHD evidence transfers to it.
And if I where to say what I subjectively feel and see.... I have ABSOLUTELY no idea how people can say that we are not seeing sparks of creativity from AIs already. If a PERSON produced some of the music, solutions or deductions that I have seen AIs do, people would have NO problem celebrating it as extremely creative.
And finally, the ADHD claim is also, at best, overstated and just as often debunked. There is some evidence that higher subclinical ADHD trait scores, often survey studies only, are associated with better performance on some divergent-thinking measures. But a review of 31 studies did not find a consistent creativity advantage for people with clinical ADHD, and it found no evidence of better convergent thinking.
Okay, I’m done… And nobody noticed that I’m not doing my job here.
For me a basic test of A.I. creativity is give the A.I. chapter 1 of a novel it has not seen and ask it to write chapter 2, then compare the quality to the original author's chapter 2. The A.I. is always terrible at this in terms of matching the author's level of quality.
I would think that’s the opposite of creativity. Rather, I would label that as pattern matching and prediction capability.
Why box in creativity basically as the ability to mimic someone else who is creative? Why not choose something that better matches the definition of creativity (the ability to make new things, think of original ideas, or show imagination that is novel, useful, or pleasing)?
And even then, I am assuming the premise that the original novels are good and creative, especially chapter 2. Most novels are not very creative.
If it could make 5 different versions of chapter 2, all with different directions for the story, and all with novel and interesting developments, wouldn’t that be a better definition of creativity than “can it read chapter 1 and be able to copy style and deduce/predict what the author is going to do in chapter 2”?
And also, that’s only a subset of creativity (storytelling). I’ve met plenty of people who are terrible at writing and storytelling but can come up with the most impressive and novel solutions to a practical problem instantly.
Final thought: I have been following the development in the anime AI generation scene for a while now. It’s not even close to anything anyone would call creative or even OK quality. But it’s also massively impressive that it’s moving in that direction really fast. And if you watch enough, you are going to start to see some truly creative sparks. And some of the mainstream stuff that gets created and labeled as creative… really... How many isekai series with the same story have humans not made already?
> If it could make 5 different versions of chapter 2, all with different directions for the story, and all with novel and interesting developments, wouldn’t that be a better definition of creativity than “can it read chapter 1 and be able to copy style and deduce/predict what the author is going to do in chapter 2”?
If I like the novel, an alternative version where things happen differently would be fine.
The problem isn't that it's different. The problem is quality.
A lot of Isekai stories are crap but you can still rank them in terms of the author's ability or inability to have a creative POV that elevates the material.
I think “what I individually like” is a poor measurement of creativity, at least alone.
And yes you can rank quality. My point was that if you made two bell curves of the distribution quality and creativity of all new manga, The one for “AI slop manga” allready started overlapping with “normal human manga”.
That’s just a fancy way of saying the absolute best AI slop is at the level of the absolute worst human creation.
The interesting thing is that the AI slop curve clearly is moving to the right every month. Where it will stop tho, impossible to say.
Is chapter 1 of a novel really enough context to generate a chapter 2 of sufficient quality to match up to an author who spent a good amount of time planning out an entire story and whose manuscript probably went through a lot of revisions?
I don't necessarily just feed it chapter 1. (And chapter length varies between novels).
And no I don't think asking the A.I. to write the next 2000 words would require it to have the entire novel planned out. Not all writers even outline in advance.
I'm an AI professor, and the same thing happened to me. I had written out contracts and specs for some novel algorithms to compare. And instead it created a new sandboxed environment with 20k lines of code to ensure we do gold standard science.... didn't implement the algorithms at all. In another case, I left it overnight with a plan, and it decided to write a 50k line new overly engineered plan. Suffice to say, I don't use any frontier models for doing coding work anymore. I use them to make detailed plans and use good worker AIs for execution, which don't overthink everything.
The average hatching egg costs a few dollars. Can go as high as nearly $20 if it's something remarkably rare and useful in terms of breed/quality.
An egg takes 21 days to hatch.
Then it's 18-24 weeks to start laying their own eggs. That's ~$5-15 in feed over the period, maybe 40-50 pounds. There's utilities, labor, land, etc., but your incremental cost to add a hen if you have the space is minimal. My numbers won't reflect a lot of operators, as scale matters.
Out of a typical 1.5-3 year lifespan that had a startup cost of a few dollars per, paying back its return by about month 6-7, that's not too bad. It's only worth doing financially in this economy at ludicrous, arguably inherently inhumane scales, but that's what it is on paper.
Not quite the same path to course-correction as a silicon shortage.
Eventually 2028 and 2029 will come around and at some point today's data center builds will be (economically) full.
In addition to that there are extra fabs build and (for example) China can use the high margin to catch up with their processes without breaking the bank.
But as a professor, enrollments are way down. In general, this was expected but not prepared for just due to the "demographic cliff" due to people having fewer children in the USA. I think university leadership had an aspirational hope that foreign enrollment would fill that gap, but instead foreign applicants have gone way down (and this was starting to happen even before the Trump administration accelerated it).
Universities are very slow moving and most professors, even in computer science, haven't really been using agents. Even as an AI professor, it is _so_ hard to keep up with the pace of change. I'd say my opinions are more dire now, than I express in that article I wrote a year ago.
reply