Anastasia Berg and Jon Baskin wrote a piece in August called “On Radical Preservation,” [1] in which they use the example of the shared concept of 'merit' to warn about the dangers of an AI-filled present and future. They anecdotally compare our situation to that of the Crow people, through the testimony of Plenty Coups, who described the fall of the buffalo and the end of the practices, rituals, concepts, and way of life that his people understood as making up what it was to live an excellent life (or not).
Likewise, they point to merit (which I think we consider very much like expertise) as one such fundamental concept in our society, which is very much undermined by unchecked use of LLMs. This coming not just via "cheating," but by the way in which we so quickly are willing to claim, and ourselves believe, that we deserve praise for that which the machine has created. On a wide scale, their use will not just compete with those who may not use the machine, but will destroy even our shared idea of personal merit.
Beyond merit alone, AI might lead generally to our "own cultural values becom[ing] not just decadent or debatable but unintelligible." At the end, Berg and Baskin basically say that hope is not sufficient (hope that the old concepts will be replaced with new ones); the proper attitude is to do everything in our power to preserve them in the present.
Serious question - what's so special about personal merit anyway? In hunter gatherers it was might, in current age it's brain, but either are equally arbitrary.
I prefer a world that's equalocracy, with a focus on specific personal-freedoms (live and let live principle).
Isolated Hunter-Gatherers tended to (and still) coalesce around the person whose opinions were correct more often then not. Violence was not a default until populations grew. The measures may be arbitrary, but can always be defined as “born with the right attributes in the right place at the right time.”
Do you have any solid evidence to back up anything you’re asserting here?
In my understanding of history (pre-enlightenment), people didn’t gather around ideas that were “correct more often” but gathered around ideas and personalities that reinforced their own ignorance, superstitions and biases and were openly hostile to “correct” ideas and truth, especially if those ideas upset tradition or dogma.
I also don’t understand how you’re getting to “violence was not a default until populations grew” since, perhaps, “default” violence was only properly recorded after populations grew and education/writing became more widespread. It’s very likely everyday violence in small groups was simply accepted as “the way things are”and simply ignored, as it often is even to this day.
Yes I do. In anthropology the surviving “undiscovered” indigenous tribes of the 20th century showed a diverse range of social configurations - matriarchies, patriarchies, everything!
Systemic violence came with the agricultural revolution, food and population stores, the ages of Bronze and Iron. Life was too precious in small bands for intense violence, even against rivals.
[Edit: 100 men armed with wooden clubs is an extremely bloody rugby match. Things generally did not start getting very-very bad until metal weapons were combined with boats or horses.]
This still feels more like an opinion than facts/proof. I was hoping for actual cited papers or studies, it feels like you just reiterated what you said above but adding "In anthropology..."
I'd like to see the studies which confirm that kingship (i.e. rule by the most mighty hunter or warrior) and violence is universal and did not explode about 5000 years ago, because that all seems to me like a mere extrapolation of the 19th century imagination of the default human condition as being "nasty, brutish, short." =)
Some interesting texts:
* The Harmless People (a classic)
* Hierarchy in the Forest
* Hunter-Gatherers of the Congo Basin
* The Foraging Spectrum
I appreciate the references but I'm guessing they don't fully support the claims you made above, at least not as strongly as you're making them seem.
While they talk about egalitarianism and resisting domineering members, and various organizational structures, they don't establish that people followed "correct" individuals or that violence increases with increased population growth/agriculture/weapons.
I think I'm safe to say that there is plenty of evidence for violence that predates agriculture and modern society, even if agriculture and nation-states increased the scale and visibility of such violence overall.
I think replacing "nasty, brutish, and short" with its opposite "noble savage" ideal is probably just as mistaken of a projection from literature. Human nature doesn't seem to change much throughout history, tools might just make that violence more efficient and more visible.
I don't think your thesis is fully wrong, but I don't think it is fully right either, which was my point in pushing back against the earlier statements you made.
A lot of the people squawking about this are in the midst of a personal identity crisis because they're not longer "special" - that is to say they no longer get special treatment because they know some arcane knowledge that only the smartest can access or some other such lie we tell ourselves.
These people are trying to be John Henry, but forget about the part of that story where he dies trying to out compete the machine.
> Understanding the concrete problem mathematicians are upset about can help us better understand the impact of AI on our own fields,
imo, The author of this essay does not understand the concrete problem that mathematicians are upset about. There is an idea that math [1] and coding [2] are human activities whose purpose is to achieve a certain kind of insight or mental clarity of things. The simplest description of this is by Feyman [3]. AI generated proofs short-circuit human understanding and therefore goes against the primary purpose. The declaration is calling this out loudly to reiterate that the purpose of the endaevor is not the generation and rewarding of proofs.
If you think the declaration is saying AI is not useful it's you that "does not understand the concrete problem that mathematicians are upset about". Terence Tao uses AI heavily and has been writing extensively about how useful it is ever since it became useful in maths.
That AI companies are pouring huge resources into strip mining outstanding problems for bragging rights rather than introducing the technology responsibly in a way that benefits the field.
Somebody will now respond that it doesn't matter that it doesn't benefit the field if it can produce the product without the mathematicians, but the whole point is that an LLM-generated Lean proof isn't the product. The proof of a singularity in Navier-Stokes is of zero commercial value (except bragging rights) in itself. If it ever results in something of commercial value it will be far downstream, after any new, valuable ideas in it have been digested, explored, explained, etc. This is work for mathematicians that LLMs cannot (currently) do.
And next someone will respond that LLMs will be able to do that work, however that is a prediction that is yet to come about. It might happen, it might not. To tear down something valuable now on the assumption that it will become obsolete in the future is wildly irresponsible.
Missing the point by the widest conceivable margin.
The million dollars is an award from the Clay Institute. It doesn't represent value in the proof itself, but rather is offered as a reward for an activity that, when done the human way, is expected to result in valuable ideas. As the Clay Institute says on their website, on the Navier-Stokes page:
> Why ask for a proof? Because a proof gives not only certitude, but also understanding.
It's also a fraction of the amount OpenAI invested in generating the proof, and its value to them is dwarfed by that of "bragging rights".
re the Feynman point, couldn't the same argument be said about, for eg, developers no longer manually writing machine code, relying on compilers instead? and the people maintaining those translation layers not knowing the phenomena that results in a transistor flip, trusting the engineering to do what it's said to do? I think an example of the kind of question this leads to would be "do you really understand software if you don't understand electrical engineering and microprocessor architectures?"
there's a level of obfuscation for any knowledge work where you rely on existing but incomprehensible-to-you systems that you just trust to work. are you unable to do any kind of mathematical work if you don't understand every single layer of proof that exists under-the-sun that touches your subject matter - or can you trust that some of these antecedents have been battle-tested and are functionally true for your purpose?
you could make an effective argument about the state of modern general-purpose LLMs that's founded on the idea that they are fundamentally untrustworthy and all results need to be validated but the larger categorical narrative, that the only true way to understand something is to know the logic from the most base principles, seems faulty
They are different -- higher level languages like C++ and Python surface the abstractions, making them more apparent to the human reading the code. LLMs typically hide the abstractions. You can still read the source code, sure, but even the latest models will produce a lot of repetition of utility functions and produce data structures of various shapes without commonality or reuse. This is moving in the opposite direction that machine code -> assembly -> C -> ... were heading.
“ Like mathematics, software engineers are going to have to rebuild our cultural sense of the kind of work we value.”
From the closing is the change I see going in at every job area and level.
My hunch is that this ultimately doubles back to those that excel at story telling and human coordination. As the AI systems “offload” not just production but I believe some initiation of what to build, the “why” and how to rally groups for any appreciably complex work matters more.
I also hope to see a plenty of solo shops succeeding in spaces that used to take entire teams, but (for now) remain convicted human coordination remains a key need for most endeavors.
>My hunch is that this ultimately doubles back to those that excel at story telling and human coordination. As the AI systems “offload” not just production but I believe some initiation of what to build, the “why” and how to rally groups for any appreciably complex work matters more.
The "why" behind most products is of little interest to the majority of workers. While it may be of the upmost importance (on the surface at least) for the leaders of a business, I see no reason why Bob from accounting is going to give a fuck about your company's grand vision.
This is especially true if your company is in a mundane lane like B2B SaaS. You could argue that workers at SpaceX care about the "why", but 99.99% of companies aren't SpaceX.
You are highlighting their point. You are pointing to current culture and behavior. The very first sentence that you responded to points to that culture and says it's gonna need to change, if you'd like to keep your job.
People skills imo will always trump other skills. The best ideas never live because people don't know how to sell. And knowing and being very successful at selling requires that excellent story telling and human coordination.
Just like how social media has possessed many people with cultivating an outward facing image that often diverges with reality, AI posses people to portray themselves as an artist/developer/musician/etc. without having put in any of the work.
> people would know you were a skilled engineer. But now projects like that are worthless, because everyone just assumes they’re vibe-coded.
I do not really care what other people think. Build something for the purity of building it for yourself or for a purpose.
Thanks to AI, I feel like I have been writing the best code in my life. Yes, writing, not vibe-coding. I mainly just ask questions and ask for hints and clues. I do not use LLMs to do the fun parts for me.
I am currently working on a game. If and when I ever finish it, I want to be able to say that I wrote every single line by hand. Will it make me better than anyone? No, not at all. I want to do it for myself.
its interesting that people view different parts of the job as 'the fun part'.
you like typing out the code, but i see that as the drudgery that I have to slog through after the fun of drawing architecture diagrams and writing out the api specs.
Different strokes for different folks. I have never written an architecture diagram or api spec in my entire career. I predict I would not want to do either.
I am 100% certain the code you write by hand will be better quality, even with AI pointing out options, than letting the AI string together completely obtuse abstractions.
I have two projects running concurrently. One is my hand-written project. The other is one "demolition zone." In the latter, I can vibe-code and do whatever I want just to see what LLMs can hack away at in comparison.
Besides the enjoyment I get from programming by hand, I have a more overarching concern when it comes to a building a full game. I am using Monogame, and I realized that is probably not the library to use if I am going to have no understanding of how things work under the hood. I tried to review LLM code, but without knowing enough of Monogame's API, I really couldn't make a good judgement call on the quality.
Looking at the LLM code, for example, were the keyboard and game pad implementations correct? I am not sure at first glance. I haven't implemented one since I used C++ in a game dev class in college over 12 years ago. The LLM's tests passed, which is fine and dandy, but how do I know the tests are truly valid if I do not understand what is fundamentally being tested?
I am not building this game entirely for fun either (though I am having a blast). I am working with a few non-technical friends on making this game. So, I do not want to disappoint them by creating something with bugs that I cannot fix because I have no earthly clue how anything works. I believe that would be embarrassing for me and disrespectful to them.
> The LLM's tests passed, which is fine and dandy, but how do I know the tests are truly valid if I do not understand what is fundamentally being tested?
Do what I do sometimes: use opaque pointers (in C, or C++), and don't give the LLM the implementation, give them only the header, when asking it for tests.
I've lost count of the number of times I'll give only the header, but the LLM insists it needs to see the implementation as well in order to write the tests.
If you're not using C or C++, well, then write interfaces, give that to the LLM.
That's fine with me. I'd even appreciate that even more. I get one life on this miserable planet. I am going to do what I enjoy for as long as I can. Call me a blacksmith or a Luddite, I couldn't care less. One thing you can never call me is unhappy.
> In a few years we'll look at this the same way as people who unicycle or blacksmith.
It's equally likely that in a few years we see everyone who doesn't have skill the same way we view people who follow the Kardashians and other influencers: vacuous and incapable of non-augmented thought.
In a few years they will be the only competent people left who can save us from the deluge of AI slop. You should be nice to them, because you'll need them someday.
Do you think information is going to be destroyed or something? If this AI stuff is so sub par, why would anyone read/use/interact with it? Why wouldn't the "better" human created stuff float to the top?
The difference is that the textile machine workers could immediately verify the work product. This is impossible when the work product is a configuration of knowledge you don't actually have. It's why, for example, amateurs "solving" these open math problems refuse to discuss the results with actual mathematicians, because they can't. So we have a bunch of Lean-verified slop that may or may not actually prove what is claimed, without experts poring over everything line-by-line and reverse engineering.
Good for you. But some people need others to know that they are skilled to get a job or to be part of a community. You must be one of those priveleged ones moderately far along in your career that you don't need to worry about that.
The fact that you 'need' that is not an inviolable feature of the universe. It's not even a good thing. It's just a thing that has evolved based on the current arrangement of society. It's as mutable as whether or not we have regular religious crusades.
Likewise, they point to merit (which I think we consider very much like expertise) as one such fundamental concept in our society, which is very much undermined by unchecked use of LLMs. This coming not just via "cheating," but by the way in which we so quickly are willing to claim, and ourselves believe, that we deserve praise for that which the machine has created. On a wide scale, their use will not just compete with those who may not use the machine, but will destroy even our shared idea of personal merit.
Beyond merit alone, AI might lead generally to our "own cultural values becom[ing] not just decadent or debatable but unintelligible." At the end, Berg and Baskin basically say that hope is not sufficient (hope that the old concepts will be replaced with new ones); the proper attitude is to do everything in our power to preserve them in the present.
[1]: https://thepointmag.com/letter/on-radical-preservation
reply