This extends beyond software to technology in general. The "tool use" theory, by which technologies are neutral instruments with no moral or political implications until put to use, is an incredibly naive, weak theory that was basically killed and buried by Langdon Winner back in the 70s. If you don't have time to read his books, at least read ["Do Artifacts have Politics?"](https://faculty.cc.gatech.edu/~beki/cs4001/Winner.pdf) which states the case very clearly.
Of course, most technologists have never read a letter of a single work in the philosophy of technology, so they don't know this, let alone who Winner is, let alone that people actually think about and theorize the impacts of technology on a broad scale seriously (even though Winner worked at MIT, the ever vaunted institution of the technical priesthood). Of course, the people building things are the very people who should be thinking about the potential effects of the things they are building deeply and seriously, but apparently that's too much to ask.
some of us have indeed read various bits of critical theory from these non-contributors and then rejected them. shockingly not everyone who reads this stuff becomes an immediate convert!
This article articulates several thoughts I share with the author well. If there is anything people in the technology industry lack, it's intellectual humility.
That, and, they are pitifully ahistorical people. Today's industry leaders are not sufficiently schooled in constructivism. No, I don't mean they all need to become radical constructivists, but they should all at least gain a passing familiarity with Foucault before they continue fucking with humanity's social mores and environments with impunity.
They won't, of course, because they are decidedly unreflective (see Andreessen), unphilosophical people. They are, oftentimes, the marquee examples of 'soulless' people fueled only by the incentive structures put before them by the economy. Money. Clout. "Progress". It's telling that, beyond the many examples of hubris like that of Gates, basically zero technologists have come out and examined the unspoken assumptions behind such terms as "progress", "alignment", "intelligence" and the difficult social problems fixing these terms would entail. Moreover, they interpret all of these terms using some reified capitalistic gloss and go on their merry way, because, in spite of their constant proclamations to the contrary, most of them are not, in fact, very intelligent beyond being savvy businessmen.
The employees bear the brunt of actual technical intelligence. Leadership bears little more than a passing ability to navigate an economic structure that, let's face it, is already very forgiving to them and essentially places little to no responsibility on them after certain levels of scale are achieved. But none of that scale is possible without the vast legions of employees. Oftentimes the 'genius idea' is about 0.5% of the actual success of an endeavor, yet we cling to an old "great men of history" form of thinking and allow the public to prop up these individuals as though they are demigods.
Yes, "predictors" or "completers" are a far more accurate metal model for what they actually do and it pays to understand how they "use tools" and what RL, SFT, etc. is and how it makes LLMs seem like much more than what they still, ultimately, are.
People latched onto the bs marketing terms like "thinking" way too easily. I don't know where all the classic hacker skepticism and desire to know how things actually work went the past four years.
Whenever people get their undies in a bundle about this argument I'm amazed at how oblivious of power differentials they are.
There is a difference between a story in which a robin hood liberated knowledge and freely, without any caveats or exchange, delivers it to the masses (think Aaron Swartz) and a story in which a power hungry empire systematically scoops up all the world's resources and decides to permit the masses access so long as they exchange (at a premium) other resources the empire needs to continually solidify its position.
If you don't see how these are different I don't know what to tell you. People making IP theft arguments are more disgruntled about the massive power imbalance between, say, a massive corporation, and a small time creator who can barely even earn a subsistence off their work in the first place, and the former taking from the latter in impunity.
The left rightly sees it as unfair because it is a power imbalance that leads to asymmetrical power of exchange. The companies get the power to continue to extract data from you, all you get is the privilege to then pay them for it.
Trying to centralize all information and "intelligence" in one or two providers, charge for "intelligence" "like a utility", and divest citizens of their own skills and recourse to resources is not "expanding access to the commons" lol.
There are several papers about "deskilling" and cognitive debt as a result of heavy LLM reliance. Ask your agent to put together a reading list. (If you ask your agent to read the papers for you, there you go, you are an existence proof of the deskilling).
That's all well and good, until the theft and spread of knowledge leaves you permanently unemployed.
Because agents can do labor this is more than just "knowledge sharing". It undercuts human labor power and allows for even grosser concentration of capital.
It has no value, and you can tell basically immediately.
People are becoming extremely lazy and shipping the first slop the model generates because they get a rush from putting their stuff in tot he world as fast as possible.
However, I would bet significant money that all the projects like this (where every aspect has hallmark slopisms (the vibe interface design, the copy) will lose out to projects that are also heavily AI coded but that took the time and care to actually polish and differentiate themselves and thus are later to the game because they shipped after a few weeks or a month of work and refinement instead of a day.
AI can save you a lot of time but it can't save you from your own lack of restraint and your inability to discern and be an objective judge of the value your thing will have to other people. For software, mere existence is no longer sufficient anymore. Anyone can generate whatever you made instantly if you put zero effort or discernment into it. This is why so many of the Ainfluencers are emphasizing "taste".
But even taste won't save you on the end either, since anyone can clone your stuff as soon as it ships.
I personally think the future of software is probably just two extremes:
1. Personal dinky vibe coded apps that all have their individual creator as the sole user.
2. Large scale apps that don't work without significant marshaling of hard physical resources (e.g. LLMs themselves).
All the middle bits, user applications that target a specific problem space but that run locally and/or don't need legions of data centers and networks to run, will probably just die eventually. Maybe a couple of established apps will stick around in business contexts because employees all using their own personal apps and workflows causes too much coordination overhead, but at the consumer level I fully except everyone will be just vibing their own stuff pretty soon here.
I doubt it, as the principle of "garbage in, garbage out" still very much applies to LLMs. With or without AI, I'm not so naive to assume I'm going to quickly produce better software to solve well-established problems than others, with access to the same tools, who have put more time and effort into producing substantially similar software.
For specific and uncommon use cases? Sure. For something as generally applicable as a photo editor or a general purpose OS? Not a chance.
> A model is _sort of_ designed to predict the next token in a string of text. But it's equally accurate to say it's designed to edit code until it passes a suite of unit tests.
Training is not design. Design is the mechanism, namely a statistical method of learning distributions, and of executing those distributions computationally as an IO function.
The model is not designed to write unit tests. It is trained to do so, i.e. the distribution learned and reinforced is one such that it maps the I distribution to the O distribution that is what human beings typically do or want when writing unit tests. The design is the suite of algorithms used for this (which is ultimately global next token prediction)
I think you are conflating design (the mechanics) with their cultural or user level function.
You are basically saying "my bike isn't just rotating wheels on a frame because it takes me places". Hopefully you can see how both parts of that statement are true and why contrasting them as if the first clause is somehow false or inaccurate is incorrect.
Both these are true statements. They operate at different levels. People are constantly conflating these levels in the "is it a next token predictor" (the answer is yes) debate.
Yeah, fine. I was using the word design loosely. I'd also happily say something like "humans are designed to like salt and fat" and feel like that was a pretty reasonable statement, even though humans aren't designed either.
reply