No one designs for users any more. It's all gas no breaks on the path to enshitification. A lot of this is the motivation of the people building software.
Raise your hand if you ever saw a PM get promoted for removing a bunch of bad and/or unused features?
Most giants in highly technological areas don’t evolve, almost by definition. They get to a point where they’re mature companies just operating an existing business, and that’s diametrically opposed to developing new businesses.
Sure, they can spend money on R&D, but then they’re competing with every other new venture in the world - the only benefit they have is money, but collectively, everyone else has more money than they do.
This is why when private equity takes over a mature business, they almost always shut down any R&D the company is doing. Silly mature business, that’s not the reason for your existence!
This appears to have been true for the last few decades, but not before.
From the first World War up to around 1980, a very large number of the most important technological innovations have been made in the research laboratories of a few giant companies, like ATT, IBM, General Electric, Westinghouse Electric.
For some reason, since around 1980 almost all big companies have focused much more on the financial results of the next quarter than on long-term research, so much less innovations were done in them (with a few exceptions, IBM has still produced a few very important innovations until around 2000, but those innovations were less important for IBM products, their impact was due to being licensed to others).
Mature businesses like e.g. tire or appliances manufacturers still have some R&D trying to make their products better. However, it may be more for appearances than for actual improvements, just because consumers like to buy a "new fancy model" over a tire or a dishwasher that have been in the market for 20 years (believing the new thing must be better).
>IBM, GE... how many former American giants failed to evolve.
"How many failed to evolve", on its own, is a bad metric for success, because it misses the how many new giants the US economy managed to create, which IMHO is the more important metric of a healthy economy.
Companies are like people, the old ones die out and are replaced by new young ones, as it should be. Worst you can do is be like Europe where all you do is try to keep ancient dying companies on life support for the sake of "stability", instead of creating new ones to replace them, which is also a fitting metaphor for EU demographics where a third of the state's budget goes to keeping boomers pensions higher than the salary of average worker(in France).
My first program was in 2nd grade on an Apple IIe in basic. It was a program to check if a word was worth a dollar (a=1, b=2, c=3...)
It became a family activity (every one taking turns at the computer) every night to just type in words and see what they came out to. It was all very boring.
I was sure I was going to win.
I lost (well 2nd place), to the kid with 6 older brothers, who had that many years of building out the family list.
> For any application at scale it'd be way more accurate to have a dedicated classifier
"at scale" or the above use case of "specialized domain"
Jev being cheaper than an a full LLM call creates places where other tools might work better later - when the volume or the available data warrants it.
System optimal, code optimal and cost optimal aren't always aligned. Pick the right tool for the job - and dont get married to it. On occasion worse is better, and moreso when your testing or it is a one off/disposable.
Caring about something also means you don't want the issue to be misrepresented.
This is the most effective altruist anti AI headline you could get out of this story.
250k trees seems like a lot from you looking out your window. From the perspective of trees we cut down every year (that we account for) it is close to a rounding error. This is a place where they take forest management seriously - and thats the only reason any one even noticed or had to get a permit.
Oof - it's not like goodfire - we have quantitative results and a clearly defined method!
But yes, we might be bad at marketing. Internally the term we use more often is "concept elements".
The intuition is that if you take a complex treatment of a topic (like an academic paper), you should be able to break it down into some more abstract self-standing ideas. Then, critically, these units are composable and easily embeddable (i.e. they don't have proper names and numbers).
Here's a couple examples of these idea atoms pulled from our paper https://lacuna.tiptreesystems.com/work/the-alien-space-of-sc...: "Scientific ideation often conflates the logical coherence of a research direction with its likelihood of being discovered by the current community. This oversight makes it difficult to identify research paths that are scientifically sound yet unlikely to be discovered due to the existing distribution of expertise and collaboration networks."
Another example: "A generative model estimates the plausibility of concept combinations by treating conceptual units as tokens and calculating their length-normalized log-likelihood. By training on random orderings of these units, the model learns to identify which components are likely to co-occur in a valid research contribution regardless of their sequence."
> Oof - it's not like goodfire - we have quantitative results and a clearly defined method!
Goodfire is a discovery from research, an ahaha moment.
"concept elements" would have been a better descriptor in both cases. The V for the cats and the cars, and for your idea.
> 16,068 LLM ... The Coherence Model is a 42.8M-parameter transformer trained
My question is this: was this the smallest model you could train on the 16k papers? Isn't there an optimal small size to identify everything that is common?
Wouldn't it make more sense to use such a model to strip OUT the commonality? Get rid of everything where the "existing distribution of expertise and collaboration networks" are already optimal - and what's left is the novelty?
Ultimately I would be curious to see if your existing "atoms" happen to follow Zipfs law...
> I made it to the first figure in the paper before I raised an eyebrow at this term.
I came up with that name, and fair eyebrow. Think of an "atom" as a unit of a working vocabulary (a "token"), not a Platonic form. It's a way of compressing the corpus (paraphrases of the same idea across papers get merged into one symbol, a bit like a tokenizer's vocabulary).
We also talk about some limitations of this framing in page 11 (towards the top).
Is formal verification the path forward? Meh... I dont think so. Does it have applications in some areas: yes. But this is just more robust "unit testing".
Anthropic keeps trying to convince you that "one agent" will do the job. It does so with the statement "We built 36 small verification tasks from real OSS-Fuzz code. Opus 5.5 proves half of them with about 40% fewer tokens than Opus 5. We expect this trend to continue as models are trained on long verification tasks."
But this is not the path forward. Robust E2E, durable (can your app go from python to rust with out loosing tests) Simulation/Emulation tests are the way. This is an "in bulk" and "brute force" activity. Go look at TigerBeetle for an example of this in practice. Go look at how oxide is doing testing.
Can a swarm of cheaper agents write these tests more effectively that Anthropics premium products. I assure you that it can.
If you look at OAI's actions (hugging face hack) and anthropic own publication on its version control AI is not a best of breed solution its a brute force hammer at it till you get what you need solution.
I would like some Russian Literature recommendations related to this. I vaguely recall We by Zamyatin addressing something like this a good few decades ago.
I'll post a comment snarkily mentioning how tedious Russian Literature is, then you flag my comment to attract dang's attention, and hopefully he will manifest and reply with some excellent Russian Literature recommendations and links to relevent hn discussions! It's one of his many special interests.
Not only that, dang is also a proud card carrying member of the Klaus Nomi Fan Club. (I shit you not!)
>"I remember Pat Gibbons, who was Bowie's manager at the time. He comes up and he goes, 'Well you're going to have to have something. You're going to have some costumes.'"
>"And he whips out a big wad of cash, and he goes, 'How much do you need?', and he starts peeling off bills. And Joey and Klaus look at him, and their eyes widen, and they look at each other, and they go, 'We're going to need shoes, too.'"
If you're going to do this: start with: Dostoevsky, Tolstoy, and "Doctor Zhivago". These are the Poe/Twain/London for Russia (to some extent).
I highly reccomend Russian film as well: Stalker and Solaris (Not Cloonie, the Russian version) are also great. I have a bit of a love of Sci Fi, may not be your tastes.
You should also hunt around for anything and everything you can find on early Russian computing, Cybernetics, their efforts and thoughts on central planning, and the ternary computing projects they did.
If you're looking for examples critical of humanity you have lots of options (mostly from the last two centuries). Notes from Underground and The Death of Ivan Ilyich fit the bill.
I reread the former relatively recently (first time since college). It's dark, not for the easily affected.
Isn’t the latter an excerpt of The Brothers Karamazov? When I read it, it felt more like a criticism on certain economic theories, rather than people specifically.
Nope, separate work! But indeed, it can be considered both... The emptiness of bourgeoisie/wealthy values and the superficiality of the humans that host them
Russian Literature is so tedious. There is no Russian writer better than America's own Adam Sandler.
People dismiss Jack and Jill as Adam Sandler in a dress, which is rather like dismissing The Brothers Karamazov as some men being upset about their father.
Dostoevsky's The Double concerns a man tormented by another version of himself. Sandler advances the inquiry by asking: what if that version came for Thanksgiving and wouldn't fucking leave?
Casting himself as both siblings makes Jack's disgust at Jill an unmistakable act of self-loathing. He longs to exclude her from his comfortable bourgeois existence, yet everywhere he looks, there is his own face. Critics complained that Jill was unbearable, apparently unaware that estrangement from the self is supposed to be uncomfortable.
Al Pacino, appearing as Al Pacino, completes the study of fractured identity. His pursuit of Jill suggests a desperate attempt to encounter something authentic beyond performance. This culminates in the Dunkin' Donuts commercial, where even his name is consumed by capital: Al becomes Dunkaccino.
When Pacino demands that the commercial be destroyed, he finally achieves the moral clarity denied to the other characters. The audience has generally reached this position much earlier.
It's so unfair that Sandler won both Worst Actor and Worst Actress, while Pacino won only one award: Worst Supporting Actor.
This too is subjective.
No one designs for users any more. It's all gas no breaks on the path to enshitification. A lot of this is the motivation of the people building software.
Raise your hand if you ever saw a PM get promoted for removing a bunch of bad and/or unused features?
reply