I think the HN crowd overestimates the quality of software. There is a (very) long tail of truly abysmal software that people are forced to use all the time. The reason it gets to be abysmal is because it’s often back-office admin software that helps people file forms for, say, logistics paperwork or school district expense tracking. AI-native engineering teams can create dramatic improvements for that long tail because the people running that software are either incompetent or checked out, and a pull towards average quality by AIs would yield great improvement.
One of the most uncomfortable truths about our profession is that there is literally no floor to how shitty software can be and yet still make people billions of dollars.
Reminds me of an incident where I had to go pick up a missed delivery (an expensive photo printer) from the dispatch warehouse because the shipping company couldn’t get a signature after three attempts.
Whatever, I drove down to this set of huge industrial warehouses with trucks coming and going by the dozen and found the customer service office.
“That’ll be half an hour, I recommend waiting at the cafe.”
“Half an hour for what? Aren’t all the packages right here?”
“They are, but looking up the package ID in the system takes at least 30 minutes, maybe 33.”
“33!? Wait… don’t tell me, let me guess: it has been getting slowly and steadily worse over years? Predictably taking longer and longer, right?”
“Yeah, six months ago it was 27 minutes! How did you know?”
“Umm… isn’t this a bit of drag on your operations?”
“Absolutely! Everything takes forever, every shipment is delayed, … …”
“I… bet. Why hasn’t anyone fixed it!? It’s a one line fix!”
“No budget!”
This was one of the biggest logistics companies in the country about to undergo a multi-billion-dollar merger.
Their core business software was missing database indexes even on primary key columns!
Usability of internal tools is just never considered. Priorities are driven by management. They invariably just don’t care about feedback from the peons that actually have to use them in their day-to-day jobs, even when it results in tremendous inefficiency.
This also allows companies like Workforce to make billions selling the most godawful software imaginable.
> There is a (very) long tail of truly abysmal software that people are forced to use all the time.
We generally call it “enterprise software,” and it’s usually abysmal because the people buying/funding it are VP-level executives who care about compliance/reporting/visibility and who never have to actually use the software.
This actually comes up on HN pretty regularly. Or used to; like many topics it’s taken a back seat to AI.
...and now with AI much more of that dreadful software can be built with all the bullshit featuritis that said managers who don't use it think they want.
Right. All the boosters crowing about how amazing and so much better LLMs are than us are just doing marketing. Our jobs will be taken or ruined and people and companies will just be forced to get used to an even lower standard of quality as they've been doing already.
I think this is an outcome of playing with systems that we don’t actually understand very well on a deep level. We have found recipes that work, and have used graduate descent (so many PhD theses) to understand and optimize these systems, but we’ve built a model of language that also encodes reasoning, and those things are extraordinarily complex. Providing proper mathematical rigor for their explanation requires, as you say, the level of commitment equivalent to a math degree. Also, because language is so complicated and subtle, you need highly precise statistical models of it, and that just takes a shitload of data; hence the scale. You can train toy models at home, but they will suck because you actually need massive amounts of data to capture the nuance that language requires. And that’s to say nothing of multimodal models.
I think the level of interaction on forums like HN is a great example of the still-yawning gulf between LLMs and humans when it comes to encoding reasoning into language. Even with the self-selection that comes with posting here, think of how many times it is almost impossible to tell if posters are shit-posting, serious, angry, sock puppets, or just weird.
You say "...because you actually need massive amounts of data to capture the nuance that language requires..." but humans don't require that! So why do LLMs?
How many books did you read in your education? Far fewer that what LLMs read, no doubt. Of course we "have a life" and so we interact with hundreds of people, which is also a form of learning.
But in any case there's are fundamental differences here between these two systems (LLMs and humans) that we have not yet discerned clearly.
Far fewer than an LLM but then I “know” far less than an LLM. Sam Altman talked about a PhD level expert on any topic; if we expect a SOTA model to answer any question on any topic in a library, presumably that requires a library worth of books go into it?
Say I ask it who are the most beloved Indian cricketers and why, it might not be correct but it will have the right vocabulary - cricket terms and gameplay I’ve not heard of, player names and styles I don’t recognise, things critics and commentators like and dislike about game strategies. And it is like that for almost any topic.
If they hallucinate a Python function, it will look like a plausible function e.g. “hn.create_user_account()”. They don’t hallucinate like
# create HN user account
# requires eye of newt and frog leg
“Oh great spirit of the internet I request and command you make a new account…”
It’s a terrific amount of facts to be able to write even plausibly about tons of topics.
We don't need LLMs that can "answer any question on any topic in a library"!
We don't even need machines that can answer any question in a specific topic area. But we could use machines that can answer "I don't know." when they cannot compute the answer.
Consider the possibility of much-smaller LLMs that answer questions only in limited domains (physics, chemistry, math, medicine, etc.)?
Could not such LLMs exist? If not, why not? Would they not likely be more useful and far, far easier and cheaper to create and run (possibly even with far less hardware) than the current "know-all" LLMs? Also might they possibly avoid certain problems ( e.g., hallucinations, deceit, apparently undesirable behavior, etc.) associated with the "know-all" LLMs we now have?
I do wonder if a medical LLM needs to know anything about popstars or films or classic literature. But your particular examples make me wonder how you could train an LLM to talk medically about "blood pressure" without knowing any physics about what pressure is, or any of the other ways the STEM subjects overlap and blend into each other without clear boundaries - chemistry of drugs, math of dose calculations, language of drug and disease naming etymology, etc.
If a patient with high blood pressure asks "will my face look like Dizzy Gillespie?" isn't it better to answer "No, that was air pressure and cheek damage" than to answer "I don't know who that is, call your doctor"?
I feel the intuition leads us astray here because the combined text of every science book ever written is much less than the size of a complete description of the human brain. To the hamburger of reason the library is a sesame seed.
We need to up the ante, we all know it's the Mumbai Indians. ;)
One can train a decent model at home, for a specific task but you are absolutely correct that in order to have a model that's an expert in any topic it requires training and data on all topics.
I nearly spit out coffee reading this. Amazing use of vocabulary; especially "... oh great spirit of the internet".
I sometimes jokingly explain to certain people not immersed in tech that sometimes saying the "magic words" (commands) from the "codex" (script) in the right sequence with a lot of hand waving allow "magic" (applications/web apps/etc.) to happen.
It generally brightens a patients day and makes things relatable to those non-technical, children, the elderly, or the like.
Humans don’t need that? How many words have you taken in from all possible sources by the time you’re 18? And many of those words came with additional data due to tone of voice, inflection and context. And then some of those (in-person, filmed art) came with incredibly subtle data from body language and facial expressions. I think we take in a massive corpus of data every year. Think just of any book you re-read: why did you do it? Did you glean additional insight?
We experience tons of data, but it's still much less than the pretraining step of LLMs
I think it's clear that LLMs are massively less intelligent than they could be, based on the amount of training data and compute they consume. Or rather, humans are orders of magnitude more efficient on training, and we don't know why
The answer is probably in the DNA. We are born highly intelligent and our brain is still growing. The DNA itself was optimized for billions of years
I think it's more accurate to say that the fundamental differences (between LLM's and humans) are abundantly clear and what we haven't really got a handle on are the similarities. Why is it that an LLM's can so convincingly mimic some things that humans do? Of course the fact that we don't really understand how humans do it in the first place is a real barrier. Perhaps looking at the intersection will shed some light on how humans work.
We have DNA though. It was subject to "pretraining" for billions of years. Alongside some epigenetic stuff
Maybe in the future we will have fewer robots interacting in the environment with continuous learning, to mimic the human experience, rather than a massive fleet of stateless, inference-only models
Sure it does! The listeners of such music have listened to lots of other music, and have developed a mental model of it that is overly predictive (too compressible) to be interesting to the information-seeking machine.
It is also possible that people are listening to the music for reasons other than the music itself (community, feelings of superiority, etc.)
The Talos Principle II has a segment where a character is frustrated with this. They complain that human fiction is extremely defeatist and hilariously memory-deprived. They do not realize that humanity had in fact, many times, kicked Death in the nuts and Death never made the ground back. They are convinced immortality MUST be a curse, and yet, our discovery of penicilin had already doubled everyone's lifespans. We were twice the humans our ancestors were, and yet we still listened to the fairy tales of those old farts, paraphrased.
I've been working on a theory that a huge fraction of human culture and values could be interpreted as coping mechanisms in the same sense (as making peace with, or finding a way to manage or compromise with, limitations, suffering, or unsatisfactoriness of aspects of life, and concluding that the things thus successfully managed or the means used to cope with them are themselves good). I've been keeping occasional notes for an article on this concept for many years. I particularly don't think that there is only one community, subculture, or ideology that exhibits this psychological tendency.
Maybe someday I can get clearer on this and organize it and write it up!
(Yes, I know about the transhumanist criticism of acceptance of mortality that this LLM's view is more specifically evoking.)
perhaps that is the original sin that leads to our ex post facto rationalisation tendencies. the moment we became prospectively rational we would be unable to content with our own forthcoming deaths.
It may be true that it might be more efficient, but it feels similar to the farming paradigm. You can actually get more calories per acre if you have ecosystems with varying heights. However, we don’t do this because it is much more scalable to drive massive machines over flat, unimpeded land.
It might be the case that my home would be more robot-friendly if I lived in a flat warehouse that was designed so that robots could easily handle tasks. But I don’t want to live in a warehouse, and I don’t want my house to be optimized for robots. And this isn’t even to mention the long tail of existing houses that do not adhere to a robot-optimized standard. The cost of replacing or retrofitting all of them would be astronomical.
"However, we don’t do this because it is much more scalable to drive massive machines over flat, unimpeded land."
It's just that it's cheaper since we're talking about land and other physical constraints. With humanoid robots we're talking about labor. Many processes done by humans are designed to be possible by humans. As soon as you remove the ability constraints of humans, you can design more efficient processes. This is evident is all jobs pre- vs post-mechanization.
For example, do you need lights in a warehouse without people? Do you need maximum shelf height or forklift speeds for safety standards? Do you even need forklifts if the bots are stronger enough; if not, aisles can be narrowed and density increased. Etc.
Even for in-home assistance, you might be better off with multiple smaller bots tailored to different tasks. A bipedal robot designed for cleaning gutters or shoveling snow sounds like a poor design decision.
This is a pretty bad-faith argument. Marriage and civilization are definitely correlated, but claiming that the breakdown of marriage is the cause of social issues needs substantiation. Marriage (and sexual abstinence) was very important for property inheritance. It was then codified into social systems as a downstream impact of property rights. A world with birth control, DNA testing, and an extensive legal system muddies this justification.
OPs description resonates with me as well, and the impact is complex, and in my case, probably more good than bad. There is a romantic notion that childish energy shouldn’t be suppressed, but childish energy is what broke my toys (and my body), and I have expensive toys now (motorcycles, house, etc.)
I have found in my own life that the practice of emotional regulation has made me a pretty good diplomat in difficult situations. Sitting down, shutting up, and following the rules when I want to yell at people is the essence of de-escalation.
Well, I feel emotional regulation is different to shutting down you emotions and your energy.
Emotional regulation is a great thing to work with (I should keep on it), but I think it's very important to do it, not like "my emotions don't belong here, I'm not valid", but knowing the differences between different people, learning how different people behave and express themselves, and finding ways of moving your emotions to different places that help you express yourself and transmit your needs. And you don't have to do it the same way than everyone else (that's one of the most important things I've learnt since my diagnose a couple years ago).
Yep. I’ve found that having an endpoint that serves a well, documented openapi.yaml is very effective for agentic usage. The biggest difference is that you can break down a REST API into RPC-like chunks and save on some tokens if you break up the tools well. But pragmatically, I think saying “tell your agent to hit /api/v3/openapi.yaml” is quite useful
We did a prototype to integrate an agent into our application and basically just gave it a tool to discover the OpenAPI spec and call endpoints. It worked surprisingly well! One caveat was that some responses were too big and would poison the context, but then I gave the agent a GraalJS engine and allowed it to save responses and post-process them using JS. For the little amount of work required this gives the agent a lot of power without having to give it full CLI and without having to create bespoke tools.
I wonder how similar the 2000s-era napster fight will look when we get a high quality music diffusion model with 0 copyright guardrails. Then I can listen to unlimited, radio of my favorite artists serenading me with songs I’d wished they’d written
The entire point of piracy and related copyright infringement is to build a path to appreciation of an artist's works without necessarily agreeing to their pricing and terms (whether imposed by the artist themselves or by publishers). What you are suggesting is the complete opposite: leveraging established or even defunct artists as mere brands to slap on generic machine-generated content that does a great job imitating said artists' style, but bears no resemblance whatsoever to the actual creative process that led to that specific style in the first place.
Record labels would actually love this model: they'd be able to put their extensive back catalogs to use by giving everyone their very own hyper-personalized, maximally-stimulating Beatles, Queen, Linkin Park and more, all without paying a single cent to the humans who actually built those acts. What breaks this utopia, and what you seem to be missing here, is that by far and large people want to listen to the real deal and not to a deepfake Chester Bennington's song on the Israel-Palestine conflict in an unprecedented act of VC-funded digital necromancy.
idk. I can imagine them getting close, but there will always be some part of me that knows that if the artist were making that song they might have chosen something different, that it's not them.
I don't want to hear Linger-er by the CrAInberries
It’s been a while, but when I was in high school, I was up at 6:30 so that I could run through the shower, get to 7AM jazz band late, be at school until 4ish, then off to my minimum-ish wage job until it was time to go to evening jazz/symphony/other bands. Home at 8 or 9; bullshit some homework for a passable grade; and then do it again (I was often doing homework during band practices). Weekends were music events.
I did not go to a top tier school, and I did not live in a top tier place. The stories I’ve heard from others indicates to me that my experience was not unique. I wasn’t even the most busy of the people I knew.
This wasn’t every single day, but it was more on days than off days. Contrarily, I had a great time in college because the pressure of classes actually felt like less than I was doing in high school.
this closely matches my experience graduating with a C average and getting accepted to a branch campus of a public university. I then crashed hard out of that school because, as you said, suddenly I actually had a lot more agency and a lot less responsibility.