Doesn't the author address this point? They speculate that it could be attained through self play back then using a ton of 1996 compute and time, but it's theoretically possible.
It comes up in evidence-based parenting programs. It helps them connect the feeling word to their internal state, which is a step toward building emotional regulation skills. You're also modeling empathy for them. But one issue rarely talked about is how parents tend to label uncomfortable emotions far more than other ones, which can inadvertently reinforce them and cause them to happen more frequently.
I have a pet theory that there's some overlap of writing patterns that A) indicate quality, clear communication and B) those that the LLMs have been converging on. As a result, we're ruining the reputation of some great writing patterns because we've associated it with AI slop.
My pet theory is that this was happening, because labs naturally worked to source highest quality data sources they could get. Even if they initially slurped everything indiscriminately, the obvious next step is to sort them by "quality", and bias towards the high end while trimming from low end.
So obviously, government documents, top publications from liberal arts scholars and STEM > Wikipedia > Reddit (and within, r/AskHistorians > r/all, etc.). The first group is the one most likely to (over)use correct / fancy typography.
I have a related pet theory: the telltale sign of AI writing is being too good for the context it's found in, in terms of vocabulary and style as well as structure.
What is jarring about this is precisely the quality - we associate certain styles with important (or self-important) venues, so it's uncanny to suddenly see random shitposts, copywriting spam, or regular chat replies look like work of NYT bestselling authors or legal scholars - it feels wrong in places we expect basic language, grug-brain reasoning, and lots of typos.
Also, what makes the above much more noticeable, is that AI systems don't have cross-session stylistic memory. From the POV of an LLM, that perfectly fitting, masterfully crafted starter is the first and only time it used it. From the user's POV, it's the tenth time they're seeing it in the span of an hour.
There's also no voice. If I get a text from a friend, I can hear them. Their tones, their mannerisms. I can't hear Claude because it's not a person that I've met and interacted with.
People use language in their own distinct fashion. To the point where you can often figure out the writer from a long enough sample. Newspapers and internet comments end up a blend of different voices but in visually distinct boxes.
LLM intentionally randomize output to cover up all the plagiarism, but this largely destroys the consistency that cause writers to feel unique. Larger patterns still emerge but it just feels stale.
Models are of course different from humans, because the process by which they were created is different. But what’s undeniable is that humans also love to quote or repeat what others have said — we learn from our own kind, that’s how we gain experience. Yet rarely does anyone hold a person to a word-for-word standard over the course of their entire life, checking how many of their sentences were picked up from other people’s conversations — and then call that infringement. Just as no one right now would say I’m plagiarizing a model’s speaking style because I use em dashes, and no one would consider it misconduct when I reference a famous person’s ideas in everyday conversation. I don’t think models intend to infringe; it’s simply how they learned language. But perhaps, just like humans, they need to learn to provide more citations for what they reference.
Even individual words can be an issue. Humans understand when you want a new story about a magic school there’s issues using Hogwarts as the name.
LLM’s will happily regurgitate multiple paragraphs practically verbatim if you don’t specifically design them not to behave in this way.
Because ultimately when you design a system to best predict the next output based on a given input perfect replication of prior training is the correct response mathematically.
That's a misconception. LLMs are trained to suppress regurgitation, not because of plagiarism concerns, but because quoting training data verbatim means the model didn't learn from it, just memorized it.
Models absolutely do have voices. Spend some time using them, and you'll see. I can easily tell Claude-written prose apart - a consequence of using it daily.
You misunderstood, “Larger patterns still emerge” means yes you notice LLM’s but it’s for different kinds of patterns than you get from people.
> not because of plagiarism concerns
I’ve second hand accounts of meetings focused on exactly the problem of how much verbatim text was being spat out and what to do about it. Some versions of specific models are far worse at this but it’s a huge legal liability not to address it.
can't say the same for newspapers or internet comments, but when reading books there's usually a voice or a "vibe", and if there isn't one then it's just an unreadable book
I didn't say it's copying anything. It's learning styles - all of them - in pre-training. Post-training is just about giving it the right defaults.
Doesn't change my point anyway - even if post-training, labs naturally want the LLM to come off as smart (well, except for Grok). Post-training is where they "untrained" em-dashes and such, due to complaints of users.
I did at the beginning, but there is only so much you can take, and no one cares if you have read books and seen it before. There were a bunch of programmers whom I thought would understand em dashes, but they did not and said no one uses such dashes only normal hyphens.
Em dashes ended up as one of the first victims of llm age
Em dashes were already "doomed" at least half a century ago, because they didn't make it into ASCII character set - which arguably is just because they were seen as redundant on ancient typewriters; that's also why they aren't there on normal keyboards, which was a double whammy: inconvenient for programmers to support, inconvenient for users to type.
In other words, since the very start of computer era, the use of em dashes was always a signal.
I think it's because it starts writing the final output and doesn't have the ability to go back and correct. It's like a stream of consciousness bandaid
An example of a past technology that there was substantial motivation to control would be napster. It changed overtime, and you could never really control online privacy. Once local models are good enough, I don't really see how you can control that.
In this analogy, the dogs understand how their leashes, fences, etc. work better than their owners. And you need only take a trip to the park to see how many owners let their dogs walk around without a leash.
Not saying you're wrong. But maybe there's a reason Tiktok got so popular. Maybe for many people this way of communicating information is more effective and doesn't feel as boring/tedious.
Overall generic drugs in the US are indeed cheaper than in Canada, but that is not the case if you don't have insurance coverage. Without that, it might be cheaper for some to order the Canadian generic if they can't afford insurance.
The other factor is not price, but availability. Some generic are in limited supply or are outright not available in the US.
Generics need to be more carefully sourced if outside the U.S. I prefer brand names from reputable manufacturing areas (Singpore, Germany, etc.), and can get them for close to the price for generics, often starting research at PharmacyChecker.com
> you then also make it so that it can't be workers, what's left?
What was the core reason that readers gave donations? Probably most donors want employees treated well and to ideally keep their jobs, and yet the main thing they probably had in mind was the existence and continuation of the service itself.
> or when they’re so bad that people might actually die
Care to elaborate? Am I understanding correctly that the implication here is that there is some condition for which a software engineer having a union agreement could prevent that worker from dying?
Yes, certainly: the ability to say “no” to abusive labor conditions without risk of irreversibly losing any source of income while living in a society that treats lack of income as deserving of whatever pains and suffering come with it.
The word “any” is doing a lot of work here—I’m implying a denylist or “social credit score” or other way of persisting punishment against an individual beyond mere job loss.
You’re right that we’re not there right now. We’re in the wide middle and still closer to the “we have lots of bargaining power because we’re in demand” end of the continuum—_directionally_ it does not feel that way, but we don’t have a ton of folks dying of Boolean lung in the code mines. But I’d not rule out the possibility. The range of possible human experiences is vast.
I’m not even sure that’s a useful question. It almost points at “what’s the other end of the continuum so we can discuss points on the axis”, which _is_ useful, but I’m not even convinced that “merely” treating those without income as deserving pain is at an end of a continuum itself. It’s a point upon it; things could be worse yet.
Instead let’s break away from opposites and into pieces. What would be _different_ in a society that doesn’t judge harshly the less fortunate?
I’d suggest that would include ensuring that nobody starves or sleeps on the street for want of work.
Richard Rorty describes liberalism as the idea that cruelty is the worst thing that exists. Cruelty, of course, has just as many forms as suffering.
Universal income protects only basic needs and it does so in part. Even if you have income, you need to have housing, clean water, food that isn't poisened and some form of healthy environment to live a good life: income doesn't guarantee it.
On the other end of the income spectrum is the unlimited amassing of wealth. After some point, wealth turns into power. For example, Musk is able to buy twitter influencing public opinion, and Starlink can make or break the battlefield.
If we are talking only about income, I think we would need both a minimum (universal) income and a maximum (a cap on wealth). If we do not, then we will eventually lose democracy.
>It looks like a healthy extended family where resources are shared towards a common goal of health and liberty.
In the Europe, like several rich western countries, we mostly replaced the extended family with the government welfare state, for better but mostly for worse.
So current young workers are contributing to a failing state ponzi pension system that will leave them in poverty by the time they retire in several decades, and they will also lack the care of the extended family unit that was the norm decades ago thanks to collapsed birth rates. Double whammy.
Unfortunately this does not work at nation scale. We are not insects. Tribes fail at levels over a few hundred and that scale only works because of blood ties. Communism has failed to produce good outcomes every time it's been tried. Even coops aren't competitive with traditional corporations.
What if health and liberty to someone looks like them not providing any net positive to society? What compels someone to work 12 hours a day to share those resources with such a person?
You’re making a point about loafers but non-wage earners are always going to outnumber wage earners, and able bodied loafers are a minute fraction of who we must provide for (the young, the elderly, students, the disabled, etc)
It mostly just feels like basic maths, not really a world model. Productivity is a finite resource. If there's a deficit in productivity, people will necessarily suffer and, in the worst case, die. The problem is world views that suggest giving everyone a free choice that enough people will choose productivity while everyone else chooses leisure. I'd be more apt to believe it if we had evidence of it being a natural outcome.
It doesn't have to be so extreme though. Basic scenario: 20% of everyone's income is taxed and then distributed equally to everyone (no matter what, and without setting any fixed dollar amount), another 10% pays for administrative functions and government stuff except welfare, and the remaining 70% is theirs to do with as they please. What would happen? Would everyone quit their job? I don't think so, because 70% is a lot. Some people with jobs that pay so poorly their 70% is less than everyone's average 20% would quit, but that's good, because those were zombie jobs.
I'd be fine with this so long as people who didn't contribute any taxes couldn't vote. Otherwise you're just signing up for those who choose to work to get exploited by the leisurely.
Are there historical examples of this ever happening in any significant numbers? This idea that people en masse just won't work and live off of the state where opportunities for work are reasonably available seems more like a boogeyman than a real thing. (obviously, if work is not really available, I can see this happening, but I wouldn't describe the people unable to find work as "leisurely" in that case)
The screen time crowds out time they could be interacting with the real world, engaging with all of their senses, and interacting with people. Two hours per day adds up over a critical time of their life that their peers are beginning to learn how to accurately learn to model the world and communicate.
At one years old, 2 hours is massive percentage of their "awake and aware" time, considering how much time they sleep and eat at this age.
reply