This democratic party is not exactly the same as any other democratic party that's been in power. There's been a pretty clear shift in party beliefs again. Democrats used to be pro tech neolibs, defending Elon on the debate stage while Mitt Romney attacked him, etc. The democratic party now is the one that got betrayed by the oligarchs and has an axe to grind and mandate to grind that axe
You are also a demagogue. I said the game is dumb and you are attributing to me some claim of 4D chess. You want them to lose, but they won't. The game is dumb and pretty much anyone could play OK, which is what they are doing.
Ooh I can say random stuff too, that has nothing to do with the article! Like, "only someone with literal shit for brains would be out here saying 'both sides same' in 2026 with a straight face!"
You must not be LGBTQ+, or a minority who can now legally be stopped purely because of their race, or a minority who has had a war declared on their ability to vote. Or an immigrant who now feels very unwelcome. Or one of the millions who have lost health insurance. Or the many whose SNAP benefits have been cut. Or an American shot down in the street for extremist belief that we need to treat immigrants with decency and within the laws of the Constitution and that we have a right to observe government law enforcement in action.
That's exactly why I keep Windows on my NAS. You can configure storage tiering with arbitrary redundancy. You can have different redundancy on SSD and HDD tiers. As far as I am concerned ReFS gives me all the features of ZFS and doesn't eat my RAM. Also BitLocker + Secure Boot aren't pain in the ass there.
USSR bit might be true, but the Russian invasion so far is the top contender for the greatest tragedy political or not of the first half of the XXI century.
hey thanks for the comment, author here. That's a fair read of the loss numbers, I could only fund up to 1.2B scale myself and I don't want to oversell the results. But note that the benefits that Soft-MTP delivers in the paper is *consistently growing across scale*. The residual is irreducible with scale and will only become noisier as model grows bigger, so removing it via Soft-MTP is intuitive. It is also easily verifiable cos it only needs minor code change, codebase: https://github.com/JaydenTeoh/Soft-MTP
Humans decidedly take much more data than LLMs. Like orders of magnitude. We happen to feed only distilled data to LLMs though. That's why they appear so efficient.
If you are counting by bytes and including all sensory input, sure. In terms of amount of text, it's not close in the other direction. It's very difficult to make an apples to apples comparison here.
What they said, which is true, is that an LLM's inputs measured in bytes of language is much higher than for humans. Throwing in all the non-language bytes of sensory input received by humans is not valid and thus is not apples to apples. Children learning from watching a dog do zoomies around the room or butterflies in the garden is not learning language.
> We happen to feed only distilled data to LLMs though. That's why they appear so efficient.
That is wild handwaving that really explains nothing.
But then, most of the comments on this page are wild handwaving, pontificating like a blind man describing an elephant. Very little engages with or matches the quality of Norvig's arguments.
> Throwing in all the non-language bytes of sensory input received by humans is not valid and thus is not apples to apples. Children learning from watching a dog do zoomies around the room or butterflies in the garden is not learning language.
Have you ever had kids? Either you never had, you forgot, or you are intentionally deceiving us (and perhaps yourself) to support your point. If you never had kids, I can tell you you have very poor imagination about how they learn language.
The BabyLM challenge has shown that multimodal models don't seem to learn in a more data-efficient way with respect to tokens. Adding an unlimited amount of images doesn't help you learn syntactic islands faster.
Not a specific entry, but in four years, there's never been a clear advantage of multimodal models and the results have been extremely lackluckster, even when you allow unbounded images alongside a fixed quantity of text.
"This year, we have merged the Multimodal and Interaction tracks into the Strict and Strict-small tracks. This is for two primary reasons: Firstly, there were insufficient submissions to warrant standalone tracks. Secondly, none of the submissions in the Interaction track outperformed models submitted to the Strict track, and *none of the submissions from Multimodal outperformed our baselines*. We will still allow participants to use multimodal data or interactive training procedures if they choose, but such data or learning procedures must conform to the rules of the Strict or Strict-small tracks.
> none of the submissions from Multimodal outperformed our baselines
I think that's a pretty weak point to your argument. They are saying that none of the contestants outperformed their baseline multimodal models and that there was very little multimodal submissions in the first place.
Democrats had plenty of terms and yet none of the illegal competitive practices have been stopped. You are being played.
reply