So let's assume a completely naive approach. modify any weight arbitrarily, run an eval if eval improves arbitrarily modify another weight and repeat ... do something like 5,000 rounds. Just brute force, zero-intelligence, random selection. how does the naive base case perform on N=20 random trials?
Unless you include a basic test like that these results are meaningless. Was there p-hacking? Why just these 3 tests? Did they try others and it failed?
I'm currently running this myself. I'll see if the naive approach improves results. Can't wait to have these in hand and have people call me stupid when I bring it up with empirical evidence, it'll be awesome.
What you're describing is a shitty version of gradient descent, which is the technique used to train LLMs and other neural networks. Proper gradient descent simultaneously calculates how much every weight improves the output, and edits them all simultaneously in proportion to that.
obviously. unless you have the equivalent of a coin toss, no metric carries implication. Does naive noise give you nothing? The null hypothesis needs to be established for these particular evals with this particular model.
Doing basic empirical hygiene shouldn't be too much to ask for.
For instance, on their base version of qwen if your turn thinking off you get a 52.0% on their evals. That single flag recovers ~72% of their entire claimed improvement - base model, toggle thinking off, scores go up 72% of their improvement.
I'm still investigating various wildly naive modifications. If I can hit their numbers doing inarguably stupid things then it puts their substantive claims in doubt.
I've personally stopped believing we will ever return to a world where anything computer isn't at extortionist nosebleed prices as if they're made by Patek Philippe.
Where can you get a EPS-32 for 1,50, with shipping, taxes and duties included?
Because in the EU whether it's amazon or AliExpress, the cheapest I can get one now would be 4,50 Euros a piece, and that's only if you buy them in bundles of two or more.
Or is it that the UK lacks the lunacy parcel taxes and duties of the EU so things are much cheaper over there?
But even paying 5-30€ for a devkit still seems insanely cheap to me; that's like several bars of chocolate at current prices, even kids can afford that (or make someone buy it for them).
There's no escaping the fact that EU fucked the hobbyists that previously were able to ship stuff not even made or available in EU at sane prices from china. I f I were to buy things I did in last 12 or so years based on current rules, I'd pay like 200% on customs/taxes and bullshit charges like 3+EUR per made up category on invoice or whatever it's now.
Hopefully, it will put a downwards price pressure on middlemen/resellers, now that buying directly is prohibitively expensive.
>There's no escaping the fact that EU fucked the hobbyists that previously were able to ship stuff not even made or available in EU at sane prices from china.
EU doesn't care about hobbyists, it cares about protecting the interests of the monopolies of the major retailers who lobby the politicians, who disliked that people could order the same Chinese shit they sell at 100x markups, directly from China, and cutting them off as middle-men.
But don't worry, in their heads, this way we'll get to on-shore our HW/electronics manufacturing industry back. Any day now. I can already see the factories pumping out ESP32s "Made in the EU", popping up everywhere across the EU, creating jobs and a competitive electronics industry. Alarm clock buzzes
"Fun" fact, they charge VAT on top of the new "handling fees" (aka tariffs). Each item category is now 6.05 euros extra in Spain (5 euros + 21% VAT).
They also seem to have no way to really tell item categories apart, so it's effectively a fee per single item. You can buy multiples and only get the extra charge once, however for example, if you order one ESP32-C3 and one ESP32-C6 from the same vendor, you will likely be hit with 2 * 6.05 even though technically they'd fall under the same item category.
Obviously, in the context of hobbyists at home, we were talking about the ESP32 SBCs, and not the bare solderable chips themselves.
>But even paying 5-30€ for a devkit still seems insanely cheap to me
Yeah but I wanted to know if the 1,50 price is realistic since it's quite a big difference, 200%+, between those two price ranges.
If price differences didn't matter for hobbyists, then Arduino wouldn't have been killed, and ESP32 taken over the world.
Clearly both people and businesses are highly price sensitive, especially in the current economy.
>that's like several bars of chocolate at current prices, even kids can afford that (or make someone buy it for them).
Needless whataboutism straying from my point, as with such line of reasoning, you can justify anything that's bad: "Getting punched in the balls isn't so bad, because you could always get cancer which is much worse."
A lot of things have changed since then: EU taxes and duties, AI-induces HW shortages and price hikes. For example RPI prices today are untenable compared to a year ago.
That's why I asked about the prices of today. Whatever people paid in the past, is not necessarily applicable anymore.
This is the way forward these days. In my opinion the situations in which a Raspberry Pi is the right answer are vanishingly small these days - for interacting with physical things like relays or LEDs use an ESP or similar, run Micropython on it if you're not into writing C and the like, or ESPHome if you want something dead simple.
If you need an actual computer there are so many options along those lines now, and almost all of them are better than a Raspberry Pi in various aspects. Finally, if you just want to put some stuff on a screen then most TVs already have an operating system and a web browser on them, target that and you can get rid of the computer dangling off the back entirely.
I’ve said this other places but an x64 processor is probably better value these days unless you need video and GPIOs on the same device for some reason.
Even then, you can get a USB to GPIO adapter. I don’t use RPis much these days and it’s kind of a shame because RPi + Python usually has a quicker dev loop than C + esp-idf. I need 5 of some of them though, and I’m not buying a dozen RPis these days.
The prices will come down eventually, even if not all the way down.
But I'd be more concerned about computing staying open in general - not only as open source but at least open and accessible standards, interfaces, open to individual development, open to tinkering.
I see a concentrated push from almost all sides to lock down any platform, lock the hardware and build black box devices, lock the software under layers of attestation and certifications, all in the name of "security". I use the quotes because I don't believe security is the top concern, or even a concern at all, it's about taking the control away from users.
For now the open-computing world is still bustling and accessible but the intention to curtail this is there, with technical or legal measures.
It is a cycle, and one may bet millions of annoyed nerds will invent something other than ddr dram... after all they always have in the past.
Blitz-scaling LLM for fictional future revenue assumes competing technology will remain immutable. It is a false premise with dire consequences for the credulous.
China is already filling the vacuum within the flash/ddr/cpu/gpu market, and quality/speed will rapidly improve with domestic use. We are already seeing blank label storage clones emerging in international markets. =3
Other datacenter prices aren't up 500%. CPUs, tape storage, mainboards, racks, cases, PSU, infiniband hardware, HVACs, all price stable.
But consumer grade RAM? Up 500%. USB disks? They're hundreds of dollars.
Also low density platter is up 2-300% as well. Don't tell me people are building out datacenters with 16tb spinning disks, I don't believe you.
It's companies that have price fixed doing it again.
Their EPS are up like 60x. Their stock is up like 10x and we're supposedly like "well I guess this thing you could sell for profit at $20 is now $960, cool"
And what's micron doing with it? $35 billion stock buy back. Alright. Asshole prices it is.
Prices of cheap 10 year old tech continues going up at crypto asshole speeds. Everyone should be insanely angry.
These prices aren't going to stop until either a non cartel player enters the market or there's a DOJ that's not just a bunch of kalshi and crypto scam bros.
We'll see what happens when everyone hears "black Friday? Nah, bro, prices are up another 20%, lol"
Micron made the mistake of letting the dog drive, and ultimately tied their brands fate to a dubious high-risk/high-reward market. The dram situation has already hit SSD/HDD, ram, and PC sales... as consumer dram chip supply chains are choked off by the hyper-scaling hardware investment loop demands.
"Pride goes before destruction, And a haughty spirit before a fall." (proverb 16:18)
>Don't tell me people are building out data-centers with 16tb spinning disks
Inexpensive JBOD HDD storage is still around $25/TiB, and while higher than tape ($7/TiB) it remains viable. The dram cache chips on the HDD PCB are likely more expensive than the old hardware now.
Personally, I have gone with old 25GB BD-R ($21.50/TiB for Ridata disc) for archival storage. It is janky, but the lowest cost cold-backup solution at the moment for home-scale projects. The 7 year disc MTBF should push past the 2032 estimated hype-bubble supply chain DDoS.
BD-R are slow to burn/access, hard to auto-load, and not a viable option for LLM/bitcoin project folks. I thought the tech was EOL 10 years ago, and yet here we are. =3
Funny story about 20 years ago I was at some party for I think EFF or PGP in sf, I had no idea what stumbleupon was and I met the ceo, it was already wildly successful and he described what it was to me and I was like "oh interesting idea, have you launched it yet" and he gave me the most confused look ... still remember it.
They have so much accountability. Everything is in public for inspection, they're subject to elections happening whether they like it or not, they have to follow strict laws and can be sued or even imprisoned, there's oversights and review boards. Depending on the position there's even television cameras broadcasting their work live. You can just walk in and sit down on them. In the federal government everything you do becomes public property and enters the public domain immediately upon creation. There's way more accountability...
In practice what the "small government" ideology actually does is get rid of these accountability mechanisms and put scammers and corrupt people in charge instead or relabels corruption as the policy through public-private partnerships.
Not everything is up to inspection; elections are about gathering votes - not actually improving infrastructure - and conflicting interest often result in heavy and broad compromises; oversight/review boards often just do the bare minimum. Get rid of intellectual property, and everything will be public domain.
Furthermore, nothing prevents applying these oversights over public-private partnerships (which is just a fancy term for outsourcing). The government decides who gets hired and on what conditions, so it can just enforce transparency requirements.
The automobile key pool used to be small. Some Eastern European car manufacturers had only a few, the joke was they wouldn't get stolen: once you've been inside one you've had enough
Yes, but coverage outside the US is surprisingly sparse. Many European countries (Italy, Spain, France, Germany, Greece…) have fewer than 20 points of interest each.
Because hosting with some other entity is not what you want for a regional focus where you want to set your own relevance criteria and licensing agreements ideally.
Germany and Austria have quite a bit of local Wikis with more than 20 active editors.
Usually as some entity adjacent to the cities archive or run by some local foundation.
Testing products, getting a feel for them, thinking about them, talking and exploring, I mean this is still there and it's long been the actual time-consuming part of making great software; where the soft skills intersect with the hard.
Maybe the technical stuff is more accessible then it was beforehand but the thousands of hours of the other stuff that leads to good software is still thousands of hours.
Quality has become less work, but only by a little.
But this doesn't mean people value the craftsmanship because they never did before...
It's always been simply a popularity contest and now it's that more than ever.
The question is, why are you spending your time building things? Is it to make things that are finely crafted? Is it the road to wealth? As a tool to get other work done? Just make sure you have an answer.
reply