What, you're all reading the books? I thought the point was to just buy them excitedly, and then let them accumulate around you and when your partner complains about them quote that thing Umberto Eco said about needing a personal library or something... :)
Recent completions:
- The Man in the High Castle by Philip K Dick. I found it well written, but ultimately lacking in something - I kind of wanted the other boot to drop at the end.
- Poor Charlie's Almanack. Stripe Press published this collection of Charlie Munger's speeches and I've found I broadly agree with him on a ton of stuff and am slightly excited by some of his ideas that are new to me.
I'm about to have some time off recovering from a planned medical procedure and expect to have a mix of sci-fi, good quality non-fiction and at least one heavy "literary" book. I'm a big fan of Borges, so I might reread some of that work, or maybe pick up some adjacent/similar authors like Calvino too.
I have roughly 48m of bookshelves stuffed with books in my apartment¹ (not to mention 25 more books sitting on my desk for which I lack the shelf space to shelve them properly and another couple dozen waiting to go to the used book store). The bookcases covering one wall of my office are about 80% unread books which I’m making slow progress at.
A sampling of the unread books (picking the rightmost book on each of the 19 shelves of unread books): The Best American Short Stories 2020, The Riverside Chaucer, Flimsy Plastic Miracles by Ron Currie Jr, I Sailed with Magellan by Stuart Dybek, My Days of Anger by James T Farrell, Antología Poetica² by Federico García Lorca, Blurred Lines* by Vanessa Grigoriadis, Just Like You by Nick Hornby, Diaries by Franz Kafka, A Different Kind of Tension by Jonathan Lethem, A Writer’s Notebook by W. Somerset Maugham, The Magician’s Wife by Brian Moore, Father Elijah by Michael D. O’Brien, Challenging Problems in Algebra by Posamentier/Salkind, Cuentos by Alonso Reyes, All the Names They Used for God by Anjali Sachdeva, Great American Desert by Terese Svoboda, A Silent Treatment by Jeanine Vanasco and The Writer’s Notebook: Craft Essays from Tin House.
⸻
1. Thanks to most of these being IKEA shelves, with one antique glass-doored bookcase inherited from my grandfather which has four four-foot shelves, it’s not too much of a challenge to get the exact total of shelf space which is 487.68cm. There are a few places where the shelves.
Yeah, I have about 5000 of them in the to-read pile. I think I'm suffering from choice paralysis right now, which was why I posted this in the first place.
Poor Charlie's Almanack is sitting in that pile somewhere. I might have to pick it up soon. And speaking of literary, maybe I should get back to that. I try to read 1-2 titles per year in that vein. Camus, maybe?
There's a lovely and eccentric quiz show on the BBC called Only Connect. Last year the host as part of her bit to fill time tried to fill it with reading excerpts from Paradise Lost. It was funny, and the in-joke I think was to remind us all of what Samuel Johnson once said of it:
> "Paradise Lost is one of the books which the reader admires and lays down, and forgets to take up again. None ever wished it longer than it is."
"working through it" does indeed seem like the right phrasing from what little I've been exposed to. Good luck!
I've suggested this elsewhere in this thread, but the analogue nature of it might be useful to current design teams to get familiar with operationally.
Getting contemporary knowledge of the navigation, hydraulic, avionics and other systems is going to be pretty powerful if you need to build something that is as battlefield-ready in the modern era. Nobody is hacking into an SR-71, and it's going to be delivering its missions just fine if GPS is knocked out. Taking the operational (and not just academic) experience of doing that and putting it into the current generation of Skunk Works engineers is going to be useful.
I'm not sure their plan is to put it into active service, but they could - the SR-72 could just be an SR-71 with upgraded engines and other systems.
However, the process of getting it ready for active service even if just as a flying museum piece could sure help a lot of engineers learn a ton of lost knowledge.
I’m going to stick with my pointless designation, sure many things can yield knowledge and if money, time, and expertise were infinite then sure, why not. But they are finite and there a million better things for these people to be doing that would yield useful knowledge more efficiency.
Engineering exists within an ecosystem and that old ecosystem is gone. Everything would be a once off. The materials science could be interesting but that’ll also be one offs and probably wouldn’t generalize.
The Russians apparently fessed up after the cold war that they knew of the SR-71 project from early on, but the only thing they knew about it in the early stages was that beautiful and distinctive outline.
The Skunk Works team knew when Russian satellites were coming up over the horizon, and so would hide the early development models they had on poles and so on. They obviously didn't want them to be pictured. However the satellites had heat-sensing cameras and so could see the heat differential the model had left as a shadow on the ground just minutes before. The Russians therefore knew there was a new plane in development, and they knew what the outline of it looked like, but nothing else for quite some time.
I bet when they first saw it, they agreed - a heck of a beautiful machine, one of my favourite aircraft of all time.
The SR-71 is mostly analogue. It has remarkable battlefield resilience including ability to operate when EMP and nuclear radiation are issues that would affect a lot of digital systems.
Worried about how to navigate if the GPS systems gets knocked out by an adversary? SR-71 has an analogue system, use that. Worried about how an adversary could use cyberwarfar against your avionics? SR-71 has you covered, use all of that.
And sure, you could take all the designs and hand them over to the SR-72 team at LM and say "yeah, all of that please", and they could figure it out, but you know what might be handy? A working SR-71 they could play with. And who knows, maybe the answer is to take an SR-71 in its entirety, and just upgrade a few parts to get the SR-72 performance (Mach 6) that you're after.
Side note: IIRC a lot of the star tracking tech still exists and is used by Trident D-5, B-52, B-1, and B-2 etc. IIRC. Most of it is analog, although I think only the B-52s version is a mechanical computer assuming its still in service. You never design your second strike capability to require a system that will be knocked out in the first round.
Crucially, the engine inlet spikes were computer controlled. And since it's an observation platform and not a bomber, is it going to be EMP hardened? That was probably classified at the time. /shrug
The SR-71 mission set included trans-attack reconnaissance for SAC, where it would overfly a radiating Europe and USSR perform nuclear bomb damage assessment for follow-on strikes. Aside from nuclear bursts and leftover detritus from the ICBMs etc., it was anticipated to operate in an environment where nuclear-tipped SA-5 and other Soviet SAMs would be bursting here and there. It's EMP-hardened.
This is obviously going to be regional, because it's cultural.
I'm old enough to have adult children, although I never did have kids of my own.
My parents would have gone to school in the knowledge that if they acted up, words would be had with their parents, and there would be consequences.
At some point, something changed, and we decided that teachers were there to be put in their place by parents, and if a teacher suggested a child was doing something wrong the parent would expect to challenge that and the teacher would be put in their place.
We're now in a situation where children being taught sometimes think its their right - perhaps even obligation - to not be instructed in appropriate behaviour, and so feel free to react the way that they do.
There's a microcosm natural experiment for this in the Pennines, the North of England. There's a small town called Hebden Bridge, which for a while had the highest suicide rate of anywhere in the UK. Some thought it was related to the environment - it sits in a valley, the persistent low grey cloud can make it feel claustrophobic - but I think another theory holds up: parents there didn't bother parenting. A community was drawn to the town by adults who were in essence, hippies: they believed in freedoms their parents didn't including for their children.
That meant that many of the kids growing up there did so in an environment without real structure, and became adolescent in an environment with easy supply to plentiful drugs and a relaxed attitude to its use.
I'm not a parent, that was an active choice I made, and so I wouldn't deign to tell parents how to bring up their children, but I would suggest parents look at the data: children thrive with structure and rules and gentle consequences for non-compliance (the naughty step, etc.), and recognition of genuinely outstanding behaviour (winning things, not participating in them). They struggle to develop as emotionally secure people without those. Hell, I'm a data point there - took me years to realise I needed therapy, and years more to benefit from it. As the old poem starts: "They fuck you up, your Mum and Dad. They don't mean to, but they do."
Anyway, teachers play a part in that journey, but they can't do the whole thing, and the sooner parents (and society in general), restore the respect to the profession it once had, the better. Are all teachers good? No, but they are professionals who have more data on more children than any parent, so maybe just respect that.
A friend of mine is on their way to becoming a teacher and already teaching at a school for one year part-time.
He tells tales about his children mostly from low income families, where school is the only structure they have as their families are completely dysfunctional. Thorn apart by the family trying or not trying to integrating into German society as well as themselves.
When he gives structure to them they behave much better, while interesting enough behaving at home worse. At least he told from stories about the parent-teacher meetings he had.
> When he gives structure to them they behave much better, while interesting enough behaving at home worse. At least he told from stories about the parent-teacher meetings he had.
Could be. A lot of kids only have so much good behavior for a day... act out at school or at home, pick one.
But also, the student who has to live with expectations at school may set expectations at home, and that may be seen as worse behavior by the parents.
I started out in a "data centre" in 1998/1999 in the North West of England. Things are radically different now.
When I joined that ISP we had ~2000 subscribers, and one modem rack that could take 30 concurrent 33.6k dial-ins. The company had just been acquired by a small independent telco in Cheshire, headquartered in a small town that is most famous for a weekly farmers market and an annual steam rally.
We shared that headquarters at first with the founder's other business - a dating agency for rich people in later years - and slowly took over.
The building was a farmhouse of sorts. We'd turned into a rabbit warren of offices, and then, we decided to expand.
First we turned one of the offices into a "machine room". We didn't even bother with fire suppression that first time, we just ripped up the carpet and laid some laminate, put some 19" racks in and got the power upgraded. The air-con was insufficient in that room and the following summer I had to wheel in portable units and had a Cisco router in there report that the inlet fan had a temperature of ~80C. That was around the same time the chief of staff was asking us why we wore jeans and t-shirts and couldn't we wear suits? No, no we could not wear suits.
Bit by bit we took over more rooms through the building, we started working in a portacabin in the car park (where the neighbouring farmer's sheepdog would join us for lunch).
At some point we needed to install ~20k modems, (Nortel CVX1800s - 56kbps had arrived!), and the telephone switch to run it, so somebody bought a second hand DMS100 that went into another room we had converted.
By that point the dating agency had moved out.
We needed another ~100 servers as well to manage email, web hosting, and so on. Those came in parts (motherboards, cases, RAM sticks, hard drives - about £500k worth in 1999 money), and we had to build them and install BSD on them (first OpenBSD, then FreeBSD, long story). We also had to make our own ethernet cables, which is where I learned I can't wire ethernet cables if my life depended on it.
Within 12 months from when we started, we were handling 750,000 dial-in customers.
Around September of 1999 I left and that was when the company had heard about a custom built data centre that was up for sale in Milton Keynes, so decided to move everything there. Around that time they were pouring concrete under the floors of all those rooms we'd thrown racks in because the timber floor joists (!) were starting to give way under the weight of the kit.
That company got acquired and then went through a weird route, but basically it's still going as a major UK ISP. I'll keep the name private to protect the guilty. :)
2000 subscribers, and one modem rack to that and a business still operating 27 years later. A lot of sausage rolls, plenty of sticking plasters for the cuts from those shitty cases, and almost every day ending with a couple of pints in a pub somewhere.
One of my old colleagues and I often think back to that being one of the best years of work in our lives. But... data centre work is horrid, dirty, hot, and underpaid for what it is. But you can - sometimes - be part of something kind of cool and fun, and something very few people get to see or understand.
But there's a reason I've stuck to software ever since. :)
This made me smile, reminds me of my 90s, early 00s in web development & hosting. Where tiny offices above shops spread out as the business grew; racks were thrown in cupboards; the noise was constant and the heat was unbearable in the summer; the pigeons nesting on the shit covered Victorian window sill had the best seat in the office and nobody really knew what they were doing but we were all incredibly cocky about being wunderkids working on the coolest new stuff that nobody else understood. Good times!
I had to go to another data centre just outside London about 10 years ago with a colleague who had never been inside one - we had some colocation kit hosted there and were going to spend most of a day swapping some cards and checking things through the KVM. He looked at me really weirdly when I offered him ear defenders on the way there. He wasn't ready for the sensory overload - the noise, the heat, the movement of air, the artificial light, the slightly weird smell, it was all a lot. 3 hours in, he wanted out. Couldn't blame him, I wasn't far off myself.
Adjacent, I work in a lab so we have dozens of ovens and freezers running constantly, so I'm intimately familiar with the constant hum, even though my office is a couple of doors away from the oven room.
We had a full site shutdown last week, which involved all power being shut off, which meant no running ovens.
The lack of sound felt so strange, like I was somewhere I shouldn't be, or in some sort of liminal space. It was unsettling in the silliest way.
>We also had to make our own ethernet cables, which is where I learned I can't wire ethernet cables if my life depended on it.
Are you me? would always take me about 10 attempts to even get one where all the pins would light up on a tester. But not good enough for high-speed data.
Kids these days, don't know when they were born. They have push-through RJ45 now, piece of piss and you never have one shorter than the other.
I don't miss on-site/DC infrastructure anymore. No more having to squeeze into tiny spaces, balance a lot of expensive kit on your head, and blood donations to the server gods.
Compute is so ubiquitous now I only have like 2 nodes in my lab anyway
We had to make our own cable. The boss had his own special way for us to order the wires when we did it. Worked okay for 10Mb/s and 100Mb/s on a good day.
I think all of these can be addressed by being able to understand which machines contributed what to the model. I know that's messy, but if I can state "this information has come from Alice and the consensus is that it's good and right as it aligns with other information from Bob and others, meanwhile this information came from Mallory and stands out as being incongruent with the rest of the information I have in the model", we can identify malicious violations. If we're then able to state either Alice or Bob is one of the copyright holders, and one of them gets paid a little more, the other a little less as the confirmation agent, well, the economics of all of this changes a little.
At the moment we have Annas Archive being paid by frontier labs and rare/second hand books being destroyed in order to support the training regime. If instead we could just pay the publishers and they could distribute royalties to authors...
Local LLMs are the future, and one of the reasons I think the data centre furore is just going to end in a market crash.
LLMs that can reliably be used for control problems are the future, and I think classic/deep RL has generally been overlooked for years for a whole host of problems by wider industry because it felt inaccessible. The first thing I thought of when I saw Jev (and then Laya), was "this might move the needle in a really, really interesting way".
Local LLMs that can reliably be used for control problems smash through a lot of barriers I'm interested in, and this intrigues me a lot. Guess I'm about to become a big Laya fan if it can run on this kind of hardware to this performance.
The future for whom? The general public? Not a chance, no way, not unless it's able to run on a phone (anywhere from 20-40% of internet users, world-wide, are phone-only).
For companies? I think that's a lot more plausible, as that's mostly just a question of money - is it cheaper to run and administrate our own models, or outsource that?
For technically inclined users? I think that's unlikely unless they're able to operate on relatively cheap hardware while still being just as good as the hosted models. And by that I don't mean "a mac studio," that's far more money than I think is reasonable. A single RTX 5080, maybe, once memory prices start to drop.
Compare the games your average high-end smartphone can run to the AAA titles of the 2010's. It's not a matter of "unless it is able to" but "when it is able to".
That's going from 150w 720p gaming to ~15w 720p gaming in ~10 years. Let's say an inference cluster draws 1500w to deliver a small-ish 500b model at reasonable speeds/quantization.
Extrapolating from your gaming example, it will take smartphones only... *checks clipboard* ...100 years to achieve datacenter-level performance at the pace of 2010's improvements.
We don't need datacenter-level performance in a handheld device just like we didn't need a storage room in our backyard for a mid-sized, tape-based NAS because technology did its thing and gave us something smarter, smaller and faster.
Your argument is true if the number of parameters in an LLM is the only measurement for quality - a bit like number of bolts per aircraft or lines of code in software. I'd bet you that an 8b parameter LLM will in 5 years outperform the datacenter-level LLMs of today.
that's not a local LLM. If it's local, it doesn't matter in this case. Laya is a System 1 "AI", namely works like a classifier, given a state and questions, it shoots probabilities for each. I publish an episode tomorrow about Laya and Typesafe AI on https://www.youtube.com/@DataScienceatHome
Microsoft got this right once, and it cost them big to turn up late - that was the Internet and Windows 95 and the massive u-turn. Otherwise, they've been defining stuff early a lot I think. It's hard to explain how big a turnaround they had to do back then, but then it says a lot that until they made that u-turn the leadership was so confident in their own hubris they thought they could take on the entire internet with their own crap version of AOL.
IBM invented the PC and then lost control of that identity. Figuring out how that happened is something they are going to have a meeting about, but they're still working through the meetings to plan that meeting, and the Gantt chart for those meetings takes meetings to adjust, and everybody is in meetings so doesn't have time for those meetings. They'll get to them though.
Apple consistently turn up late, but redefine and improve the status quo. They sometimes prototype early in public (for example, the Newton), but normally they show up with the version 3 that everyone else wished they'd thought of that then goes on to define a sector quite cleanly (Mac, iMac, Mac OS System X, iPod, iTunes, iPhone, iPad, AirPods, Mac Mini). They Newton'ed Siri, but I'm pretty sure they're going to come back strong in the next 18 months, unless the enshitification we've all witnessed over the last few years really has taken hold and Siri is just going to automate that at scale. Maybe we will all be killed by AI, but not through malevolent evil, but widely distributed incompetence.
Meta are constantly taking swings at the latest thing the founder is excited by - the Metaverse (now with legs!), pervert glasses, and so on - but they mostly prop up a bunch of expensive failures on the back of a hugely successful advertising business. I don't think they come late to the party per se, they're just not very good at anything that isn't killing & harming children, or undermining democracy. They should stay in their lane and keep doing that, I guess.
Recent completions:
- The Man in the High Castle by Philip K Dick. I found it well written, but ultimately lacking in something - I kind of wanted the other boot to drop at the end.
- Poor Charlie's Almanack. Stripe Press published this collection of Charlie Munger's speeches and I've found I broadly agree with him on a ton of stuff and am slightly excited by some of his ideas that are new to me.
I'm about to have some time off recovering from a planned medical procedure and expect to have a mix of sci-fi, good quality non-fiction and at least one heavy "literary" book. I'm a big fan of Borges, so I might reread some of that work, or maybe pick up some adjacent/similar authors like Calvino too.
reply