REPACK is my favorite new feature. Vacuum was the bane of my existence when I ran Postgres at reddit. Partly because I didn't know what vacuum was for the first two years.
So I thought, "huh, every time vacuum runs it slows down, I should turn this off". Yeah it was dumb to not even google what it was for. About two years in, I hit the point where I ran out of row IDs, and that's when I finally learned what vacuum was for (and reddit was down for a bunch of hours).
Ha, you're not alone to have gone through "autovacuum slows things down let's turn it off". Yes, it's fundamental to how Postgres works, yes it's a pain in the ass so often.
I was so worried to talk much about repack when things started getting reverted. This felt like the riskiest thing of all the headline features in PG19 in my view. Not the graph support, not the split/merge partitions, not group by for all, repack felt like it wouldn't make it lone behold it was the one that did survive of so many headline features. Fingers crossed all the others hold up for PG20 and it ends up as packed with features as 19 was looking like, but at least for now we got repack.
What gets me is that by default the id counter starts at 1 rather than -9223372036854775808 (for BIGINT) so you have immediately lost half of the space.
Biggest differences are that Neon can scale to zero (and not charge you) and it has branching as a native concept. This means that it can provision a new DB nearly instantly (as opposed to waiting a few minutes for Aurora) and it can do instant copy on write branching.
So Neon is a lot better for AI agents that are running experiments.
How many different "software factories" does the world need? How many engineers to maintain them?
I assume the premise of a software factory is that it can churn out any software idea you have. You won't need to build a new distinct factory for each new distinct piece of software you dream up. Otherwise it wouldn't be called building a software factory, it would just be called building software.
So if somebody gets this right (this remains to be seen, the various Gas Towns of the world have not proven their value), won't one good software factory be enough for everyone to download and use? Wouldn't we all then be done with the labor of making both software and software factories?
Actually, if we really follow the "factory" idea, wouldn't that lead to the idea that each "software factory" churns out one specific standardized type of software in a particular way? That's how real factories work.
For example, I could clearly see having a "Static web page with HTML and CSS only software factory" that just churned out that kind of software. Or a "Postgresql optimized SQL" software factory. And so on.
I have had this thought as well. There would be AGENTS.MD files with an appropriate document hierarchy per language, and then people would just clone these repos and use them to generate their specific applications.
For example I pointed an agent at the bulletproof react website and had it generate an AGENTS.MD with the top level best practices and links to specific docs ("when creating hooks, read docs/hooks.md"), and a note to consult bulletproof react when in doubt. My react code was pretty solid, even though I haven't coded javascript in 8 years or so and was out of the loop on current techniques.
agreed. i was just hypothesizing about a future where all i.e. java code looks the same, using the same formatting, naming conventions, basic patterns, etc. we all sorta just pick some conventions and stick with them, so more attention can go to the higher level architecture and we don't have people continually complaining about AI slop code.
it just sounds like reusable components that you can build on, to me anyways
every company is making a totally different product so its hard to see how every company is its own 'software factory' outside of such components (which they were already doing even before llms)
Maybe. But then you train a model on the different open-source factories and end up with a factory factory, and then you're back to what the parent said.
But how long until the models can do that too? I’d guess months. No one person’s taste an experience will have much value compared with the combined taste and expertise of the entire internet.
Even if it takes years, it doesn't matter. And once that is solved, you can make hardware do anything you want, and the next 10-20 years will be all out automation of every physical job as well.
I don't really see a future where human labour at all can still be competitive beyond like 2050 in any field. Anyone who's ridden in a Tesla in FSD knows transport jobs are defacto over, the tech works and it will be copied and deployed en masse sooner or later. Humans don't scale and we haven't meaningfully mentally improved in 100k years, we're limited to 20W of compute, take 20+ years to train and go EoL in 40 after that. It's inevitable that we'll be outclassed and it only needs to happen once. Once a model exists that is slightly better at just one specific task, it only gets cheaper over time and it can be used for that forever.
No one knows how to do this, and at the moment I don't think it's possible with current AI. Good taste has to either be trained in (can't be prompted), or might be one of those actual human abilities that can't be artificialized.
Well I keep trying and I can't do it! Plus I've never seen something Claude made that has good taste without substantial human involvement providing the good taste. I'd love to know how you do it.
Unless we organize to prevent it, technology will continue be commoditized like all other technology into the hands of a few players.
And especially with AI it’s going to look more and more like nuclear or biological technology and that’s why you see so much push for regulatory capture.
Literally all I do is spend my time trying to figure out how to prevent this, and still make progress towards AGI for everyone, but it doesn’t seem like people care
Everyone just wants their bag everything else be damned, and that’s what we’re seeing
I'm curious why you say this? Why would my wealth be relevant to the advice given?
I would understand this comment if I was telling you how to live frugally because it's not my lived experience. But in this case, it is my lived experience. I literally work at a job where I have to build a software factory to be successful.
You are speaking of being successful at your current pursuit, but you don't need to be succeed. You're building for fun. You are giving advice to people who need to think about how they will provide for themselves and their dependents for the next few decades.
You're rich enough that you don't have to think about what you'll be working on a decade from now, or even a year from now. Anyone saying that building a software factory is the future of software engineering is looking days, perhaps weeks, into the future. Software factories are a meme that'll be dead in a month. Saying software engineers concerned about their career should learn to make software factories is hollow.
> You're rich enough that you don't have to think about what you'll be working on a decade from now, or even a year from now.
Perhaps. But I work right now, and I actually care about people other than myself. I would like to see the industry that made me successful keep making others successful.
We'll have to wait a month or two I suppose to find out if you're right, but I think all companies will be moving to the model of software factories. And It's not something I thought of recently -- I've been building a software factory with AI for a year now, and working with others who are building entire businesses on it.
And FWIW, pretty much every tech company is now moving towards the same model.
Maybe I am off but software factories isn't what we are doing per se. I am not sure how successful we are with AI (yet) but our token cost vs results is very exciting.
This is my first reading about your work but I like your attitude. Databricks is also a good service.
From what I am seeing, those who embrace AI are embracing the 'computer' from the 80's. The landscape and reality is still being defined.
To the AI haters and doomers I say buy the largest house with cash you can to enable your retirement. The stress that people experience is the thought of their crushing debt or obligations that can be met with a 90% reduction in spend. Of course, that requires a lifestyle change that most are unwilling to make.
Are they factories tailored to the specific product you are creating? I feel like I get good results with a harness, and lessons from factories can be incorporated into models and harnesses as long as the lessons are general.
Success in a prior time does not guarantee success in the present or future as too many variables are different
Demographics have shifted wildly and most users are not a soft spoken grama needing help setting up pop3 in Outlook Express
Look around you; everyone uses their phone at whatever level of expertise they need now
Goodhart's Law has come for how we used to measure need and todos in tech. We hit the top of the S-curve relevant to this form of computing devices and there is nowhere else to go with it
The proverbial ladder is being pulled up. Not necessarily by "jedbergh" (disclaimer: i have no idea who that guy is other than having apparently won the game of capital) but it's happening and none of his life experience is going to be replicable so his advice to anybody in a substantially different economic strata is effectively worthless.
It's like that meme from about a decade back about the millionaire saying that the only reason millennials can't afford houses is that they eat too many avocados. That advice makes sense in a long-gone era where houses are cheap and avocados are expensive, but in the 21st century the price of personal avocado consumption would never add up to a house in a reasonable amount of time no matter how excessive your avocado lifestyle may be. jedbergh is going to be an "avocado guy" from now on (again not that I actually know anything about him).
I disagree. I currently work with a bunch of fresh college grads. We are teaching them to build AI software factories so they can be good at it and grow their careers.
What is falling away is that we just won't need as many SWEs as we used to. The way to make sure you don't "fall off the ladder" is to make sure you're the best at the new way of doing things.
That's probably as good advice as anybody could give at this point, but it's still largely speculative rather than experience-based. Is there any reason to expect that building AI software factories is itself a viable career path? Do we eventually reach a point where the AI doesn't need human guidance? One of the things that took me by surprise the last few years is that even driving the AI and telling it what to do is subject to being replaced by AI.
I can see this going a few different ways. We could be approaching a future where the machines can drive themselves and the computer goes from being a tool to a partner (ie Data from Star Trek). Or maybe we're approaching the future where computers still require humans to tell them what to do but they're good enough at figuring things out that nobody even needs to understand how a computer works to build the next enterprise-grade large-scale thingamabob.
The future you describe where software engineers still exist but they've carved out a niche managing AI software factories only makes sense if we've just reached the end-state of generative AI and there will only be incremental improvements from here on out. Otherwise I don't think you can safely assume that the AI software factory actually needs anybody with an advanced skillset to tell it what to do.
I agree with snickerbockers. Easy to kick jedberg here, but will mention that I asked him/you for a referral twice last year and was unceremoniously ignored.
It doesn't matter how good you are at anything when you can't even get a reply from a person. And absolutely forget about learning anything on the job. The ladder is gone. Combined with the war on the middle class, am looking at a retirement in poverty. Even social security is scheduled to be empty in a few years when available.
> but will mention that I asked him/you for a referral twice last year
Sorry, I just checked and I don't see anything from you. Where did you ask? I try to respond to everything I get, but sometimes gmail doesn't surface random messages.
I sent two emails to the main domain on your pages at jobs@${username}.net and they didn't bounce. Though I'd appreciate even a low paying job, I'm focused on https://trustworthy.technology/ and FOSDEM27 at the moment. Your interest in any of these would be appreciated.
Not really, no response again. A polite no or "maybe later" would be fine.
As mentioned, the ladder is gone. Also, judging by the number of news articles, people care about 5x more about young people not able to work over older.
Backend dev opinions/arbitrary strings are being compressed out of the stack.
Devs that don't understand hardware development are naive about the endgame; black boxes that boot to a Google Opal like GUI.
Boot to Blender-like/WYSIWG environment loaded from a model itself. All the context we put in files added as vectors on the presentation layer.
Users will conjure a word processor or email UI and then a video game.
Network gear will boot a model to handle the specific job the model was trained on. That's what all the Stargate stuff is about but obfuscated behind marketing to hide implementation goals details from knowledgeable labor to avoid public scrutiny. Vague scrutiny without concrete proof is fine but if we could point to the details too they'd be naked.
50 million devs globally among 8 billion that do not want a subscription service to an expert when they have no such leverage over the expert themselves.
This isn't empty speculation. Hardware industry executives are responding to social pressure at scale.
America was able to leverage its clout to empower IT to pretend it's more than just math relative to memory and display matrices but America no longer has the global clout to assure that. To Chinese engineers it's just math.
American software devs were engaged in sophistry and social moat building. Ya got disrupted by the world. Believed they would ignore their eyes for the rhetoric and hype forever; rather Orwellian really.
Lambda is built for the generic case. Amazon was kind enough to open source firecracker, which is what they built Lambda on. If you want to run workloads that are more specific than what Lambda supports, it's pretty straightforward to use firecracker to build your own optimized Lambda.
> other than the lack of a prenup meaning she owns half his stock
They were married years before he founded the company, and lived in California, a community property state. And they got married right out of college. Unless he somehow knew that he would start a company and got a prenup that says "if I start a company it's all mine", a prenup wouldn't matter anyway (and would probably but null and void anyhow).
Most of the finance companies left for more tax friendly states. In the early 2000s, SF had two rush hours. One around 5:30am and another at the normal 8/9am. The 5:30 rush was all the finance people getting from Walnut Creek to SF to be there by the market open at 6:30am.
This is absolutely correct. I recently found myself asking my agent to modify a local copy of an open source project to fix bugs that I found. It fixed the bugs, and it rebases against the public repo on occasion.
Why haven't I submitted my patches? Because my agent wrote them and I didn't review them. I would feel bad submitting my patches without looking at them, and I don't have time to look at them.
I've actually been doing this in Clojure and that avoids the major pitfall I've seen: the interfaces it makes are clunky. Re-using an existing library starts with a sane interface.
I've seen the same thing in Go, for what it's worth.
The code totally works, I'm not a denier there, but the interfaces are heinous if you don't keep an eye out. Mine wrote a Go function that passed a name variable as a string, but also embedded in 2 separate structs that were arguments. I've also seen it do re-validation of data multiple times in internal functions so everything has to return an error.
It's also stuff that sort of doesn't matter if an LLM is the only user. It probably costs some extra quantity of tokens, but not enough to be worth hand-grooming the code.
KMyMoney stopped importing (some) OFX files for me. I asked an LLM to debug.
It was a wild journey. An obscure bug in some QT library that was triggered because of something completely unrelated that I had installed briefly in 2019 (and uninstalled years ago).
I had it fix the issue by cleaning the leftovers from 2019, but it's still a QT bug. I had it do a writeup, but I'll never submit it. It's fairly sophisticated and I don't know QT code at all.
If it can put together a minimal reproduction of the bug I hope you might submit that. Tossing a commit full of slop over the fence is a dick move, sure, but tossing a small script over the fence that has approximately no dependencies and reliably reproduces the bug is pretty much guaranteed to be helpful. Leave it to the experts to figure out how to actually solve the problem.
I never fixed the bug. I just removed the thing that triggered it.
I believe I do have some sort of a reproducer, but I likely wouldn't understand even the reproducer script. I think it won't be taken well if someone files a bug, says "Here's a script that reproduces it", but himself doesn't understand what the script is doing.
That logic is true of "contributions" - here's some undocumented code, it seems to fix my own poorly specified problem, I have no idea how it works, please merge.
For bug reports (at least IMO) a minimal example to reliably reproduce the error without any fuss is the basic building block. The reporter doesn't need to understand why it works let alone offer an explanation. In fact for complex projects more often than not any purported explanation will be subtly incorrect and actively misleading. If anything further "documentation" of bugs is unwelcome. Unambiguously demonstrate the defect and GTFO.
Well sure, if you use a library in a borderline incomprehensible and clearly unintended manner and encounter an edge case then perhaps it isn't useful to report it. But while obscure, the usage you described (ie encountering stale config or cache data from a decommissioned install) can hardly be said to be unintended.
If an end user encounters an error while staying within the bounds of "expected usage" then the bug is guaranteed to be valid. You just don't necessarily know whose bug it actually is (for example a broken library versus broken usage of the library by the consuming program).
Why not ask someone involved if they want it or tell someone the prompt that made it? Someone who does the work of checking it and sanitizing it can take the credit.
Depends on the project but some actually adopt an outright hostile response to this. Sometimes justifiably. Sometimes not.
It's honestly not worth it. I understand both sides here, but people are going to have to come up with new ways of collaborating. Pull requests are likely not it anymore.
So I thought, "huh, every time vacuum runs it slows down, I should turn this off". Yeah it was dumb to not even google what it was for. About two years in, I hit the point where I ran out of row IDs, and that's when I finally learned what vacuum was for (and reddit was down for a bunch of hours).
Having REPACK would have been a godsend.
reply