I was in a Chromebook pilot program when I was at Google. They gave me a different model every month or three as long as I kept giving them feedback. The build quality and usability varied wildly. Some models were so good I forgot I wasn’t on a real laptop. Some were so slow and unusable even on basic sites. Unfortunately the latter tended to be the ones that were so cheap that they ended up in schools.
> all relevant facts will be cited and checked easily by humans
I've talked to a lawyer about how they handle this. They do indeed double-check everything, since it'd be embarrassing (or worse) to send hallucinated statements to opposing council or to the court. They still find the assembly a huge time saver
But based on stories in the news on the subject, not everyone has this same level of diligence
These numbers are useful proxies for how likely you are to have your work disrupted outside of your own control.
If you do something 100 times a day against a four-nines service, you can reasonably expect that everything will succeed.
If you do something 10,000 times a day against a two-nines service, you can expect to hit a substantial number of errors during that day, or even have long periods where your work cannot happen at all.
People aren't frustrated with Github because Github has 98% uptime or whatever the specific number is. They're frustrated because it regularly interferes with their ability to work. The 98% number is just a concise way to say it.
Before 9's became a thing, people built systems that required outages, and those outages would happen outside of business hours.
However that's also why some transactions had to complete the next business day after they were registered. Because the whole business was running on offline processing (aka batch processing) that could be interrupted for upgrades, but had to be completed by the start of business the following morning.
My dad did one of those jobs, and my brain has made a bigger deal out of the times I awoke in the middle of the night to find him on the phone at the kitchen table at 2 am dealing with a war room call because an upgrade broke things that needed to be done in 5 hours. It probably only happened 3 times that I know about, and I probably knew about at least half of them, but it felt like it happened twice a year.
In my experience it’s a safe way to do something useful while everyone is getting their bearings. It immediately partitions the situation space between being persisted or systemic vs local or caused by long-running processes. Plus everyone’s going to ask if you’ve tried that already, so you might as well get it out of the way if it makes any amount of sense
I hate when companies do this. Instagram is a black box for me. Whenever someone sends me a link the web app is completely nonfunctional; clicking links just reload the page, it constantly tries opening the app, etc
The last time I signed up for an Instagram account (2016) it got immediately flagged as a bot account. I think I was supposed to appeal it to have it reinstanted, but why bother if this is how badly they treat users?
Can you imagine going to work as Instagram's web developer, spending your day finding new ways that the app accidentally works and disabling them?
I know there are like 5 trillion Instagram viewers. But I've never actually needed to use any of them because there is absolutely nothing so important on Instagram that I must see it. The entire site is optional.
My kids school puts some information that I need to see (not just "would like to see") on Instagram. Same with the sports team they play for.
I've complained to both about it but change is glacial. I'm lucky in that I know how to work around it, and I do my best to keep others who aren't so tech savvy updated about things they need to know but may have missed.
Irony is they moved most of this info from Twitter/X to Instagram.
What!? It's the entire, singular purpose and meaning of life for millions of people to make themselves look glamorous on Instagram, and you say it's optional?! /s
I'm sorry that I keep posting this but it gets said often enough that I'm just going to hopefully kill it permanently
People say repeatedly Instagram was a 'great acquisition' (it was. It destroyed Facebook's competition. Great for them, not for users). That Zuckerberg bought it when it was 'barely a company'. That it's somehow not antitrust (Zuckerberg in emails: "Instagram can hurt us"). People loved Instagram. Not anymore, not for a long time. Stop giving him the benefit of the doubt
I would listen to every single episode of a podcast that could get the real skinny, not the sanitized feel-good post-facto rationalization, from founders who had a legitimate shot at being a major player in an industry but chose to sell out during their meteoric rise.
There are messages, by the way. I'm paraphrasing, but it's something like "So, Mark messaged me saying he thinks it would be a good idea if he bought Instagram, what if I tell him no" "He'll go into destroy mode" "What if I " "Then he'll go further into destroy mode"
But of course you can't tell the public this, Zuck's reputation in the Valley. So it indeed becomes the feel-good story about Our Beautiful Journey
I built my own personal Google Docs clone to use as the editor for my blog
I used to work on the Google Docs team, so I basically did an 8-hour Q/A with Sol spanning like 400 questions. Did my best to reach back 10 years into my memories to see how things were implemented, and discussing architecture and tradeoffs. Then went through 5 ChatGPT resets implementing it on Astra to get a feel for the model.
I have a long way to go. But I'm writing my first post on it this week, and then I'll do a writeup of the editor project.
My workhorse is Opus orchestrating Sonnet or Sol orchestrating Luna, depending on which ecosystem I'm using this week. I'm a backend engineer, so for tasks that either interact with the machine learning or client codebases, I have Fable (Medium) review across the product spec, tech spec, and all the codebases after the implementation is done (haven't tried this workflow with Astra yet, don't know if it's a good reviewer).
As a side project, I'm doing an experimental task now (having Astra implement my own personal Google Docs clone, OT and everything, for my blog), and it feels more capable than Fable but needs to be watched closer than Fable. It is obsessed with verification and evidence to the point that I've needed to make some absolute rules to stop it from repeatedly e.g. making unit tests with 10,000 test cases with 5+ hours of runtime
For well-defined coding tasks, I'd default to either Luna or Sonnet, or hell, just writing it by hand.
Unlike the vanilla read of the OpenAI press release, it is much more unfiltered and outlines some particularly aggressive behavior by specific OpenAI employees
Which seems to be entirely true by their own admission! [0] Both the comments about him risking his career and about Levent's authorship seem to have indeed occurred.
> 2) I never ever asked for Levent to be removed from authorship of his own work (as indicated by my text). I was surprised to learn during the call with Tristan that they had only solved Euler and not Navier-Stokes; after learning this we brainstormed possible paths forward. One option we discussed was that Tristan could be the lead author on a rewrite of OpenAI’s Navier-Stokes proof. It is in that context that I said “it would be simpler if Levent was not an Anthropic employee” because I felt it would be inappropriate for an Anthropic employee to author OpenAI’s work. Importantly it was admitted that internal Anthropic models had been used in their proof of Euler blowup; I therefore felt I could not consider Levent to be an independent academic. Another option I wanted to propose (but got cut short) is to offer access to our internal model so that they could try to finish their proof and bridge the gap between Euler and NS. Again I did not know how to navigate giving access to internal OpenAI IP to an Anthropic employee.
> 3) To reiterate it plainly: as my text clearly indicates, and as I said during our call, OpenAI's intention was to do everything possible to celebrate their mathematical achievements and the heroic efforts that they made on Euler. In the call I was immediately met with a litany of slander, including direct threats that if we were to announce Navier-Stokes he would immediately go to the press with a barrage of unfounded accusations. I refuted all these accusations but he replied “there is nothing you can do, I simply do not trust you”. I was confused why one would turn an incredible source for celebration (of their achievements!) into such bickering, which is when I said that I did not understand why one would risk their career [over unfounded accusations]. Genuinely, at that moment, I was trying to care for him and do a last ditch attempt to get a chance to give them all the credits that they deserve. I deeply apologize for this extremely poor choice of words, it is the opposite of what I was trying to convey. (I should say that I retracted them on the spot by the way.)
Seems like OpenAI did a boring normal corporate thing (find out your competitor made a breakthrough, try to replicate it) and then when the other mathematicians found out OpenAI had beat them to Navier-Stokes, they decided to lie about what happened because they were upset they didn't get to make the big breakthrough themselves.
The available information is essentially that story.
OpenAI's account: they heard a rumor that a Millennium Prize problem had been solved, so they tried to do it themselves and succeeded. Then they contacted the other researchers and were surprised to discover those guys hadn't actually cracked it, but offered the one of them who's not an Anthropic employee a co-authorship anyway. The conversations got testy.
Buckmaster's account: totally unsubstantiated accusations of plagiarizing from chat logs and plainly false accusations of OpenAI trying to get Alpoge removed as coauthor of a thing he was not an author of in the first place, and threats to ruin people's careers.
I think the synthesis is basically that Buckmaster and Alpoge had not quite solved the Navier-Stokes problem yet but thought they were really close, and had told friends as much, which is how the rumors got out. Now they're mad they got scooped. They aren't getting the money and recognition they thought they had locked down, and are engaging in a smear campaign.
I don't think your reading is correct. ML history is related with examples of models using hints from side channels.
If it would be possible to replicate OpenAIs success with a model cut-off earlier than the rumours, and without hinting from informed mathematicians, then we can consider it original. Otherwise it's indefensible.
By Buckmaster's own account, OpenAI (a) wanted him and Alpoge to publish their Euler result first and (b) wanted Buckmaster to author the Navier-Stokes paper, and (c) when he refused both of those, tried to get Alpoge to talk him into it.
Why would OpenAI go burn another $20 million trying to prove they didn't accidentally train on synthetic data derived from Buckmaster's codex sessions? They're desperately trying to give him credit anyway!
As someone who also spent a bunch of time benchmarking various techniques, this is as good as you can do without publishing a formal versioned benchmark suite that you want to maintain and run forever at immense cost to yourself. If you actually go to benchmark your own flow as you mentioned, you will quickly run into like a dozen problems that discourage you from publishing.
- Are you sure that temperature and other nondeterminism isn't affecting your output?
- Are you sure you're not being routed through an A/B test at this moment?
- Are you sure there's not a bug affecting the model at this moment?
- Are you sure that you picked the right model and effort level?
- Are you sure that your result generalizes across providers?
- Are you sure that you set up the correct level of sandboxing and the agent can't e.g. look at a sister directory or git history in the current directory for answers?
- Are you sure that the agent isn't leaking answers in memory or its conversation history?
- Are you sure that tool calls aren't somehow affecting results?
- Are you sure that your results are robust, i.e. you see the same results with mild tweaks to the prompt?
- Are you comfortable keeping your blog post live when your results are invalidated next week with the next model launch?
And that's just a quick list off the top of my head.
I personally decided that it wasn't worth it, I'm glad that Dan decided to publish his. Frankly I think we could use a lot more of these "I ran these 2 techniques side by side and here's what I saw" anecdata, because most people who promote prompt techniques can't produce a single prompt they ran twice because they never actually tested it per se.
I am really grateful that he’s publishing as well.
I am just asking for the bare minimum to actually understand what has been tested and for reproducibility of methods- the publication of the prompts. It would take a lot less time than all the guess work analysis write up and be a lot more useful.
Without the prompts the rest of your questions about reproducibility are moot.
This post boils down to, "I tried a bunch of things that didn't work very well". But we don't actually know what he tried. Everything hinges on Dan's prompting skills.
This was my instant reaction when I started using a Mac. But 15 years later, even knowing more tricks for interacting with it, it still baffles me. Once you've used Windows Explorer and a few Linux distro navigators, there's no denying that Finder is unquestionably worst in class.
As a Mac user since the 1980s, I feel feel like Finder is easily best in class, even after ten or fifteen years of design. I would describe Windows Explorer as “embarrassingly bad” in comparison, despite having a good hold on the shortcuts.
I honestly think it is a matter of what you like rather than what is good or bad.
> As a Mac user since the 1980s, I feel feel like Finder is easily best in class
Once internal muscle memory is established, the usability problems can be easily dismissed; but for the rest of us, who got used to better file managers, Finder is atrocious.
As a Windows user in the 90s, Linux user in the 2000s, and Mac user since the 2010s, I dread the moments when I have to use Finder.
What do you like about the other file managers over the Finder?
Finder has column view and quicklook. If there’s something that I’m missing out on from, say, Explorer or Thunar or whatever, I’d love to hear it. Whenever I have to sort through a lot of files on Linux or Windows, I get a better experience by exporting a SMB share and sorting on a Mac.
The only file managers that I’ve used that felt competitive with the Mac are tools like Midnight Commander (or the original Norton version).
But also I can use JK to navigate down/up instead of moving my hand of their resting positions to the cursor.
And NM you jump faster by 5 files.
And the ability to change shortcuts easily
And the ability to use double tap C to copy file name, and tap C then tap F to copy full file path.
And turn column view into a flat list of all files at or below the current dir for easier search
And instant search (with the help of Everything) limited to the current dir+children, shown in the same file manager list with the same navigation option
And the ability to tap Cmd-R multiple files to select rename file full name/name/extension
And the ability to autoreplace illegal filesystem file chars with my own alternative to preserve those Divide/Rule common document names
And the ability to automatically control how much space a column is wasting.
Etc., a million of other things that differentiate a real pro file manager from the toy of Finder
To be honest, a lot of these seem to have the shape, “I want to be able to press button A to do action X,” but for the Finder, you have to press button B instead. I was hoping for something a little more interesting.
Things like, “In a Real Pro file manager, you can double-tap C to copy a filename, but in Finder, you have to press command-C.” Do you see why I’m a little disappointed by this list?
I’m interested to hear what file manager you’re using because it sounds like you’d hate Explorer, Nautilus, and Thunar too.
Don't close your mind to new knowledge and it might just get interested!
> but in Finder, you have to press command-C.” Do you see why I’m a little disappointed by this list?
Sure I do, you're doing the lazy cut. The original quote was:
> double tap C to copy file name, and tap C then tap F to copy full file path.
And of course you can't do that in Finder, that's just nonsense. Now also add a few more actions: copy file name without the extension, copy the extension, copy full path quoted for pasting in your terminal - and
A. You've run out of keys and modifiers, and definitely out of memory to remember which action does what
B. C. D. It's actually not trivial to extend Finder in this way, and of course you can't easily change the keybinds, and of course you can't group those keybinds in a conveniently located submenu, and of course you can't assigne custom key modifiers to those menus for ease of navigation..
Also pray do tell which command-C you need to press to achieve this:
> (cycle) rename file full name/name/extension
There are also a few non-keybinding-related things in the list which you've conveniently ignored
Your list seems very preference-shaped, not pointing to actually-missing features I can see. You could make a case about discoverability, but it's hard to see how letter combinations that only work within one application and no others are easily memorized. If your habits mean you spend a lot of time working with files in a UI, then memorizing double-tap C and C+F might be easy for you. Otherwise, using the same Cmd+C and Opt+Cmd+C would have the benefit of doing what it does in nearly every other program. Or, as the other responder said, dragging auto-escapes appropriately.
I spend most of my time using a CLI, and I'm not claiming Finder is the best keyboard-driven file manager out there, but I will say that it's consistent, and works the way macOS works. Key combos work the way they work in every app, and holding down Opt updates the Edit menu to make it clear what would be different. You can do the most common things pretty easily and clearly, even if sometimes you have to use a mouse and right-click. There are also third-party apps for focused file management for pros, including some that use tuis, like ranger.
> Opt+Cmd+C would have the benefit of doing what it does in nearly every other program
So what do you think it does in Preview? In TextEdit? Are you not aware that you imaginary benefit doesn't exist even in the system default open file Finder window where you can't copy file path?
> Finder is fine: not the best, not the worst.
Very astute observation relevant to none of the things being discussed.
> Your list seems very preference-shaped, not pointing to actually-missing features I can see
Look closer! Besides that hard to say what you missed because you haven't said anything specific
For example, what is "preference-shaped" about this missing feature
> turn column view into a flat list of all files at or below the current dir for easier search
> it's hard to see how letter combinations that only work within one application and no others are easily memorized
It's very easy to see if you don't ignore use cases and add extra conditions. Like why does it universally have to be easy?? I mean, it doesn't even have to be easy for me, the extra convenience might be worth the effort of having to remember it
> might be easy for you. Otherwise
What do you mean "otherwise"? Why would you care about others to the detriment of your own ease?
> and holding down Opt updates the Edit menu
What does this have to do with anything? You can retain the visual updates of these menus even if you use sequence of keys! Press C and the copy command menu expands into a submenu of what you could copy next (path, extension, name). You'll retain your consistency at a higher ergonomic baseline for Finder, nothing stops the improvements here
> Or, as the other responder said, dragging auto-escapes appropriately.
And as I've replied there, it doesn't do that "appropriately"
> third-party apps for focused file management for pros
But that's exactly what I said!
> differentiate a real pro file manager from the toy of Finder
> Don't close your mind to new knowledge and it might just get interested!
Please be respectful in these conversations—I’m interested in hearing more about this, but the comments you’ve been making are emotionally charged and insulting.
I’m interested to hear what file manager you’re using because it sounds like you’d hate Explorer, Nautilus, and Thunar too. Sorry for repeating the question—but I’m not going to accuse you of “conveniently ignoring” the question. Accusing someone of conveniently ignoring a question is, I don’t know, at best just rude. Don’t do that.
I know that there are things that Finder is not good at, that’s fine by me. What baffles me is the comparison with Windows Explorer, and that’s why I responded, and that’s the context for my comments. If you have a comparison with a specific other file manager in mind I’d like to hear it. For my use cases, I am often sorting through large quantities of media files, so I value things like column view and Quicklook.
(Copy without extension is enter, command-C. Full path 95% of the time, I am pasting into the Terminal, and so I drag it in, and the terminal automatically escapes it.)
You spend more time saying you're interested than on discussing the topic of interest! And if you care about respect, don't start by trivializing the the argument
> but I’m not going to accuse you of “conveniently ignoring” the question.
You just have, though in a worse "passive aggressive" way that's harder to address, but also in a way that doesn't even make sense - how is focusing only on features, not apps, "convenient" for me??? It was for you because otherwise your defense of Finder doesn't work.
Also it's not just one app (though primarily the list is based on Opus), some of the improvements are sprinkled across different file manager, like on Mac Marta (while being rather incomplete) has a few things I miss when using Finder.
You also don't seem to know how hate sounds, I don't hate either of those apps (not even Finder). Why are you emotionally charging the conversation?
> What baffles me is the comparison with Windows Explorer
Then talk to the people doing the comparison! I'm not one of them, I've responded to a different point.
> For my use cases, I am often sorting through large quantities of media files, so I value things like column view and Quicklook.
And one of the points I raised in my comparison that is not a fake "but Cmd-C is the same as C" is the ability to customize UI to give more space to your precious media files, especially when you add various columns that have media information that can be blank or very long!
Or if you value quicklook you might be interested in an embedded alternative that doesn't obscure the file list by always popping in the center instead of a dedicated panel. Or the that doesn't reset zoom on file change. Or the one that does resize whitespace to only fit the image. Or the one that doesn't waste your time and mess with your focus with animations.
There is, of course, an endless list of deficiencies behind any serious workflow in Finder with better alternatives in other apps
> Copy without extension is enter, command-C.
Nice try, but no. That's two commands, which means you can't use a better key bind, but also the second command fails if the first one does, which it does, for example, in admin folders
>Full path 95% of the time, I am pasting into the Terminal, and so I drag it in, and the t
Besides having no option for the other 5% of the time, this also fails in shells using different escaping mechanism, and even in bashes it uses the ugliest escape mechanism of adding visual noise of \ for every space. Quoting was mentioned for a reason!
Also dragging instead of C, Cmd-V is a big ergonomic failure
Lastly, I don't think that's Finder that's doing the substandard escaping
Our KDE development motto (and a contrast explained in our HIG) is "Simple by default, powerful when needed". It's nice to see someone re-state it so naturally and eloquently in the wild!
I have been a hardcore Konqueror user back in the days. Early dolphin felt like a back step. Nowadays, thought, it easily outshines Konqueror. I still miss a few KIOs which were showcased in Konqueror. I guess they were discontinued.
