After the recent somewhat discouraging stuff with clickbench that made the news, I must say this blog post is very cool.
Sounds like you found an excellent way of structuring this problem to solve it with Claude (for those who don't click into the article - writing hand/claude-optimized Rust code for each query, checking its assembly, and trying to make their own optimizer generate something close to that same assembly), and the writeup is very interesting and readable too!
Kudos, looking forward to more blog posts like this one.
Quickly reading the article, one notable limitation seems to be that these checkpoints are 512-1024 tokens context size models, while Jev is seemingly 32k.
That's a pretty big limitation, I would argue, unless I'm misunderstanding and it can be worked around easily somehow? I'm surprised it isn't surfaced more prominently in the comparison.
Jev has 64k total token request budget and I do wonder how it will handle highly specialised inputs.
This Jev waitlist that Typesafe AI are utilising is surely going to raise questions pretty soon - it's hard to sell this to bosses when it looks like a pop-up restaurant
We're a VC-funded startup (recently raised $51M Series C) building an infrastructure orchestrator and collaborative management platform for Infrastructure-as-Code – from OpenTofu, Terraform, Terragrunt, CloudFormation, Pulumi, Kubernetes, to Ansible.
On the backend we're using 100% Go with AWS primitives. We're looking for backend developers who like doing DevOps'y stuff sometimes (because in a way it's the spirit of our company), or have experience with the cloud native ecosystem. Ideally you'd have experience working with an IaC tool, i.e. Terraform, Pulumi, Ansible, CloudFormation, Kubernetes, or SaltStack.
Overall we have a deeply technical product, trying to build something customers love to use, and have a lot of happy and satisfied customers. We promise interesting work, the ability to open source parts of the project which don't give us a business advantage, as well as healthy working hours.
Yeah, I think I agree with this. I think making GUIs and reverse engineering proprietary APIs are two things that frontier models make really pleasant - e.g. I made a menu-bar app (very similar to your temperatures dropdown, similar graph, SwiftUI) which connects via bluetooth to my Inkbird CO2 sensor, graphs the CO2 level over time, and looks fabulous.
The skills are useful, thanks for linking them! They'll be useful for my future projects like this. FWIW I've had good luck with using Claude Design first to do mocks.
At the same time, I think there's still the "is this worth doing a hyper-customized version of this software, if I'm also then responsible for fixing the bugs?". But I think increasingly as we improve our agentic workflows and make them more self-contained loops driving the result to completion, this becomes kind of a non-issue.
In general, I do think LLMs are bringing about a new age of hyper-customization, which I welcome.
Maybe it is not listed by the OSI, but it is 'open source' in practice, if the source is also in the public domain. The FSF's definition of free software makes more sense not only ideologically, but also because it does not try to be exhaustive.
One problem is that a large part of Europe doesn't understand the concept of public domain. I'm not legally allowed to not own my creation, except by transferring it to someone else. That's why licences like CC0 exist.
Love them, but I wish they had a way to upload new sieve rules via an API. I'm probably going to try them with my own domain at some point since I think they have an option to just deliver all mail bound for that domain, which makes setting up random emails for dodgy sites really easy.
We're a VC-funded startup (recently raised $51M Series C) building an infrastructure orchestrator and collaborative management platform for Infrastructure-as-Code – from OpenTofu, Terraform, Terragrunt, CloudFormation, Pulumi, Kubernetes, to Ansible.
On the backend we're using 100% Go with AWS primitives. We're looking for backend developers who like doing DevOps'y stuff sometimes (because in a way it's the spirit of our company), or have experience with the cloud native ecosystem. Ideally you'd have experience working with an IaC tool, i.e. Terraform, Pulumi, Ansible, CloudFormation, Kubernetes, or SaltStack.
Overall we have a deeply technical product, trying to build something customers love to use, and have a lot of happy and satisfied customers. We promise interesting work, the ability to open source parts of the project which don't give us a business advantage, as well as healthy working hours.
Sounds like you found an excellent way of structuring this problem to solve it with Claude (for those who don't click into the article - writing hand/claude-optimized Rust code for each query, checking its assembly, and trying to make their own optimizer generate something close to that same assembly), and the writeup is very interesting and readable too!
Kudos, looking forward to more blog posts like this one.
reply