When I started using Perplexity when they started out, it was such an awesome product. And they held that position for some time.
For some reason, I feel like it has gotten worse than it was. I don't know if it's just me.
Maybe chatgpt/claude just got increasingly better. Yet still feel they should have been able to keep up with it at least? but now they are just so much behind in terms of user experience and general product quality.
They substituted models quite often, and imposed low usage limits for users who had been onboarded for free. Supposedly it was for specific adverse situations such as heavy usage or overall load. [1]
I highly recommend! I've been writing Rust professionally for 5 years now. Compile times are slow but I love the language. https://cheats.rs/ is a resource i go back to often in case you're interested in learning more. Especially since as you get more into it, Rust has a lot of weird types like RefCell and Cell and OnceCell and disambiguating between them is a chore sometimes. Hope it helps!
Right! Have the same experience myself. I think we approach dangerous levels when people outsource thinking to a model. There is great value in having one's own judgement, and that takes more than just reading and consulting with an LLM.
> Earlier in the year, we had largely handwritten the first version of the git-to-Scion ingestion pipeline as a prototype. We thought it was good—until we got our hands on an early internal snapshot of Claude Mythos Preview. We asked it to rewrite the whole thing, and with no further instructions, it produced a version that consumed 20x less memory and was 4x faster, while preserving all of the implicit invariants around ingestion ordering and data structures.
I feel like the advantage of having these models in house can't be understated. Companies with access to inferior models, trying to compete, will be left in the dust when it comes to general development.
The question of what to build rather than how is still central to development. If the article does point to success it's not as much a technical one but of product. That said I'd take these articles with a lot of grains of salt, knowing first-person how corporations spin their successes, including internally.
That's been my thinking lately. The SOTA models are good enough that they "solve" coding but still need directing. The product is still important. Products are human too, and need human attention (still).
Maybe that'll change in the future, but while AI can make awesome images without a good "director" they're pretty cookie-cutter and bland.
Yes, I think we'll find that technical limitations were never the bottleneck. I work for a large corporation and that has been the case here so far. Yes, a lot of things are faster, but issues with product vision, roadmap and cross-team friction aren't gone. They are actually maybe even harder to get right.
Yep, agreed. Like seemingly is the general opinion, taste and direction I think still matter a lot. Efficiency in it of itself is kind of ”solved”, but effectiveness, doing the right things, is not.
There actually was lot more hand-writing of the code back then! Or, at least, only moderately assisted by Claude with a lot of human-still-typing-things-in-an-editor. The ability to delegate larger chunks of code to Claude had a noticeable, step function increase with Mythos Preview.
Others can confirm, but I suspect a lot of inefficiencies in large company programming projects are due to multiple people working on the same code - where you ultimately have to let both above and below average programmers work, and fixing bugs or improving existing implementations is more difficult because of company politics or interactions with others.
This is true for consumer software. I would imagine for developer tooling and especially internal tools these metrics are rather different?
A 10 minute CI pipeline is great and fine, but everyone absolutely cares if you can bring it down to 30 seconds. This software, by its very nature, is measured by speed.
I used to think this as well, but until it turns out to be a problem for the business worth solving (eg impacting $ machine), it will not get solved. There has to be a why now, otherwise plenty of "problems" get shoved under rug.
Internal tools, IME, are almost always short budget. unless it is in path of revenue, then there is $ and performance matters a great deal *if* more performance means more $.
We ran the repo ingestion in a loop for several months while we were split between Scion and git to keep things in sync, so this perf win was actually impactful for us!
Are you already operating with multiple levels of orchestrators and agents? i.e. one human talks to one or more orchestrators who then talks to orchestrators who handles teams of agents?
Yes, this is a common pattern. There is still an opportunity to improve how these agents write code without needlessly conflicting between themselves and we are exploring some ideas.
For some reason, I feel like it has gotten worse than it was. I don't know if it's just me.
Maybe chatgpt/claude just got increasingly better. Yet still feel they should have been able to keep up with it at least? but now they are just so much behind in terms of user experience and general product quality.
reply