To my understanding it has nothing to do with output and entirely to do with the understanding of the input together as basically one input stream. An LLM with a vision encoder or audio encoder takes the media and turns them into tokens the LLM reads and then turns into its text latents (basically).
It’s like the difference between me looking at a picture, describing it to you, and then you trying to form that picture in your mind. You might get close, but it will never be the picture I’m looking at. If I then ask you questions about the picture, you can’t answer any questions about any details that I didn’t describe. I believe that is exactly the information and understanding loss that an Omni model attempts to address, which doesn’t have anything to do with output.
Some models do also offer “Omni” output, but that is more like a feature of the model than what Omni models are by design I think.
I have no doubt they can build an excellent machine. I’ve also received enough plated brass/pot metal “100% stainless steel” items from China to know I would have zero trust in their suppliers even if I trusted one company enough to buy a car.
I took a break from commercial plans and was using local qwen/glimmer/gemma and hosted glm/DS4.1-flash for things; opus 5.5 is night and day difference to return to. It makes some mistakes certainly, but its commits are often exactly the change I would have made with far less babysitting. It also shows that fable spark of common sense of “hey…you realize this other thing is broken too right?” without getting lost in the weeds immediately like other models seem do. Still an idiot sometimes though... “Oh no, I’ve discovered the exact same thing we have discussed and you have approved 5 times, and I’ve made notes about…I don’t know what to do, can you tell me to read my own note I’ve just reviewed and ignored?”
Exactly. And without setting a /goal for the harness to loop it, it also did essentially this today after a few hours of work:
I finished optimization A. Optimizations B and C that we planned look super promising. Shall we continue?
50 minutes ago
It's also happy to write multiple paragraphs about different implementation options, where the ticket's acceptance criteria already rules out all but one.
The outputs are still often verbose and the language always is convoluted Claude-speak.
Even for typical day-to-day software engineering work, there is still much room for improvement.
Yes…and no. If you put 5 stupid people together, chances are you can get a smarter answer than individuals. If you put 5 smart people together, they will argue about the wording of the question.
I kid, but also not. If the smart model already has a smart plan, odds aren’t great more smart models are going to create an even smarter plan. Not entirely true because that is the goal of parallelizing discovery attempts, but the point is there is more room to gain benefit from adding additional inference-time compute with a smaller model than one that already had a ton of compute to begin with.
Both sides have a point. If I ask you for feedback on the food sample I just gave you, would you automatically assume that I was recording audio and video, will be using your response in training videos, and will be used for AI training? Or might one reasonably assume the “feedback” is what is being given and not a blanket right to anything about them and their answer and using their likeness for training videos because they said “Eh.”
It seems like your counter-point is “they already use everyone’s data, why didn’t you just assume they would”, which is a bit weird from a victim-blaming perspective but valid nonetheless.
A closer analogy would be: If you make a food sample based on a description I gave you, and we typically agree not to record anything about either my description or your resulting recipe, and then you asked for (recorded) feedback - would I expect that I'm implicitly allowing you to record my requirements and your recipe with it?
I mean, still perhaps not. But it's not quite as ridiculous as your analogy makes it sound.
I’d be surprised if an investor lawsuit doesn’t come out of this in order to get depositions from Musk under oath. I would certainly sign on, being an investor in SpaceX whose investment was put at risk by these antics, regardless of who ordered or didn’t. If it was because of government order, I have no problem because the company had no choice, if it is because he decided to invoke his personal views this way, I got a big problem as that puts at risk every lucrative government contract.
“Perfectly safe” eh… freeze distillation doesn’t remove methanol so it is far from perfectly safe, but closer to “probably be fine” than poorly handled alembic would be. I think people should have the right to make their own alcohol if they want, but I’m probably not rushing to drink other people’s distilled trashcan mash.
It’s like the difference between me looking at a picture, describing it to you, and then you trying to form that picture in your mind. You might get close, but it will never be the picture I’m looking at. If I then ask you questions about the picture, you can’t answer any questions about any details that I didn’t describe. I believe that is exactly the information and understanding loss that an Omni model attempts to address, which doesn’t have anything to do with output.
Some models do also offer “Omni” output, but that is more like a feature of the model than what Omni models are by design I think.
reply