I suspect they’re ignoring it because there isn’t money in it.
Jev is super cool, and it’s super cool that it can handle novel categorization. Most organizations aren’t spending the big bucks on novel classification, though. They’re doing the same thing a billion times, which means Jev has to compete with having a custom model built and doing your own inference.
It’s very hard to build something generic like Jev, but it’s fairly manageable to build something more specific that handles a single kind of task. Especially at the scale where OpenAI would care about your attention.
I don’t really understand neural networks and I managed to train one to do hand gesture recognition on an old Leap Motion. It’s a pretty limited domain, but it trains on the CPU of my decade old laptop in under 3 minutes. The accuracy is quite good too, especially when you discount errors from the Leap (like misidentifying which finger is extended).
When talking about Jev, people love to bring up the whole "bitter lesson" idea that generalization will win out. Maybe it will in the end, but I tried most of the recent Jev-likes on my own not-very-complex use case and they weren't competitive with domain-specific classification at all.
General classifiers are actually incredible for two things:
1. inference on arbitrary context / world knowledge with fast, structured output
2. I almost don't want to say it because I don't know if people realize this yet... but that makes traces from any kind of non-AI process or system "learnable", you can easily model it by by just asking it to predict the next outcome during learning and compare against what happened.
which also means it could also learn new strategies through rollouts in a way that generalizes across semantic space which means...
so these models are actually quite promisingly useful for existing companies because they do have general traces of what happened in their company and want to discover ways they can tweak their business processes, and then there is durable demand for models you can embed within a product or UX flow without exposing your user to slop + then train or customize and build real IP or custom capabilities
general classification itself is quite interesting IMO for applications that need to make judgment calls or select one of many UX actions, and a much better fit for this than LLMs. So if you think about it, this is the type of thing businesses might be calling a lot - every trace (eg user session, package shipment, sales motion, marketing campaign, job outreach) + proactively throughout dynamic experiences to decide whether to eg end a call or which of multiple UIs to manipulate
this is just an unfamiliar product experience to most people because it hasn't really been bootstrappable with a strong general baseline until now. But now it's posible and I think it will be very col.
I think you’re misunderstanding me; I’m not saying places will use an LLM instead (whether cloud-based or on-premise).
I’m talking about them training their own models with a much more specific purpose.
You brought up choosing a menu option in a UI. Jev is really good at something like “find the menu option that best fits the users prompt”. That’s general, and requires a big complicated model.
Companies are usually more interested in something less variable. Ie the menu option they’re looking for is fixed. “Take an arbitrary website and find the Login button”. That’s a _way_ simpler model, and well within the abilities of many large companies to train.
You only need to pay Jev if there are so many variables the model basically has to be general.
Eg classifiers are a separate thing from LLMs and are literally only capable of structured output. It will output one of the X options it was trained to recognize.
I think some of this is that the Venn diagram of people who like AI and the people who like machine learning isn’t a circle. There’s a lot of “intelligences” out there that aren’t an LLM or decision model.
Jev is super cool, and it’s super cool that it can handle novel categorization. Most organizations aren’t spending the big bucks on novel classification, though. They’re doing the same thing a billion times, which means Jev has to compete with having a custom model built and doing your own inference.
It’s very hard to build something generic like Jev, but it’s fairly manageable to build something more specific that handles a single kind of task. Especially at the scale where OpenAI would care about your attention.
I don’t really understand neural networks and I managed to train one to do hand gesture recognition on an old Leap Motion. It’s a pretty limited domain, but it trains on the CPU of my decade old laptop in under 3 minutes. The accuracy is quite good too, especially when you discount errors from the Leap (like misidentifying which finger is extended).