I mean artificial is in their name....

seems to be…

at first i got very excited about a new fast transcription model (voz) but turns out its just parakeet v3 with some new inference code which is macOS/iOS specific

It's an ANE optimized version of Parakeet, with our own inference, which enabled us to push performance to about 300x realtime speed on an iPhone 16/17. Our next gen Voz model is trained from scratch and will be at least twice as fast. Android and other platforms will land soon.

I understand. And inference optimizations are great. It's just that it's not a new model, and that's what I was excited about. I work on cross-platform transcription inference and was hoping for something more than just a re-badged parakeet v3

Look forward to the next model

This seems to be a common theme with these "European sovereign AI" companies. Mostly built on top of other open source work and slightly adjusted, often to collect grant money (although I'm not sure about that last part in this case).

Even worse, they made their additions proprietary:

> How much does Voz cost?

> Every model is free up to 100k monthly active devices per SDK. Unlimited inference per user. Contact us for custom licenses.

Nice, now I know what I actually need.

I was also thinking that this is almost too good to be true

it's unbelievable this was not pointed out earlier in this thread, long live tom 7

what do you mean cheaper/faster is not really coming? the cost of the same level of intelligence steadily decreases year over year. computer hardware also advances at the same time enabling cheaper and faster serving (or move to local)

people are lazy

people don't want to maintain infra (ie adding new models all the time)

people want their queries to work without thinking

if a provider goes down, openrouter queries dont (ideally)

it's really not that complex to understand

Yeah, but who wants to deploy a model just like that. I mean, what if that model doesn't handle the same guard rails. There's a million reasons why switching models should not be a snap decision.

The back-up model is not a bad idea, but nonetheless...

As a maintainer just saying no and closing PRs is largely the solution

Sure, for FOSS. But at a business I'm not sure you'll stay employed long if you do this regularly and don't try to find ways to adapt to the new world we live in.

To be clear, I don't have an answer either. Reviews have definitely become a bottleneck, and the agent-created code has absolutely not reached a "broadly trustworthy" state for complex problems and maintainable contributions.

Depends on the corporate environment. Refusing to look at oversized PRs makes good business sense, there's an argument to be made that working like this sabotages the quality of the product.

Agree, I was not talking about a business context.

But if people are submitting garbage in a business context you got other problems you need to address.

Unfortunately you are wrong. Time marches on and progress in silicon does too. You may not like the timeline, but it will happen.

While not exactly an “SSD”, High Bandwidth Flash will make it to market and make big sparse MOE models even more accessible. Everyone knows bandwidth is the game right now and you think hardware vendors are going to sit on their heels and not take part of the market? Absolutely silly to discount progress both on the software and hardware sides.

all day yesterday, Taiwan time Sol was having significant issues.

“Overloaded”, maybe half of requests or more wouldn’t go through

That’s what I was seeing too, and my cache hit rate was below 25% during that same time, leading to significant burn of my weekly limit (Pro 5x plan) via all the uncached input. Doesn’t prove that the overload caused the cache failure, but it does seem to point to some common infrastructure cause. No such problems (cache hit or system overload) with Terra or Luna.

Was that Codex/subsidised-usage or API? I do get overloaded in Codex-account from time to time, but API is rock solid.

They obviously load shed a bit of Codex-sub during peak times, and for the amount of tokens you get for a sub, I don't mind. I just mean the API where you pay-per-token is rock stable.

Yep the latest version has support! Virtually all of the SOTA open models are supported by Handy including the streaming ones like

Nemotron Streaming

Parakeet Unified

Voxtral Mini Realtime

If something you want is not supported, open an issue on transcribe.cpp!