I was dev, and now I am Senior level manager.
Open Weight models are current main focus for many companies with full alignment with top management for very simple reasons:
- stable and predictable performance (no pre-launch models degradation)
- ability to tune them for specific business cases (though still rare tbh)
- better (at least 60% Opus vs Kimi (real,3rd party)) and more competitive pricing
- flat pricing if tokenusage is big enough to justify renting GPU
- decent quality
- much higher guarantees that data will not be sent somewhere (assuming 3rd party inference providers)
- and cherry on top: flat and minimal pricing with absolute confidentiality using Alibaba Apsara stack of recently released AMD Instinct Coder box[1]
It doesn't make any sense unless they are going to exit from inference market. They will be literally one of the costliest option (by output, for flash) if use openrouter as source.
I've put some effort to integrate it to my agentic workflow. The problem, however, with docker in smolvm: it work-ish (there is example), but quite hacky.
Another problem which I wasnt able to solve - persistent image without Dockerfile. CloudInit will be ideal.
Documention at this moment in an early stage.
Overall, its a great project but for me was simpler just use Virtual Machine Manager (libvirt GUI).
I wish all luck to the maintainers, but probably DX-wise I will prefer to have more granular or predictable controls (eg micro cloud from Canonical).
[1] https://www.amd.com/en/ecosystem/oem/supermicro/amd-instinct...