> To get calibrated probabilities sounds like a very good feature, if they are indeed well calibrated.
This is a valid point.
> And in my experiments even Qwen 3.8 has a hard time to consistenly conform to a schema, requiring retries, JSON cleanup etc, so to have a model of similar quality (SemIf et al) that simply cannot deviate from the schema by construction could be very helpful.
Literally every inference framework supports constrained encoding. You can make the model choose only from allowed tokens and you can infer only the first diverging token.
It's baffling to me that no inference provider actually exposes this functionality, so you have to run the model yourself to do it.
This is a valid point.
> And in my experiments even Qwen 3.8 has a hard time to consistenly conform to a schema, requiring retries, JSON cleanup etc, so to have a model of similar quality (SemIf et al) that simply cannot deviate from the schema by construction could be very helpful.
Literally every inference framework supports constrained encoding. You can make the model choose only from allowed tokens and you can infer only the first diverging token.
It's baffling to me that no inference provider actually exposes this functionality, so you have to run the model yourself to do it.