Not just structured output. Dropping down to logprobs, prompting the model to emit one word as the answer, and then ranking the output tokens to pick your answer works great on small Qwen & Gemma models.
The fascinating part to me is that Jev seems like this technique plus post-training to get multiple independent confidence values for each possible answer.
Yeah encryption isn't the issue. The only way I see to fix this is if you stop the user from switching models mid-session, or strip out the thoughts when switching models. Either way you're degrading the user experience.
reply