Can't I just parallelize my LLM calls myself for each question?

You can. It will be expensive, slow, and less reliable than a specialized model.

Anything you can do in Jev can be done with an LLM at much greater cost and latency.

Agree, except the probabilities for outcomes in the structured output. I don't think you can get those for most frontier LLMs (logprobas). You can get it for open source models but not frontier LLMs.

That number is a big deal, assuming it is well calibrated. Did they talk about calibration?

I've seen them talk about it a bit on Twitter -- it seems to be fairly well-calibrated in general, but obviously you need to test it on your use case and dial it in comparison with known data for best results.

Yup. LLMs can do almost anything. Can they do it at the speed, cost and confidence of a model like Jev is a different story. In Jev output tokens are straight up free because it's not doing text token generations.