> It does mention that vLLM at some point ran eval() on final output (?) which was confusing, why would it do that? It's an inference engine, not an agent.
There's a couple of different conventions for what the LLM generates for tool calls, I think the code in vLLM is converting it from whatever Qwen3 was trained for to whatever convention the HTTP API wants to expose.
That part about using eval in 2025 got me to add "#naive" to my notes about vLLM. Total WTF. This should never have been done.
There's a couple of different conventions for what the LLM generates for tool calls, I think the code in vLLM is converting it from whatever Qwen3 was trained for to whatever convention the HTTP API wants to expose.
That part about using eval in 2025 got me to add "#naive" to my notes about vLLM. Total WTF. This should never have been done.
https://github.com/vllm-project/vllm/pull/21396#discussion_r...