Yesterday I installed llama.cpp to test it with local AI Data Analyst that I'm building. I was also testing other open LLM providers: Ollama, Jan, vLLM, LM Studio. I had older NVIDIA card (RTX 3070) and llama.cpp instalation was smooth, contrary to vLLM which required me to reinstall CUDA drivers because by default it installed the latest one. I'm curious if there is a speed difference between the same open LLM model served with different runners.

> I had older NVIDIA card (RTX 3070) and llama.cpp instalation was smooth

what model was it that you were able to run with the rtx 3070?

I was able to fit only small models Qwen3.5-4B in RTX3070 which is not very useful for Python and SQL generation thought. When I wan to test larger open LLM models I often just use cloud resources.

Just FYI lm-studio is a GUI wrapper on top of a copy of llama-server that the lm-studio developers compile and distribute

Python outputs is also very versatile. You can use Python to build command line script, web application, desktop app with GUI, notebook with data analysis, or Python package and share with others. It is many ways how Python code can be used by final user.