> It took him two hours of passing errors to Claude for the endpoint to start working

What? It's literally three actions and you're good: download llama.cpp, download the model on Huggingface, and run it with.

I have no idea how it's supposed to take two hours (unless you have a slow connection and the model download takes this much time, that is).