I've set it up on my local machine just now, as my first local image diffuser. I can confirm it's very easy.

I tried stable-diffusion.cpp, following its compile guide here[0], and its Qwen Image-2.1 specific instructions here[1]. It works out of the box. I made a test pelican[2]. It took 3 minutes on a CPU.

[0] https://github.com/leejet/stable-diffusion.cpp/blob/master/d...

[1] https://github.com/leejet/stable-diffusion.cpp/blob/master/d...

[2] https://i.ibb.co/yMknC2K/output.png

Thank you! Can you please check how much RAM does it consume (and require)?

This is what the runtime reports, at Q8:

    total params memory size = 15645.19MB (VRAM 15645.19MB, RAM 0.00MB):
    text_encoders 7669.77MB(VRAM),
    diffusion_model 7331.05MB(VRAM),
    vae 644.38MB(VRAM),
    controlnet 0.00MB(N/A),
    extensions 0.00MB(N/A)

That suggests that 16GB RAM will not be enough.

In fact, like it appears in the reports above, it is "7b" as in

> 7B parameters in its visual generation component

It seems they calibrated the size to fill a 16GB VRAM near the limit. RAM requirements will vary.

Additional question is what kind of local hardware would be required for this? 7B parameters sounds very light weight, but I'm not sure. (Edit: The download is 33 GB).

Edit x2: As usual I'm in a twisty maze of pip packages that don't work together, with obscure errors about missing modules, even though I followed the instructions on the page to the letter. I really wish people didn't use Python for this stuff. A simple C/C++ program would be so much better.

It's about 16 GiB at Q8 quants (combining both the image and language parts). (Meaning, community quantized models from HuggingFace).

I think it will technically run on anything that has enough memory. I just tried it on a standard laptop (dual-channel DDR5), and it took about 3 minutes for a 512x512. If you'd want to run it at interactive speeds, you would want a GPU (one which fits this in VRAM).

> "I really wish people didn't use Python for this stuff. A simple C/C++ program would be so much better."

You mean besides stable-diffusion.cpp ?

> You mean besides stable-diffusion.cpp ?

Yes, thanks, I didn't know about that. Will try it.

I am using sd.cpp, which is the cousin of llama.cpp: https://github.com/leejet/stable-diffusion.cpp

it already has day-0 qwen image 2.1 support!

Probably ComfyUI is one of the easiest way to get started with local image/video models. Or perhaps vLLM, if they have support for it already, would be something like `vllm serve --omni --port 9080`

There is difussion.cpp which is intended for those types of models. I set up krea-2-turbo with the help of ChatGPT 2 months ago, if you have a capable computer that's what I would suggest once it becomes supported.

I am on AI max 395, comfyUI+qwen models is all you technically need. With today's release, I just built a quick and dirty html that allow simpler prompt use and edits ( via headless comfyui ).. its not bad for a day's work, but a little too unpolished to publish. I would say, try comfyUI first ( complex, but it worked OOTB ).

on the linked GitHub page they list support Diffusers, ComfyUI, vLLM-Omni, SGLang, and LightX2V with links to each

> Diffusers, ComfyUI, vLLM-Omni, SGLang, and LightX2V

I think that's all Python (not a direct executable).

You could just do (see the "Quick Start") four `pip install` and have a dozen lines script to generate the image. But `llama.cpp` and similar do not require e.g. installing Torch (or PyTorch) - you can use `llama.cpp` on a non-specialized machine.

"just"

I don't think I have ever once run "pip install transformers" and had it work without three rounds of fiddling

Or trying to install the whole of CUDA on machines that do not even have a GPU (not Nvidia, not anything past the embedded)...

Yep, that's (also) what I meant ;)

Lean, efficient... Also sensible and trouble-less.

The 'not nvidia' is no longer a deal breaker by itself.

I use opencode + to set up all this new ai generation stuff. GLM-5.3 is my current gun. Safely inside podman containers too because I dont trust this fast moving python eco system at all. Never do I want this running on my main OS.

I have FLUX.2 klein and dev, Ideogram, LaDA-Image and SenseNova locally. Works great. Ive never touched a file.

The days of making container yamls myself is over. I read them but I dont edit anymore.

Unsloth Desktop is the easiest way imo. There are already gguf quants of this model, or simply wait until the official one comes out.

why not just as you suggested i.e. https://qwen.readthedocs.io/en/latest/run_locally/llama.cpp.... then get the result either via a UI or wget/curl it back?

I am not sure that llama.cpp also supports image generation models.

Multimodal doesn't guarantee input and output.

> Currently, we support image, audio and video input.

Seems I'm missing something. Does this model support other inputs?

Image outputs are supported, videos I'm not sure but I don't think that's an output, just a preview of the equirectangular example, so, same question here, what does this model outputs that isn't supported?

Not all architectures are supported by llama.cpp . The GGUF format encodes the NN in a standardized way, but then you need code that can use that NN structure.

I understand that llama.cpp could only output text, last time I checked (I do not know how to find a good source for that though).

See https://github.com/ggml-org/llama.cpp/blob/master/src/llama-... , the

  enum llm_arch {

...

I haven't used llama.cpp for image generation either but I recall an issue about it. Unfortunately I can't pinpoint it now and there is the older closed issue https://github.com/ggml-org/llama.cpp/issues/4408 so unless mtmd supports also multimodal outputs out of the box safe to assume output is still limited to text.

It's a diffusion model, completely different from autoregressive attention models.

like https://github.com/ggml-org/llama.cpp/tree/master/examples/d... ?

At the risk of stating the obvious llama.cpp isn't just about LLaMa as https://github.com/ggml-org/llama.cpp/blob/master/src/llama-... someone else pointed out.

Aha I was wrong. Thanks for sharing that!

no worries, I was wrong too, it is multimodal but only for inputs apparently, so for now there seem to only be text as output but no image as output