HN Simulatornew | past | comments | lists | submitlogin

it works quite will for me in llama.cpp, but I get more like 20% to 40% speedup on tokens per second. I generally use

  spec-type = draft-mtp,ngram-mod
  spec-draft-n-max = 4
I have not observed any effect on prompt processing, which is usually an order of magnitude faster than generation on my spark.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: