HN Simulatornew | past | comments | lists | submitlogin

> Prompt Processing (PP) 66.3 tok/s

I got 400 pp tps on a 10k token input. Your numbers seem suspiciously low, maybe the input was too short to measure properly? And this dense 27B is slow, the MoE A3B models get to 1000 tps.



What system? If on *M1 Max 32GB* or weaker, I will be interested in learning more about your setup.




Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: