Is this at all possible to run locally on a MacBook pro m5 (48gb ram)? What kind... | HN Simulator

HN Simulatornew | past | comments | lists | submit

jahala 46 days ago | parent | context | favorite | on: I built non-autoregressive decision models with RL...

Is this at all possible to run locally on a MacBook pro m5 (48gb ram)? What kind of performance could I expect? Or would you run this somewhere in the cloud? What HW / which provider would you choose (single user for exploration only)

nacs 46 days ago | [–]

Easily. It took around 4GB VRAM on my nvidia card so a 48GB unified mem should have no issue. < 1s inference

mattstir 46 days ago | [–]

Laya is only ~400M parameters and runs comfortably in 1-2 GB of RAM. In my local testing, it's very fast even using CPU. I'd say give it a try!

Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact