I've run local video generation models on an 8GB graphics card and know firsthand that nothing runs smoothly when memory is insufficient. So seeing 14GB of weights crammed into 2GB of RAM is impressive.
If running continuously for over an hour (like an overnight batch task), will a fanless MacBook Air overheat and throttle? Can the SSD handle the continuous weight reads and sustained output speeds?
I think it will throttle quite soon, but I haven't tried runs longer than 30minutes with this engine.
However, there is no constant load on ssd or gpu. i/o and gpu work are alternating and there is a brief idle periods for each i/o and gpu during inference (because gpu waits for i/o and after that i/o waits for gpu)
I use two AI CLIs every day (one for main tasks and the other for review), and context migration is a real pain point.
My current workaround is to write the context as a Markdown file and save it to disk, then let both of two CLI to read. I'd like to ask you, what are the advantages of this tool compared to using a file as a bridge?
I use both of subscription and API services. on last month, i chat with CLI and let it to do something. After that, maybe in one days pass, i received the $32 USD bill. it cause my left my API key and CLI call the API to do job not through subscription.
If running continuously for over an hour (like an overnight batch task), will a fanless MacBook Air overheat and throttle? Can the SSD handle the continuous weight reads and sustained output speeds?
Great work, congratulations on the release!