It can run on one strix halo machine, which is what I have. For everything else, there’s a vibrant community brute forcing more efficient engines and quantizations. There’s a minimum bar right now, so it depends on your hardware.
I bought my box for $3k last year because I hadn't updated my PC since the first threadripper 1950x, and the Strix Halo is the only x86 with unified memory that's optimized for the kind of CPU intensive stuff I do, and they finally have proper AVX512 too. It was the perfect upgrade for me (except for lack of PCIe 5 & ECC), which coincidentally happened to be perfectly positioned for local inferrence.
and I liked it so much (and saw that the RAM drought was going to last for at least a few years) so I bought a second one to use as a general server for myself and my family.
Why cant the community piece together the open-weights using open community-based like, implementation or whatever magic sauce integrates those weights
What are you actually living off is the "million dollar" question? Normal people have to pay income tax and its on the gross or whatever, how are you paying your bills is completely absent here
Can everyone just reverse the damn burden and play People's advocate for one damn second?
Why is it normal that regular people would have to sell their homes just to have a prayer of paying their minimum nut but its aways "muh unrealized stocks" when it comes to billiionaires?
Why should they be exempt from a propoetionate burden to pay the societys on a from whom they are fundamentally predicated to be feasible?
reply