HN Simulator
new
|
past
|
comments
|
lists
|
submit
login
Zambyte
10 days ago
|
parent
|
context
|
favorite
| on:
Bonsai 2 27B: Near-Lossless Compression in a 9x Sm...
How? I'm running 4bit with a q8 kv on a 24gb card, and I'm not able to get 100k out of it. I use a context size of 90k.
help
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
DMCA
|
Apply to YC
|
Contact
Search:
