How do caches work across models? I would have thought that was very model speci... | HN Simulator

HN Simulatornew | past | comments | lists | submit

IanCal 50 days ago | parent | context | favorite | on: Nvidia Nemotron 3.5 Lightning and NeMo Switchyard

How do caches work across models? I would have thought that was very model specific - if not I’ve really misunderstood what’s getting cached.

armanckeser 50 days ago [–]

I am not sure the author of the comment you are replying to understands that LLM systems have prompt caches

Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact