ghostcacher
david-spies
GhostCacher is a distributed Key-Value (KV) prompt caching orchestrator that dramatically reduces LLM inference latency and cost by storing and reusing the computed attention states of frequently used prompt prefixes across a distributed GPU cluster.
Growth over time
2 data points · 2026-07-24 → 2026-08-30How do you feel about this project?
Ask AI about ghostcacher
Question copied to clipboardWhat is the david-spies/ghostcacher GitHub project? Description: "GhostCacher is a distributed Key-Value (KV) prompt caching orchestrator that dramatically reduces LLM inference latency and cost by storing and reusing the computed attention states of frequently used prompt prefixes across a distributed GPU cluster.". Written in Rust. Explain what it does, its main use cases, key features, and who would benefit from using it.
Question is copied to clipboard — paste it after the AI opens.
🏆 Show off this rank
Add it to a GitHub README or profile — the card updates as the ranking changes:
How to clone ghostcacher
Clone via HTTPS
Clone via SSH
Download ZIP
Download main.zipFound an issue?
Report bugs or request features on the ghostcacher issue tracker:
Open GitHub Issues