KVSplit
dipampaul17
Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.
Growth over time
3 data points · 2025-09-25 → 2026-04-21How do you feel about this project?
Ask AI about KVSplit
Question copied to clipboardWhat is the dipampaul17/KVSplit GitHub project? Description: "Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.". Written in Python. Explain what it does, its main use cases, key features, and who would benefit from using it.
Question is copied to clipboard — paste it after the AI opens.
How to clone KVSplit
Clone via HTTPS
Clone via SSH
Download ZIP
Download master.zipFound an issue?
Report bugs or request features on the KVSplit issue tracker:
Open GitHub Issues