zxuhan

zxuhan

👤 Developer

1 repository on SrcLog

View on GitHub
1 Repos
4 Stars
1 Forks
0 Watchers

🏆 Show off this rank

zxuhan on SrcLog (holo style)

Add it to a GitHub README or profile — the card updates as the ranking changes:

Repositories (1)

llm-router zxuhan/llm-router Go

Cache-aware router for OpenAI-compatible LLM servers, in Go. Per-worker radix trees route each request to the worker holding its KV prefix. Validated on 4x A100 + vLLM and Apple Silicon + llama.cpp.

4