inference

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

artificial-intelligence machine-learning deployment

View on GitHub Website

9.2k Stars

813 Forks

9.2k Watchers

Python Language

apache-2.0 License

100 SrcLog Score

Cost to Build

$3.77M

Market Value

$27.32M

How is this calculated?

Growth over time

3 data points · 2025-03-01 → 2026-04-01

Stars Forks Watchers

💬

How do you feel about this project?

Ask AI about inference

Question copied to clipboard

What is the xorbitsai/inference GitHub project? Description: "Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.". Written in Python. Explain what it does, its main use cases, key features, and who would benefit from using it.

Question is copied to clipboard — paste it after the AI opens.

How to clone inference

Clone via HTTPS

git clone https://github.com/xorbitsai/inference.git

Clone via SSH

[email protected]:xorbitsai/inference.git

Download ZIP

Download master.zip

Found an issue?

Report bugs or request features on the inference issue tracker:

Open GitHub Issues

Similar to inference

tensorflow keras scikit-learn caffe TensorFlow-Examples tesseract machine-learning-for-software-engineers awesome-deep-learning-papers face_recognition CNTK incubator-mxnet data-science-ipython-notebooks lectures cs-video-courses julia Qix Screenshot-to-code spaCy cheatsheets-ai awesome-deep-learning python-machine-learning-book WaveFunctionCollapse handson-ml ML-From-Scratch gun dive-into-machine-learning tfjs tflearn awesome-datascience openpose