TransformerEngine

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory utilization in both training and inference.

machine-learning deep-learning

View on GitHub Website

3.3k Stars

700 Forks

3.3k Watchers

Python Language

apache-2.0 License

100 SrcLog Score

Cost to Build

$947.5K

Market Value

$6.25M

How is this calculated?

Growth over time

3 data points · 2025-04-01 → 2026-04-01

Stars Forks Watchers

💬

How do you feel about this project?

Ask AI about TransformerEngine

Question copied to clipboard

What is the NVIDIA/TransformerEngine GitHub project? Description: "A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory utilization in both training and inference.". Written in Python. Explain what it does, its main use cases, key features, and who would benefit from using it.

Question is copied to clipboard — paste it after the AI opens.

How to clone TransformerEngine

Clone via HTTPS

git clone https://github.com/NVIDIA/TransformerEngine.git

Clone via SSH

[email protected]:NVIDIA/TransformerEngine.git

Download ZIP

Download master.zip

Found an issue?

Report bugs or request features on the TransformerEngine issue tracker:

Open GitHub Issues

Similar to TransformerEngine

tensorflow keras scikit-learn caffe TensorFlow-Examples tesseract machine-learning-for-software-engineers awesome-deep-learning-papers face_recognition CNTK incubator-mxnet data-science-ipython-notebooks lectures cs-video-courses julia Qix Screenshot-to-code spaCy cheatsheets-ai awesome-deep-learning python-machine-learning-book WaveFunctionCollapse handson-ml ML-From-Scratch gun dive-into-machine-learning tfjs tflearn awesome-datascience openpose