awesome-tokenizers

awesome-tokenizers

NLPOptimize

A curated list of tokenizer libraries for blazing-fast NLP processing.

collections
View on GitHub
14 Stars
0 Forks
1 Watchers
50.1 SrcLog Score
Cost to Build
$654
Market Value
$590
How is this calculated?

Growth over time

9 data points  ·  2026-04-08 → 2026-11-11
Stars Forks Watchers
💬

How do you feel about this project?

Ask AI about awesome-tokenizers

Question copied to clipboard

What is the NLPOptimize/awesome-tokenizers GitHub project? Description: "A curated list of tokenizer libraries for blazing-fast NLP processing.". Explain what it does, its main use cases, key features, and who would benefit from using it.

Question is copied to clipboard — paste it after the AI opens.

🏆 Show off this rank

NLPOptimize/awesome-tokenizers on SrcLog (holo style)

Add it to a GitHub README or profile — the card updates as the ranking changes:

How to clone awesome-tokenizers

Clone via HTTPS

git clone https://github.com/NLPOptimize/awesome-tokenizers.git

Clone via SSH

[email protected]:NLPOptimize/awesome-tokenizers.git

Download ZIP

Download main.zip

Found an issue?

Report bugs or request features on the awesome-tokenizers issue tracker:

Open GitHub Issues

Similar to awesome-tokenizers

awesome-python awesome-flutter windows-kernel-exploits Collections-C vvedenie-mashinnoe-obuchenie streamex Buckets-JS Koloboke cyclops Amazing-Windows-Apps Ardent cdsa explore Knapsack laravel-helpers swift-nonempty vue-mc AwesomeArticles stream-parser SmoothieMap kotlinx.collections.immutable awesome-jupyterlab dictomaton f .NET-Big-O-Algorithm-Complexity-Cheat-Sheet miniboxing-plugin CodeJam grizzly essentials4j paginable-collections