awesome-tokenizers

NLPOptimize

A curated list of tokenizer libraries for blazing-fast NLP processing.

collections

View on GitHub

14 Stars

0 Forks

1 Watchers

50.1 SrcLog Score

Cost to Build

$632

Market Value

$571

How is this calculated?

Growth over time

4 data points · 2026-04-08 → 2026-07-22

Stars Forks Watchers

💬

How do you feel about this project?

Ask AI about awesome-tokenizers

Question copied to clipboard

What is the NLPOptimize/awesome-tokenizers GitHub project? Description: "A curated list of tokenizer libraries for blazing-fast NLP processing.". Explain what it does, its main use cases, key features, and who would benefit from using it.

Question is copied to clipboard — paste it after the AI opens.

How to clone awesome-tokenizers

Clone via HTTPS

git clone https://github.com/NLPOptimize/awesome-tokenizers.git

Clone via SSH

[email protected]:NLPOptimize/awesome-tokenizers.git

Download ZIP

Download main.zip

Found an issue?

Report bugs or request features on the awesome-tokenizers issue tracker:

Open GitHub Issues

Similar to awesome-tokenizers

awesome-python awesome-flutter windows-kernel-exploits Collections-C vvedenie-mashinnoe-obuchenie streamex Buckets-JS Koloboke cyclops Amazing-Windows-Apps Ardent cdsa explore Knapsack laravel-helpers swift-nonempty vue-mc AwesomeArticles stream-parser SmoothieMap kotlinx.collections.immutable awesome-jupyterlab dictomaton f .NET-Big-O-Algorithm-Complexity-Cheat-Sheet miniboxing-plugin CodeJam grizzly essentials4j paginable-collections