CICD_LLM_DATA_SCRAPER
devo8604/CICD_LLM_DATA_SCRAPER
Python
Automated pipeline for generating high-quality Q&A training data from Git repositories. Processes source code with LLMs to create fine-tuning datasets. Features smart caching, resume support, MLX (Apple Silicon) & llama.cpp backends, multiple export formats (Alpaca, ChatML, etc).