crawlee

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

npm crawling automation scraping crawler nodejs javascript typescript

View on GitHub Website

18.5k Stars

889 Forks

18.5k Watchers

TypeScript Language

apache-2.0 License

Cost to Build

$15.01M

Market Value

$79.82M

How is this calculated?

Growth over time

30 data points · 2019-01-01 → 2025-07-01

Stars Forks Watchers

💬

How do you feel about this project?

Ask AI about crawlee

Question copied to clipboard

What is the apify/crawlee GitHub project? Description: "Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.". Written in TypeScript. Explain what it does, its main use cases, key features, and who would benefit from using it.

Question is copied to clipboard — paste it after the AI opens.

How to clone crawlee

Clone via HTTPS

git clone https://github.com/apify/crawlee.git

Clone via SSH

[email protected]:apify/crawlee.git

Download ZIP

Download master.zip

Found an issue?

Report bugs or request features on the crawlee issue tracker:

Open GitHub Issues

Similar to crawlee

freeCodeCamp bootstrap vue react You-Dont-Know-JS javascript electron node atom axios three.js webpack html5-boilerplate meteor express angular material-ui Chart.js TypeScript free-programming-books-zh_CN ionic-framework nw.js lodash materialize yarn javascript-algorithms element echarts Front-End-Checklist babel