web-crawler
xiaomaozjj666
Scrapling 风格隐身爬虫库：自适应选择器 · TLS 指纹 HTTP · JS 渲染 · 回调式 Spider 框架（重试/robots/去重/中间件管道）· JS 逆向 Agent · 验证码识别 · MCP 服务 · 本地 Web UI · 授权渗透侦察 · 默认 SSRF 防护
4
Stars
1
Forks
0
Watchers
Python
Language
mit
License
64
SrcLog Score
Growth over time
2 data points · 2026-09-12 → 2026-10-05
Stars
Forks
Watchers
💬
How do you feel about this project?
Ask AI about web-crawler
Question copied to clipboardWhat is the xiaomaozjj666/web-crawler GitHub project? Description: "Scrapling 风格隐身爬虫库：自适应选择器 · TLS 指纹 HTTP · JS 渲染 · 回调式 Spider 框架（重试/robots/去重/中间件管道）· JS 逆向 Agent · 验证码识别 · MCP 服务 · 本地 Web UI · 授权渗透侦察 · 默认 SSRF 防护". Written in Python. Explain what it does, its main use cases, key features, and who would benefit from using it.
Question is copied to clipboard — paste it after the AI opens.
How to clone web-crawler
Clone via HTTPS
git clone https://github.com/xiaomaozjj666/web-crawler.git
Clone via SSH
[email protected]:xiaomaozjj666/web-crawler.git
Download ZIP
Download master.zipFound an issue?
Report bugs or request features on the web-crawler issue tracker:
Open GitHub IssuesSimilar to web-crawler
scrapy
newspaper
colly
headless-chrome-crawler
Lulu
isp-data-pollution
Sasila
webster
cdp4j
crawler
spidy
stopstalk-deployment
N2H4
memorious
easy-scraping-tutorial
antch
pomp
Harvester
diffbot-php-client
talospider
corpuscrawler
Python-Crawling-Tutorial
learn.scrapinghub.com
crawling-projects
dig-etl-engine
crawlkit
scrapy-selenium
spidyquotes
zcrawl
podcastcrawler