Write web scrapers in Ruby using a clean, AI-assisted DSL. Kimurai uses AI to figure out where the data lives, then caches the selectors and scrapes w...
小说下载|小说爬取|起点|笔趣阁|导出Markdown|导出txt|转换epub|广告过滤|自动校对
ai reverse 逆向一把梭
Google play scraper for Python inspired by <facundoolano/google-play-scraper>
SiteOne Crawler is a cross-platform website crawler and analyzer for SEO, security, accessibility, and performance optimization—ideal for developers,...
A scalable, mature and versatile web crawler based on Apache Storm
A tool for pixiv.net. 人人可用的P站爬虫
SpiderSuite (web security crawler) releases, wiki and roadmap
✌️ Python3 BitTorrent DHT crawler
A high performance web crawler / scraper in Elixir.
爬虫逆向案例,已完成:TLS指纹|瑞数|震坤行 | 网易易盾 | 微信小程序反编译逆向(百达星系) | 同花顺 | rpc解密 | 加速乐 | 极验滑块验证码 | 巨量算数 | Boss...
Pure-Python protocol tools and research notes for X.com, with VibeLoft Twitter account datasets
High-performance asynchronous Douyin(抖音) TikTok Xiaohongshu(小红书) Kuaishou(快手) Weibo(微博) Instagram YouTube(油管) Twitter(X) Captcha Solver(验...
一个方便安全研究人员获取每日安全日报的爬虫和推送程序,目前爬取范围包括先知社区、安全客、Seebug Paper、跳跳糖、奇安信攻防社区、棱角社区以及绿盟、腾讯玄...
A multi-thread crawler framework with many builtin image crawlers provided.
zhihu-crawler是一个基于Java的高性能、支持免费http代理池、支持横向扩展、分布式爬虫项目
ChatWeb can crawl web pages, read PDF, DOCX, TXT, and extract the main content, then answer your questions based on the content, or summarize the key...
一个超级轻量的百度图片爬虫
A Tumblr Blog Backup Application
HTTP API for Scrapy spiders
Crawler (Bot) searching for leaks on paste sites.
Kernel-level antidetect browser with the Playwright API you already write. Python + Node SDKs, MCP-ready, unlimited local profiles. Linux x64 + arm64,...
A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places...
Open source SEO audit tool.
Crawl BookCorpus
ArrowDL (Arrow Downloader) is a download manager for Windows, MacOS and Linux
Krawl is a customizable, lightweight, cloud-native web deception server and anti-crawler that creates fake web applications with low-hanging vulnerabi...
Social media (Weibo) comments analyzing toolbox in Chinese 微博评论分析工具, 实现功能: 1.微博评论数据爬取; 2.分词与关键词提取; 3.词云与词频统计; 4.情...
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to...
SEO & Security Audit for Websites. Lighthouse & Security Headers crawler, Sitemap/Keywords/Images Extractor, Summarizer, etc ...
Simple but useful Python web scraping tutorial code.
Stealth Chromium engine that stops scrapers and browser agents from getting blocked, with one line of code change.
DataHen Till is a companion tool to your existing web scraper that instantly makes it scalable, maintainable, and more unblockable, with minimal code...
Doujinshi downloader 绅士漫画下载
Java API For Chrome and Firefox
🎓 中国大学MOOC、学堂在线、网易云课堂、好大学在线、爱课程 MOOC 课程下载。
[Unmaintained] A simple and clean video/music/image downloader 👾
🛑 image collector, which supports custom acquisition source configuration and is compatible with MacOS and Windows operating systems.
🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON.
A simple and flexible web crawler that follows the robots.txt policies and crawl delays.
The Big Brother is a weaponized OSINT platform featuring username enumeration (473+ platforms), quad-vector visual intelligence, Sky Radar tracking, c...
js cookie逆向利器:js cookie变动监控可视化工具 & js cookie hook打条件断点
:paw_prints: Creeper - The Next Generation Crawler Framework (Go)
《爬虫逆向进阶实战》书籍代码库
BaiduSpider,一个爬取百度搜索结果的爬虫,目前支持百度网页搜索,百度图片搜索,百度知道搜索,百度视频搜索,百度资讯搜索,百度文库搜索,百度经验搜索和百...
A lightweight web crawler framework.(Java爬虫框架)
:newspaper: Let LLM Summarize Hacker News for You
A Tumblr and Twitter Blog Backup Application
The best PTT library
获取免费socks/https/http代理的网站集合