Flexible Node.js crawler library
Diskover Community Edition - Open source file indexer, file search engine and data management and analytics powered by Elasticsearch
简单易用的Python爬虫框架,QQ交流群:597510560
[爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized c...
⚡️Lightning-fast async download tool for bilibili and more
Boss直聘爬虫 / BOSS直聘职位数据抓取工具,基于 Chrome CDP 协议复用真实登录态,绕过字体反爬,输出明文薪资 JSON/CSV + 薪资技能分析。A Chrome-CDP-based BO...
Async Python 3.6+ web scraping micro-framework based on asyncio
Python binding to Modest and Lexbor engines. Fast HTML5 parser with CSS selectors for Python.
CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)
Google, Naver multiprocess image web crawler (Selenium)
python爬虫,目前库存:网易云音乐歌曲爬取,B站视频爬取,知乎问答爬取,壁纸爬取,xvideos视频爬取,有声书爬取,微博爬虫,安居客信息爬取+数据可视化,哔哩...
❤️ Fredy - [F]ind [R]eal [E]state [D]amn Eas[y] - Fredy keeps searching for new apartments, houses, and flats in Europe on platforms like ImmoScout24,...
抖音爬虫——采集账号主页、喜欢、收藏、音乐原声、话题、搜索、合集、作品、关注、粉丝等公开数据。
The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns
Crawl a website and run it through Google lighthouse
Scrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!
An ergonomic, privacy-aware Python HTTP Client
Collection of patches for puppeteer and playwright to avoid automation detection and leaks. Helps to avoid Cloudflare and DataDome CAPTCHA pages. Easy...
Elasticsearch File System Crawler (FS Crawler)
小红书数据采集、网站图片、视频资源批量下载工具,颜值超高的数据采集工具(批量下载,视频提取,图片)Telegram:https://t.me/+ZtLSwuIKTo44MDY1
A web privacy measurement framework
浏览过的精彩逆向文章汇总,值得一看
K 哥爬虫代码分享,JS 逆向,爬虫进阶。关注公众号:K哥爬虫
🤖 Scrape data from HTML websites automatically by just providing examples
It makes a preview from an URL, grabbing all the information such as title, relevant texts and images.
Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.
📝 quickly crawl the information (e.g. followers, tags etc...) of an instagram profile.
The fastest dork scanner written in Go.
The Prime Cross Site Request Forgery (CSRF) Audit and Exploitation Toolkit.
Python爬虫,京东自动登录,在线抢购商品
Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible...
收集各种免费的 Python 爬虫项目
Beanbun 是用 PHP 编写的多进程网络爬虫框架,具有良好的开放性、高可扩展性,基于 Workerman。
基于appium的app自动遍历工具
👧 美女写真套图爬虫(二)
massive SQL injection vulnerability scanner
一个好用的哔哩哔哩漫画下载器,拥有图形界面,支持关键词搜索漫画和二维码登入,黑科技下载未解锁章节,多线程下载,多种保存格式,本地漫画管理,一键检查更新...
🤖 Fake fingerprints to bypass anti-bot systems. Simulate mouse and keyboard operations to make behavior like a real person.
Run a high-fidelity browser-based web archiving crawler in a single Docker container
Browser4 — an AI-native browser engine for autonomous agents, intelligent extraction, and large-scale web automation.
:beers: bilibili video (including bangumi) and danmaku downloader | B站视频(含番剧)、弹幕下载器
A Simple Mihomo GUI. 一个简易的 Mihomo 桌面客户端
📰 Newspaper4k a fork of the beloved Newspaper3k. Extraction of articles, titles, and metadata from news websites.
Easily download all the photos/videos from tumblr blogs. 下载指定的 Tumblr 博客中的图片,视频
Open-source, self-hosted enterprise & site search server built on OpenSearch. Crawls web / file / DB / cloud sources, 20+ languages, REST API, and AI/...
Scalable Python web scraping scripts for +80 popular domains
Crawly, a high-level web crawling & scraping framework for Elixir.
An ergonomic, privacy-aware Rust HTTP Client
Golang短视频去水印:抖音,皮皮虾,火山,微视,最右,快手,全民小视频,皮皮搞笑,西瓜视频,虎牙,梨视频,acfun,好看视频...