crawler
Repositories (1431)
Backup your friends' Instagram Stories forever and get to keep them even after 24 hours.
Crawlzone is a fast asynchronous internet crawling framework for PHP.
xSMTP 🦟 Lightning fast, multithreaded smtp scanner targeting open-relay and unsecured servers in multiple network ranges.
一个用于 哔哩哔哩漫画 B漫 的下载脚本
超高速异步协程Python爬虫
Search through all your personal data efficiently like web search.
百度贴吧分布式爬虫，用于贴吧数据挖掘。从贴吧维度和用户维度进行数据分析
Golang爬虫 爬取汽车之家 二手车产品库
Slideshare to PDF downloader. Using Selenium and auto scroll-down to get the entire slides completely.
NTU CEIBA 資料下載工具
Some scrapy and web.py exmaples
Scrape Learning (ctrip)
a puppeteer walker 🕷 🕸
Powerful web scraping framework for Crystal
fetchman is a simple crawler system/简单好用的爬虫框架
tumblr解析网站
JS逆向技巧：页面跳转JS代码定位通杀方案
基于NodeJS的基金数据爬虫，爬取的数据存于github的@nullpointer/fund-data。
Just a crawler based on tg-cli for Telegram. Deprecated by now, please use telegram-export.
python requests + Django+ nodejs koa+ mysql to crawl eastmoney fund and stock data,for data analysis and visualiaztion .
Tumblr Download Tool with High Speed and Customization. 高性能&高定制化的Tumblr下载工具。
Parse government documents into well formed JSON
深圳大学抢课系统
Privacy Web Search Engine (not meta, own crawler)
Your preferred open source focused crawler for the deep web.
Vietnamese text data crawler scripts for various sites (including Youtube, Facebook, 4rum, news, ...)
Some classic web crawler projects.一些经典的爬虫
台股機器人，提供即時個股及大盤報價、走勢、新聞、盤後資料等 Telegram bot to query real-time TW stock quotes, charts, news, and other related informatio...
A PasteBin scrapper that doesnt rely on the PasteBin scrape API
crawl QR-codes from search engines and look for bitcoin private keys
Automatically download all PDF files of searching results & their patent families found on Google Patents.
News extraction and scraping. Article Parsing
A crawler for automated functional testing of a web application
雪球爬虫 高效爬取近期沪深A股股票评论并自动生成PDF版情感分析报告
Instagram user's photos and videos downloader. Download all media files from any username. Working 2022!
Extract data from websites in LLM ready JSON or CSV format. Crawl or Scrape entire website with Website Crawler
A bot that automatically sends emails to new ads posted in any desired xe.gr search url.
a simplified directed customizable website crawler
动漫之家漫画站电脑版原图爬虫
[码云](https://gitee.com/generals-space/site-mirror-py) 通用爬虫, 仿站工具, 整站下载
NEU new ipgw python manager
Midnight Sea: navigating in the waters of dark web markets
Wrapper for TikTok API
🔥 PHP library to warm up caches of URLs located in XML sitemaps
steam 特惠游戏榜单（自动刷新）
可配置的小说下载及电子书生成工具
A command line tool based on the crypto-crawler library.
Implement scrapy with asyncio
Download HD images from pinterest by your favorite keywords
👾 CLI MetaSpy (Facebook, Instagram) scraper and crawler - instagram account, facebook accounts, pages and search