crawler

A self-hosted, extensible manga reader and download tool with plug-in support.

Multi-threaded web scraper to download all the tutorials from www.learncpp.com and convert them to PDF files concurrently.

Findpapers: A tool for helping researchers who are looking for related works

Public Opinion Mining System of Taiwanese Forums

Sample of using proxies to crawl baidu search results.

一只优雅的正方教务系统爬虫。

爬虫代理IP池服务，可供其他爬虫程序通过restapi获取

下载搜狗、百度、QQ输入法的词库文件的 python 爬虫，可用于构建不同行业的词汇库

✴️ An experimental graph database

gocrawler, go分布式爬虫框架

Download books from bookwalker.jp/bookwalker.com.tw

🌱 goClone - clone websites in seconds

Processes XML sitemaps and extracts URLs. Includes features such as support for both plain XML and compressed XML files, multiple input sources, prote...

Cross-platform persistent and distributed web crawler :link:

Powerful dork searcher and vulnerability scanner for windows platform

Project thu thập điểm chuẩn đại học 2014 - 2018 và phân tích dữ liệu

Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler

LFITester is a Python3 program that automates the detection and exploitation of Local File Inclusion (LFI) vulnerabilities on a server.

Scrapy + Puppeteer

wxpath - declarative web crawling with XPath; a Web Query Language (WQL)

Proxy List Scrapper

Golang pkg to quickly return a preview of a webpage (title/description/images)

A universal solution for web crawling lists. 抓取网页列表的通用解决方案

qcrawl - fast async web crawling & scraping framework for Python.

A complimentary proxy to help to use SPM with headless browsers

✨ BOSE IS SWISS ARMY KNIFE 🔪 FOR BOT DEVELOPMENT. THE ULTIMATE BOT DEVELOPMENT FRAMEWORK. 🤖

爬虫, http代理, 模拟登陆!

AI-powered web scraping CLI. Describe what you want, get a production-ready Scrapy spider. Write once, reuse forever.

使用asyncio和aiohttp开发的轻量级异步协程web爬虫框架

A command-line utility designed to recursively spider webpages for URLs. It works by actively traversing websites - following links embedded in webpag...

Simple Weibo Scraper

Spider ported to Python

This package is a complete tool for creating a large dataset of images (specially designed -but not only- for machine learning enthusiasts). It can cr...

Get Aliexpress product details in JSON

针对某亿些小说网站的爬虫

免费 IP 代理池。Scrapy 爬虫框架插件

A command-line tool to crawl websites using puppeteer.

Java 網路資料爬蟲包

Parsed data from website https://jadwalsholat.org

练手项目：Comment of Interest 电商文本评论数据挖掘 （爬虫 + 观点抽取 + 句子级和观点级情感分析）

A package to get list of user agents based on filters such as operating system, software name etc..

Continuously search imageboards threads for images/webms and download them

Google Arts & Culture high quality image downloader

Turn any developer documentation into a GPT

Anti-detect browser for web scraping and automation. Engine-level fingerprint masking for Chromium and Firefox. Self-hosted, Docker-ready. Integrates...

Python: An all-in-one Web Crawler, Web Parser and Web Scrapping library!

Grabs current REWE discounts and saves them in a markdown file || Holt sich aktuelle REWE-Angebote und exportiert sie in eine Markdown-Liste

A spider on Dcard. Strong and speedy.