Most popular crawler repositories and open source projects

vcrawler-engine

database and sites api + WPF client

1   9   9  

ferret

A modern pythonic lib to extract data from news pages

0   9   9  

ticketnak

Ticketswap Facebook crawler

3   9   9  

manga-s

个人漫画管理应用 || 漫画平台 || 爬虫

1   9   9  

scrawler

Declarative, scriptable web robot (crawler) and scrapper

1   9   9  

wangyin-blog-spider

爬取王垠的博客，输出pdf文档

4   9   9  

2-Distributed-Crawler

A distributed crawler to capture screenshots and log the redirection

4   9   9  

go_slack_bot

고언어 기반 슬랙 크롤링 봇입니다. Slack interactive bot made by go, in...

1   9   9  

SimpleSpider

lagou spider

3   9   9  

beian-domain

获取最新可备案域名列表爬虫

0   9   9  

arena-of-valor-data-scraper

Scrape data from Arena of Valor's official website.

2   9   9  

psi-report

Crawls a website, gets PageSpeed Insights data for each page, and expo...

4   9   9  

porn

auto download 91porn hot movies

4   9   9  

Times-Of-India-Web-Crawler-for-Articles

Crawls through Web pages of Times of India Website and saves articles...

0   9   9  

stackshare

A simple Web crawler for stackshare.io using scrapy .

2   9   9  

offensive-fortune

A script for generating fortune cookie from the the funniest and most...

0   9   9  

ipgw-py-manager

NEU new ipgw python manager

2   9   9  

SSN-Intranet-Downloader

Python Script to download all files for a given branch & semester from...

4   9   9  

stay-reader

📚Miniprogram Book Reader

5   9   9  

NaverRealTimeRanking

Naver Keyword Crawler (네이버 실시간 검색어 순위 크롤러)

1   9   9  

codeforces-management-tools

Conmato: A Command Line Interface (CLI) for Codeforces Management Tool...

0   9   9  

ImageSpider

超轻量级多协程百度图片爬虫

4   9   9  

immobilienscout24-tracker

A php based web crawler to track Immobilienscout24.de website for new...

2   9   9  

Abyss-Watcher

Abyss Watcher - Malware Downloader

2   9   9  

Go-IMDb-Crawler

Want to know which celebrities have a common birthday with yours? 👀...

0   9   9  

frontendmasters-crawler

A demo of a serverless crawler built on AWS Lambda (scheduled tasks) a...

0   9   9  

ScrapySplashWrapper

A wrapper that uses scrappy and splash to crawl a website.

10   9   9  

AirSpider

A Fast and Light Python Spider Framework 🕷️

7   8   8  

sprite

基于python协程池、用法灵活的高性能爬虫框架

2   8   8  

crawler

A node.js crawler support custom special crawl rules.

0   8   8  

Crawler_weibo

Python 抓取新浪微博m站微博信息

2   8   8  

aragog

Distributed web scraping framework

1   8   8