Most popular scraping repositories and open source projects

spidyquotes

Example site for web scraping tutorials

15   31   31  

fb-event-scraper

Recursively scrape facebook events

3   31   31  

tiktok-trending-data-api

Scraping the TikTok Discovery Data API every 1 hour using Github Actio...

6   31   31  

moneyman

Automatically save transactions from all major Israeli banks and credi...

19   31   31  

naos

📉 Uptime and error monitoring CLI

1   30   30  

video-crawler

Crawl websites for videos from Youtube, Vimeo, Soundcloud, etc

4   30   30  

api-flight.com

Main API Flight Git Repository

9   30   30  

Whatsapp-Scraper

Scraps all the open chats, and their last n messages, and saves them i...

12   30   30  

etf4u

📊 Python tool to scrape real-time information about ETFs from the web...

5   30   30  

jmd_imagescraper

Image scraping library for creating deep learning datasets

12   30   30  

gopher-parse-sitemap

A high effective golang library for parsing big-sized sitemaps and avo...

16   30   30  

scaling-to-distributed-crawling

Repository for the Mastering Web Scraping in Python: Scaling to Distri...

7   30   30  

insta-scrape

Scrape Instagram

14   29   29  

htmlspitter

NodeJS server to spit out HTML from loaded JS using Puppeteer

7   29   29  

Scraper-Projects

🕸 List of mini projects that involve web scraping 🕸

33   29   29  

tripadvisor-scraper

Scrape the hotel reviews of a whole city on TripAdvisor

13   29   29  

asyncio-hn

Python (asyncio) wrapper for hackernews api

2   28   28  

Iranian-politicians-twitter-dataset-persian

Iranian politicians twitter dataset persian | دیتاست کامل توییت های سی...

6   28   28  

m2m

Transform a Medium post into a Markdown file

3   28   28  

shup

A POSIX shell script to parse HTML

4   28   28  

serritor

Serritor is an open source web crawler framework built upon Selenium a...

15   28   28  

TorScrapper

A Scraper made 100% in Python using BeautifulSoup and Tor. It can be u...

10   28   28  

SneakerBot

Buy limited edition sneakers

8   28   28  

instagram-without-api-node

A simple Node.js code to get unlimited instagram public pictures by ev...

19   28   28  

KC-Scraper

A powerful open-source proxy scraper

9   28   28  

scrap

Scrapping Facebook with JavaScript.

8   27   27  

ferret-server

Advanced declarative web scraping

6   27   27  

google-search-results-nodejs

Scrape and parse Google search results in Node.JS

3   27   27  

geetest-captcha-solver

Solve the Geetest slider captcha with Puppeteer

7   27   27  

BSCWhalesMonitor

BSC whales monitor and detector written in Python.

16   27   27  

goGetJS

a tool for extracting, searching, and saving JavaScript files (with op...

5   27   27  

AyugeSpiderTools

scrapy 扩展库：其主要功能使 scrapy 开发不用在意 item，pipeline，middle...

3   27   27  

CodeChef-Gitter

:books: Save all and update solutions submitted on Codechef to github.

5   26   26  

gitfollow

Github follower and following

6   26   26  

shorter.recipes

A website dedicated to making recipes from any website easy to read.

3   26   26  

stock-news-analysis

📰 Web app built with Flask that given a stock ticker will scrape dozen...

8   26   26  

policy-data-analyzer

Building a model to recognize incentives for landscape restoration in...

10   26   26  

scrape_discord

Scrape discord channels

7   26   26  

Crawling-Emails

Very simple bash script to crawl email addresses from a specific websi...

14   26   26  

facebook-snooper

Scrapes Facebook profile pages using a normal login (without an API ke...

7   26   26  

cambridge

Terminal version of Cambridge Dictionary by default. Also supports Mer...

4   26   26  

garlic

🧄🧛 protect your website from being scraped by bots.

0   26   26  

MenuGen

An intelligent generator of well-balanced meals.

6   25   25  

trypophobia

Trypophobia images detector based on deep neural networks and utilitie...

2   25   25  

scrapy_facebooker

Collection of scrapy spiders which can scrape posts, images, and so on...

6   25   25  

pickall

.NET agile and extensible web searching API

3   25   25  

anime-scraper

[partially working] Scrape and add anime episode stream URLs to uGet (...

5   25   25  

Scrapping

Mastering the art of scrapping 🎓

40   25   25  

proxy-db

Manage free and private proxies on local db for Python Projects

7   25   25  

browser-automation-api

Browser automation API for repetitive web-based tasks, with a friendly...

3   25   25