Most popular scraping repositories and open source projects

Scraper-Projects

🕸 List of mini projects that involve web scraping 🕸

31   29   29  

pickall

.NET agile and extensible web searching API

3   29   29  

skyscanner-flights-scraping

A scraping tool (for personal use, not commercial) to get some informa...

11   29   29  

Instagram-Network_scraping_and_analysis

Python script to scrape Instagram network

5   29   29  

pastebin-bisque

Download all of a given user's public Pastebin pastes

5   29   29  

ted-scraper

🎙️ TED Talks web scraper

8   29   29  

asyncio-hn

Python (asyncio) wrapper for hackernews api

2   28   28  

Iranian-politicians-twitter-dataset-persian

Iranian politicians twitter dataset persian | دیتاست کامل توییت های سی...

6   28   28  

stock-news-analysis

📰 Web app built with Flask that given a stock ticker will scrape doze...

8   28   28  

scrap

Scrapping Facebook with JavaScript.

8   28   28  

trypophobia

Trypophobia images detector based on deep neural networks and utilitie...

2   28   28  

scrape_discord

Scrape discord channels

8   28   28  

dmi-instascraper

A GUI for Instaloader to scrape users and hashtags with on Instagram

7   28   28  

scrapeadvisor

A user-friendly python-based GUI which provides sentiment analysis of...

7   28   28  

ProductHunt-scraper

Producthunt.com famous website scraper script. Scrap all offers and sa...

9   28   28  

bet365-api-scraper

This project is a scraper of the Bet365 API to collect data from live...

9   28   28  

bet365_web_scraping

Ainda tem como raspar dados do site bet365.com - tutorial educacional...

2   28   28  

substack_scraper

A scraper for Substack article text content

2   27   27  

Doublegram-Startup-Telegram-Scraper-Adder

The Professional Telegram Bulk Members Adder & Scraper. More on double...

5   27   27  

marktplaats-py

Small Python package to request listings & users from marktplaats.nl

9   27   27  

proxy-scraper

This is an application that scrapes various Proxy API Endpoints, then...

5   27   27  

python-web-scrapping

Detailed web scraping tutorials for dummies with financial data crawle...

2   27   27  

AyugeSpiderTools

scrapy 扩展库：其主要功能使 scrapy 开发不用在意 item，pipeline，middle...

3   27   27  

gitfollow

Github follower and following

6   27   27  

MenuGen

An intelligent generator of well-balanced meals.

6   27   27  

ferenda

Transform unstructured document collections to structured Linked Data

11   27   27  

MotoGP-API

MotoGP Api: Library that reads the results of the MotoGP, 500cc, Moto2...

1   27   27  

Google-Web-Scraper

This Python code scrapes Google search results then applies sentiment...

6   27   27  

htmltab

Command-line utility to convert HTML tables into CSV files

4   27   27  

shorter.recipes

A website dedicated to making recipes from any website easy to read.

3   26   26  

scrapy_facebooker

Collection of scrapy spiders which can scrape posts, images, and so on...

6   26   26  

web_scraping_python

Techniques for Scraping the Web in Python

22   26   26  

froxy

Hide your IP with free proxies using Froxy 🔄

2   26   26  

html2rss-configs

📇 A growing collection of html2rss feed configs. Generate configs wit...

7   26   26  

RSI-Scraper

Web Scaper for RSI

8   26   26  

emailscraper

Minimalistic library to scrape emails from websites with headless brow...

6   26   26  

mangareader-api

A Python based web scraping api built with fastapi that provides easy...

8   26   26  

cambridge

Terminal version of Cambridge Dictionary by default. Also supports Mer...

4   26   26  

board-game-scraper

Board game data scraper

5   26   26  

duckduckgo

A simple DuckDuckGo URL scraper.

3   25   25  

Telegram-search

Simple web scrapping to search from telegram

6   25   25  

automation-samples

Using clicknium to automate platforms like Linkedin, twitter, Slack, Y...

8   25   25  

CodeChef-Gitter

:books: Save all and update solutions submitted on Codechef to github.

5   25   25  

Babler

Data Collection System For NLP/Speech Recognition

12   25   25  

amazon_tracker

A simple amazon tracker that sends you an email when prices of your fo...

6   25   25  

super-scraper

Generic REST API for scraping websites. Drop-in replacement for Scrapi...

7   25   25  

dom-content-extraction

DOM Based Content Extraction via Text Density

2   25   25  

dynamic-rendering-ultra-generation__vite-react

Advanced SEO for Vite + ReactJS Project

8   25   25  

Philia

An easy to use imageboard scraper.

1   25   25  

GitStats

Generate regularly updated visualizations of personalized GitHub stati...

10   25   25