Most popular scraping repositories and open source projects

Scrapping

Mastering the art of scrapping 🎓

40   25   25  

proxy-db

Manage free and private proxies on local db for Python Projects

7   25   25  

browser-automation-api

Browser automation API for repetitive web-based tasks, with a friendly...

3   25   25  

imslp

🎼 The clean and modern way of accessing IMSLP data and scores program...

5   25   25  

flutter_notification_listener

Flutter plugin to listen for and interact with all incoming notificat...

13   25   25  

youtube-comment-scraper

Scraping comments from Youtube.

3   24   24  

dmi-instascraper

A GUI for Instaloader to scrape users and hashtags with on Instagram

6   24   24  

scrapingant-client-python

ScrapingAnt API client for Python.

3   24   24  

Python-Web-Scraping-Tutorial

In this Python Web Scraping Tutorial, we will outline everything neede...

6   24   24  

getter

Get and put users (scraping) to the target group/channel efficiently,...

11   24   24  

github-languages

Tiny little ruby on rails website that crawls though your public githu...

0   23   23  

web_scraping_python

Techniques for Scraping the Web in Python

22   23   23  

fb-github-trends

Automatically post Github trends on Facebook page

3   23   23  

skyscanner-flights-scraping

A scraping tool (for personal use, not commercial) to get some informa...

9   23   23  

PastebinScrapy

Threat hunting tool for scraping latest scrapes from Pastebin

12   23   23  

scrapeadvisor

A user-friendly python-based GUI which provides sentiment analysis of...

6   23   23  

Champ

A Telegram bot combined with python to serve some basic functions like...

22   22   22  

nest

High-level, robust framework for web scraping in Node.js

3   22   22  

Babler

Data Collection System For NLP/Speech Recognition

12   22   22  

crawling-framework

Easily crawl news portals or blog sites using Storm Crawler.

3   22   22  

exoskeleton

A Python framework to build polite, but tenacious crawlers / scrapers...

1   22   22  

chopper

Chopper is a tool to extract elements from HTML by preserving ancestor...

2   22   22  

osint-series

Source codes related to the articles about OSINT. Using social media A...

6   22   22  

hodor

🕷Configuration based html scraper

3   22   22  

ferenda

Transform unstructured document collections to structured Linked Data

7   22   22  

Goirate

[DEPRECATED] Pillaging the seven seas for torrents, pieces of eight an...

0   22   22  

amazon_tracker

A simple amazon tracker that sends you an email when prices of your fo...

5   22   22  

LInkedIn-Reverese-Lookup

🔎Search LinkedIn profile by email address📧

6   22   22  

Realestate

Contains a set of Python scripts for real estate data analysis, visual...

12   22   22  

CraigslistScraper

Craigslist Scraper is a web scraper for craigslist, it allows users to...

8   22   22  

youtube-comment-scraper

This script will dump youtube video comments to a CSV from youtube vid...

11   22   22  

imdb-api

[🚧 WIP] Cross-platform microservice to scrape the IMDb website.

1   22   22  

react-node-web-scraper

Final Year project, scraping data of e-commerce stores and display in...

5   21   21  

Pinterest-infinite-crawler

An infinite Pinterest crawler/scraper. Crawl image with inifnite-scrol...

3   21   21  

coches-net-dashboard

Sample project that use Dagster, dbt, DuckDB and Dash to visualize car...

1   21   21  

zcrawl

An open source web crawling platform

4   21   21  

scrapman

Retrieve real (with Javascript executed) HTML code from an URL, ultra...

2   21   21  

raspagem-de-dados-fatec

:notebook: Minicurso de raspagem de dados web com Python ministrado n...

20   21   21  

NBA-Fantasy-Optimizer

NBA Daily Fantasy Lineup Optimizer for FanDuel Using Python

12   21   21  

darwin

Projet pour générer la page http://clementgrimal.fr/darwin/

7   21   21  

PrawWallpaperDownloader

Download images from reddit

3   21   21  

househunterbot

Use Python, Google Spreadsheet, Google Shortener and CALLR API to auto...

7   20   20  

node-github-trend

node.js library for scraping GitHub trending repositories.

3   20   20  

Justdail-scrapper

A 100% working Justdial scrapper, Just enter the url and it'll extract...

15   20   20  

Tweets-Scrapper

Get all the tweets from your favorite twitter accounts.

6   20   20  

proxycrawl-node

ProxyCrawl Node library for scraping and crawling

5   20   20  

fbspider

Scraping Facebook information

7   20   20  

lookout_hunter

Find availability on Recreation.gov to camp in decommissioned fire-loo...

0   20   20  

email-scraper

Simple Python library to scrape email addresses from HTML

11   20   20  

price_tracker

Price Tracker app for Android and iOS. Built with Flutter

2   20   20