Most popular scraping repositories and open source projects

api

👥 A better directory of Yale personnel, with a clean API to match. Use...

20   10   10  

scrapy-proxycrawl-middleware

Scrapy middleware interface to scrape using ProxyCrawl proxy service

3   10   10  

Advanced_PHP_Scrapping

Enhanment Scrapping API for six hotel booking website from Expedia.com...

8   10   10  

BetWaves

Simple betfair python scripts

2   10   10  

fb-scraping-tools

A unit tested set of tools written in Python to scrape / collect data...

2   10   10  

pukpuk

HTTP discovery and change monitoring tool

3   10   10  

web-ctf-help

Collection of scripts to help with web based ctfs.

3   10   10  

BestCarDeal

:moneybag: Scraping, Visualizing, and Analyzing 1,700,000 Entries of U...

4   10   10  

manifold

Manifold is a plug-and-play end-to-end real estate asset tracker, from...

0   10   10  

web-scraper

This is a python based website crawling script equipped with Random ti...

5   10   10  

Faceapp-Gender-Swap-Detection

Detecting fake photos generated by FACEAPP gender swap feature using D...

1   10   10  

FirstCyclingAPI

An unofficial Python API wrapper for firstcycling.com

0   10   10  

tarantula

Python web crawler tool

3   10   10  

quora-plus-bypass

A python script to extract answers to any question on Quora (Quora+ in...

2   10   10  

isoxya-api

Isoxya Crawler API

0   10   10  

python-scrapfly

Scrapfly Python SDK for headless browsers and proxy rotation

2   10   10  

package-bot

A Discord Parcel Tracking Bot

4   10   10  

apracticalguidetoevil

An tool for creating Pandoc compatible markdown files of https://pract...

1   10   10  

web-scraper

simple web scraping: extract texts and links as CSV, and save images o...

2   9   9  

Esolat-Malaysia

Packagist for E-Waktu Solat in Malaysia. Data from Jakim

6   9   9  

cotoami-scraper

Cotoami Scraper is a Chrome Extension that scrapes web pages to genera...

2   9   9  

nextjs_scraping_marketplace

Scraping product from Tokopedia & Bukalapak

2   9   9  

FreshProxies

Fresh Proxies | Proxied Browser | HTTP/HTTPS/SOCKS4/SOCKS5

5   9   9  

bookmeter-recommender

python3 bookmeter scraping tools

0   9   9  

SiteRank-Alexa

This is vanilla Python library for gathering data about websites and d...

2   9   9  

scrapy-scraperapi-proxy

Scrapy middleware interface to a scraper API proxy service

4   9   9  

nhkore

:jp::newspaper::mount_fuji: NHK News Web (Easy) word frequency (core l...

1   9   9  

simple-email-scraper

Simple Email Scraper

3   9   9  

partisan

An analysis of the partisan behavior for the U.S. House of Representat...

1   9   9  

node-scraper

Scraping websites made easy! A minimalistic yet powerful tool for coll...

0   9   9  

Tap-News

Real Time News Scraping and Recommendation System

4   9   9  

DatingAI

Learning & researching personal taste in online dating, developing a p...

2   9   9  

ImageScraper

Dumps all images from a WebSite.

2   9   9  

ticker-iq

📈 Stock screener and portfolio analyzer, providing key insights on fin...

5   9   9  

curlx

CurlX a basic Curl syntax

7   9   9  

archive-your-ig

Download all instagram image and video post

1   9   9  

paytm-scraping-offers

Scraping & crawling all of the products (and their coupons, categories...

4   9   9  

India-Trade-Data

A web scraper written in Python to gather trade data for India across...

6   9   9  

Scraping-Google-Play-Store

scraping applications data on google play store

3   9   9  

Syo

Download web novels from Syosetu

4   9   9  

vaksincovid19-api

API VAKSIN COVID-19 INDONESIA ( Scrapping Website Kemenkes )

3   9   9  

Django-Search-Engine

Django Search Engine, Scrape links and text from different search engi...

10   9   9  

alibaba_scraper

Alibaba scraper with using of rotating proxies and headless Chrome fro...

3   9   9  

tubby

:house_with_garden: A utility for the Genshin Impact Housing system

0   9   9  

italian_dictionary

Package that helps getting the meaning of italian words.

4   9   9  

bidbybid

💸 Python tool which scrapes sold eBay auctions for minimum, maximum an...

0   9   9  

News-Article-Scraper

Learn about web scraping news articles using Python and JavaScript.

2   9   9  

Stock_Data_Scraper

Fast and multi threaded stock data scraper written in Java using HTMLU...

1   9   9  

webscraping-py

Web Scraping scripts for all Google, other search engines, and other w...

2   9   9  

telegram_members_scrapper

Python Script to scrape members from a selected Telegram group.

10   9   9