Most popular scraping repositories and open source projects

pro-penetration

Penetration research against NSFW websites

2   10   10  

api

šŸ‘„ A better directory of Yale personnel, with a clean API to match. Us...

20   10   10  

scrapy-proxycrawl-middleware

Scrapy middleware interface to scrape using ProxyCrawl proxy service

3   10   10  

Advanced_PHP_Scrapping

Enhanment Scrapping API for six hotel booking website from Expedia.com...

8   10   10  

BetWaves

Simple betfair python scripts

2   10   10  

fb-scraping-tools

A unit tested set of tools written in Python to scrape / collect data...

2   10   10  

pukpuk

HTTP discovery and change monitoring tool

3   10   10  

web-ctf-help

Collection of scripts to help with web based ctfs.

3   10   10  

manifold

Manifold is a plug-and-play end-to-end real estate asset tracker, from...

0   10   10  

web-scraper

This is a python based website crawling script equipped with Random ti...

5   10   10  

Faceapp-Gender-Swap-Detection

Detecting fake photos generated by FACEAPP gender swap feature using D...

1   10   10  

FirstCyclingAPI

An unofficial Python API wrapper for firstcycling.com

0   10   10  

tarantula

Python web crawler tool

3   10   10  

quora-plus-bypass

A python script to extract answers to any question on Quora (Quora+ in...

2   10   10  

isoxya-api

Isoxya Crawler API

0   10   10  

python-scrapfly

Scrapfly Python SDK for headless browsers and proxy rotation

2   10   10  

package-bot

A Discord Parcel Tracking Bot

4   10   10  

apracticalguidetoevil

An tool for creating Pandoc compatible markdown files of https://pract...

1   10   10  

News-Article-Scraper

Learn about web scraping news articles using Python and JavaScript.

2   9   9  

Stock_Data_Scraper

Fast and multi threaded stock data scraper written in Java using HTMLU...

1   9   9  

webscraping-py

Web Scraping scripts for all Google, other search engines, and other w...

2   9   9  

telegram_members_scrapper

Python Script to scrape members from a selected Telegram group.

10   9   9  

Impressionist

Impressionist is a JavaScript library that allows you to scrape data i...

1   9   9  

Go-MultiTor

Multiple tor circuit with golang

1   9   9  

CMScrape

A simple python CardMarket price scraper.

2   9   9  

python-automation-and-scraping

Sending Emails with Python

3   9   9  

fake-http-header

A python package to generate random request fields for a http header.

0   9   9  

Disboard-searcher

Script for get some discord server on disboard with keyword

1   9   9  

Redbubble-automatisation-bot

Upload automatisation for RedBubble

2   9   9  

Web_Scraper_Tutor

This repo basically contains details on how to work with WEB SCRAPER a...

0   9   9  

instagram-scraping-fish

A tutorial for scraping Instagram profile information and posts using...

1   9   9  

ScrapingUPF

Repositorio de la aplicaciĆ³n que permite transferir los horarios de la...

0   9   9  

scrapeaw

ScrapeAW is a framework that without API scrape IPs across the world u...

5   9   9  

scrape-facebook-by-playwright

This is web app by streamlit, this app scrape facebook page and show s...

3   9   9  

dm-feeds

An RSS feed for websites with broken or unavailable RSS feeds

0   9   9  

scrapeblocks

Scraping automation framework based on Playwright

0   9   9  

rug

Library for fetching various stock data from the internet (official an...

0   9   9  

Web-Scraping-API

Web Scraping API code examples for Python, PHP and Node.js

2   9   9  

cloudfire

A proxy server to bypass cloudflare I am under attack mode using playw...

2   9   9  

AniSync

Mapping sites to AniList and back.

2   9   9  

serpapi-javascript

Scrape and parse search engine results using SerpApi.

1   9   9  

nekopoi-scraper

API scraper for nekopoi.care

3   9   9  

web-scraper

simple web scraping: extract texts and links as CSV, and save images o...

2   9   9  

Esolat-Malaysia

Packagist for E-Waktu Solat in Malaysia. Data from Jakim

6   9   9  

cotoami-scraper

Cotoami Scraper is a Chrome Extension that scrapes web pages to genera...

2   9   9  

nextjs_scraping_marketplace

Scraping product from Tokopedia & Bukalapak

2   9   9  

FreshProxies

Fresh Proxies | Proxied Browser | HTTP/HTTPS/SOCKS4/SOCKS5

5   9   9  

bookmeter-recommender

python3 bookmeter scraping tools

0   9   9  

SiteRank-Alexa

This is vanilla Python library for gathering data about websites and d...

2   9   9  

scrapy-scraperapi-proxy

Scrapy middleware interface to a scraper API proxy service

4   9   9