Most popular scraping repositories and open source projects

scraping lenoarditrryh

The Ultimate Guide to Web Scraping APIs in 2025

18 3 2

tap LeonTing1010 JavaScript

Capture a logged-in browser task once — replay it forever at zero LLM tokens. Local-first browser-automation MCP for Claude Code, Cursor & any MCP hos...

18 2 1

magpie Magpie-Tools Shell

Multi-user AIO Proxy Manager

18 5 2

go-scrapy kabelsea Go

Web crawling and scraping framework for Golang

17 2 1

WebHere rdlopes Objective-C

HTML scraping for Objective-C.

17 6 5

hass-multiscrape danieldotnl Python

Home Assistant custom component for scraping multiple values (from a single HTTP request) with a separate sensor for each value.

17 4 1

monitor_web_page_changes nyavramov Python

I thought it would be nice to get an email alert when a new job posting appears on a website, so I can use this to check for the newest job openings o...

17 3 2

scrapyd-mongodb Tiago-Lira Python

Library designed to replace the SQLite backend by a MongoDB backend on Scrapy queue management

17 8 1

skrapr BaristaLabs C#

DotNet Core based declarative web scraping using the Chrome Debugger Protocol

17 3 0

precios-claros martjanz Python

Precios Claros (http://preciosclaros.gob.ar) scraper / data downloader

17 5 3

deephotel gkzz Python

scraping TripAdvisor, Booking.com with Scrapy

17 10 0

Scraping-Jumia-Ecommerce krizten Python

Using the Scrapy framework to scrape data consisting of name, brand, rating, price, product URL and image URLs of laptops on Jumia e-commerce (https:/...

17 14 0

xiaomi-mi-fit-data-export zoilomora

Export data from Xiaomi Mi Fit in the most automated way possible

17 1 1

scraper nirantak Python

Python web scrapers

17 8 2

telegram_members_scrapper gurjeet29 Python

Python Script to scrape members from a selected Telegram group.

17 10 2

Zeiver ZimCodes Rust

A Scraper, Downloader, & Recorder for static open directories.

17 1 1

Scrape-GettyImages-Using-Selenium KryeKuzhinieri Python

Using python and selenium download images from gettyimages.com

17 5 1

Investopedia-Bot bassel27 Python

Pick the best stocks and automate Investopedia

17 14 1

Earnings_Call_Analyzed_By_NLP nilijing Jupyter Notebook

Earnings Call Sentiment Analysis. This repository includes my work on extracting the focus area of companies from their earnings calls transcripts.

17 3 1

people blasferna Python

Api de datos de contribuyentes del Paraguay

17 5 1

YiraBot-Crawler OwenOrcan Python

YiraBot: Simplifying Web Scraping for All. A user-friendly tool for developers and enthusiasts, offering command-line ease and Python integration. Ide...

17 0 4

dynamic-rendering-ultra-generation__webpack-vue anhchangvt1994 TypeScript

Advanced SEO for Webpack + VueJS Project

17 0 1

langchain-scrapegraph ScrapeGraphAI Python

ScrapeGraph client langchain integration

17 4 1

idealista_data_extraction laurabarredaagusti Python

Data extraction from Idealista, using their API and web scraping (python). It contains both the datasets and the extraction files.

17 9 1

NewApkPure Alnyz Python

Search and download applications from apkpure.com

17 2 1

scrapy-scraper ivan-sincek Python

Web crawler and scraper based on Scrapy and Playwright's headless browser.

17 7 1

web-scraping-lvl1 FirasKahlaoui Jupyter Notebook

The "web-scraping-lvl1" project is a beginner-level exercise in web scraping using Python's Beautiful Soup library and Requests module.

17 0 1

scrapetoon RoloEdits Rust

A tool for scraping information from Webtoons.

17 2 1

python-selenium-scraper-template therealthoren Python

Watch your python scraper with selenium in a docker container

17 8 1

WebRunner sp34rh34d Python

Web scraping | Website cloner | Path Traversal Scanner

17 4 1

scrapemed danielfrees Python

ScrapeMed: Data scraping for PubMed Central.

17 1 1

OnionScanner N4rr34n6 Python

An enhanced version of the OnionScanner tool for scanning and analyzing .onion V3 domains on the Dark Web. This version features SQLite database integ...

17 6 2

Real-Estate-AI fortunatoman TypeScript

AI-powered real estate GPT that provides property insights, market analysis, pricing guidance, and smart recommendations to help buyers, sellers, and...

17 0 0

open-source-legislation spartypkp Python

Open-source global legislation data in an SQL knowledge-graph format ideal for use with LLMs: Download legislation data in bulk and immediately start...

17 2 2

browser-agent lightfeed TypeScript

Replayable Browser Agent

17 0 1

litehook tensake Rust

Lightweight social media monitoring tool built with Rust

17 0 1

awesome-scraping ScrapeRouter Python

The definitive list of the latest libraries, tools, APIs and providers for web scraping. The only daily-updated collection of web scraping resources.

17 2 0

scrapeman scrape-do TypeScript

Postman-grade API client for every developer — local-first, git-friendly collections, built-in load testing, unlimited history

17 0 0

medium-scraper sarperavci Python

A high-performance, async Medium scraper that simplifies converting articles to Markdown. Offers flexible request handling and a user-friendly web int...

17 2 0

abx-plugins ArchiveBox Python

🧩 Plugins and extractors that ArchiveBox + abx-dl use: chrome, ytdlp, wget, singlefile, readability, forum-dl, gallery-dl, papers-dl, and more...

17 3 0

zendriver-rs TurtIeSocks Rust

Async-first, undetectable browser automation in Rust via the Chrome DevTools Protocol. Stealth-by-default port of zendriver — no WebDriver, no JS shim...

17 4 0