scraping

Web scraper with a simple REST API living in Docker and using a Headless browser and Readability.js for parsing.

This script allows you to automate the creation of Gmail accounts using the Selenium automation framework with the Chrome WebDriver. It navigates thro...

Tools to build web AI agents that can authenticate, interact with and extract data from any website.

A high-performance proxy rotation engine with automated IP management and real-time health monitoring

Facebook Group Members Extractor. Download Facebook group members in CSV.

In this Python Web Scraping Tutorial, we will outline everything needed to get started with web scraping. We will begin with simple examples and move...

一个灵活、友好的爬虫框架

🧰 A collection of automation tools for Instagram 📱| Written in Python 🐍 | Don't forget to ⭐ the repo !

Scrapes all the data of followers of any instagram account

Turn Webpage to LLM friendly input text. Similar to Firecrawl and Jina Reader API. Makes RAG, AI web scraping, image & webpage links extraction easy.

Automate webpages at scale, scrape web data completely and accurately with high performance, distributed RPA.

Web data extraction tool implemented as chrome extension

Real-time detection of anti-bot systems, CAPTCHAs & fingerprinting techniques. Identifies Cloudflare, Akamai, DataDome, reCAPTCHA, hCaptcha, Shape Se...

SEO python scraper to extract data from major searchengine result pages. Extract data like url, title, snippet, richsnippet and the type from searchre...

OWASP D4N155 - Intelligent and dynamic wordlist using OSINT

scan for webcams on the internet

A playwright bot which is implemented to scrape linkedin and store advertisement data in a database and telegram channel

Antch, a fast, powerful and extensible web crawling & scraping framework for Go

Python library for automated email account creation. Create multiple accounts easily with support for major email providers.

Attack Surface Discovery tool built on a microservice approach, utilizing multi-threading for fast, internet-scale asset indexing

Crawl all unique internal links found on a given website, and extract SEO related information - supports javascript based sites

A modern Python library for writing maintainable web scrapers.

The Kemono and Coomer Downloader simplifies downloading posts from Kemono and Coomer websites, allowing users to download individual or multiple posts...

Face recognition system using MTCNN, FACENET, SVM and FAST API to track participants of Big Brother Brasil in real time.

MinerU-HTML: An SLM-powered HTML main content extractor that outputs clean HTML bodies. Perfect for Deep Research Agents, RAG applications, and traini...

A python utility for downloading Common Crawl data

Jsoup Annotations POJO

Node.js package to bypass CloudFlare's anti-bot JavaScript challenges

Anime-dl is a command-line program to download anime from CrunchyRoll and Funimation.

Image Dataset Tool (idt) is a cli tool designed to make the otherwise repetitive and slow task of creating image datasets into a fast and intuitive pr...

Universal scraping tool, which allows you to extract data using multiple environments

Free Palestine. 📖 This tool is to download course from educative.io for offline usage. It uses your login credentials and download the course.

Experience for effectively fetching Facebook data by Querying Graph API with Account-based Token and Operating undetectable scraping Bots to extract C...

Automated web scraping spider generation using Browser Use and LLMs. Streamline the creation of Playwright-based spiders with minimal manual coding. I...

Simple scripts for Level UP your scraping Skills, and source code for Level UP playlist on Youtube

Transistor, a Python web scraping framework for intelligent use cases.

Grawler is a tool written in PHP which comes with a web interface that automates the task of using google dorks, scrapes the results, and stores them...

Perform Google Dork search with Dorkify

📚 Python tool to download book summaries and audio from Blinkist.com, and generate some pretty output

Linkedin Learning videos downloader

simple multi-level scraper json input/output for Cheerio

An Unofficial REST API for vlr.gg, a site for Valorant Pro Esports match results and news.

The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de...

▞ Command line tool to scrape crosswords from online solvers and save them as .puz files ▚

estela, an elastic web scraping cluster 🕸

📜 Framework-agnostic API scraper to load items from any paginated JSON API into a Laravel lazy collection via async HTTP requests.

Powerful Telegram bot for web scraping and crawling. Fast, easy, and loved by thousands!

自学入门 Python 优质中文资源索引，包含 书籍 / 文档 / 视频，适用于 爬虫 / Web / 数据分析 / 机器学习 方向

Examples for using Hyperbrowser

Spotify scraper