scraping
Repositories (1837)
Official JavaScript/TypeScript library for interacting with Kameleo Client
A headless pure-python browser for the web
Unofficial Python client for Twitter
The intelligence layer for any web scraper. Pair with Scrapling, Playwright, or httpx to crawl smarter.
xDSL Prometheus Exporter
CLI recon tool for scraper developers. Detects TLS fingerprinting, JS challenges, bot protection, and rate limits across 4 stages
Extract structured data from any unstructured web page
Learn about how to rotate proxies by using Python.
A python package to generate random request fields for a http header.
Tutorial for web scraping / crawling with Node.js.
With Selenium headless browsing and CAPTCHA solving
This script automates the installation of 50 OSINT tools for reconnaissance and information gathering.
Generic REST API for scraping websites. Drop-in replacement for ScrapingBee, ScrapingAnt, and ScraperAPI services. And it is open-source!
A POSIX shell script to parse HTML
ScrapingAnt API client for Python.
Zyte API integration for Scrapy
Tool for conveniently downloading audios from r/gonewildaudio and similar subreddits
PHP Laravel Library for Scrapingbee Web Scraping API. AI querying supported. Also support Google, Walmart, Amazon, YouTube scraping
An admin UI project for a configurable web crawler platform
MITM HTTP(S) proxy with integrated load-balancing, rate-limiting and error handling. Built for automated web scraping.
A python package with client to scrape the israeli supermarkets data
A Scraper made 100% in Python using BeautifulSoup and Tor. It can be used to scrape both normal and onion links. Happy Scraping :)
Chew is a Go library for processing various content types into markdown/plaintext.
Open-source AI browser agent for web reverse engineering. Turn websites into browser-free API clients and crawlers with CDP, network tracing, and Java...
This repository is deprecated
a tool for extracting, searching, and saving JavaScript files (with optional headless browser)
UFC Fights Dataset Collection and Analysis
Self-hosted, open-source platform for running Apify Actors. Drop-in compatible with the Apify SDK.
Fast TikTok NO Watermark Video Downloader (username or url)
Unsupervised clustering of movie posters with features extracted from Convolutional Neural Network
Hyper Solutions SDK for Playwright - Bypass Akamai Bot Manager, Incapsula, Datadome and Kasada.
Web scraping API for building AI applications.
Introduction to web scraping
Production-ready web scraping in a single function call. Built on Crawlee.
The most advanced raiplay.it downloader
Generate JSON representations of HTML tables
A high effective golang library for parsing big-sized sitemaps and avoiding high memory usage. The sitemap parser was written on golang without extern...
Unofficial NHentai mobile app with flutter and bloc
TikTok LIVE API Client - The #1 Worldwide freemium SaaS API for TikTok LIVE data retrieval, offered in all major languages!
Cleaning tool for web scraped text
🔎 um bot de Web Scraping para mostrar vagas do LinkedIn
A drop-in replacement for puppeteer patched with rebrowser-patches. It allows to pass modern automation detection tests.
👒 One Piece TCG data scraper written in Rust
Collection of some simple python scripts to create https://myanimelist.net/ anime and user data set.
This repository provides minimal working examples for bypassing Cloudflare 1020 errors using Playwright in both Python and Node.js. The focus is on sh...
This is everand book downloader
📊 Python tool to scrape real-time information about ETFs from the web and mixing them together by proportionally distributing their assets allocation
CobWeb is a Python library for web scraping. The library consists of two classes: Spider and Scraper.
Zoominfo scraper with using of rotating proxies and headless Chrome from ScrapingAnt
Enhanced LinkedIn Job Search Chrome Extension