scraping
Repositories (1837)
Project built part of the "Web Scraping in JavaScript – How to Use Puppeteer to Scrape Web Pages" article.
An advanced Python tool for extracting data from websites, cleaning the content, and converting it to high-quality Markdown for optimal use by LLM sys...
A simple python CardMarket price scraper.
An open source web crawling platform
Chopper is a tool to extract elements from HTML by preserving ancestors and CSS rules
🕷Configuration based html scraper
Scrape and parse HTML tables with the Puppeteer table parser.
Allows to bypass Cloudflare checks
A web automation library that lets AI agents browse the web using declarative JSON, with a unified dual-engine API and powerful anti-bot evasion.
Querying, scraping, and parsing of HTML. Good for snapshot testing too!
Downloads all refcardz from https://dzone.com/refcardz
ProxyCrawl Node library for scraping and crawling
Scraping Code4rena contest audits reports for stats, fun (and profit ?)
:ram: Simple PHP Email Grabber to get emails from a txt file containing the list of urls (add one url per line).
[DEPRECATED] Pillaging the seven seas for torrents, pieces of eight and other bounty.
Dataset and visualizations of Nintendo Games and ratings, scraped from metacritic.com
MCP server for Olostep — the web scraping, crawling, and search infrastructure used by top AI companies. Gives any MCP-compatible AI agent the ability...
Code examples and general information
IG Index Scalping/Scraping Bot, Written in Python3/Cross Platform/REST API
Quick and dirty date parsing Python library to parse HTML dates really fast
Price Tracker app for Android and iOS. Built with Flutter
Command line program to download documents from web portals
A powerful, stealthy website cloner/scraper built with TypeScript that downloads entire websites for offline use. Supports HTTP proxy authentication,...
Download images from reddit
eCommerce Scraping API code examples for Python, PHP and Node.js
Automatically fetch and update proxy lists from multiple sources every 6 hours using GitHub Actions
A Telegram bot combined with python to serve some basic functions like weather, music charts, cricket score and much more.
[🚧 WIP] Cross-platform microservice to scrape the IMDb website.
SOAP - A Sockpuppet Auditing Tool for Very Large Online Platforms
This Python script is designed to scrape articles from The Guardian's technology section using their API. It fetches article data, extracts the titles...
A 100% working Justdial scrapper, Just enter the url and it'll extract business info from it
Toolkit to assist in stock checking and checkout automation of various retail sites.
Scrape structured data from HTML documents automatically
Tiny little ruby on rails website that crawls though your public github repos to find out what your favourite languages are.
Python library to scrape social media data via the EnsembleData API.
A Python framework to build polite, but tenacious crawlers / scrapers with a MariaDB backend
Retrieve real (with Javascript executed) HTML code from an URL, ultra fast and supports multiple parallel loading of webs
Easily crawl news portals or blog sites using Storm Crawler.
🤖 A Discord bot that allows you to access solutions to homework problems from Chegg.
node.js library for scraping GitHub trending repositories.
JustDial Scraper to scrap all the requested data which includes their name, address, email address and phone number.
Lightweight light novel, pdf, manwha, manga and epub reader with offline AI summaries.
A small CLI app to scrap high-quality movie snapshots from various websites.
API for scrapping common 🇵🇱 meme sites
A python library for scraping videos from JW Player
ProxyCrawl PHP library for scraping and crawling websites
SERP Scraping API code examples for Python, PHP and Node.js
A tutorial for scraping Instagram profile information and posts using Scraping Fish API: https://scrapingfish.com
Automatically post Github trends on Facebook page
Scraping of 5 types of fuel :fuelpump: from 8 different fuelcompanies in Denmark :denmark:.