scraping

A web crawling framework implemented in Golang, it is simple to write and delivers powerful performance. It comes with a wide range of practical middl...

AI-native web scraper. Single binary with a bundled Claude Code skill. MIT-licensed alternative to Firecrawl.

Scrape data from Goodreads using Scrapy and Selenium :books:

The Scripting Engine that Combines Speed, Safety, and Simplicity

An all-in-one scraper/downloader for Fansly written in go with the aid of A.I. Download content, Record lives, and Interact with post from your favori...

Free proxy list updated every 15 minutes | HTTP, HTTPS, SOCKS4, SOCKS5, elite & anonymous proxies. Powered by VPSLab.

Sasori is a dynamic web crawler powered by Puppeteer, designed for lightning-fast endpoint discovery.

Site que agrega filmes em cartaz em algumas das diversas salas de cinema de Porto Alegre.

Spiderbuf 是一个专注于 Python 爬虫练习的网站。提供丰富的爬虫教程、爬虫案例解析和爬虫练习题。Python爬虫开发强化练习，在矛与盾的攻防中不断提高技术水平，...

Simple Query Scraping with CSS and Go Reflection (MOVED to Gitlab)

arxiv_miner is a toolkit for mining research papers on CS ArXiv.

⬇️ A simple all-in-one CLI tool to download EVERYTHING from a URL (like youtube-dl/yt-dlp, forum-dl, gallery-dl, simpler ArchiveBox). 🎭 Uses headless...

Distributed crawler, database and web frontend for public directories indexing

📰 Build RSS 2.0 feeds from websites (and JSON APIs) automatically or with a few CSS selectors.

Nim library for querying HTML using CSS-selectors (like JavaScripts document.querySelector)

A test suite of common scraper detection techniques. See how detectable your scraper stack is.

Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.

Use AWS Lambda functions as a proxy pool to scrape web pages.

Learn how to send POST requests with cURL.

An Telegram Mass Members Adding/Scraping Tool Written In Python Using Pyrogram Library.

Automate your LinkedIn job applications with AI! This bot utilizes GPT models such as GPT-4, GPT-3.5, and Google's Gemini Pro for Easy Apply form fill...

Scraping the justETF

Web Data Scraper - no-code internet scraping. Extract and export to CSV, Excel, JSON, Google Sheets, and Webhook.

Use the MapReduce's Java interface to distributed crawle the data of Chinese universities and learn basic knowledge of hdfs.

Python framework to scrape Pastebin pastes and analyze them

Scraping assistant tool. Editing and maintaining CSS/XPath selectors across webpages.

Machine learning for beginner(Data Science enthusiast)

Library with a set of tools for scraping information about Nintendo games and its prices across all regions (NA, EU and JP).

An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site

Monitor instagram user account and automatically post new images to discord channel via a webhook. Working 2022!

The code used to create and update the Open Australian Legal Corpus, the first and only multijurisdictional open corpus of Australian legislative and...

🌱 goClone - clone websites in seconds

Simple library for exploring/scraping the web or testing a website you’re developing

Detect bots, vision AI agents, and headless browsers through 40+ behavioral signals and SHA-256 proof of work. Self-hosted, privacy-first, and fully o...

htmlSQL is a experimental PHP library which allows you to access HTML values by an SQL like syntax.

ASP.NET View State Decoder

A web crawling programming language

This tutorial shows how to automate your web scraping processes using AutoScaper – one of Python web scraping libraries available.

An advanced tool for checking GitHub repositories, with star statistics, including fake star analysis and data visualization.

Cloudflare Turnstile solver & bypass — Python, real Chrome browser, no paid APIs. Local HTTP API service included. Auto-solves invisible and managed (...

Web scraping using rust !

Search geolocations for (social) media posts in databases like Bellingcat, Cen4InfoRes etc.

ScapeGraph MCP Server

qcrawl - fast async web crawling & scraping framework for Python.

Nodejs web scraper. Contains a command line, docker container, terraform module and ansible roles for distributed cloud scraping. Supported databases:...

API ketersediaan rumah sakit dan tempat tidur rumah sakit untuk pasien covid-19 ataupun non-covid yang berada di Indonesia

Scrapy + Puppeteer

A curated list of AI-powered web scraping tools, LLM-friendly crawlers, MCP servers, and infrastructure for turning the web into data.

Telegram CC Scrapper - Debit/Credit Card [channel public or private / group ]