scraping
Repositories (1626)
Tools for various online judges. Downloading sample cases, generating additional test cases, testing your code, and submitting it.
Lightweight library for scraping web-sites with LLMs
Crawly, a high-level web crawling & scraping framework for Elixir.
🧹 Python package for text cleaning
Scrape the Instagram frontend. Inspired from twitter-scraper by @kennethreitz.
Watch everything from your terminal.
OSINT cheat sheet, list OSINT tools, wiki, dataset, article, book , red team OSINT for hackers and OSINT tips and OSINT branch. This repository will g...
Collection of patches for puppeteer and playwright to avoid automation detection and leaks. Helps to avoid Cloudflare and DataDome CAPTCHA pages. Easy...
🚀 Web scraping for humans
Your browser anime experience from the terminal
Linkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy
📄 Python tool to turn Notion.so pages into lightweight, customizable static websites
AgentQL is a suite of tools for connecting your AI to the web. Featuring a query language and Playwright integrations for interacting with elements an...
:rocket: An open source alternative to searx which provides a modern-looking :sparkles:, lightning-fast :zap:, privacy respecting :disguised_face:, se...
DataHen Till is a companion tool to your existing web scraper that instantly makes it scalable, maintainable, and more unblockable, with minimal code...
[Unmaintained] A simple and clean video/music/image downloader 👾
Generate Free Edu Mail(s) within minutes
Simple but useful Python web scraping tutorial code.
Kuwala is the no-code data platform for BI analysts and engineers enabling you to build powerful analytics workflows. We are set out to bring state-of...
modular service framework to move and transform network packets
:scissors: High performance, multi-threaded image scraper
A completely revamped and redesigned fork, reimagined from scratch based on the original onlyfans-scraper
🥫 The simple, fast, and modern web scraping library
A CLI toolset to generate table of contents for PDF files automatically.
🎭 Intelligent browser header & fingerprint generator
The web scraper that's nearly impossible to block - now called @ulixee/hero
📰 Newspaper4k a fork of the beloved Newspaper3k. Extraction of articles, titles, and metadata from news websites.
Lookyloo is a web interface that allows users to capture a website page and then display a tree of domains that call each other.
Extract structured data from web sites. Web sites scraping.
Automatically archive links to videos, images, and social media content from Google Sheets (and more).
python script for Google Dorking
🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON.
😈📚 A curated library of research papers and presentations for counter-detection and web privacy enthusiasts.
Google Search Results via SERP API pip Python Package
Open Source Bulk Auto Gmail Creator Bot with Selenium & Seleniumwire ( Python ). Feel free to contact me with Django/Flask, ML, AI, GPT, Automation, S...
:card_index: Extract social media profiles and more with regular expressions
Social media scraping / data collection library for Facebook, Twitter, Reddit, YouTube, Pinterest, and Tumblr APIs
The simplest demo of chrome automation by python and selenium in AWS Lambda
Comic-dl is a command line tool to download manga and comics from various comic and manga sites. Supported sites : readcomiconline.to, mangafox.me, co...
Free Web Scraping Tool with Java
Scalable Python web scraping scripts for +40 popular domains
A universal web-util for PHP.
Scrapy Extension for monitoring spiders execution.
Dive into web scraping and build a Next.js 13 eCommerce price tracker within a single video that teaches you data scraping, cron jobs, sending emails,...
Analyze facebook copy of your data with ruby language. Download zip file from facebook and get info about friends ranking by message, vocabulary, con...
Jekyll-based static site for The Programming Historian
List of anti-detect and humanizing tools and browsers, including captcha solvers and sms-activation.
Python quick start guides to get the most out of Oxylabs' Web Scraper API free trial.
A framework for creating semi-automatic web content extractors
Web scraping library made by the Phantombuster team. Modern, simple & works on all websites. (Deprecated)