Most popular scraping repositories and open source projects

scraping-das-mei engmsilva JavaScript

Raspagem do Documento de Arrecadação do Simples Nacional MEI

12 3 12

OSINT AiGptCode Python

⭐ Project Review: OSINT Data Collection Tool ⭐ This project provides a foundational tool for OSINT (Open-Source Intelligence) data collection, usin...

12 1 12

duckduckgo-scraper omkarcloud Python

Get real-time DuckDuckGo search results as structured JSON — organic results, knowledge panels, sponsored links, and related queries in one call. No h...

12 3 12

hypersolutions-examples Hyper-Solutions Go

Hyper Solutions SDK Examples — Working code examples in Go, Python, and Node.js for integrating the Hyper Solutions API to automate bot protection cha...

12 4 12

litehook KITFC-dev Rust

Self-hosted Telegram channel scraper server that monitors channels and sends webhooks for new posts. Docker, proxies, and web dashboard. Built with Ru...

12 1 12

covid19br-pub fgrehm Ruby

Projeto de monitoramento de publicações oficiais relacionadas a COVID-19 no Brasil.

11 0 11

crawlerUtils Tyrone-Zhao Python

Utils for programming web crawler

11 2 11

gathering_data dkhramov R

Примеры к книге "Сбор данных в Интернете на языке R".

11 3 11

Predicting-Trump mm909 Python

Predicting Trump's Tweets With A Character Level Recurrent Neural Network - Generating author and task specific text with a LSTM RNN.

11 2 11

hll-algorithm-sample gvolpe Scala

HLL Algorithm and Web Scraping sample

11 2 11

scraper-engine lukluk JavaScript

Async Scraper Framework Based Nodejs

11 7 11

Reddit-Scraper jamesbower Python

Reddit Scraper is a Python script that utilizies the PRAW Python library to interact with Reddit's API.

11 1 11

journal raphaelberly Python

A movie journal coupled with open IMDb data, and a Flask web-app for easy movie insertion.

11 0 11

BK wb-08 Python

Finds a fork in the tennis matches betting marathonbet and plusminus

11 3 11

vidpy preeteshjain Python

A Python based customizable script for scraping links to videos hosted on any website. Based on Scrapy and BeautifulSoup.

11 3 11

scrapy-proxycrawl-middleware crawlbase Python

Scrapy middleware interface to scrape using ProxyCrawl proxy service

11 3 11

Advanced_PHP_Scrapping FavyTeam PHP

Enhanment Scrapping API for six hotel booking website from Expedia.com, Booking.com, Bookhotelbeds.com. Hotels.com, Bestday.com, despegar.com

11 8 11

Nepali-news-portal-kbd hemanta212 Python

Online webapp that scrapes news from different new portals of Nepal and worldwide. Hosted at heroku.

11 3 11

pukpuk tasooshi Python

HTTP discovery and change monitoring tool

11 3 11

music-festivals BBC-Data-Unit Python

Festivals dominated by male acts, study shows, as Glastonbury begins

11 1 11

TED-Scraper The-Gupta Jupyter Notebook

Complete Web Scraping of TED.com for Metadata, Transcript, Audio, Video, Images using Parallel Programming

11 6 11

get-dream Tiger-0512 Python

予約困難な東京ディズニーリゾートの予約を自動化

11 4 11

OneFreeMonth CamTosh Python

Get free month on Netflix, OCS... using Selenium and Python

11 12 11

atmmi-info-api moebiusmania Go

Node.js webservice that turns the ATM Milano news website in a REST API JSON data.

11 2 11

Saffron UniStudents HTML

A fairly intuitive & powerful framework that enables you to collect & save articles and news from all over the web.

11 0 11

tarantula rly0nheart Python

Python web crawler tool

11 3 11

tabroom-API Debate-Land Python

tournaments.tech's API for scraping tabroom.com

11 3 11

webscraping-py dimitryzub Python

Web Scraping scripts for all Google, other search engines, and other websites (currently outdated, something may not be working).

11 2 11

GoogleNewsScraper GeminidSystems Python

A Python package that scrapes Google News article data while remaining undetected by Google. Our scraper can scrape page data up until the last page a...

11 5 11

cheinstein.py DouglasTaylorSupportGroup Python

🧠 A parsing library for Chegg.com written in Python.

11 3 11

Scrappp Jojiv2 Python

A Very simple free proxy list scraper.

11 3 11

sih2022-interface aaparikh Python

An interface for viewing different problem statements of smart india hackathon 2022 with basic summary built with ❤️ using streamlit

11 4 11

ScrapingUPF Sklyvan Python

Repositorio de la aplicación que permite transferir los horarios de la página web de upf.edu a Google Calendar.

11 0 11

scrapeaw Loubaris Python

ScrapeAW is a framework that without API scrape IPs across the world using Shodan

11 5 11

vidio_google_play_store_reviews darren7753 Jupyter Notebook

Analyzing Google Play Store reviews of the Indonesian streaming platform Vidio through topic modeling with the assistance of GPT.

11 1 11

CodeHarvest gui25 Ruby

CodeHarvest is a command-line tool for extracting and visualizing the structure and content of GitHub repositories. Easily explore and analyze files...

11 0 11

lambda-selenium teticio Python

Use AWS Lambda functions as a proxy pool to scrape web pages with Selenium.

11 0 11

etsy-sales-scraper rehan-nasir Python

A python program that scrapes an Etsy URL and displays all sellers and their info. (Works as of Feb 2023)

11 2 11

liveuamap-analysis conflict-investigations Jupyter Notebook

Territory control via liveuamap, NZZ.ch, Novaya Gazeta

11 0 11

mtg-dataset nmichlo Python

🧙‍♀️🃏✨ Magic the Gathering image dataset with automatic multithreaded downloading and optional conversion to HDF5. Powering my other projects like...

11 2 11

just-scrape ScrapeGraphAI TypeScript

CLI for AI-powered web scraping, data extraction, search, and crawling powered by the ScrapeGraph AI API. Supports smart scraping, agentic bro...

11 0 11

darknet_forum octokami Jupyter Notebook

Scraping darknet forums has been the object of research in the field of intelligence and security informatics. In this repository we scraped three di...

11 2 11