Most popular crawling repositories and open source projects

DataScrapingCrawling changwookjun HTML

Data Scraping 정리 자료

7 6 2

golang-scraping-colly itwars Go

Exemples de récupération de données non structurées avec le framework Golang COLLY

7 0 1

instagramProfileCrawler czPechy PHP

Get latest media from instagram profile without API

7 0 1

Cars.com-Crawling zhangyaqi1989 Python

A python crawler for cars.com

7 1 2

leo-bot D3vle0 JavaScript

📢 디스코드 공식 리오봇 📢

7 1 7

napi-rs-trafilatura gorango Rust

Fast and accurate web content extraction

7 0 0

web-scraping-template omkarcloud Python

🚀 THIS WEB SCRAPING TEMPLATE PROVIDES YOU WITH A GREAT STARTING POINT WHEN CREATING WEB SCRAPING BOTS. 🤖

7 5 1

DBpia_crawler chanhee-kang Python

국내 논문 서지정보 사이트 DBpia 크롤링 프로그램

7 3 1

jsonld-extract capturr TypeScript

A damn simple tool to extract json-ld metadata from webpage using jquery like api (jQuery, Cheerio, CashDom ...).

7 0 1

web-crawlers blmarquess JavaScript

Web Crawl

7 0 1

craw-Pinterest RomySaputraSihananda Python

melakukan web scraping dan mengambil gambar berdasarkan keyword pencarian pinterest.

7 1 1

leetcode-summary-crawler HUANGXUANKUN Python

A leetcode crawler built with selenium and requests. Generate a revision guide for your coding interview

6 0 1

ScienceProject Kimdonghyeon7645 Python

🔭🌦 과학프로젝트, 날씨에 학교를 더하다 (with Django)

6 1 1

proxypool franklingu Python

A proxy poll: get free and high quality proxies

6 4 2

chrome-php helloiamlukas PHP

A PHP Wrapper for Chrome Headless. Get the DOM of any webpage.

6 8 1

crawling-scraping-scripts soccer-it JavaScript

Collection of brazilian soccer data crawling/scraping scripts.

6 0 0

Scrapy Decodo Python

Scrapy proxy authentication example for Decodo

6 0 0

scrap-superloto erseler Python

A web scrapping project to fetch all lottery winning numbers, date, prizes etc.

6 1 2

actual-deeplearning fredisbusy Jupyter Notebook

파이썬을 이용한 머신러닝, 딥러닝 실전개발 입문

6 14 2

Booklify.me LillySchramm TypeScript

Booklify.me is an open-source platform for keeping track of everything in your bookshelf.

6 1 2

craw-BadanPusatStatistik RomySaputraSihananda Python

craw-BadanPusatStatistik adalah program untuk mengambil data dari website Badan Pusat Statistik Indonesia.

6 1 1

SLR-Tools maurice-schleussinger Python

Python scripts to perform a systematic literature review for Google Scholar and others

6 2 6

vba-crawler bokhua Visual Basic

VBA web crawler using http GET/POST

6 4 1

dotlas_odyssey dotlas Jupyter Notebook

⛵️ A take-home assignment for the full-time Data Engineering position at Dotlas

6 2 1

bot-detector lula73 Shell

🚫 IP list to block bots, scrapers, AI crawlers & malicious traffic. +17.000 IP Updated every week.

6 0 0

Playwright Decodo JavaScript

Playwright proxy authentication & scraping example for Decodo

6 0 0

Scrapy-Middleware Decodo Python

Scrapy Middleware for proxy authentication with Decodo

6 1 0

Puppeteer Decodo JavaScript

Puppeteer proxy authentication example for Decodo

6 3 0

namu-soup anteater333 JavaScript

숲Soup - 나무위키 인기 검색어 크롤러

6 0 1

telegramBot_instaDP codenashwan PHP

A simple BOT Telegram to downloading Instagram profiles photo

6 0 6

quotes-crawler dori-dev Python

Quotes crawler using scrapy and python.

6 0 1

knu-lms-scheduler HyeokjaeLee JavaScript

:mortar_board: 공주대학교 온라인 강의 시스템 편의성 향상 프로그램

6 1 1

woocommerce-scraper vanquan805 PHP

The best scraping solution for WooCommerce

6 1 1

Advanced-proxy-Scraper FuckingToasters

Advanced Proxy Scraper Crawler fetcher

6 1 1

Crawling-Data-From-Tokopedia aqilwahid Jupyter Notebook

Repositori ini berisi proyek web scraping atau crawling data dari situs Tokopedia. Proyek ini bertujuan untuk mengumpulkan informasi produk seperti na...

6 2 1

Web-Crawler 0MeMo07 Python

Web Crawler with Python

6 0 1

GitHub_Crawling_TextMining_Project park1997 Jupyter Notebook

Data collection and processing for intelligent technology ecosystem analysis

6 2 1

Cloud_Player_V2 amirhoseinsb Python

You can use the cloudplayer tool to listen to the music of the singer you want without going to a specific website and at a very high speed.

6 0 1

PlatformsCrawler eric2788 Go

多平台爬蟲 + 模塊化管理，用於搜集資料並經 redis pubsub 發送

6 2 1

crawl-agoda mandes95 Python

6 3 1

awesome-web-crawler ilovedevs

List of best web crawlers to extract data from the web. Find web crawling tools for different needs.

5 2 0

crawl-for-vector-db habanoz Jupyter Notebook

A web site crawler for semantic search.

5 0 1

gumbo-parser-cpp cschanaj C++

C++ Library to Extract Information from the Google Gumbo HTML Parse Tree

5 2 0

go-web-scrapper zahidhasann88 Go

A web scraping API using Golang with Gin and ChromeDP for dynamic site scraping.

5 0 1

imperium-crawl ceoimperiumprojects TypeScript

The most powerful open-source CLI toolkit for web scraping. 28 tools — stealth, ARIA snapshots, AI extraction, API discovery, YouTube, Reddit, Instagr...

5 0 5

craw-kompas RomySaputraSihananda Python

crawling and scrapping data from the kompas news website

5 0 1