crawler

Get notified of new listings on popular German real estate portals.

Google Play Unofficial Python 3 API Library

A Python based Web Application security scanner

crawling App by uiautomator2 & mitmproxy

Source code for the IndieWeb search engine.

Explore a website recursively and download all the wanted documents (PDF, ODT…)

A okcoin crawler based on websocket, save data to mysql

A crawler that extracts data from a dynamic webpage. Written in node js.

Node 批量爬取头条视频

Better.php crawler for Redacted that uses WhatManager

A Python framework to build polite, but tenacious crawlers / scrapers with a MariaDB backend

Crawler and scraper of the public directory of companies on LinkedIn.

Multiplexing web client supporting HTTP/2 and WHATWG URL compliant parser written in C

:camera: Tistory photo grabber

🧹 A libp2p DHT scraper as a service allowing anyone to collect, consume and use to generate useful reports & visualisations.

A toolkit for web reconnaissance, it's fast and easy to use.

backup posts and comments of specify user in sina

虎扑步行街爬虫

A personal submission codeforces parser for CF, parsed by individual contests.The user is prompted for the username and has the flexibilty to parse la...

高校高校统一身份认证 Node.js 优雅可扩展示例，已集成今日校园签到(支持多平台一键部署)

Easily crawl news portals or blog sites using Storm Crawler.

A Soccer Dashboard created by scraping EPL website using Akka backend and ReactJS frontend and IBM Cloudant for object storage. IBM Cloud Foundry is u...

🇰🇷 Realtime integrated information analysis service

Search for Slack Webhooks token publicly exposed on Github

web crawler for Nodejs

爬取QQ用户信息（qq号、昵称、生日、地址等基本信息）并做简要analysis。

Crawl your own website with various clients for SEO and indexing purposes.

:bug:一只登录正方教务管理系统，爬取数据的小爬虫

A web page content extractor

A useful website for finding movie's rating in Chinese and English. By crawling Yahoo, Ptt, IMDB.

코로나 확진자 수/정보 크롤링

Taiwan Lottery Crawler 🐛 (Crawler for Various Types of Lotteries in Taiwan)

先知文章爬虫项目-[包含2021年7月之前所有文章]

Apify actor to scrape Youtube search results. You can set the maximum videos to scrape per page as well as the date from which to start scraping.

ProxyCrawl PHP library for scraping and crawling websites

本项目使用scrapy对 古诗文网 进行爬虫，获取不同分类（爱情、七夕等）的宋词的：词牌、作者、正文、注释、创作背景。

Scraping the real estate agencies for up-to-date house listings as soon as they arrive!

Crawling and scraping the Web for fun and profit

When you solve the problem of the Codeforces site, it automatically commits and pushes to the remote repository.

🎉 开箱即用的高性能可自定义的笔趣阁小说爬虫 快速下载无广告小说

整理本人在2021年10月-12月期间写的一些爬虫demo，比如用于渗透测试中SQL注入的URL收集脚本(爬取必应和百度搜索结果的URL)，子域名爆破demo，各大高校漏洞信息收...

Crawling client's groups and channels to invite their members to a target group.

A torrent site written in the python language & douban scraper

Lightweight library for scalable crawlers in Go.

A simple and flexible web crawler framework for java.

Scrapy as a Web Crawler for Azure Search Samples

botsonar analyse open api

A Python DNS crawler to find identical domain names under different TLDs.

百万英雄答题助手 - 兼容全部答题 APP

Flutter爬虫框架，帮助开发者快速在移动设备上构建爬虫，单线程版本