go-crawler-distributed

分布式爬虫项目，本项目支持个性化定制页面解析器二次开发，项目整体采用微服务架构，通过消息队列实现消息的异步发送，使用到的框架包括：redigo, gorm, goquery, easyjson, viper, amqp, zap, go-micro，并通过Docker实现容器化部署，中间爬虫节点支持水平拓展。

crawler elasticsearch

View on GitHub

53 Stars

8 Forks

53 Watchers

Go Language

mit License

Cost to Build

$6.59M

Market Value

$13.93M

How is this calculated?

Growth over time

9 data points · 2021-08-01 → 2025-09-01

Stars Forks Watchers

💬

How do you feel about this project?

Ask AI about go-crawler-distributed

Question copied to clipboard

What is the golang-collection/go-crawler-distributed GitHub project? Description: "分布式爬虫项目，本项目支持个性化定制页面解析器二次开发，项目整体采用微服务架构，通过消息队列实现消息的异步发送，使用到的框架包括：redigo, gorm, goquery, easyjson, viper, amqp, zap, go-micro，并通过Docker实现容器化部署，中间爬虫节点支持水平拓展。". Written in Go. Explain what it does, its main use cases, key features, and who would benefit from using it.

Question is copied to clipboard — paste it after the AI opens.

How to clone go-crawler-distributed

Clone via HTTPS

git clone https://github.com/golang-collection/go-crawler-distributed.git

Clone via SSH

[email protected]:golang-collection/go-crawler-distributed.git

Download ZIP

Download master.zip

Found an issue?

Report bugs or request features on the go-crawler-distributed issue tracker:

Open GitHub Issues

Similar to go-crawler-distributed

elasticsearch grafana kibana awesome-aws NewsBlur analysis-ik docker-elk dejavu elasticsearch-sql dev-setup Linux-Tutorial Opserver searchkit elasticsearch-HQ elasticsearch-dump pipeline peek elastic vue-storefront elasticsearch-rails reactivesearch xmall awesome-elasticsearch elasticsearch-dsl-py elasticsearch-net elasticsearch-gmail elasticsearch-py cerebro search-guard janusgraph