wind-bell

distributed crawler

风铃虫是一款轻量级的爬虫工具，似风铃一样灵敏，如蜘蛛一般敏捷，能感知任何细小的风吹草动，轻松抓取互联网上的内容。它是一款对目标服务器相对友好的蜘蛛程序，内置了二十余种常见或不常见的浏览器标识，能够自动处理cookie和网页来源信息，轻松绕过服务器限制，智能调整请求间隔时间，动态调整请求频率，防止对目标服务器造成干扰。此外，风铃虫还是一款对普通用户十分友好的工具，它提供的大量链接提取器和内容提取器让用户可以随心所欲地快速配置，甚至于只要提供一个开始请求地址就能配置出自己爬虫程序。同时，风铃虫也开放了许多自定义接口，让高级用户能够根据需要自定义爬虫功能。最后，风铃虫还天然支持分布式和集群功能，让你突破单机环境的束缚，释放出你的爬虫能力。可以说，风铃虫几乎能抓取目前所有的网站里的绝大部分内容。

How to download and setup wind-bell

Open terminal and run command

git clone https://github.com/yishuifengxiao/wind-bell.git

git clone is used to create a copy or clone of wind-bell repositories. You pass git clone a repository URL.
it supports a few different network protocols and corresponding URL formats.

Also you may download zip file with wind-bell https://github.com/yishuifengxiao/wind-bell/archive/master.zip

Or simply clone wind-bell with SSH

[email protected]:yishuifengxiao/wind-bell.git

If you have some problems with wind-bell

You may open issue on wind-bell support forum (system) here: https://github.com/yishuifengxiao/wind-bell/issues

Similar to wind-bell repositories

Here you may see wind-bell alternatives and analogs

tensorflow scrapy CNTK diaspora Qix handson-ml Sasila Price-monitor infinit diplomat olric qTox LightGBM h2o-3 catboost distributed tns webmagic colly headless-chrome-crawler scrapy-cluster Lulu newcrawler scrapple goose-parser arachnid gopa scrapy-zyte-smartproxy EvaEngine.js dgraph