Distributed web crawler admin platform for spiders management regardless of languages and frameworks.
How to download and setup crawlabOpen terminal and run command
git clone https://github.com/crawlab-team/crawlab.gitgit clone is used to create a copy or clone of crawlab repositories. You pass git clone a repository URL.
it supports a few different network protocols and corresponding URL formats.
Also you may download zip file with crawlab https://github.com/crawlab-team/crawlab/archive/master.zip
Or simply clone crawlab with SSH
If you have some problems with crawlabYou may open issue on crawlab support forum (system) here: https://github.com/crawlab-team/crawlab/issues
Similar to crawlab repositories
Here you may see crawlab alternatives and analogsmeteor N-blog scrapy NodeBB mongo webmagic colly headless-chrome-crawler mean habitica node-elm reaction backup generator-angular-fullstack boardgame.io dev-setup laravel-mongodb node-crawler codecombat node-express-mongoose-demo arachni lila newspaper fiora annie cloudtunes practicalnode eve NewsBlur spring-boot-examples