pycrawler
powerful python crawler: proxy-ip,mutiprocessing+Queue+yaml configurable crawler, readability, bs4(beautiful soup), pybloom, PooledDB, MysqlDb, selenium-webdriver-phantomjs, reids,anti-geetest, yaml, email
How to download and setup pycrawler
Open terminal and run command
git clone https://github.com/zyq001/pycrawler.git
git clone is used to create a copy or clone of pycrawler repositories.
You pass git clone a repository URL. it supports a few different network protocols and corresponding URL formats.
Also you may download zip file with pycrawler https://github.com/zyq001/pycrawler/archive/master.zip
Or simply clone pycrawler with SSH
[email protected]:zyq001/pycrawler.git
If you have some problems with pycrawler
You may open issue on pycrawler support forum (system) here: https://github.com/zyq001/pycrawler/issuesSimilar to pycrawler repositories
Here you may see pycrawler alternatives and analogs
nylas-mail prettier responsive-html-email-template PHPMailer grav awesome-falsehood just-dashboard structured-text-tools countries bojler community-edition alertmanager cli mailinabox light-push Pico tmuxp yaml XcodeGen rq cv-boilerplate YamlDotNet ParameterHandler yaml.js bashful yq metta invoice-boilerplate ios-dev-playbook yamllint