13 Forks
27 Stars
27 Watchers

scrapy_pro

关于5000+站点的scrapy爬虫开发,涉及一些技术架构搭建以及各种反爬方案,详见readme文件

How to download and setup scrapy_pro

Open terminal and run command
git clone https://github.com/yangge11/scrapy_pro.git
git clone is used to create a copy or clone of scrapy_pro repositories. You pass git clone a repository URL.
it supports a few different network protocols and corresponding URL formats.

Also you may download zip file with scrapy_pro https://github.com/yangge11/scrapy_pro/archive/master.zip

Or simply clone scrapy_pro with SSH
[email protected]:yangge11/scrapy_pro.git

If you have some problems with scrapy_pro

You may open issue on scrapy_pro support forum (system) here: https://github.com/yangge11/scrapy_pro/issues

Similar to scrapy_pro repositories

Here you may see scrapy_pro alternatives and analogs

 scrapy    Sasila    Price-monitor    webmagic    colly    headless-chrome-crawler    Lulu    newcrawler    scrapple    goose-parser    arachnid    gopa    scrapy-zyte-smartproxy    node-crawler    arachni    newspaper    webster    spidy    N2H4    easy-scraping-tutorial    antch    pomp    talospider    podcastcrawler    FileMasta    lux    scrapy-redis    haipproxy    DotnetSpider    TumblThree