python3-concurrency-pics-02

爬取 www.mzitu.com 全站图片，截至目前共5162个图集，16.5万多张美女图片，使用 asyncio 和 aiohttp 实现的异步版本只需要不到2小时就能爬取完成。按日期创建图集目录，保存更合理。控制台只显示下载的进度条，详细信息保存在日志文件中。支持异常处理，不会终止爬虫程序。失败的请求，下次再执行爬虫程序时会自动下载

python3 requests

View on GitHub Website

312 Stars

82 Forks

312 Watchers

Python Language

Cost to Build

$6.1K

Market Value

$13.1K

How is this calculated?

Growth over time

10 data points · 2021-08-01 → 2023-07-01

Stars Forks Watchers

💬

How do you feel about this project?

Ask AI about python3-concurrency-pics-02

Question copied to clipboard

What is the wangy8961/python3-concurrency-pics-02 GitHub project? Description: "爬取 www.mzitu.com 全站图片，截至目前共5162个图集，16.5万多张美女图片，使用 asyncio 和 aiohttp 实现的异步版本只需要不到2小时就能爬取完成。按日期创建图集目录，保存更合理。控制台只显示下载的进度条，详细信息保存在日志文件中。支持异常处理，不会终止爬虫程序。失败的请求，下次再执行爬虫程序时会自动下载". Written in Python. Explain what it does, its main use cases, key features, and who would benefit from using it.

Question is copied to clipboard — paste it after the AI opens.

How to clone python3-concurrency-pics-02

Clone via HTTPS

git clone https://github.com/wangy8961/python3-concurrency-pics-02.git

Clone via SSH

[email protected]:wangy8961/python3-concurrency-pics-02.git

Download ZIP

Download master.zip

Found an issue?

Report bugs or request features on the python3-concurrency-pics-02 issue tracker:

Open GitHub Issues

Similar to python3-concurrency-pics-02

requests requests-html zulip MechanicalSoup php-curl-class cpr requestium Just grequests performance-bookmarklet uplink lassie requests-respectful httmock curl pycookiecheat node-request-retry curequests khttp Sasila requests-threads robotframework-requests Requester ngx-resource AutoLine human_curl webspider saber asks assent