DotnetCrawler

crawling crawler csharp scraping

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

How to download and setup DotnetCrawler

Open terminal and run command

git clone https://github.com/mehmetozkaya/DotnetCrawler.git

git clone is used to create a copy or clone of DotnetCrawler repositories. You pass git clone a repository URL.
it supports a few different network protocols and corresponding URL formats.

Also you may download zip file with DotnetCrawler https://github.com/mehmetozkaya/DotnetCrawler/archive/master.zip

Or simply clone DotnetCrawler with SSH

[email protected]:mehmetozkaya/DotnetCrawler.git

If you have some problems with DotnetCrawler

You may open issue on DotnetCrawler support forum (system) here: https://github.com/mehmetozkaya/DotnetCrawler/issues

Similar to DotnetCrawler repositories

Here you may see DotnetCrawler alternatives and analogs

scrapy requests-html mathnet-numerics WaveFunctionCollapse Sasila Price-monitor prettydiff arl webmagic colly headless-chrome-crawler Embed artoo instagram-scraper django-dynamic-scraper scrapy-cluster Lulu newcrawler panther facebook_data_analyzer ImageScraper scrapple parsel nickjs jsoup-annotations jekyll Musoq goose-parser arachnid lambdasoup