spidr

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

ruby crawler web

View on GitHub

815 Stars

108 Forks

815 Watchers

Ruby Language

mit License

Cost to Build

$60.6K

Market Value

$185.1K

How is this calculated?

Growth over time

11 data points · 2021-08-01 → 2025-03-01

Stars Forks Watchers

💬

How do you feel about this project?

Ask AI about spidr

Question copied to clipboard

What is the postmodern/spidr GitHub project? Description: "A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.". Written in Ruby. Explain what it does, its main use cases, key features, and who would benefit from using it.

Question is copied to clipboard — paste it after the AI opens.

How to clone spidr

Clone via HTTPS

git clone https://github.com/postmodern/spidr.git

Clone via SSH

[email protected]:postmodern/spidr.git

Download ZIP

Download master.zip

Found an issue?

Report bugs or request features on the spidr issue tracker:

Open GitHub Issues

Similar to spidr

rails jekyll discourse gitlabhq awesome-interview-questions devise ruby ruby-style-guide brew diaspora selenium capistrano sinatra spree rubocop awesome-ruby sidekiq grape postal activeadmin Command-line-text-processing kaminari simple_form faker fluentd factory_bot awesomo graal chef puma