Web crawlers rankings
Web crawlers, ordered by GitHub stars · 8 projects
Sort
1
phoneinfogasundowndevPhoneInfoga is one of the most advanced open-source tools for scanning international phone numbers, gathering basic info and attempting to identify VoIP providers or number owners.Go 18.0k 2024.02.212
feroxbusterepi052A fast, simple, recursive content discovery tool written in Rust for forced browsing and directory enumeration.Rust 8.1k 2025.12.133
bitmagnetbitmagnet-ioA self-hosted BitTorrent indexer, DHT crawler, content classifier, and torrent search engine with Web UI, GraphQL API, and Servarr stack integration.Go 4.4k 2026.05.214
waybackwabarcA web archiving tool with an IM-style interface that prioritizes privacy and accessibility, integrated with Internet Archive, archive.today, IPFS, and other archival services.Go 2.2k 2026.07.055
SpiderSuitespidersuiteAn advanced cross-platform and multi-feature web crawler designed for security professionals. 981 2026.08.016
siteone-crawlerjanregesA cross-platform website crawler and analyzer written in Rust, ideal for SEO, security, accessibility, and performance optimization, with a single binary and zero runtime dependencies.Rust 938 2026.06.277
nimbus-eth2status-imA Nim implementation of the Ethereum Beacon Chain consensus layer client, optimized for resource-restricted devices including Raspberry Pi, with low resource usage.Nim 666 2026.10.178
crawleys0rgA Unix-way web crawler that uses a fast HTML SAX parser to extract links, API endpoints from JS/CSS, images, videos, and more, with support for depth scanning, robots.txt, and proxy authentication.Go 339 2026.06.12
phoneinfogasundowndevPhoneInfoga is one of the most advanced open-source tools for scanning international phone numbers, gathering basic info and attempting to identify VoIP providers or number owners.Go 18.0k 2024.02.212feroxbusterepi052A fast, simple, recursive content discovery tool written in Rust for forced browsing and directory enumeration.Rust 8.1k 2025.12.133bitmagnetbitmagnet-ioA self-hosted BitTorrent indexer, DHT crawler, content classifier, and torrent search engine with Web UI, GraphQL API, and Servarr stack integration.Go 4.4k 2026.05.214waybackwabarcA web archiving tool with an IM-style interface that prioritizes privacy and accessibility, integrated with Internet Archive, archive.today, IPFS, and other archival services.Go 2.2k 2026.07.055SpiderSuitespidersuiteAn advanced cross-platform and multi-feature web crawler designed for security professionals. 981 2026.08.016siteone-crawlerjanregesA cross-platform website crawler and analyzer written in Rust, ideal for SEO, security, accessibility, and performance optimization, with a single binary and zero runtime dependencies.Rust 938 2026.06.277nimbus-eth2status-imA Nim implementation of the Ethereum Beacon Chain consensus layer client, optimized for resource-restricted devices including Raspberry Pi, with low resource usage.Nim 666 2026.10.178crawleys0rgA Unix-way web crawler that uses a fast HTML SAX parser to extract links, API endpoints from JS/CSS, images, videos, and more, with support for depth scanning, robots.txt, and proxy authentication.Go 339 2026.06.12