Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

新闻网页正文通用抽取器 Beta 版.

蓝天采集器是一款开源免费的爬虫系统，仅需点选编辑规则即可采集数据，可运行在本地、虚拟主机或云服务器中，几乎能采集所有类型的网页，无缝对接各类CMS建站程序，免登录实时发布数据，全自动无需人工干预！是网页大数据采集软件中完全跨平台的云端爬虫系统

A Unix-style personal search engine and web crawler for your digital footprint.

SpiderSuite releases, wiki and roadmap

HTTP API for Scrapy spiders

Uscrapper Vanta: Dive deeper into the web with this powerful open-source tool. Extract valuable insights with ease and efficiency, from both surface and deep web sources. Empower your data mining and analysis with Vanta's advanced capabilities. Fast, reliable, and user-friendly, Uscrapper Vanta is the ultimate choice for researchers and analysts.

🌈Python3网络爬虫实战：QQ音乐歌曲、京东商品信息、房天下、破解有道翻译、构建代理池、豆瓣读书、百度图片、破解网易登录、B站模拟扫码登录、小鹅通、荔枝微课

Open-source Enterprise Grade Search Engine Software

Python Pastebin Webcrawler that returns list of public pastebins containing keywords

逆向

《Python爬虫开发 从入门到实战》配套源代码。

An R web crawler and scraper

Price Crawler - Tracking Price Inflation

O maior livro de receitas culinárias em língua portuguesa

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

📲 Bot to help solve HQ trivia

This program provides efficient web scraping services for Tor and non-Tor sites. The program has both a CLI and REST API.

Bookmarklet and manual webcrawler to aid in web research

中国大陆大学列表爬虫

.NET based webcrawler

Browse the web, fetch API data, and more on your Flipper Zero.

网页版nodejs爬虫工具

A php crawler that finds emails on the internets

A web crawling framework written in Kotlin

Web scraper, crawler and parser in C#. Designed as simple, declarative and scalable web scraping solution.

An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site

Simple web crawler written in Python

使用 Scrapy 写成的 JK 爬虫，图片源自哔哩哔哩、Tumblr、Instagram，以及微博、Twitter