Found 53,382 repositories(showing 30)
firecrawl
🔥 The Web Data API for AI - Power AI agents with clean web data
unclecode
🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN
D4Vinci
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
ScrapeGraphAI
Python scraper based on AI
Panniantong
Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
getmaxun
🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥
FriendsOfPHP
Goutte, a simple PHP Web Scraper
BruceDone
A collection of awesome web crawler,spider in different languages
alirezamika
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
matthewmueller
The next web scraper. See through the <html> noise.
jaypyles
Self-hosted webscraper.
rchipka
Web scraper for NodeJS
ruipgil
A complete and versatile web scraper.
arpit-omprakash
A list of practical knowledge-building projects.
gosom
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
php-curl-class
PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs
oxylabs
Scrape Google AI Mode responses without blocks on a large scale.
itsOwen
A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama
oxylabs
Easily collect responses from ChatGPT scraper by providing a prompt along with valid Web Scraper API credentials.
oxylabs
Structured data gathering from any website using AI-powered scraper, crawler, and browser automation. Scraping and crawling with natural language prompts. Equip your LLM agents with fresh data. AI Studio python SDK for intelligent web data gathering.
The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.
spider-rs
Web crawler and scraper for Rust
lncrawl
Generate and download e-books from online sources.
anaskhan96
Web Scraper in Go, similar to BeautifulSoup
website-scraper
Download website to local directory (including all css, images, js, etc.)
oxylabs
A code for extracting best-selling items, search results, and currently available deals from Amazon using Python and Oxylabs E-Commerce Scraper API.
oxylabs
A guide for extracting titles, authors, and citations from Google Scholar using Python and Oxylabs SERP Scraper API.
martinsbalodis
Web data extraction tool implemented as chrome extension
felipecsl
Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.
gildas-lormeau
CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)