Found 228,277 repositories(showing 30)
firecrawl
๐ฅ The Web Data API for AI - Power AI agents with clean web data
unclecode
๐๐ค Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN
huginn
Create agents that monitor and act on your behalf. Your agents are standing by!
NaiboWang
A visual no-code/code-free web crawler/spiderๆ้้๏ผไธไธชๅฏ่งๅๆต่งๅจ่ชๅจๅๆต่ฏ/ๆฐๆฎ้้/็ฌ่ซ่ฝฏไปถ๏ผๅฏไปฅๆ ไปฃ็ ๅพๅฝขๅ็่ฎพ่ฎกๅๆง่ก็ฌ่ซไปปๅกใๅซๅ๏ผServiceWrapper้ขๅWebๅบ็จ็ๆบ่ฝๅๆๅกๅฐ่ฃ ็ณป็ปใ
iawia002
๐พ Fast and simple video download library and CLI tool written in Go
cheeriojs
The fast, flexible, and elegant library for parsing and manipulating HTML and XML.
feder-cr
AIHawk aims to easy job hunt process by automating the job application process. Utilizing artificial intelligence, it enables users to apply for multiple jobs in a tailored way.
gocolly
Elegant Scraper and Crawler Framework for Golang
ScrapeGraphAI
Python scraper based on AI
apify
CrawleeโA web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
Evil0ctal
๐ใDouyin_TikTok_Download_APIใๆฏไธไธชๅผ็ฎฑๅณ็จ็้ซๆง่ฝๅผๆญฅๆ้ณใๅฟซๆใTikTokใBilibiliๆฐๆฎ็ฌๅๅทฅๅ ท๏ผๆฏๆAPI่ฐ็จ๏ผๅจ็บฟๆน้่งฃๆๅไธ่ฝฝใ
getmaxun
๐ฅ The open-source no-code platform for web scraping, crawling, search and AI data extraction โข Turn websites into structured APIs in minutes ๐ฅ
codelucas
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
alex000kim
Collection of scripts to aggregate image data for the purposes of training an NSFW Image Classifier
pwxcoo
:orange_book: ไธญๅๆฐๅๅญๅ ธๆฐๆฎๅบใๅ ๆฌๆญๅ่ฏญ๏ผๆ่ฏญ๏ผ่ฏ่ฏญ๏ผๆฑๅญใ
guyueyingmu
AV ็ตๅฝฑ็ฎก็็ณป็ป๏ผ avmoo , javbus , javlibrary ็ฌ่ซ๏ผ็บฟไธ AV ๅฝฑ็ๅพไนฆ้ฆ๏ผAV ็ฃๅ้พๆฅๆฐๆฎๅบ๏ผJapanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database
FriendsOfPHP
Goutte, a simple PHP Web Scraper
apify
CrawleeโA web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
BruceDone
A collection of awesome web crawler,spider in different languages
alirezamika
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
go-rod
A Chrome DevTools Protocol driver for web automation and scraping.
mishushakov
Turn any webpage into structured data using LLMs
MontFerret
Declarative web scraping
matthewmueller
The next web scraper. See through the <html> noise.
yujiosaka
Distributed crawler powered by Headless Chrome
JustAnotherArchivist
A social networking service scraper in Python
drawrowfly
TikTok Scraper. Download video posts, collect user/trend/hashtag/music feed metadata, sign URL and etc.
niespodd
Analysis of Bot Protection systems with available countermeasures ๐ฟ. How to defeat anti-bot system ๐ป and get around browser fingerprinting scripts ๐ต๏ธโโ๏ธ when scraping the web?
jaypyles
Self-hosted webscraper.
fent
YouTube video downloader in javascript.