Found 190 repositories(showing 30)
scrapy-plugins
🎭 Playwright integration for Scrapy
taskPyroer
爬虫管理系统,爬虫管理平台,分布式爬虫管理平台, openclaw龙虾,AI助手,可视化操作,完整监控,灵活的Python环境管理,,环境隔离,资源占用小,支持 Scrapy 等主流爬虫框架,支持 Selenium、Playwright、DrissionPage 等浏览器自动化工具,支持node环境下的js逆向代码,支持windows分布式调度
Boris-code
爬虫管理系统，支持集群，弹性伸缩。支持运行feapder、scrapy、selenium、playwright等各种框架及脚本
Gerapy
Downloader Middleware to support Playwright in Scrapy & Gerapy
elacuesta
Trying scrapy-playwright on Scrapy Cloud
ivan-sincek
Web crawler and scraper based on Scrapy and Playwright's headless browser.
Wesson1337
Scrapy + Playwright + FastAPI scrapper example
fioreagui
At the company we work at, BuscoJobs, we applied Scrapy Playwright on 48 spiders. We have created a guideline to help users get started with this library. Our intention is that this guideline serve as reference and a starting point for further exploration.
eupendra
No description available
Hhhexx
亚马逊电商平台商品数据智能采集系统，基于 Scrapy 和 Playwright 构建的专业数据采集解决方案。支持动态页面渲染、多层反爬策略、分布式架构与自动化调度，可高效采集商品价格、用户评论、促销活动等信息，帮助市场调研实时掌握市场动态，优化定价策略。
HritikShukla02
This Scrapy spider conquers dynamic websites! Enter a keyword, and it leverages Playwright to render JavaScript-heavy pages. It then scrapes data across the first 10 pages of search results, wielding Scrapy's parsing power. Unleash it for keyword research, price comparisons, or content analysis!
zubedev
Scrapy dappy doo crawler for proxy sites
renatodvc
A wrapper for scrapy-playwright with stealth features.
champion-yang
scrapy & pyppeteer & playwright-python
ParkingReformNetwork
Pipeline to populate, update, maintain a database of parking requirements across the United States.
ai-to-ai
No description available
GarajedeIdeas
No description available
zyq5428
scrapy使用playwright进行动态网页爬取的模版
ai-to-ai
No description available
Mahmoud-T-Almetwally
Image & Audio Web scrapper using Scrapy and Playwright, and a Feature Extractor using MambaVision & Clap. all tied to a Go Backend To Provide the API
A1Elite
No description available
ShahidulDotTech
Here is the code to scrape the website doctorgrades.com. If it is helpful please click the star on the top right corner
Pronewing
A Scrapy–Playwright based web crawler for Practo that scrapes doctor profiles across multiple paginated search results, extracting structured data such as name, degree, specialization, consultation fees, experience, hospital details, location, ratings, and profile URLs, and exporting the data to CSV.
priyanshu-maity
A sophisticated Medium.com scraper built with Scrapy and Playwright. 🚀 Extract articles, metadata, and engagement metrics based on keywords. Features include article summarization using BART, smart retry mechanisms, and comprehensive data validation. Perfect for content analysis, research, and monitoring trending topics on Medium.
darkstar360
No description available
GerritGeeraerts
"Playing it right" Efficiently crawling Javascript-heavy websites. [Python] [Scrapy] [Playwright]
NicoNicoNico123
No description available
eupendra
No description available
cloud-browser
No description available
No description available