Web Scraper Quality Assurance
Enforces a rigorous, test-driven development workflow for building and maintaining web scrapers and data extraction agents.
The source repository doesn't declare a license. Check its terms before reusing the code.
Key features
- Mandates a test-first approach, defining use cases before implementation
- Provides standardized templates for scope analysis, test cases, and final reports
- Includes a reference of common CLI commands and test configurations for efficiency
- Implements a structured five-phase workflow: Scope, Setup, Implement, Verify, Report
- Defines required test categories, including dry-run and live verification
Use cases
- Fixing a bug in a platform-specific scraper (e.g., for UberEats)
- Adding a new data field to be extracted by the discovery agent
- Refactoring Puppeteer logic to handle changes on a target website
FAQ
What key capabilities does it provide?
It provides a structured five-phase workflow (Scope, Setup, Implement, Verify, Report), standardized templates for scope analysis and test cases, required test categories like dry-run and live verification, and a handy reference of CLI commands to streamline the entire QA process.
When should I use this skill?
You should use this skill for any task involving the creation, modification, or fixing of web scraping, data discovery, or extraction code. It is essential for work on components like Puppeteer fetchers, platform adapters, and search engine integrations to prevent bugs and regressions.
How does this skill improve my development workflow?
It adds crucial structure by forcing a 'test-first' approach. By defining success criteria before you code, you catch bugs earlier, reduce debugging time, and build more robust, maintainable scrapers. The standardized five-phase process eliminates guesswork and ensures consistent quality.
What does the Web Scraper Quality Assurance skill do?
This skill enforces a professional, test-driven development (TDD) workflow for building and modifying web scrapers. It ensures that every code change is validated against pre-defined use cases, dramatically increasing the reliability and quality of your data extraction agents.
Related skills
Crawl4AI
Scrapes websites, extracts structured data, and automates web data collection pipelines using the Crawl4AI library.
Web Scraping Data Collection412 ptsFirecrawl API
Extracts clean web content, crawls entire domains, and searches the web directly through the Firecrawl API using terminal commands.
Web Scraping Data Collection612 ptsFirecrawl Web Scraping
Scrapes web content, maps site structures, and extracts structured data using advanced crawling and search capabilities.
Web Scraping Data Collection612 pts