Measure useful rows
Raw request success is not enough. A benchmark should count delivered rows that contain the fields your workflow needs, such as product_id, title, price, currency, rating, reviews_count, stock_status and product_url.
For search and category pages, also measure position and source_url. Those fields make the dataset useful for rank tracking and market mapping.
Measure export readiness
A scraper API that returns JSON but leaves spreadsheet users with cleanup work will slow down ecommerce teams. Include CSV and XLSX readiness in the benchmark if analysts or operators use the data.
ParseShelf is strongest when the benchmark values dashboard visibility, live row preview and downloadable files, not only low-level scraping infrastructure.
Measure repair cost
Track fallback rows, missing critical fields and duplicate ASINs. The real cost of an Amazon data provider includes the time spent repairing data, re-running scripts and explaining failures to non-technical stakeholders.
A fair test should run the same keyword, category and ASIN inputs across providers and compare the final usable dataset, not only provider marketing claims.
Production checklist
Start with one small input and compare the delivered fields with the report, catalog or pipeline you actually need.
Keep the job ID, source URL, mode and run date with downstream exports so operators and developers can audit the same result.