Introduction

What is Hyperscrape?

Hyperscrape is a cloud platform for extracting data from websites. It gives you three things:

1. A catalog of in-house scrapers — maintained programs that each know how to extract a specific kind of data (a website's text, a subreddit's posts, a product's price, and so on).
2. A run engine that executes those scrapers, stores their results, meters usage, and gets through anti-bot defenses with a tiered web unlocker.
3. A REST API and console so you can start runs, schedule them, and pull results from anywhere.

Core concepts

• Scraper — a program in the catalog. You configure its input and run it.
• Run — one execution of a scraper. Runs move through QUEUED → RUNNING → SUCCEEDED (or FAILED/ABORTED/TIMED_OUT).
• Dataset — the ordered list of result items a run produces. Export as JSON or CSV.
• Page scrape — the metering unit. One request a scraper makes to a target page is one page scrape, billed at a flat $0.005.
• Web unlocker — the mechanism that gets through blocks by escalating from direct requests to proxies to a full unlocker.

Your first run

1. Create a free account.
2. Open the Store, pick a scraper (try Website Content Crawler), and click Run.
3. Watch the live logs, then browse the resulting dataset and export it.

That's it — no infrastructure to manage.