Amazon Scraper API vs Proxy: Choose a Product Data Source
Compare seller APIs, licensed Amazon product data and proxy page checks by record, access scope and reuse rights.
Read guideQuick answer
There is no universal permission to scrape a website. Check the source’s terms, access controls, robots rules, applicable privacy and intellectual-property obligations, and planned reuse for this specific dataset. A reachable page or proxy route does not grant permission; record a source-specific go, modify, or stop decision before collection.
Check the source and the planned use
Start with the exact domain, pages, account state, and fields the job will touch. Look for a first-party API or licensed feed that covers the intended records. Then review the current terms for automated collection, storage, redistribution, and commercial use. Terms can differ by product, account, or region, so save the applicable version and review date.
| Signal | What it helps answer | What it does not settle |
|---|---|---|
| First-party API or data agreement | Whether the documented route covers the requested fields and use | Rights beyond the API terms or missing fields |
| Site terms and account agreement | Contract conditions tied to the source or account | Every privacy, copyright, or jurisdiction question |
robots.txt |
Which crawler paths the publisher asks compliant crawlers to avoid | Access authorization or a license; RFC 9309 says its rules are not access authorization |
| Authentication, challenge, or explicit denial | Whether the source is restricting this request | A reason to switch identities, IPs, or routes |
| Data category and destination | Whether personal data, copyright, database, or reuse obligations may apply | A final legal conclusion for every jurisdiction |
The Ninth Circuit’s 2022 hiQ opinion addressed specific public-profile access allegations under one US statute and procedural record. It is not a blanket permission for other sites, accounts, laws, or uses. For EU-regulated personal-data processing, GDPR Article 6 requires a lawful basis; public visibility alone does not establish one. The applicable assessment can involve additional obligations.
Write a decision record before collecting
Record the source and URL scope, whether authentication is involved, fields collected, request cadence, retention period, recipients, intended reuse, terms and robots review, jurisdiction questions, and the person responsible for approval. If any answer is unknown, assign an owner and resolve it before the affected collection begins.
| Decision | Use when | Next action |
|---|---|---|
| Go | The source route and intended storage and reuse have been reviewed and approved | Save the basis, scope, date, and rate limits with the job |
| Modify | A licensed feed, narrower fields, lower cadence, shorter retention, or aggregate output addresses the open issue | Revise the collection plan and review the changed scope |
| Stop | An access denial, conflicting term, unresolved legal question, or missing approval remains | Do not change routes to defeat the restriction; ask the source or qualified counsel |
Worked example: a price-trend report
This example is illustrative, not legal advice or a statement about a named retailer. A team proposes one daily request per product page and plans to publish a trend chart. Its decision record says pages=20, account_required=no, personal_fields=no, robots=checked, terms=reviewed, rate=one fetch per SKU per day, retention=30 days, and reuse=aggregated chart.
If the retailer offers a permitted price feed, document that route instead. If the terms do not cover the proposed reuse or the source denies automated access, pause for a source-specific review.
Treat a CAPTCHA or explicit block as a stop signal for that request path. An HTTP 200 response only reports the response status; it does not answer permission, data accuracy, or reuse rights. Respect HTTP 429 and Retry-After by reducing or pausing requests when the source provides a wait instruction.
If collection involves profiles, reviews, or other identifiable people, resolve the data-protection purpose, lawful basis, minimization, retention, and disclosure questions with qualified counsel. For a separate decision about comparable price records, use the price-monitoring workflow.
Sources and further reading
ProxyLaneTraffic never expiresPremium residential proxies
From $2/GB
Scale your workflows with the tools you already use. Rotating or sticky sessions up to 72 hours.
