Amazon Scraper API vs Proxy: Choose a Product Data Source
Distinguish authorized Amazon APIs, licensed product data and proxy-based page checks by record quality and access rights.
Read guideA proxy quote tells you the unit on the invoice. Before you request capacity, you need a range for the traffic your job may produce. A page body, its images and scripts, redirects, retries and browser state can all change that range. The useful question is not “how many gigabytes does this provider advertise?” It is “how many decimal GB could this permitted workload consume under the client settings we will actually run?”
This is a planning model. The values below are labelled assumptions. This is not a provider benchmark and contains no ProxyLane measurement.
Start with the request your job will make
Write the unit that will consume traffic: one product page, one search result, one browser checkout you own, or one API response. Count the pages or records in the next run, then record the client and its behavior.
| Input | What to record | Why it changes the forecast |
|---|---|---|
| Work units | Pages, records or browser journeys | Sets the number of primary requests |
| Response bytes | Body and headers from the same client | A browser page and an HTTP response fetch different resources |
| Browser assets | Images, scripts, stylesheets, fonts and XHR or fetch calls | A rendered page can load much more than its document |
| Retries | Retry limit and the conditions that trigger one | A failed attempt can still consume traffic |
| Redirects | Redirect count and whether each hop uses the route | A final URL does not erase earlier transfers |
| Cache policy | Cold or warm cache, persistent context and cache bypass | A cache hit can reduce observed transfer while the next worker is cold |
Keep the target, parser, browser or HTTP client, headers, session policy and concurrency fixed while you measure. A bandwidth estimate made from a command-line document request should not be presented as a browser-journey estimate.
Use a range model, not one precise number
For a first forecast, use decimal GB and state each assumption. Define base_bytes as the document response plus its response headers for one initial attempt. Apply an asset multiplier only when you measured the document separately and need to model the browser resources. If you measured all browser resources directly, use that total and set the asset multiplier to 1. Do not multiply a full-journey measurement by an asset factor a second time.
forecast_gb =
work_units × base_bytes_per_initial_attempt × asset_multiplier × retry_multiplier
÷ 1,000,000,000
asset_multiplier is 1 for a document-only client or for a browser run whose complete resource total is already measured. In a document-only sample, it represents the extra browser resources you decide to count. retry_multiplier is 1 plus the expected extra attempts. Keep redirects and failed requests in the measured bytes when the provider bills them; otherwise put them in a separate scenario and ask the provider how its meter works.
| Scenario | Work units | Mean transferred bytes | Asset multiplier | Retry multiplier | Modelled traffic |
|---|---|---|---|---|---|
| Document-only check | 1,000 | 250 KB document + headers | 1.0 | 1.05 | 0.2625 GB |
| Browser page modelled from document sample | 1,000 | 750 KB document + headers | 2.0 | 1.20 | 1.8 GB |
| Heavy page modelled from document sample | 1,000 | 1.5 MB document + headers | 3.0 | 1.40 | 6.3 GB |
These are worked assumptions so a buyer can see the arithmetic. They are not measurements of a named target, client or provider. Replace them with your own sample before choosing a package. If the job has a long tail, carry a low, expected and high case into the purchase discussion rather than averaging away the heavy pages.
Measure the client that will run the job
For an HTTP client, log request headers, response headers and response body bytes for a permitted sample and keep redirects and retries as separate rows. Decide whether your forecast includes request uplink bytes, then record that choice. Requests documents per-request proxy configuration and timeout behavior, but the library’s response body length is not a provider invoice. A proxy may count bytes differently from your application, and a provider may apply minimums or include failed attempts.
For a browser, collect resource timing after the page settles and record the resource type and URL host in a redacted log. The browser’s PerformanceResourceTiming.transferSize includes response headers and payload for a fetched resource. MDN also documents zero for some cache hits and cross-origin resources without Timing-Allow-Origin, so it is an observation with limits. It does not include every possible uplink or provider-side accounting rule, and it is not proof of the provider’s billing meter.
Playwright can monitor HTTP and HTTPS traffic and configure a proxy globally or per browser context in its network documentation. Use that instrumentation to count the resources your acceptance rule needs. Decide whether analytics, third-party fonts, video, prefetches and service-worker traffic belong in the forecast. Record the rule so a later run remains comparable.
Make retries visible in the estimate
A retry is work, even when the final record is accepted. Keep these values separate:
| Field | Example meaning |
|---|---|
attempts |
Every request sent, including retries |
retry_reason |
Timeout, proxy authentication, rate limit, target error or parser failure |
transferred_bytes |
Bytes observed by the client for that attempt |
accepted_record |
Whether the required fields passed validation |
provider_billed_unit |
The unit and amount shown by the provider for the run |
A timeout may have no body but still use connection and request traffic. A target can return a response that your parser rejects. Do not estimate only from successful records, and do not turn one retry ratio into a provider success rate. It belongs to this workload, client and period.
Reconcile the estimate before buying
Run a small permitted sample, then compare three totals:
- The client-side transferred bytes, split by request headers, response headers, body, asset, redirect and retry
- The expected decimal GB after any unit conversion, with uplink included or excluded by an explicit rule
- The provider’s reported or invoiced usage for the same window and its documented billing boundary
Investigate the difference before scaling. Common explanations include compression, cached resources, cross-origin timing gaps, background requests, DNS or TLS retries, provider rounding and resources fetched outside the proxy. Keep page content and credentials out of general telemetry. The proxy monitoring playbook shows how to separate gateway, target and validation evidence.
This forecast answers a pre-purchase question. The existing cost per successful request guide answers a different post-run question: what did each distinct validated result cost after retries, compute and support time? Link the two in your procurement notes so a low traffic estimate is not mistaken for low cost per useful result.
Bring the sample size, low/expected/high GB range, geography, session behavior, retry rule and billing questions to the provider. ProxyLane’s approved public price wording is From $2.50/GB and the offer describes non-expiring traffic, HTTP and SOCKS5, but current package terms and the provider’s meter still need to be checked against your workload. Request access with the client and forecast you intend to test.