Amazon Scraper API vs Proxy: Choose a Product Data Source
Distinguish authorized Amazon APIs, licensed product data and proxy-based page checks by record quality and access rights.
Read guideSupport can investigate a reproducible issue faster when the packet identifies the failing phase and preserves one control request. Include the proxy host, port, and scheme, but never include its password. A screenshot of “proxy failed” rarely says whether the problem was DNS, authentication, TLS, the target, or the client’s validator.
Send enough detail to repeat one permitted request. Redact secrets and private content before the packet leaves your system. Never send a password, API key, cookie, bearer token, or credential-bearing URL.
Collect one failing sample and one control
Use the same client, target class, proxy scheme, and deadline for both requests. The control can be a direct request to a harmless endpoint or a known-good route that your policy allows. Its purpose is to separate a client-wide problem from a route-specific observation.
| Field | Failing sample | Control |
|---|---|---|
| Run ID and UTC time | Stable ID and exact timestamp | Separate stable ID and timestamp |
| Client | Name, version, runtime, OS, and proxy adapter | Same values |
| Proxy | Scheme, host label, port, and auth method name | Direct or known-good route label |
| Target | Redacted host/path and method | Same target class or harmless control URL |
| Phase | DNS, connect, auth, TLS, target, read, or validation | Phase reached |
| Status | HTTP status and proxy status | HTTP status and proxy status |
| Timing | Name lookup, connect, TLS, first byte, total | Same fields |
| Error | Exact redacted client error line | Exact redacted client error line |
| Expected result | The status or field that would pass | The control’s expected result |
| Observed result | What happened and how often in the bounded sample | What happened |
HTTP 407 is proxy authentication. A 429 may be a rate-limit response, but record whether the proxy or target sent it before assigning ownership; Retry-After can be part of the evidence. A 200 response still needs the application’s validation result. RFC 9110 and RFC 6585 are the status semantics references for this distinction.
Use a small reproducible command
The command below emits phase timing and status without writing the response body. Keep the proxy variable free of credentials and use one permitted target.
This example assumes an IP-allowlisted or no-auth route. If the endpoint requires username and password, provide them through the client’s private credential store or secret binding supported by your runtime; do not put them in PROXY_URL, shell history, or logs.
set -eu
: "${PROXY_URL:?Set PROXY_URL to scheme://host:port without credentials}"
: "${TARGET_URL:?Set TARGET_URL to one permitted HTTPS URL}"
curl --disable --proxy "$PROXY_URL" --noproxy '' \
--connect-timeout 10 --max-time 30 \
--silent --show-error --output /dev/null \
--write-out 'status=%{response_code} proxy_status=%{http_connect} dns=%{time_namelookup}s connect=%{time_connect}s appconnect=%{time_appconnect}s starttransfer=%{time_starttransfer}s total=%{time_total}s\n' \
"$TARGET_URL"
For browser failures, attach the client version, the awaited operation, the request URL host, and redacted request/response event summaries. Do not attach a HAR or screenshot until you have removed cookies, authorization headers, query secrets, private response bodies, and any credential-bearing proxy URL. OWASP recommends masking or removing access tokens, passwords, connection strings, keys, session identifiers, and sensitive personal data from logs.
Copy this ticket template
Subject: [proxy] <short phase and symptom> · <UTC date> · <run ID>
Impact:
- Permitted workflow and affected target class:
- First observed UTC time and frequency in the bounded sample:
- Records or requests blocked:
Client:
- Client, version, runtime, OS, and proxy adapter:
- Proxy scheme and port:
- Proxy host label: <redacted or provider endpoint label>
- Auth method name only: <do not include credentials>
- Timeout settings: connect <seconds>, total <seconds>
Failing request:
- Run ID:
- Target host and path shape: <redacted as needed>
- Method:
- Phase reached:
- HTTP status / proxy status:
- DNS, connect, TLS, first-byte, and total timings:
- Exact redacted error line:
- Expected result:
- Observed result:
Control:
- Control run ID and route:
- Same client settings? <yes/no; explain>
- Result and timings:
Reproduction:
- Minimal command or steps with placeholders only:
- Reproduces on <one run / N of M bounded runs>:
- Direct or known-good route result:
Question:
- Which phase or account-side condition should be checked next?
- What additional redacted field would make this reproducible?
Redact before escalation
| Remove or mask | Keep instead |
|---|---|
| Proxy password, API key, bearer token, cookie, session ID | Auth method name and a support-safe run ID |
| Credential-bearing URL | Scheme, host label, and port |
| Full private path, query values, or response body | Host, path shape, parameter names, status, and byte count |
| Personal data or customer identifiers | Synthetic non-identifying ID when correlation is required |
| Unbounded trace or repeated retries | One failing sample, one control, and the bounded attempt count |
Escalate when the packet contains a reproducible phase, the client settings, a control result, and a specific question. If the issue is intermittent, include the count and window instead of calling it a success rate. Do not promise a support SLA or a response time that your service terms do not state.
The timeout guide explains the phase timings; the TLS guide separates proxy and destination trust. For repeated incidents, the monitoring playbook covers operational alerts. Sign up to choose your workflow and traffic package during onboarding.