Amazon Scraper API vs Proxy: Choose a Product Data Source
Compare seller APIs, licensed Amazon product data and proxy page checks by record, access scope and reuse rights.
Read guideSeparate destination trust from HTTPS-proxy trust, then repair the CA bundle or hostname while certificate verification stays enabled.

A certificate error can come from the HTTPS proxy or the destination reached through it. Identify which Transport Layer Security (TLS) peer failed, then correct that endpoint name or its trusted certificate authority (CA) source; keep certificate and hostname verification enabled throughout diagnosis.
Use curl 8.7.1 and Python 3.12.0, tested on macOS 26.5.2. The curl route example requires an HTTPS proxy and reviewed CA files.
An HTTPS destination usually has its own TLS connection even when a proxy carries the traffic. With an HTTPS proxy, the client first verifies the proxy endpoint, then establishes the destination connection through the tunnel. An HTTP proxy carrying HTTPS uses CONNECT but has no TLS certificate of its own on that client-to-proxy leg.
| Route | TLS peer to identify | Trust setting to inspect |
|---|---|---|
| HTTP proxy to HTTPS destination | Destination, after CONNECT | Destination CA source and target hostname |
| HTTPS proxy to HTTPS destination | HTTPS proxy, then destination | Proxy CA source, then destination CA source |
| SOCKS5 proxy to HTTPS destination | Destination | Destination CA source and target hostname |
| Direct HTTPS request | Destination | Destination CA source and target hostname |
A certificate chain and a hostname answer different questions. The X.509 path-validation standard, RFC 5280, describes certificate paths. TLS 1.3, RFC 8446, describes the handshake. A trusted chain can still identify the wrong hostname, and a matching hostname does not make an unknown issuer trusted.
First record the proxy URL scheme, target hostname, curl version, and exact sanitized error. Use curl's verbose diagnostics only in a private terminal, because connection traces may contain sensitive endpoint details. Redact them before sharing.
This example assumes an HTTPS proxy and two existing Privacy-Enhanced Mail (PEM) files: one CA bundle for the target, one for the proxy. Set PROXY_URL without credentials, TARGET_URL to a permitted HTTPS URL, and the file variables to reviewed CA bundles. If both connections use the normal system store, remove the two CA options and their variables.
The command sends CONNECT through TLS to the proxy, then validates the destination certificate inside the tunnel. In a local loopback test with separate trusted CA bundles, it returned target=200 connect=200; using the target CA for the proxy returned target=000 connect=000, while using the proxy CA for the target returned target=000 connect=200. Those two exit-60 failures locate the rejected peer in this fixture only; they are not measurements of a provider route.
set -eu
: "${PROXY_URL:?Set PROXY_URL to an HTTPS proxy without credentials}"
: "${TARGET_URL:?Set TARGET_URL to one permitted HTTPS URL}"
: "${TARGET_CA_FILE:?Set TARGET_CA_FILE to a reviewed destination CA bundle}"
: "${PROXY_CA_FILE:?Set PROXY_CA_FILE to a reviewed proxy CA bundle}"
curl --disable --proxy "$PROXY_URL" --noproxy '' \
--connect-timeout 5 --max-time 20 \
--cacert "$TARGET_CA_FILE" \
--proxy-cacert "$PROXY_CA_FILE" \
--silent --show-error --output /dev/null \
--write-out 'target=%{response_code} connect=%{http_connect}\n' \
"$TARGET_URL"
curl verifies certificates by default. --cacert supplies the CA source for the destination connection. --proxy-cacert is the separate trust option for an HTTPS proxy. Use it only when the proxy URL is HTTPS and its trust differs from the target's. If the endpoint name is wrong, use the intended DNS name; do not replace a name check with a CA bundle that merely makes the chain validate.
This curl example is specific to the HTTPS proxy scheme. For an HTTP proxy, remove proxy-certificate options and diagnose the destination TLS connection after CONNECT. For SOCKS5, the client still verifies the HTTPS destination, but curl's HTTP CONNECT status does not apply.
Python's ssl.create_default_context() creates a client context with certificate validation and hostname checking enabled. Save the snippet as inspect_tls_context.py, then run python3 inspect_tls_context.py. The paths in default_verify_paths depend on the Python build and host; the sample output below records one local macOS run. This inspection does not make a proxy request or reveal which CA bundle another HTTP library uses.
import ssl
context = ssl.create_default_context()
print({
"verify_mode": context.verify_mode.name,
"check_hostname": context.check_hostname,
"default_verify_paths": ssl.get_default_verify_paths(),
})
Representative output (the CA paths are specific to one local Python installation):
{'verify_mode': 'CERT_REQUIRED', 'check_hostname': True, 'default_verify_paths': DefaultVerifyPaths(cafile='/Library/Frameworks/Python.framework/Versions/3.12/etc/openssl/cert.pem', capath=None, openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/Library/Frameworks/Python.framework/Versions/3.12/etc/openssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/Library/Frameworks/Python.framework/Versions/3.12/etc/openssl/certs')}
If the application uses a custom CA, add it to the specific client's supported trust configuration and document who owns and rotates it. Keep the application adapter and proxy URL in the reproduction. A successful direct request does not prove that the proxied route trusts both TLS peers.
| Evidence | Possible cause | Safe check |
|---|---|---|
| Unknown issuer or missing local issuer | The selected CA source cannot build a trusted chain | Check the intended CA bundle and required intermediate certificates |
| Hostname mismatch | The requested DNS name is absent from the certificate identity | Check the proxy endpoint name separately from the target URL |
| Expired or not-yet-valid certificate | Certificate validity window or local clock | Check system time and the endpoint certificate lifecycle |
| HTTPS proxy TLS fails before CONNECT | Proxy trust or hostname | Check the HTTPS proxy URL, proxy CA source, and curl backend support |
| Direct succeeds, proxied request fails | Proxy route changes the TLS peer or trust path | Compare scheme, certificate peer, and CA source at each leg |
TLS succeeds, then 407 or 403 arrives |
Authentication or target policy | Follow the HTTP layer that returned the status |
Warning: Disabling verification with --insecure, --proxy-insecure, -k, or a library equivalent removes the peer-identity check and can expose data to an impostor. Keep the test limited to the intended endpoint, restore the reviewed CA or hostname, and rerun with verification enabled.
Save the client and version, route scheme, endpoint name, failing phase, CA source, and redacted error. The proxy timeout guide helps locate a failure that occurs before any HTTP status; the 407 guide covers authentication challenges.
Scale your workflows with the tools you already use. Rotating or sticky sessions up to 72 hours.
Compare seller APIs, licensed Amazon product data and proxy page checks by record, access scope and reuse rights.
Read guideChoose the right anti-detect browser proxy guide, map the shared verification steps, and keep account access separate from the network route.
Read guideOur picks for market research: ProxyLane for small, intermittent projects, plus options for regional tracking, monthly collection and managed extraction.
Read guide