Amazon Scraper API vs Proxy: Choose a Product Data Source
Distinguish authorized Amazon APIs, licensed product data and proxy-based page checks by record quality and access rights.
Read guide →Connect CloakBrowser to a residential proxy, choose session behavior by workload, verify the exit in the same browser, and clean up safely.
CloakBrowser gives a Python or JavaScript program a Playwright-compatible browser. Its documented launch() API takes the proxy, while launchPersistentContext provides the JavaScript persistent option. A residential proxy supplies the network exit. To get useful browser data, bind both in the same launch and check the exit and run a target-content smoke check before building record extraction.
Python:
python -m pip install cloakbrowser
JavaScript requires Node.js 20 or newer:
npm install cloakbrowser playwright-core
The wrapper downloads its browser binary on first launch. Set PROXY_SERVER, PROXY_USER, PROXY_PASSWORD, EXIT_CHECK_URL, EXPECTED_EXIT_MARKER, TARGET_URL and EXPECTED_TARGET_MARKER in the runtime environment. The two marker variables are required by the examples. A username and password are optional only when the purchased endpoint is allowlisted; if either is set, set both.
Residential is a sensible choice when the permitted workflow depends on a documented residential or access-network signal, or when a region-sensitive page must be checked from the purchased route. Datacenter can be the better fit for open public pages when the target accepts cloud egress and throughput or cost matters more than that network property. Neither label tells you whether the target will return the record you need.
| Workload | Start with | Keep stable or change | Acceptance check |
|---|---|---|---|
| Public pages and independent records | Datacenter, if accepted by the target | One route per independent request | Expected title, status and fields |
| Region-sensitive pages | Residential only when that property is part of the route you bought | Keep the same route through related pages | Same-browser exit and expected locale or record |
| Login, pagination or checkout-like sequence you own | Whichever route the target and provider permit | Sticky session and persistent profile | Exit remains stable and state survives the sequence |
| Independent country or market observations | Route matching each observation | New session or provider-defined rotation between items | Record exit, final URL and field-level result |
CloakBrowser does not provide proxy rotation. Rotation belongs to the proxy provider's endpoint or session parameters, and the browser must be relaunched or reconfigured according to that provider's contract. Do not rotate between a login page, pagination request and detail page when those pages share state.
The official Python wrapper accepts a proxy URL or a Playwright-style object with server, bypass, username and password. The JavaScript types expose the same shape. Build that object from the runtime environment so the password never enters page code, a committed file or a log.
import os
from urllib.parse import urlsplit
from cloakbrowser import launch
required = [
"PROXY_SERVER",
"EXIT_CHECK_URL",
"EXPECTED_EXIT_MARKER",
"TARGET_URL",
"EXPECTED_TARGET_MARKER",
]
missing = [name for name in required if not os.environ.get(name, "").strip()]
if missing:
raise RuntimeError("Missing required environment variables: " + ", ".join(missing))
if bool(os.environ.get("PROXY_USER")) != bool(os.environ.get("PROXY_PASSWORD")):
raise RuntimeError("PROXY_USER and PROXY_PASSWORD must be set together")
proxy = {"server": os.environ["PROXY_SERVER"]}
if os.environ.get("PROXY_USER"):
proxy["username"] = os.environ["PROXY_USER"]
proxy["password"] = os.environ["PROXY_PASSWORD"]
browser = launch(proxy=proxy)
try:
page = browser.new_page()
route = page.goto(
os.environ["EXIT_CHECK_URL"],
wait_until="domcontentloaded",
timeout=20_000,
)
route_body = page.locator("body").inner_text()
expected_exit = os.environ["EXPECTED_EXIT_MARKER"]
route_ok = (
route is not None
and route.status == 200
and route_body.strip() == expected_exit.strip()
)
if not route_ok:
raise RuntimeError("The same-browser exit check failed")
target = page.goto(
os.environ["TARGET_URL"],
wait_until="domcontentloaded",
timeout=20_000,
)
target_marker = os.environ["EXPECTED_TARGET_MARKER"]
target_body = page.locator("body").inner_text()
data_ok = (
target is not None
and target.status == 200
and target_marker in target_body
)
if not data_ok:
raise RuntimeError("The target response failed validation")
print({
"route_status": route.status,
"route_ok": route_ok,
"target_status": target.status,
"target_path": urlsplit(page.url).path,
"data_ok": data_ok,
})
finally:
browser.close()
Set EXIT_CHECK_URL to https://api.ipify.org, whose official documentation specifies a plain-text IPv4 response, or a trusted endpoint with that same response format. Set EXPECTED_EXIT_MARKER to the complete expected IPv4 address. Both examples compare the entire trimmed response with that value; a partial IP cannot pass. This checks the exit of that browser request. A provider's rotating endpoint may use another exit for the next request, including the target.
Set EXPECTED_TARGET_MARKER to a distinctive expected record value, such as a product's exact SKU. Generic site text is insufficient. The target check establishes only HTTP 200 and the presence of that substring in the page body. It does not extract or validate record fields; add those assertions before accepting records. Keep any retained exit evidence redacted.
For JavaScript, the launch and cleanup boundary is equivalent:
import { launch } from 'cloakbrowser';
const required = [
'PROXY_SERVER',
'EXIT_CHECK_URL',
'EXPECTED_EXIT_MARKER',
'TARGET_URL',
'EXPECTED_TARGET_MARKER',
];
const missing = required.filter((name) => !process.env[name]?.trim());
if (missing.length) {
throw new Error(`Missing required environment variables: ${missing.join(', ')}`);
}
if (Boolean(process.env.PROXY_USER) !== Boolean(process.env.PROXY_PASSWORD)) {
throw new Error('PROXY_USER and PROXY_PASSWORD must be set together');
}
const proxy = { server: process.env.PROXY_SERVER };
if (process.env.PROXY_USER) {
proxy.username = process.env.PROXY_USER;
proxy.password = process.env.PROXY_PASSWORD;
}
let browser;
try {
browser = await launch({ proxy });
const page = await browser.newPage();
const exitResponse = await page.goto(process.env.EXIT_CHECK_URL, {
waitUntil: 'domcontentloaded',
timeout: 20_000,
});
const exitBody = await page.locator('body').innerText();
const exitOk =
exitResponse?.status() === 200 &&
exitBody.trim() === process.env.EXPECTED_EXIT_MARKER.trim();
if (!exitOk) throw new Error('The same-browser exit check failed');
const response = await page.goto(process.env.TARGET_URL, {
waitUntil: 'domcontentloaded',
timeout: 20_000,
});
const targetBody = await page.locator('body').innerText();
const dataOk =
response?.status() === 200 &&
targetBody.includes(process.env.EXPECTED_TARGET_MARKER);
console.log(JSON.stringify({
exitStatus: exitResponse?.status() ?? null,
exitOk,
targetStatus: response?.status() ?? null,
targetPath: new URL(page.url()).pathname,
dataOk,
}));
if (!dataOk) throw new Error('The target response failed validation');
} finally {
if (browser) await browser.close();
}
CloakBrowser also documents inline credential URLs, but an environment-built object makes accidental disclosure harder. Its optional geoip mode can derive timezone and locale from the proxy route, so treat that lookup as part of the route check. Its current source routes SOCKS5 through Chrome's --proxy-server path. Credentialed HTTP or HTTPS uses Chrome inline authentication only when the selected binary supports it; older platform binaries can fall back to Playwright's proxy object. Keep the protocol exactly as supplied by the proxy provider. Pass raw PROXY_USER and PROXY_PASSWORD values in object fields; do not percent-encode them. Percent-encoding applies only when constructing inline credential URLs.
Use a persistent profile when cookies and local storage must survive a restart:
from cloakbrowser import launch_persistent_context
context = launch_persistent_context(
"./browser-profile",
proxy=proxy,
)
try:
page = context.pages[0] if context.pages else context.new_page()
page.goto(os.environ["TARGET_URL"])
finally:
context.close()
A persistent profile preserves browser state; it does not prove that the proxy exit is sticky. Run the same-browser exit check twice during a controlled sequence and compare the observed exit. If the provider rotates by request, keep only independent records in that session. For a stable workflow, use the provider's documented sticky-session setting and keep one profile and route through the related pages.
| Observation | Inspect first | Next action |
|---|---|---|
| Binary or import fails | Wrapper version, platform and cache | Run python -m cloakbrowser info --json or npx cloakbrowser info |
| 407 or proxy authentication error | Protocol, username, password and URL encoding | Check the provider credential pair and the selected binary's auth path |
| Exit check fails in the browser | Proxy host, port, bypass and actual egress | Repeat the check in a clean browser and compare the route record |
| Exit passes but target data is wrong | Final URL, status, locale and required field | Inspect redirects, cookies and page content separately |
| Exit changes mid-sequence | Provider session or rotation policy | Keep a sticky route for related pages or split the job |
| Browser remains after an exception | Missing close path | Put browser.close() or context.close() in finally and remove only task-owned temporary profiles |
The repository README currently identifies wrapper v0.5.10 and a Pro Stable Chromium 151.0.7922.108.6 build, while Python source exposes a bundled baseline 146.0.7680.177.5 in CHROMIUM_VERSION. Platform details include Linux arm64, and the first launch can download roughly 200 MB into the local cache. Use the CLI or binary_info() to see what will actually launch. CLOAKBROWSER_VERSION or browserVersion can pin a Chromium version; CLOAKBROWSER_BINARY_PATH can select a local binary. CLOAKBROWSER_LICENSE_KEY selects license-backed access, and CLOAKBROWSER_RELEASE_CHANNEL can select the Preview channel.
The Python and JavaScript wrappers are MIT licensed. The compiled browser binary has a separate license, and its terms call the third-party distribution permission an OEM/SaaS license for bundling, embedding or providing browser functionality to customers. The official /free path and paid plans describe different current access routes, so confirm the entitlement before packaging the binary into a service. The proxy route, browser state and target permission remain separate decisions.
For the network classification decision, see the residential versus datacenter guide. For session policy, read the rotation and sticky sessions guide. The Playwright proxy guide, browser diagnostics guide and 407 troubleshooting guide cover the next checks when the route does not behave as expected.
Distinguish authorized Amazon APIs, licensed product data and proxy-based page checks by record quality and access rights.
Read guide →Connect a buyer-owned proxy to an Apify Actor, keep the session boundary clear, and validate records instead of counting requests.
Read guide →Separate Australian egress from en-AU content, AUD pricing, GST display, postcode validation and the state delivery context.
Read guide →