Amazon Scraper API vs Proxy: Choose a Product Data Source
Distinguish authorized Amazon APIs, licensed product data and proxy-based page checks by record quality and access rights.
Read guide →Route Patchright Chromium sessions through an authenticated proxy, preserve session state, and validate the response and page content.
Patchright Python is a patched Chromium-focused automation runtime with a Playwright-shaped API. The proxy is configured at browser launch, while the browser context carries cookies and the page validates the result. That division matters when a flow includes login, pagination or a region-specific page.
The guide uses Patchright’s async Python package. Patchright only patches Chromium based browsers, so the recipe does not imply Firefox or WebKit support. It also does not make a proxy or a browser universally accepted by a target.
python -m pip install -U patchright
patchright install chromium
Set PROXY_SERVER, PROXY_USERNAME, PROXY_PASSWORD, TARGET_URL and EXPECTED_MARKER in the worker’s secret environment. Patchright passes the proxy object through the Playwright launch contract. Keep the URL and credentials out of logs.
import asyncio
import json
import os
from patchright.async_api import async_playwright
REQUIRED = (
"PROXY_SERVER",
"PROXY_USERNAME",
"PROXY_PASSWORD",
"TARGET_URL",
"EXPECTED_MARKER",
)
async def main() -> None:
missing = [name for name in REQUIRED if not os.environ.get(name)]
if missing:
raise SystemExit(f"Missing required environment variables: {', '.join(missing)}")
async with async_playwright() as playwright:
browser = await playwright.chromium.launch(
proxy={
"server": os.environ["PROXY_SERVER"],
"username": os.environ["PROXY_USERNAME"],
"password": os.environ["PROXY_PASSWORD"],
},
headless=True,
)
try:
page = await browser.new_page()
ip_response = await page.goto(
"https://api.ipify.org?format=json",
wait_until="domcontentloaded",
timeout=25_000,
)
if ip_response is None or not ip_response.ok:
raise RuntimeError("Exit-IP navigation did not return a successful response")
exit_ip = json.loads(await page.locator("body").inner_text(timeout=5_000)).get("ip")
if not exit_ip:
raise RuntimeError("The same browser returned no exit-IP body")
target_response = await page.goto(
os.environ["TARGET_URL"],
wait_until="domcontentloaded",
timeout=25_000,
)
if target_response is None or not target_response.ok:
status = target_response.status if target_response else "no response"
raise RuntimeError(f"Target navigation failed with status {status}")
body = await page.locator("body").inner_text()
if os.environ["EXPECTED_MARKER"] not in body:
raise RuntimeError("Target marker was not found in the received page")
print({"exit_ip_present": True, "target_status": target_response.status, "content_valid": True})
finally:
await browser.close()
asyncio.run(main())
The response object from page.goto() makes HTTP status available, but a successful status is only a transport signal. The marker is the application-level check. The explicit 25-second navigation bound and browser.close() keep a failed route from consuming an unbounded worker.
Use the residential versus datacenter comparison to choose by workload and measured results. For multi-step flows, the sticky-session guide explains how provider affinity differs from browser cookies.
| Workload | Proxy choice | Session policy | Acceptance check |
|---|---|---|---|
| High-volume internal QA | Datacenter | New context per independent case | Status and expected marker |
| Geo-sensitive catalog or ad check | Residential | One context and sticky route per flow | Page region and marker agree |
| Login plus several browser steps | Residential or static ISP where allowed | Reuse one context and provider session | Cookies survive every step |
| Separate regions in parallel | One route per context | Isolate contexts and profiles | Each context reports its own route |
Patchright’s README recommends Chrome with a persistent context, headless=False, no_viewport=True and no custom browser headers or user agent for its documented setup. Adopt those choices only when they fit your workload; do not add a persistent profile merely to keep cookies if the job needs isolation.
Issue #112 reports a specific Python path: an add_init_script call activates the lazy install_inject_route implementation, whose missing patchrightInitScript flag breaks authenticated-proxy navigation. Avoid that path in this baseline or isolate it in a reproduction; check your installed Patchright version and the issue’s fix status before combining init scripts with proxy authentication. This report does not establish that every version or ordinary proxied navigation is affected.
| Symptom | Likely layer | Next check |
|---|---|---|
| Browser install or launch fails | Runtime | Re-run the package and Chromium install in the same environment |
| Proxy returns 407 | Credentials or endpoint | Check the launch proxy fields and provider auth scheme |
Authenticated navigation breaks after add_init_script |
Patchright route edge case | Isolate install_inject_route and follow the documented issue before adding that path |
| Status is good but marker is missing | Target or parser | Inspect the received page, redirects and selector assumptions |
| Flow loses login state | Session policy | Reuse one context and request a sticky provider session |
| Browser survives an exception | Cleanup | Keep browser.close() in finally and cap the worker deadline |
Patchright changes browser runtime behavior, but it does not replace permission checks, parsing or proxy monitoring. Count a useful run only after the status, marker and any business fields required by your job pass together.
Distinguish authorized Amazon APIs, licensed product data and proxy-based page checks by record quality and access rights.
Read guide →Connect a buyer-owned proxy to an Apify Actor, keep the session boundary clear, and validate records instead of counting requests.
Read guide →Separate Australian egress from en-AU content, AUD pricing, GST display, postcode validation and the state delivery context.
Read guide →