aiohttp Proxy Guide: Async Requests with Safe Auth and Cleanup

An aiohttp proxy setting belongs to the request that needs it. That makes the route easy to reason about, but it also means you need an explicit policy for environment variables, proxy authentication, timeouts, response validation, retries, and session shutdown. A successful coroutine is not automatically a useful record.

The example below makes one idempotent GET to https://example.com/. Use a target you are permitted to access, and replace the body marker with the field your worker must accept. The code reports only a status and body size; it never prints the proxy endpoint or authentication headers.

Route a request with the current aiohttp API

The stable aiohttp guide documents proxy= for plain HTTP proxies and HTTP proxies that establish an HTTPS target through CONNECT. It also documents proxy_headers with encode_basic_auth(). The older proxy_auth parameter is deprecated since aiohttp 3.14, so new code should use the header form. Install or pin aiohttp 3.14 or newer for this example.

Inject PROXY_URL, PROXY_USER, and PROXY_PASSWORD through your worker's secret binding. They are read in memory and never passed on a command line, interpolated into source, or included in a log message:

import asyncio
import os

import aiohttp
from aiohttp import encode_basic_auth


TARGET = "https://example.com/"
PROXY_URL = os.environ["PROXY_URL"]
proxy_user = os.environ.get("PROXY_USER")
proxy_password = os.environ.get("PROXY_PASSWORD")
proxy_headers = None
if proxy_user is not None and proxy_password is not None:
    proxy_headers = {
        "Proxy-Authorization": encode_basic_auth(proxy_user, proxy_password)
    }


async def main() -> None:
    timeout = aiohttp.ClientTimeout(total=15, connect=5, sock_read=10)
    connector = aiohttp.TCPConnector(limit=8, limit_per_host=4)
    async with aiohttp.ClientSession(
        connector=connector,
        timeout=timeout,
        trust_env=False,
    ) as session:
        async with session.get(
            TARGET,
            proxy=PROXY_URL,
            proxy_headers=proxy_headers,
            allow_redirects=False,
        ) as response:
            body = await response.read()
            if response.status != 200:
                raise RuntimeError(f"target returned HTTP {response.status}")
            if b"Example Domain" not in body:
                raise RuntimeError("accepted status, but the expected marker was absent")
            print({"status": response.status, "response_body_bytes": len(body)})


try:
    asyncio.run(main())
except (aiohttp.ClientError, asyncio.TimeoutError, RuntimeError):
    raise SystemExit("request failed") from None

trust_env=False keeps HTTP_PROXY, HTTPS_PROXY, and no_proxy from changing this explicit request. aiohttp defaults to not reading environment proxy settings; trust_env=True opts into urllib.request.getproxies() and .netrc handling. Use that mode only when its precedence and credential sources are part of your deployment design. An HTTP proxy scheme is not interchangeable with a SOCKS scheme, and this API example makes no claim about SOCKS authentication.

The async with blocks matter. A ClientSession owns a cookie jar and connection pool, while the response context manager releases the response connection after its body is consumed. The aiohttp guide also describes a short event-loop delay for graceful shutdown in programs that stop immediately after closing an SSL session. Let the application lifecycle finish instead of leaving a session open per request.

Bound time and concurrency

ClientTimeout separates the total request budget from connection acquisition, socket connection, and socket read budgets. TCPConnector controls the total pool and the per-host pool. These are worker limits, not target guarantees: a remote service may continue handling a request after your client times out.

Keep one session for a bounded unit of work when cookies and pooled connections should be shared. Do not create a new session for every retry. If the worker has a hard wall-clock deadline, wrap the operation in an outer task deadline as well; the total timeout alone is scoped to an individual request.

Retry only an idempotent request

aiohttp does not turn a proxy failure into a safe retry by itself. For a GET, retry a small set of transport failures and temporary gateway statuses. A 407 is an authentication or proxy-policy result, not a reason to resend the same secret. The loop below drains a retry response before its context closes, caps the delay, and hides exception details:

RETRY_STATUSES = {502, 503, 504}
RETRY_EXCEPTIONS = (
    aiohttp.ClientConnectorError,
    aiohttp.ServerDisconnectedError,
    asyncio.TimeoutError,
)


async def get_with_retries(
    session: aiohttp.ClientSession,
    url: str,
    proxy: str,
    proxy_headers: dict[str, str] | None,
    max_attempts: int = 3,
) -> tuple[int, str]:
    for attempt in range(max_attempts):
        try:
            async with session.get(
                url,
                proxy=proxy,
                proxy_headers=proxy_headers,
                allow_redirects=False,
            ) as response:
                if response.status in RETRY_STATUSES and attempt + 1 < max_attempts:
                    retry_after = response.headers.get("Retry-After")
                    try:
                        server_delay = float(retry_after) if retry_after else 0.0
                    except ValueError:
                        await response.read()
                        return response.status, ""
                    await response.read()
                    if server_delay > 2.0:
                        return response.status, ""
                    await asyncio.sleep(min(max(server_delay, 0.5 * (2**attempt)), 2.0))
                    continue
                body = await response.text()
                return response.status, body
        except RETRY_EXCEPTIONS:
            if attempt + 1 == max_attempts:
                raise RuntimeError("request exhausted its bounded transport retries") from None
            await asyncio.sleep(min(0.5 * (2**attempt), 2.0))

    raise RuntimeError("request did not produce a response") from None

Only call this helper for a request whose method and side effects make repetition safe. Do not retry 429 with an unbounded sleep, and do not change exits to override a target's rate limit. If a service sends Retry-After, keep the delay inside the worker's job deadline or hand the item back to a scheduler that owns that deadline.

Read the failure in the right phase

For an established route, ProxyLane offers HTTP/SOCKS5 traffic from $2.50/GB with country, city, ISP, rotating, and sticky options. Traffic does not expire; registration is free and setup continues in onboarding. Payment is separate. Sign up to choose your workflow and traffic package during onboarding.

Compare the same permitted URL with the HTTPX proxy guide if your workload is synchronous or needs HTTPX's mount-based routing. In either client, count a result only after the transport phase, response status, and required fields all pass.