Go HTTP Proxy: Configure net/http with Safe Auth and Bounded Requests

Go's net/http already knows how to send an HTTP or HTTPS request through a proxy. The part that needs design is the boundary around it: which proxy is selected, where credentials live, how much time and body data one request may consume, and what counts as an accepted result.

The example below makes a bounded, idempotent GET to https://example.com/. Replace the target and marker with an endpoint you are allowed to access and a field your worker must accept. It reports only a status and body size. The proxy URL and password stay in memory and never enter a log or command argument.

Use an explicit http.Transport

http.ProxyURL creates a proxy function for a fixed URL. The transport uses that function for HTTP requests and for an HTTPS target reached through CONNECT. Go also supports proxy selection from HTTP_PROXY, HTTPS_PROXY, and NO_PROXY through ProxyFromEnvironment, but an explicit URL makes this one request's route visible in code.

The baseline uses only the standard library and assumes Go 1.23 or later. It expects these environment bindings:

PROXY_URL=http://proxy.example:3128
PROXY_USER=account-name             # optional
PROXY_PASSWORD=bound-secret         # optional

Keep the URL scheme, host, and port in PROXY_URL; provide credentials through the secret store or process environment. Do not put user:password@ in a shared configuration value.

Save this as main.go:

package main

import (
	"context"
	"fmt"
	"io"
	"net"
	"net/http"
	"net/url"
	"os"
	"strings"
	"time"
)

const maxBodyBytes = int64(1 << 20)

func requiredEnv(name string) string {
	value := strings.TrimSpace(os.Getenv(name))
	if value == "" {
		panic("missing required environment binding: " + name)
	}
	return value
}

func readBody(response *http.Response) (string, error) {
	defer response.Body.Close()

	body, err := io.ReadAll(io.LimitReader(response.Body, maxBodyBytes+1))
	if err != nil {
		return "", err
	}
	if int64(len(body)) > maxBodyBytes {
		return "", fmt.Errorf("response body exceeded local cap")
	}
	return string(body), nil
}

func main() {
	proxyURL, err := url.Parse(requiredEnv("PROXY_URL"))
	if err != nil || proxyURL.Host == "" || proxyURL.User != nil || (proxyURL.Scheme != "http" && proxyURL.Scheme != "https") {
		panic("PROXY_URL must use an HTTP or HTTPS scheme, host, and no credentials")
	}

	proxyUser, hasUser := os.LookupEnv("PROXY_USER")
	proxyPassword, hasPassword := os.LookupEnv("PROXY_PASSWORD")
	if hasUser != hasPassword {
		panic("PROXY_USER and PROXY_PASSWORD must be supplied together")
	}
	if hasUser {
		proxyURL.User = url.UserPassword(proxyUser, proxyPassword)
	}

	transport := &http.Transport{
		Proxy:                 http.ProxyURL(proxyURL),
		DialContext:           (&net.Dialer{Timeout: 5 * time.Second}).DialContext,
		TLSHandshakeTimeout:   10 * time.Second,
		ResponseHeaderTimeout: 10 * time.Second,
		MaxIdleConns:          8,
		MaxIdleConnsPerHost:   4,
	}
	defer transport.CloseIdleConnections()

	client := &http.Client{
		Transport: transport,
		Timeout:   15 * time.Second,
		CheckRedirect: func(_ *http.Request, _ []*http.Request) error {
			return http.ErrUseLastResponse
		},
	}

	request, err := http.NewRequestWithContext(
		context.Background(),
		http.MethodGet,
		"https://example.com/",
		nil,
	)
	if err != nil {
		panic("request configuration failed")
	}

	response, err := client.Do(request)
	if err != nil {
		panic("request failed in transport or proxy negotiation")
	}

	status := response.StatusCode
	if status == http.StatusProxyAuthRequired {
		response.Body.Close()
		panic("proxy authentication or CONNECT negotiation returned HTTP 407")
	}
	body, bodyErr := readBody(response)
	if bodyErr != nil {
		panic("response body failed local acceptance checks")
	}
	if status != http.StatusOK {
		panic("target returned a non-accepted HTTP status")
	}
	if !strings.Contains(body, "Example Domain") {
		panic("target marker was absent")
	}

	fmt.Printf("status=%d body_bytes=%d\n", status, len(body))
}

The Client.Timeout covers connection, redirects, and body reading. The transport timeouts give the failure a more useful phase, while CloseIdleConnections releases pooled sockets when this short-lived program exits. In a worker, reuse one configured client and transport for a bounded unit of work instead of creating a transport per request.

This core example uses one attempt. If a worker adds retries for an idempotent GET, classify the failure first, honor a valid Retry-After value, and keep a job-level deadline. Transport, TLS, configuration, and proxy-authentication errors should stop immediately because repeating them does not repair the route. A 407 is returned by the proxy authentication boundary, so changing the secret or endpoint is the next action. Do not copy a retry policy to a POST or another operation with side effects.

Accept only a useful result

The example accepts one result only when all three checks pass:

input:  permitted target URL + expected marker + explicit proxy binding
output: status=200, marker present, body <= 1 MiB

status=200 is an HTTP result, not proof that the record your application needs is present. The marker is intentionally simple; replace it with the field, schema, or checksum that makes the result useful to your job. The body cap is an application bound, not a proxy-account quota.

For a wider protocol comparison, see HTTP versus SOCKS5 proxies. The curl proxy guide is useful for a hop-by-hop terminal probe, and HTTP 407 troubleshooting explains why a CONNECT challenge can surface as a Go transport error instead of a response object.

ProxyLane's public residential offer starts from $2.50/GB with HTTP and SOCKS5 access, country, city, ISP, rotating and sticky options, and traffic that does not expire. Create a free account, then choose a traffic package separately; no payment is required at signup. Those are route and commercial inputs, not a Go benchmark or a target acceptance promise. When the target and acceptance rule are authorized, Sign up.