# Wget Proxy: Use WGETRC Safely and Verify the Download

[All guides](https://proxylane.dev/blog) 

DEVELOPER GUIDES · 6 MIN READ

Configure GNU Wget with an explicit proxy, keep credentials in a protected temporary file, limit attempts, and inspect response phases.

**Founder, ProxyLane** Published September 18, 2026

On this page [Put proxy settings in a protected Wgetrc](https://proxylane.dev/blog/wget-proxy#put-proxy-settings-in-a-protected-wgetrc)  [Use --spider as a separate header check](https://proxylane.dev/blog/wget-proxy#use-spider-as-a-separate-header-check)  [Read the failure at the right hop](https://proxylane.dev/blog/wget-proxy#read-the-failure-at-the-right-hop)

**Residential proxies from $2.50/GB**

Non-expiring traffic, location targeting and rotating or sticky sessions for your existing tools.

 [Sign up](https://proxylane.dev/register?interest=proxies)

Wget can use a proxy with one environment variable, but a production check needs to make more visible decisions: which proxy settings win, where authentication is stored, how many attempts are allowed, how redirects are bounded, and whether the downloaded file is the expected resource.

 

The script below uses a temporary `WGETRC` file with mode `0600`, makes one authorized HTTPS fetch, captures server headers for classification, checks the downloaded file against an application size rule, and deletes its temporary files on exit. It assumes GNU Wget 1.21 or later and uses settings documented in the current GNU Wget 1.25.0 manual. Replace the target and marker with a resource you are allowed to access.

 

## Put proxy settings in a protected Wgetrc

 

GNU Wget documents `http_proxy`, `https_proxy`, and `no_proxy` for environment-driven routing. A Wgetrc file can override those values and can hold `proxy_user`, `proxy_password`, `timeout`, `tries`, and `waitretry`. `WGETRC` selects a specific startup file, which lets a job create a short-lived file without changing a developer's home configuration.

 

Keep the endpoint and credentials in separate environment bindings:

 

```text
PROXY_URL=http://proxy.example:3128
PROXY_USER=account-name             # optional
PROXY_PASSWORD=bound-secret         # optional
NO_PROXY=localhost,127.0.0.1         # optional
TARGET_URL=https://example.com/      # optional
```

 

Save this as `fetch-through-proxy.sh`:

 

```sh
#!/bin/sh
set -eu

: "${PROXY_URL:?Set PROXY_URL to an HTTP proxy URL without credentials}"
line_feed=$(printf '\nX')
line_feed=${line_feed%X}
carriage_return=$(printf '\rX')
carriage_return=${carriage_return%X}
check_single_line() {
  case "$1" in
    *"$line_feed"*|*"$carriage_return"*) printf '%s\n' 'proxy settings must not contain newlines' >&2; exit 2 ;;
  esac
}
check_single_line "$PROXY_URL"
check_single_line "${PROXY_USER:-}"
check_single_line "${PROXY_PASSWORD:-}"
check_single_line "${NO_PROXY:-}"
case "$PROXY_URL" in
  http://*@*) printf '%s\n' 'PROXY_URL must not contain userinfo' >&2; exit 2 ;;
  http://*) ;;
  *) printf '%s\n' 'PROXY_URL must be an HTTP proxy URL' >&2; exit 2 ;;
esac

target_url=${TARGET_URL:-https://example.com/}
output_file=${OUTPUT_FILE:-proxy-check.out}
check_single_line "$target_url"
check_single_line "$output_file"
umask 077
if [ -n "${PROXY_USER:-}" ] || [ -n "${PROXY_PASSWORD:-}" ]; then
  if [ -z "${PROXY_USER:-}" ] || [ -z "${PROXY_PASSWORD:-}" ]; then
    printf '%s\n' 'PROXY_USER and PROXY_PASSWORD must be supplied together' >&2
    exit 2
  fi
fi
config_file=$(mktemp "${TMPDIR:-/tmp}/wgetrc.XXXXXX")
trace_file=$(mktemp "${TMPDIR:-/tmp}/wget-trace.XXXXXX")
cleanup() {
  rm -f "$config_file" "$trace_file"
}
trap cleanup EXIT HUP INT TERM

{
  printf 'http_proxy = %s\n' "$PROXY_URL"
  printf 'https_proxy = %s\n' "$PROXY_URL"
  printf 'no_proxy = %s\n' "${NO_PROXY:-}"
  if [ -n "${PROXY_USER:-}" ]; then
    printf 'proxy_user = %s\n' "$PROXY_USER"
  fi
  if [ -n "${PROXY_PASSWORD:-}" ]; then
    printf 'proxy_password = %s\n' "$PROXY_PASSWORD"
  fi
  printf '%s\n' 'use_proxy = on' 'timeout = 15' 'tries = 1' 'waitretry = 0'
} > "$config_file"

if ! WGETRC="$config_file" wget \
  --server-response \
  --max-redirect=5 \
  --output-document="$output_file" \
  -- "$target_url" 2>"$trace_file"; then
  if grep -Eq 'HTTP/[0-9.]+ 407' "$trace_file"; then
    printf '%s\n' 'request failed at proxy authentication or CONNECT negotiation' >&2
  else
    printf '%s\n' 'request failed before an accepted target response' >&2
  fi
  exit 1
fi

if grep -Eq 'HTTP/[0-9.]+ 407' "$trace_file"; then
  printf '%s\n' 'proxy authentication or CONNECT negotiation returned HTTP 407' >&2
  exit 1
fi
if grep -Eq 'HTTP/[0-9.]+ (401|403)' "$trace_file"; then
  printf '%s\n' 'target returned an authentication or permission response' >&2
  exit 1
fi
if ! grep -q 'Example Domain' "$output_file"; then
  printf '%s\n' 'target marker was absent' >&2
  exit 1
fi

body_bytes=$(wc -c < "$output_file" | tr -d '[:space:]')
if [ "$body_bytes" -gt 1048576 ]; then
  printf '%s\n' 'download exceeded the local body cap' >&2
  exit 1
fi
printf 'status=accepted body_bytes=%s\n' "$body_bytes"
```

 

Run it after making the script executable:

 

```sh
chmod 700 fetch-through-proxy.sh
./fetch-through-proxy.sh
```

 

The temporary config is created with `umask 077` and removed by the trap. The single-line checks prevent a secret or endpoint value from injecting another Wgetrc directive. The password is never placed in a command argument or printed. `tries = 1` is intentional: Wget cannot know from a generic exit whether the failure was a proxy-auth challenge, a CONNECT failure, or an origin error, so the script classifies the trace before anyone decides whether a repeat is safe. If a job adds retries, make them an explicit policy for an idempotent operation, honor `Retry-After` where the target provides it, and never repeat a rejected proxy secret.

 

The final byte check is an acceptance rule applied after Wget writes the file; GNU Wget's quota does not cap a single-file download, so this script does not present it as a hard transfer limit. If a hard wall-clock deadline is required, wrap the command in a supervisor supplied by your runtime. `timeout = 15` bounds applicable network operations, not the full job lifetime. `--max-redirect=5` prevents an unbounded redirect chain. `--server-response` captures headers for a short-lived local trace; do not publish that trace if the target URL or surrounding environment is sensitive.

 

## Use `--spider` as a separate header check

 

When you need to test route and response headers without saving a body, use the same protected config with Wget's spider mode:

 

```sh
WGETRC="$config_file" wget --spider --server-response --max-redirect=5 "$target_url"
```

 

Run that command inside the script before the download if your workflow needs a header-only preflight, then keep the download's marker and byte checks. `--spider` is not proof that a later fetch will return the same content; it is a separate request with its own target and policy.

 

## Read the failure at the right hop

 

| Observation | Boundary | Next decision |
| --- | --- | --- |
| Wgetrc cannot be read or proxy URL is malformed | Local configuration | Check the temporary file, scheme, host, and port without printing credentials |
| 407 in server-response trace | Proxy authentication or CONNECT | Repair proxy credentials or entitlement; do not treat it as target login |
| 401 or 403 after a successful route | Target policy | Check target credentials, permission, or target-side rules |
| No HTTP status, DNS error, or timeout | Client to proxy/network | Check endpoint reachability and protocol; keep `tries` bounded |
| 200 but marker is absent | Application validation | Reject the file and inspect redirects, content, or parser assumptions |
| File exceeds 1 MiB | Application acceptance policy | Reject the file or use a downloader with a hard transfer cap |

 

The  [curl proxy guide](https://proxylane.dev/blog/curl-proxy)  shows a compact CONNECT probe, while  [HTTP 407 troubleshooting](https://proxylane.dev/blog/proxy-error-407)  explains why a proxy challenge can appear before an origin response. For application-owned bypass rules, compare  [proxy client settings](https://proxylane.dev/blog/proxy-client-settings) .

 

ProxyLane's public residential offer starts from **$2.50/GB** with HTTP and SOCKS5 access, country, city, ISP, rotating and sticky options, and traffic that does not expire. Create a free account, then choose a traffic package separately; no payment is required at signup. These are route and commercial inputs, not a Wget benchmark or a target acceptance promise. When your target and acceptance rule are authorized,  [Sign up](https://proxylane.dev/register?interest=proxies) .

 

## Sources and further reading

- [https://www.gnu.org/software/wget/manual/html_node/Proxies.html](https://www.gnu.org/software/wget/manual/html_node/Proxies.html)

- [https://www.gnu.org/software/wget/manual/html_node/Wgetrc-Location.html](https://www.gnu.org/software/wget/manual/html_node/Wgetrc-Location.html)

- [https://www.gnu.org/software/wget/manual/html_node/Wgetrc-Commands.html](https://www.gnu.org/software/wget/manual/html_node/Wgetrc-Commands.html)

- [https://www.gnu.org/software/wget/manual/html_node/HTTP-Options.html](https://www.gnu.org/software/wget/manual/html_node/HTTP-Options.html)

- [https://www.gnu.org/software/wget/manual/html_node/Basic-Startup-Options.html](https://www.gnu.org/software/wget/manual/html_node/Basic-Startup-Options.html)

- [https://www.rfc-editor.org/rfc/rfc9110.html#section-15.5.8](https://www.rfc-editor.org/rfc/rfc9110.html#section-15.5.8)

[Sign up](https://proxylane.dev/register?interest=proxies)

## Keep reading

[Use cases · 3 min read

### Amazon Scraper API vs Proxy: Choose a Product Data Source

Distinguish authorized Amazon APIs, licensed product data and proxy-based page checks by record quality and access rights.

 Read guide](https://proxylane.dev/blog/amazon-scraper-api-vs-proxy)   [Integrations · 5 min read

### Apify Custom Proxy Setup: Use Your Own Residential Route

Connect a buyer-owned proxy to an Apify Actor, keep the session boundary clear, and validate records instead of counting requests.

 Read guide](https://proxylane.dev/blog/apify-custom-proxy)   [Proxy fundamentals · 6 min read

### Australia Residential Proxies: Verify the AU Exit and State-Level Result

Separate Australian egress from en-AU content, AUD pricing, GST display, postcode validation and the state delivery context.

 Read guide](https://proxylane.dev/blog/australia-residential-proxies)

Canonical source: https://proxylane.dev/blog/wget-proxy

Documentation index: https://proxylane.dev/llms.txt