All guides
GUIDES & FIELD NOTES · 13 MIN READ

cURL Using Proxy: How to Route Command-Line Scraping Safely Through Residential IPs

If you want to run command-line scraping through residential IPs, cURL already has everything you need: HTTP and SOCKS5 proxies, authentication, headers, and…

cURL Using Proxy: How to Route Command-Line Scraping Safely Through Residential IPs

ProxyLane Residential Proxies Review 2026 — cURL, Pricing, Features, and Examples

If you want to run command-line scraping through residential IPs, cURL already has everything you need: HTTP and SOCKS5 proxies, authentication, headers, and diagnostics. Combined with a provider like ProxyLane, you can turn ad‑hoc terminal commands into safe, geo‑aware checks, diagnostics, and small‑batch data pulls without building a full scraper.

This guide walks through cURL using proxy step by step, shows how to use cURL with proxy authentication, gives a curl --proxy socks5h example, and then converts those calls to Python and Node for repeatable workflows. Along the way, you’ll see how ProxyLane’s pricing and features fit small-batch, evidence‑based scraping.


ProxyLane proxy service review: pricing, features, and usage model

ProxyLane is a premium residential proxy provider designed for developers and data teams who care about verified local responses and cost per successful result.

Key points for 2026 (based on the current residential proxies page):

  • Pricing model:

    • Pay‑per‑GB residential traffic (no subscriptions required).
    • Public copy mentions "from $2/GB" and a 350 MB trial for $1.95.
    • Traffic is sold in GB packages; smaller batches are common for test runs.
  • Unused traffic policy:

    • Traffic never expires — unused GB stays available for future jobs.
    • This is especially useful for teams that run irregular batches, POCs, or seasonal scraping.
  • Residential proxy features:

    • Real P2P residential IPs, currently advertised as 28M+.
    • 195 countries with city, ISP, and ZIP targeting.
    • Sticky sessions up to 72 hours for multi‑step or logged‑in flows.
    • HTTP(S) and SOCKS5 support; works with tools like curl, Python, Playwright, and n8n.
  • Session & authentication model:

    • You choose country → city → ISP in the dashboard; changing country resets city/ISP.
    • Sticky sessions keep the same home IP up to 72h while the session remains active.
    • Auth is typically via username/password or token, supplied as proxy credentials.

This combination — non‑expiring GB, granular geo targeting, and session control — makes ProxyLane well suited for command-line workflows where you:

  • Run quick geo checks and diagnostics.
  • Pull small batches for AI retrieval or enrichment.
  • Measure cost per validated result instead of just bandwidth.

Best residential proxy providers 2026: how ProxyLane compares

Many teams now use multiple providers. Apify’s 2026 state‑of‑web‑scraping report notes that 43.1% of respondents used 2–3 proxy providers and 12.1% used 4–5, while 65.8% increased proxy usage and 58.3% increased proxy budgets between 2024 and 2025.

Here’s a simplified comparison for residential proxy pricing per GB, session length, geo granularity, and usage policy.

Residential proxy providers web scraping review (comparison)

Provider Price from (per GB) Session / IP duration Geo granularity Usage policy / notes
ProxyLane From $2/GB Sticky up to 72h Country, city, ISP, ZIP Traffic never expires
SOAX ~$3.00/GB (US T1) Smart rotation, stateful Country; some region/city Credits + geo tier pricing
ProxyEmpire From $1.50/GB Rotating + static options Country, city, ZIP, ISP, carrier Mix of GB and static IP plans
HydraProxy From $2.5/GB Rotating or sticky Country‑level No monthly commitment
Proxy‑Cheap From $0.78/GB Rotating residential 195+ countries No monthly commitment
TrueProxies From $0.45/GB Time‑based unlimited options City, ASN GB and hourly unlimited models

ProxyLane is not the cheapest per GB. Its differentiator is:

  • Non‑expiring traffic (shared only with a few competitors like GonzoProxy).
  • Strong geo and ISP targeting.
  • Clear session control (rotating vs sticky) and integration docs.

For small‑batch cURL jobs that need verified location and repeatability, that mix matters more than headline price.


cURL using proxy: core patterns you’ll reuse

At the command line, almost every residential proxy workflow with curl boils down to four building blocks:

  1. Selecting the proxy and scheme (--proxy).
  2. Adding proxy authentication (--proxy-user).
  3. Choosing HTTP vs SOCKS5/socks5h for DNS behavior.
  4. Setting headers (User‑Agent, Referer, etc.).

cURL treats --proxy as an HTTP proxy by default if you don’t specify a scheme. To avoid ambiguity and make your intent explicit, it’s best to include the scheme in the URL, e.g. http://proxy.example.com:8080 or socks5h://....


What is a command line interface in this context?

A command line interface (CLI) is a text‑based way to interact with tools like curl.

For scraping and diagnostics:

  • You run single commands to fetch pages or APIs.
  • You add flags to control proxies, headers, and output.
  • You can turn those commands into scripts, cron jobs, or CI checks.

Using curl with a residential proxy network lets you test the “local web” from different cities or ISPs without building a full scraping framework.


How to use curl with proxy authentication

Proxy providers typically require credentials. With curl, you pass them using --proxy-user.

Basic HTTP proxy with authentication

curl \
  --proxy 'http://proxy.proxylane.example:8000' \
  --proxy-user 'session-username:session-password' \
  'https://target-site.example/products?city=paris' -v

Key points:

  • --proxy sets the proxy host, port, and scheme.
  • --proxy-user sends credentials to the proxy, not the origin site.
  • -v enables verbose mode so you can inspect the handshake.

If authentication fails, curl will surface a 407 Proxy Authentication Required status. That’s your primary signal that:

  • The username/password are wrong.
  • The session token expired.
  • The proxy endpoint or port doesn’t match the protocol you selected.

Security caveat: credentials in commands

Putting credentials directly in the command:

  • Makes them visible in shell history.
  • May expose them in process listings (ps, activity monitors) while the command runs.

Safer patterns include:

  • Use environment variables for secrets and reference them:

    export PROXY_USER='session-username'
    export PROXY_PASS='session-password'
    
    curl \
      --proxy 'http://proxy.proxylane.example:8000' \
      --proxy-user "$PROXY_USER:$PROXY_PASS" \
      'https://target-site.example/'
    
  • Store credentials in a protected config file (600 permissions) and source them.

  • In CI, inject secrets via the platform’s secrets manager rather than hard‑coding.

ProxyLane’s own cURL guidance also recommends avoiding plain‑text passwords in history and using secure secret injection for unattended jobs.


curl --proxy socks5h example: remote DNS and “local web” behavior

For residential proxies, the difference between local DNS and remote DNS is critical.

  • socks5:// — DNS resolved locally; only traffic goes through the proxy.
  • socks5h:// — DNS resolved by the proxy (“hostname” mode); avoids local DNS leakage.

To see what a site looks like from a specific city or ISP, you typically want socks5h:// so that both DNS and HTTP traffic originate from the residential IP.

Example: routing through a Paris residential IP with remote DNS

curl \
  --proxy 'socks5h://session-username:session-password@proxy.proxylane.example:9000' \
  'https://target-site.example/pricing' \
  -H 'User-Agent: curl-local-check/1.0' \
  -H 'Accept-Language: fr-FR,fr;q=0.9' \
  -v

Assuming your ProxyLane session is pinned to France / Paris / ISP X, this command:

  • Resolves target-site.example via the Paris residential network.
  • Sends HTTP traffic from that residential IP.
  • Adds headers to mimic a French user.

Use this pattern for:

  • Geo‑specific price checks.
  • Content localization verification.
  • Geo‑aware diagnostics (e.g., whether a feature is available in that market).

curl set header User-Agent and Referer for realistic scraping

Many sites behave differently based on headers. With curl, you can set them explicitly.

headers curl basics

Use -H or --header:

curl \
  --proxy 'http://proxy.proxylane.example:8000' \
  --proxy-user "$PROXY_USER:$PROXY_PASS" \
  'https://target-site.example/listings' \
  -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64)' \
  -H 'Referer: https://search.example/' \
  -H 'Accept: text/html,application/xhtml+xml' \
  -H 'Accept-Language: en-US,en;q=0.9' \
  -v

Tips:

  • Start with a browser‑like User‑Agent to avoid being treated as a bot.
  • Set Referer only when it’s consistent with real navigation.
  • Keep headers simple; over‑optimizing can look artificial.

Remember that curl distinguishes proxy headers from origin headers. In most residential workflows, you only need origin headers; advanced setups may use proxy‑specific headers for custom routing.


ProxyLane-style jobs: command-line scraping through residential IPs

ProxyLane emphasizes small-batch, verified-result workflows, not just bulk scraping.

Here are three common job patterns you can implement entirely with curl:

1. Quick geo/location checks

Goal: confirm that your site or partner site renders the correct content from a local perspective.

Example:

  • Run the same curl command via two sessions:
    • Session A: US / New York / ISP 1.
    • Session B: DE / Berlin / ISP 2.
  • Compare HTML snippets or JSON payloads.

2. Diagnostics for blocked or throttled traffic

Goal: determine whether your IP, region, or header set is causing issues.

Steps:

  • Use -v to see HTTP status codes and headers.
  • Try rotating residential IPs vs a sticky session.
  • Change User‑Agent or Accept‑Language to see if responses change.

3. Small-batch data pulls

Goal: fetch a limited, cost‑aware dataset for enrichment or AI retrieval.

Pattern:

  • Use curl with --proxy and headers.
  • Pipe output to a file or JSON processor (e.g., jq).
  • Run a short script to iterate over URLs, each request independent or under a sticky session.

Non‑expiring traffic means you can pre‑buy GB for these jobs and keep the remainder for later without worrying about monthly reset.


curl converter: moving from one-off commands to scripts

Once you have a working curl command, you can convert it to code.

Popular approaches:

  • Use your browser’s “Copy as cURL” and a curl converter tool to transform it.
  • Manually map flags to Python Requests, Node fetch, or other libraries.

Below are direct examples for HTTP and SOCKS proxies.


curl to Python Requests proxy (HTTP and SOCKS)

Basic HTTP(S) proxy

Suppose you start with:

curl \
  --proxy 'http://proxy.proxylane.example:8000' \
  --proxy-user 'session-username:session-password' \
  'https://target-site.example/api/items'

A Python Requests equivalent:

import requests

proxies = {
    'http': 'http://session-username:session-password@proxy.proxylane.example:8000',
    'https': 'http://session-username:session-password@proxy.proxylane.example:8000',
}

headers = {
    'User-Agent': 'python-requests/2.x via residential proxy',
}

resp = requests.get('https://target-site.example/api/items', proxies=proxies, headers=headers)
resp.raise_for_status()

print(resp.json())

Using SOCKS proxies with Python Requests

Python’s requests does not support SOCKS out of the box. You need the requests[socks] extra (PySocks) or an equivalent tunnel.

Install:

pip install 'requests[socks]'

Then use URLs like socks5h://user:pass@host:port:

import requests

proxies = {
    'http': 'socks5h://session-username:session-password@proxy.proxylane.example:9000',
    'https': 'socks5h://session-username:session-password@proxy.proxylane.example:9000',
}

resp = requests.get('https://target-site.example/pricing', proxies=proxies)
resp.raise_for_status()

print(resp.text)

If you’re concerned about credentials in code:

  • Load them from environment variables.
  • Use a secrets manager in production.
  • Avoid committing proxy URLs with credentials to version control.

curl convert to Node fetch proxy

Node’s modern fetch (via node-fetch or built‑in in newer runtimes) doesn’t handle proxies directly. You use agents instead.

Example: HTTP proxy via http-proxy-agent

First, the curl:

curl \
  --proxy 'http://session-username:session-password@proxy.proxylane.example:8000' \
  'https://target-site.example/search?q=shoes'

In Node:

npm install node-fetch http-proxy-agent
import fetch from 'node-fetch';
import { HttpProxyAgent } from 'http-proxy-agent';

const proxyUrl = 'http://session-username:session-password@proxy.proxylane.example:8000';
const agent = new HttpProxyAgent(proxyUrl);

async function run() {
  const resp = await fetch('https://target-site.example/search?q=shoes', {
    agent,
    headers: {
      'User-Agent': 'node-fetch via residential proxy',
    },
  });

  if (!resp.ok) {
    throw new Error(`HTTP ${resp.status}`);
  }

  const text = await resp.text();
  console.log(text);
}

run().catch(console.error);

Node + SOCKS5 proxies

For SOCKS, use a SOCKS agent library (e.g., socks-proxy-agent or socks5-http-client).

npm install node-fetch socks-proxy-agent
import fetch from 'node-fetch';
import { SocksProxyAgent } from 'socks-proxy-agent';

const proxyUrl = 'socks5h://session-username:session-password@proxy.proxylane.example:9000';
const agent = new SocksProxyAgent(proxyUrl);

async function run() {
  const resp = await fetch('https://target-site.example/pricing', { agent });

  if (!resp.ok) {
    throw new Error(`HTTP ${resp.status}`);
  }

  console.log(await resp.text());
}

run().catch(console.error);

Security note for Node proxy URLs

Just like with curl:

  • Embedding credentials in proxyUrl risks exposure via stack traces, logs, or inspection tools.
  • Prefer environment variables or secret management systems.
  • Consider constructing the URL at runtime rather than hard‑coding.

For example:

const proxyUrl = `socks5h://${process.env.PROXY_USER}:${process.env.PROXY_PASS}@proxy.proxylane.example:9000`;

Residential proxy pricing per GB and unused traffic never expires

For teams searching explicitly on "residential proxy pricing per GB" and "unused traffic never expires", here’s the concise picture for ProxyLane:

  • Per‑GB pricing:

    • Residential proxies are sold per gigabyte of data transfer.
    • Public materials currently reference entry pricing from $2/GB, with trial volumes around 350 MB for $1.95.
  • Unused traffic policy:

    • Purchased GB does not expire — unused traffic is kept for future runs.
    • That’s different from many providers that reset quotas monthly or attach expiration dates.

In practice, this lets you:

  • Buy bandwidth for specific jobs (e.g., a 5 GB batch) rather than ongoing subscriptions.
  • Measure cost per successful result (e.g., per validated listing or lead) across multiple jobs, reusing leftover traffic.

For always‑current numbers and tiers, check the pricing details on proxylane.dev.


Geo-targeting and cURL: country, city proxy examples

With a provider that supports country, city, and ISP targeting, you can tie a cURL command to a precise local web.

curl geo-targeting country city proxy pattern

  1. In your proxy dashboard, create a session with:

    • Country: DE (Germany).
    • City: Berlin.
    • ISP: a specific residential provider.
  2. Use the appropriate session credentials with curl:

curl \
  --proxy 'socks5h://berlin-session-user:berlin-session-pass@proxy.proxylane.example:9000' \
  'https://target-site.example/local-offers' \
  -H 'Accept-Language: de-DE,de;q=0.9' \
  -v

This way, when the target site inspects your IP and DNS origin, it sees a Berlin home IP on that ISP, and your response reflects what a local user would receive.


FAQs: cURL using proxy and residential IPs

1. Is it legal to use residential proxies with cURL for web scraping?

Generally, using residential proxies is legal in many jurisdictions if:

  • Proxy participants have opted in to the network.
  • You access public data without circumventing authentication.
  • You respect applicable laws (data protection, computer misuse) and site terms.

Context.dev’s legal framing is useful here: the legality of the proxy itself and the legality of how you use it are separate questions. Always consult your legal team for your specific use case.

2. How do I know my cURL request actually went through the residential proxy?

Use these checks:

  • Hit an IP‑echo service (https://ifconfig.me or similar) via your proxy.
  • Compare the IP and location to your proxy dashboard.
  • Use -v in curl to inspect the CONNECT and ensure your traffic is routed.

If the IP matches the residential pool and geo you configured, your cURL traffic is flowing as expected.

3. Why does cURL return 407 Proxy Authentication Required?

A 407 means the proxy rejected your credentials. Common causes:

  • Wrong username or password.
  • Using account password instead of proxy/session password.
  • Expired or disabled session.

Double‑check the proxy auth details, protocol (HTTP vs SOCKS), and endpoint in your provider’s dashboard.

4. Should I use HTTP or SOCKS5/socks5h for residential scraping with cURL?

Choose based on goals:

  • Use HTTP when you only need traffic to appear from a different IP and local DNS isn’t a concern.
  • Use socks5h when you want the proxy network to perform DNS resolution, avoiding local DNS leakage and better matching real local behavior.

For most "local web" checks, socks5h is the safer, more realistic option.

5. How can I keep proxy credentials safe in scripts and CI?

Follow these practices:

  • Do not hard‑code credentials in scripts committed to version control.
  • Use environment variables or your CI platform’s secret storage.
  • Restrict permissions on any files that hold proxy URLs or passwords.
  • Regularly rotate proxy credentials and audit where they are referenced.

By combining cURL’s proxy features with a residential provider that offers non‑expiring traffic, geo targeting, and session control, you can turn simple command-line calls into a reliable, evidence‑based way to see what your users see — and keep your scraping workflows measured and compliant.

ProxyLaneTraffic never expires

Premium residential proxies
From $2/GB

Scale your workflows with the tools you already use. Rotating or sticky sessions up to 72 hours.

Try 350 MB for $1.95 View plans

Keep reading

Amazon Scraper API vs Proxy: Choose a Product Data Source

Compare seller APIs, licensed Amazon product data and proxy page checks by record, access scope and reuse rights.

Read guide

Antidetect browser proxy setup: choose a profile guide

Choose the right anti-detect browser proxy guide, map the shared verification steps, and keep account access separate from the network route.

Read guide

Best Proxies for Market Research: Choose by Sample and Cost

Our picks for market research: ProxyLane for small, intermittent projects, plus options for regional tracking, monthly collection and managed extraction.

Read guide