# Proxies for Job Market Data: Count Vacancies, Not Job Posts

[All guides](https://proxylane.dev/blog) 

USE CASES · 6 MIN READ

Collect job postings with bounded pacing, deduplicate reposts and track closed status before drawing a labor-market conclusion.

**Founder, ProxyLane** Published September 18, 2026

On this page [Define the unit before collecting](https://proxylane.dev/blog/proxies-for-job-market-data#define-the-unit-before-collecting)  [Use publisher fields as signals](https://proxylane.dev/blog/proxies-for-job-market-data#use-publisher-fields-as-signals)  [Deduplicate with evidence, then review](https://proxylane.dev/blog/proxies-for-job-market-data#deduplicate-with-evidence-then-review)  [Track closed status as a time series](https://proxylane.dev/blog/proxies-for-job-market-data#track-closed-status-as-a-time-series)  [Describe the sample before the conclusion](https://proxylane.dev/blog/proxies-for-job-market-data#describe-the-sample-before-the-conclusion)

**Residential proxies from $2.50/GB**

Non-expiring traffic, location targeting and rotating or sticky sessions for your existing tools.

 [Request access](https://proxylane.dev/register?interest=proxies)

Job pages are easy to count and easy to misread. One vacancy can appear on an employer site, an applicant-tracking system and several boards. A stale page can still return `200`, while a new URL can be a repost of the same requisition. A proxy may help a permitted fetch reflect a target region. The output still needs unique-vacancy logic and a sample-bias note because posting rows do not describe the whole labor market. Treat sample bias as part of the output, not a footnote.

 

## Define the unit before collecting

 

Start with the decision: compare hiring activity for named employers, map role demand in a region or monitor a set of public career pages. Freeze the employers, role vocabulary, locations, source types and observation dates. Check official feeds or APIs and the source terms first. Read the  [robots rules](https://www.rfc-editor.org/rfc/rfc9309.html) , keep request pacing bounded and pause when a target asks you to slow down.

 

| Input | Action | Accepted output |
| --- | --- | --- |
| Employer, role and location frame | Collect a permitted single-job page and its visible fields | A dated posting record with source URL |
| Multiple URLs for one apparent role | Normalize title, employer, location and source signals | A provisional vacancy group with merge evidence |
| Recheck date and page status | Revisit the same URL or canonical page | `open`, `closed`, `uncertain` or `not_checked` with evidence |

 

Keep `posting_id` and `vacancy_id` separate. The first identifies a page observation. The second is a hypothesis that several observations refer to one hiring request. If the evidence is weak, leave the group provisional instead of presenting a guessed unique vacancy count.

 

## Use publisher fields as signals

 

Google's  [JobPosting documentation](https://developers.google.com/search/docs/appearance/structured-data/job-posting)  says structured data belongs on a page dedicated to a single job posting, not a search or list page. It documents fields such as `datePosted` and `validThrough`, and recommends canonical job URLs and accurate change timestamps.  [Schema.org's JobPosting type](https://schema.org/JobPosting)  provides a vocabulary for job-related fields.

 

These fields are useful observations, not a unique-vacancy key. Store the visible title, employer name, location or remote requirement, employment type, `datePosted`, `validThrough`, application URL, canonical URL, retrieval time and the structured-data version when present. Compare those fields with the page text. A page's markup can be missing, stale or copied across a board.

 

## Deduplicate with evidence, then review

 

Create a candidate key from normalized employer, title, location, employment type and a time window. Add a requisition ID, canonical URL or application URL when the publisher exposes one. A title alone is too weak: “Data Analyst” can describe several teams, locations and levels. A URL alone is also weak when a board regenerates links during a repost.

 

Use the merge result as a review queue:

 

| Observation | Suggested state | Why |
| --- | --- | --- |
| Same requisition ID and application URL | `same_vacancy` | Strong publisher identifier |
| Same employer, title and location; close dates | `likely_duplicate` | Useful hypothesis, still reviewable |
| Same title but different location or team | `separate_or_unknown` | Do not collapse by title |
| New URL with older text and no new identifier | `repost_candidate` | Preserve both posting observations |

 

Here is a hypothetical batch, included to show the denominator rather than report a market result: 120 posting URLs produce 86 candidate groups after normalization. Review accepts 73 groups for the stated frame and leaves 13 unresolved. A row-level audit also flags 19 duplicate links, eight repost candidates and seven ambiguous groups; those flags can overlap, so they are diagnostics rather than extra rows to subtract. The 73 groups are operational vacancy groups, not a headcount: one requisition can fill several seats, and one posting can describe a role family. The example is synthetic and says nothing about the size of a real market.

 

## Track closed status as a time series

 

An absent `validThrough` date does not prove that a role is open. Record explicit “no longer accepting applications” text, a closed application form, a redirect to a careers index, `404` or `410`, and the last date on which a valid job description was observed. Google's guidance says expired job postings should not remain presented as open; its documented handling includes current `validThrough` data or removing the expired posting. Your collector still needs its own status field and recheck schedule.

 

Store `first_seen`, `last_seen`, `status_checked_at` and `status_evidence`. Keep an older valid observation when a temporary fetch fails, but mark its age. A `200` page containing a login wall or a list of other jobs is not an open vacancy record. The  [cost-per-successful-request guide](https://proxylane.dev/blog/proxy-cost-per-successful-request)  shows why a successful transport response is different from an accepted business record.

 

## Describe the sample before the conclusion

 

A board-only sample favors employers that publish there. A career-site sample favors companies with discoverable pages. Search results favor indexed and crawlable postings. Geography, language, remote eligibility, salary disclosure, duplicate policy and observation time create more selection effects. Report the employer list, source mix, role and location filters, collection window, exclusions, missing fields and unique-vacancy rule before saying that demand rose or fell.

 

If a response is `429`,  [RFC 6585](https://www.rfc-editor.org/rfc/rfc6585.html)  defines it as “Too Many Requests” and allows a `Retry-After` field. Honor that signal across the target schedule. Switching exits to defeat a target limit corrupts the observation plan and can create a more biased sample. A proxy belongs only where the permitted fetch needs a different route or regional presentation check.

 

Use the  [market research guide](https://proxylane.dev/blog/proxies-for-market-research)  for sampling frames and named exclusions. The  [RAG crawling guide](https://proxylane.dev/blog/rag-web-crawling-proxies)  shows how to retain URL and freshness evidence when the job inventory is refreshed. The  [permission checklist](https://proxylane.dev/blog/web-scraping-permission-checklist)  covers source terms, robots and personal-data questions; legal conclusions remain source- and jurisdiction-specific.

 

ProxyLane offers HTTP and SOCKS5 residential access with rotating or sticky sessions, location targeting and non-expiring traffic. Plans start from $2.50/GB.  [Request access](https://proxylane.dev/register?interest=proxies)  after defining the source set, pacing rule and accepted vacancy record.

 

## Sources and further reading

- [https://developers.google.com/search/docs/appearance/structured-data/job-posting](https://developers.google.com/search/docs/appearance/structured-data/job-posting)

- [https://schema.org/JobPosting](https://schema.org/JobPosting)

- [https://www.rfc-editor.org/rfc/rfc9309.html](https://www.rfc-editor.org/rfc/rfc9309.html)

- [https://www.rfc-editor.org/rfc/rfc6585.html](https://www.rfc-editor.org/rfc/rfc6585.html)

[Request access](https://proxylane.dev/register?interest=proxies)

## Keep reading

[Use cases · 3 min read

### Amazon Scraper API vs Proxy: Choose a Product Data Source

Distinguish authorized Amazon APIs, licensed product data and proxy-based page checks by record quality and access rights.

 Read guide](https://proxylane.dev/blog/amazon-scraper-api-vs-proxy)   [Integrations · 5 min read

### Apify Custom Proxy Setup: Use Your Own Residential Route

Connect a buyer-owned proxy to an Apify Actor, keep the session boundary clear, and validate records instead of counting requests.

 Read guide](https://proxylane.dev/blog/apify-custom-proxy)   [Proxy fundamentals · 6 min read

### Australia Residential Proxies: Verify the AU Exit and State-Level Result

Separate Australian egress from en-AU content, AUD pricing, GST display, postcode validation and the state delivery context.

 Read guide](https://proxylane.dev/blog/australia-residential-proxies)

Canonical source: https://proxylane.dev/blog/proxies-for-job-market-data

Documentation index: https://proxylane.dev/llms.txt