Give your research agent source-linked records it can check

{
    "source_url": "https://example.com/retrieval-guide",
    "retrieved_at": "2026-09-17T10:00:00Z",
    "text": "Synthetic guidance: keep retrieval time separate from page publication time.",
    "content_hash": "sha256:b6386f800e0d671f16feecb72d366f00a7d9fe52e6325f14389f5736bbfc31eb",
    "fetch_status": "fetched",
    "published_at": "2026-08-01"
}

Fit this workflow to your retrieval job

This package fits a developer building a research agent, RAG ingestion step, or source-monitoring workflow that needs each retrieved passage tied to a URL and fetch time. The proposed record keeps source_url, retrieved_at, text, content_hash, and fetch_status together. That gives your downstream code something concrete to inspect before it uses text.

It also fits teams that need failed pages and changed content to remain visible. A fetched page can still be wrong or outdated. The pilot can focus on the record contract and review path around your existing agent.

If you need a managed API or committed production coverage today, this page does not make that commitment. ProxyLane's current public offer is a pilot scoping conversation; feasibility, price, and acceptance criteria are agreed with a human before any work.

What you can do now

Download the synthetic sample JSON or the sample CSV, rename the downloaded ai-retrieval-sample.json to sample.json in your working folder, then run the local check below. The sample is a schema example, not measured output. It uses only example.com and example.org URLs and fictional text.

Use the Request access button to open an email draft with your vertical, requested source list, required fields, geography, refresh interval, output format, expected volume, and acceptance rules. You review and send it yourself.

Record fields and acceptance checks

The fields and rules remain proposed until your pilot worksheet is agreed. Passing these checks does not prove answer correctness; route changed, ambiguous, or failed pages to human review.

Try the sample locally

Put the renamed sample.json in your working folder, then run the standard-library script below from that directory. It prints an outcome for each record and treats missing values as review cases instead of guessing.

import json
import hashlib
from datetime import datetime
from pathlib import Path
from urllib.parse import urlparse

sample = json.loads(Path("sample.json").read_text(encoding="utf-8"))
assert sample.get("sample_type") == "synthetic", "Expected a synthetic schema example"
records = sample.get("records")
assert isinstance(records, list) and records, "Expected at least one record"

for number, record in enumerate(records, start=1):
    if not isinstance(record, dict):
        print(f"record {number}: review")
        print("  reason: record is not an object")
        continue
    required = ("source_url", "retrieved_at", "text", "content_hash", "fetch_status")
    missing = [field for field in required if not record.get(field)]
    parsed_url = urlparse(record.get("source_url", ""))
    host = parsed_url.hostname
    timestamp_ok = False
    try:
        timestamp = (record.get("retrieved_at") or "").replace("Z", "+00:00")
        timestamp_ok = bool(timestamp) and datetime.fromisoformat(timestamp).tzinfo is not None
    except (AttributeError, ValueError):
        pass
    hash_ok = bool(record.get("text")) and record.get("content_hash") == "sha256:" + hashlib.sha256(record["text"].encode("utf-8")).hexdigest()
    accepted = (
        not missing
        and parsed_url.scheme == "https"
        and host in {"example.com", "example.org"}
        and timestamp_ok
        and hash_ok
        and record["fetch_status"] == "fetched"
    )
    outcome = "accepted" if accepted else "review"
    print(f"record {number}: {outcome}")
    if missing:
        print("  missing:", ", ".join(missing))
    if record.get("fetch_status") != "fetched":
        print("  fetch_status:", record.get("fetch_status", "unknown"))

The check deliberately accepts only HTTPS synthetic hosts, timezone-aware retrieval timestamps, a non-empty record, matching SHA-256 text, and the fetched status. “Accepted” here means the local schema checks passed; it is not a production or answer-quality result. Extend the checks with your approved source list and human review conditions during scoping. The script makes no ProxyLane API call.

Define the data requirements

A pilot worksheet can cover one source list, the fields above, geography, refresh interval, output format, expected volume, and rules for accepted, failed, changed, and unknown records. The output should be a reviewed sample in the agreed schema plus an exception log. The pilot brief contains the questions and blank worksheet.

Keep unit economics explicit: pilot cost = setup effort + retrieval effort × expected volume + review effort × exception volume. Replace each term with agreed assumptions after feasibility review. Measure whether accepted records reduce manual research time or improve a paid workflow, not whether a page was fetched.

Questions buyers usually ask

Can I use this as training data?

The proposed workflow is for retrieval records. Your team remains responsible for deciding whether and how any text enters model training.

Does a fetched page guarantee a correct answer?

No. The record proves what was fetched and when, subject to the agreed checks. Your acceptance rules should route stale, changed, ambiguous, and failed pages to review.

What happens after I send a request?

A human reviews the source list and requirements, then confirms whether the proposed scope is feasible and what price and acceptance criteria would apply. No delivery or purchase terms are assumed in the request.

Can I test the format before that conversation?

Yes. Run the synthetic sample locally and use the pilot brief to prepare the questions you want answered. Then use the email draft to request access.