Async jobs and webhooks

Create a job

One endpoint, an array of parameter objects, and optionally a webhook. You get 202 and a job id straight away.

curl -X POST ${BASE}/jobs \
  -H "Authorization: Bearer $${KEY}" \
  -H "content-type: application/json" \
  -d '{
    "endpoint": "google-maps/place",
    "inputs": [
      { "place_id": "pl_googlemaps_1g3mpl" },
      { "place_id": "pl_googlemaps_1r1l3qd" }
    ],
    "webhook_url": "https://yours.example/hooks/scraping"
  }'

Credits are reserved up front

The whole job is charged when you create it — {inputs} × the endpoint’s credit cost. If the account cannot afford it, the job is refused before it starts and nothing is charged; running out half way through would be worse for you than being told no.

Every input that fails is refunded as it fails, and cancelling returns the credits for whatever never ran. credits_reserved and credits_refunded on the job are the whole story.

Poll it

{
  "data": {
    "id": "job_9bb0b0206f7641859c3c426f",
    "endpoint": "google-maps/place",
    "status": "done",
    "total": 2,
    "completed": 2,
    "failed": 0,
    "credits_reserved": 6,
    "credits_refunded": 0,
    "results_url": "/v1/jobs/job_9bb0b0206f7641859c3c426f/results",
    "webhook": { "url": "https://yours.example/hooks/scraping", "state": "delivered" },
    "created_at": "2026-09-20T22:14:02Z",
    "finished_at": "2026-09-20T22:14:09Z"
  }
}

The results

NDJSON, one line per input, in the order you gave them — so you can zip inputs to results by index. A failure is a line, not a missing line, which means a half-broken run is still a usable file.

Results are kept for seven days after the job finishes, then deleted — the job says when, in results_expire_at. After that the download returns results_expired; run the job again if you need them. We hold scraped data as briefly as we can, and say so in the privacy notice.

{"input":{"place_id":"pl_googlemaps_1g3mpl"},"data":{"id":"pl_googlemaps_1g3mpl","name":"Ember Books", ... }}
{"input":{"place_id":"pl_googlemaps_bad"},"error":{"type":"not_found","code":"place_not_found","message":"No place matches that id. …"}}

Webhooks

If you pass a webhook_url (https only), we POST the finished job to it. The payload is signed with HMAC-SHA256 over timestamp.body and carried in the ScrapeField-Signature header as t=…,v1=….

The timestamp is inside the signed material on purpose: without it, a payload captured once can be replayed forever. Verify it like this — this is the same implementation we sign with:

import { createHmac, timingSafeEqual } from "node:crypto";

export function verify(rawBody, header, secret) {
  const t = /t=(\d+)/.exec(header)?.[1];
  const v1 = /v1=([a-f0-9]{64})/.exec(header)?.[1];
  if (!t || !v1) return false;

  // reject anything older than five minutes: the timestamp is inside the
  // signed material, so a captured payload cannot be replayed
  if (Math.abs(Date.now() / 1000 - Number(t)) > 300) return false;

  const expected = createHmac("sha256", secret).update(`${t}.${rawBody}`).digest("hex");
  return timingSafeEqual(Buffer.from(expected), Buffer.from(v1));
}

We retry five times with doubling backoff — 2, 4, 8, 16, 32 seconds — and then stop and record the failure on the job. A 4xx other than 429 stops us immediately, because that is you telling us never to send it again. A webhook that retries forever is a denial-of-service against a customer whose endpoint is already down.