Webhooks
Function jobs and monitors can POST what happened to your URL. Every request is signed with your account's secret, carries an id that stays the same when it is retried, and is retried until your endpoint answers 2xx or the schedule runs out. Jobs and monitors come with paid plans.
What arrives
A JSON POST to the public https URL you gave in "deliver": {"webhook": {"url": "https://…"}}. Job webhooks are {type, job_id, created_at, data}, monitor webhooks {type, created_at, data}:
type | When | data |
|---|---|---|
job.done | A job ends (with "events": ["done"], the default) | status (done, stopped, failed or interrupted), progress |
job.page | Each page of a batch or crawl ("page") | That page's result, as in results.jsonl; over 1 MB, a result_url |
job.event | A watch's live events, at most one request per 2 s ("event") | A batch of events |
job.archive | The zip of a job with "format": "archive" is ready ("archive") | url, bytes |
monitor.change | A monitor's page changed | monitor_id, change_id, url, at, added, removed, diff, snapshot_url |
monitor.error | A monitor failed three checks in a row (sent once) | monitor_id, url, error, failures |
Two of the deliveries a two-page batch sent on 2026-11-14, byte for byte:
{"type":"job.page","job_id":"f1e49e3c-820e-4fcf-b0a0-7c9b3aedc20f","created_at":"2026-11-14T02:05:08.979Z","data":{"url":"https://books.toscrape.com/","status":200,"extract":{"title":"All products"}}}
{"type":"job.done","job_id":"f1e49e3c-820e-4fcf-b0a0-7c9b3aedc20f","created_at":"2026-11-14T02:05:09.087Z","data":{"status":"done","progress":{"done":2,"failed":0,"queued":0,"total":2}}}
Headers
x-cdpfleet-event— thetype, so you can route before parsing.x-cdpfleet-delivery— a UUID for this delivery; every retry of it carries the same one.x-cdpfleet-signature—t=<unix seconds>,v1=<hex>.content-type: application/json,user-agent: cdpfleet-webhooks/1.
Verify the signature
Your secret is a whsec_… string, one per account for all your jobs and monitors: GET /v1/me/webhook-secret returns it (and creates it the first time). Treat it like an API key. For each request:
- Take
tandv1fromx-cdpfleet-signature. - Compute HMAC-SHA256 of
"<t>.<raw body>", keyed with the wholewhsec_…string, as lowercase hex. - Accept only if it equals
v1(compare in constant time) andtis within 5 minutes of your clock.
import crypto from 'node:crypto';
// rawBody: the request body exactly as received (a Buffer or string), before any JSON parsing.
export function verifyWebhook(rawBody, header, secret, toleranceSeconds = 300) {
const parts = Object.fromEntries(String(header ?? '').split(',').map((kv) => kv.trim().split('=', 2)));
const t = Number(parts.t);
if (!Number.isInteger(t) || !/^[0-9a-f]{64}$/.test(parts.v1 ?? '')) return false;
if (Math.abs(Date.now() / 1000 - t) > toleranceSeconds) return false;
const expected = crypto.createHmac('sha256', secret).update(`${t}.`).update(rawBody).digest();
return crypto.timingSafeEqual(expected, Buffer.from(parts.v1, 'hex'));
}
import hashlib
import hmac
import time
# raw_body: the request body bytes exactly as received, before any JSON parsing.
def verify_webhook(raw_body: bytes, header: str, secret: str, tolerance_seconds: int = 300) -> bool:
parts = dict(p.strip().split("=", 1) for p in (header or "").split(",") if "=" in p)
try:
t = int(parts["t"])
except (KeyError, ValueError):
return False
if abs(time.time() - t) > tolerance_seconds:
return False
expected = hmac.new(secret.encode(), f"{t}.".encode() + raw_body, hashlib.sha256).hexdigest()
return hmac.compare_digest(expected, parts.get("v1", ""))
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.util.HexFormat;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
public final class Webhooks {
// rawBody: the request body bytes exactly as received, before any JSON parsing.
public static boolean verify(byte[] rawBody, String header, String secret, long toleranceSeconds) throws Exception {
String t = null, v1 = null;
for (String part : (header == null ? "" : header).split(",")) {
String[] kv = part.trim().split("=", 2);
if (kv.length != 2) continue;
if (kv[0].equals("t")) t = kv[1];
if (kv[0].equals("v1")) v1 = kv[1];
}
if (t == null || v1 == null) return false;
long ts;
byte[] given;
try {
ts = Long.parseLong(t);
given = HexFormat.of().parseHex(v1);
} catch (IllegalArgumentException e) {
return false;
}
if (Math.abs(System.currentTimeMillis() / 1000 - ts) > toleranceSeconds) return false;
Mac mac = Mac.getInstance("HmacSHA256");
mac.init(new SecretKeySpec(secret.getBytes(StandardCharsets.UTF_8), "HmacSHA256"));
mac.update((t + ".").getBytes(StandardCharsets.UTF_8));
return MessageDigest.isEqual(mac.doFinal(rawBody), given);
}
}
using System.Security.Cryptography;
using System.Text;
public static class Webhooks
{
// rawBody: the request body bytes exactly as received, before any JSON parsing.
public static bool Verify(byte[] rawBody, string? header, string secret, long toleranceSeconds = 300)
{
string? t = null, v1 = null;
foreach (var part in (header ?? "").Split(','))
{
var kv = part.Trim().Split('=', 2);
if (kv.Length != 2) continue;
if (kv[0] == "t") t = kv[1];
if (kv[0] == "v1") v1 = kv[1];
}
if (!long.TryParse(t, out var ts) || v1 == null) return false;
if (Math.Abs(DateTimeOffset.UtcNow.ToUnixTimeSeconds() - ts) > toleranceSeconds) return false;
byte[] given;
try { given = Convert.FromHexString(v1); } catch (FormatException) { return false; }
using var hmac = new HMACSHA256(Encoding.UTF8.GetBytes(secret));
var expected = hmac.ComputeHash([.. Encoding.UTF8.GetBytes(t + "."), .. rawBody]);
return CryptographicOperations.FixedTimeEquals(expected, given);
}
}
package webhooks
import (
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"strconv"
"strings"
"time"
)
// Verify checks x-cdpfleet-signature. rawBody is the request body exactly as received, before any JSON decoding.
func Verify(rawBody []byte, header, secret string, tolerance time.Duration) bool {
var t, v1 string
for _, part := range strings.Split(header, ",") {
k, v, ok := strings.Cut(strings.TrimSpace(part), "=")
if !ok {
continue
}
switch k {
case "t":
t = v
case "v1":
v1 = v
}
}
ts, err := strconv.ParseInt(t, 10, 64)
if err != nil {
return false
}
if age := time.Since(time.Unix(ts, 0)); age > tolerance || age < -tolerance {
return false
}
given, err := hex.DecodeString(v1)
if err != nil {
return false
}
mac := hmac.New(sha256.New, []byte(secret))
mac.Write([]byte(t + "."))
mac.Write(rawBody)
return hmac.Equal(mac.Sum(nil), given)
}
Sign the raw body, not parsed JSON. The signature covers the exact bytes we sent; JSON parsed and serialised again almost never matches. Where to get the raw bytes:
- Node.js: collect the request stream (below), or
express.raw({ type: 'application/json' })on that route. - Python:
request.get_data()in Flask,await request.body()in FastAPI and Starlette,request.bodyin Django. - Java:
@RequestBody byte[] bodyin Spring. - C#: copy
Request.Bodyinto aMemoryStreamin ASP.NET Core and useToArray(). - Go:
io.ReadAll(r.Body), thenr.Header.Get("X-Cdpfleet-Signature").
Tested on 2026-11-14 with real deliveries (two job.page, a job.done and a monitor.change): each of the five functions accepted all four, and rejected the same body with one value changed, the body parsed and serialised again with spaces, a signature with one hex digit changed, a correct signature dated 10 minutes earlier, and a missing header.
A receiver
A complete one in Node.js, no dependencies: verify, answer at once, then skip deliveries you've already handled.
import http from 'node:http';
import { verifyWebhook } from './verify.mjs';
const secret = process.env.CDPFLEET_WEBHOOK_SECRET; // whsec_…, from GET /v1/me/webhook-secret
const seen = new Set(); // delivery ids already handled; keep them in your database in production
http.createServer((req, res) => {
const chunks = [];
req.on('data', (c) => chunks.push(c));
req.on('end', () => {
const raw = Buffer.concat(chunks);
if (!verifyWebhook(raw, req.headers['x-cdpfleet-signature'], secret)) return res.writeHead(401).end();
res.writeHead(204).end(); // answer first, work after
const id = req.headers['x-cdpfleet-delivery'];
if (seen.has(id)) return; // a retry of one already handled
seen.add(id);
const event = JSON.parse(raw);
console.log(event.type, event.job_id ?? event.data.monitor_id, JSON.stringify(event.data));
});
}).listen(3000);
To try it before any job runs, sign a request yourself with the secret in CDPFLEET_WEBHOOK_SECRET:
body='{"type":"job.done","job_id":"test","created_at":"2026-11-14T00:00:00Z","data":{"status":"done"}}'
t=$(date +%s)
sig=$(printf '%s' "$t.$body" | openssl dgst -sha256 -hmac "$CDPFLEET_WEBHOOK_SECRET" | sed 's/^.* //')
curl -s -o /dev/null -w '%{http_code}\n' http://localhost:3000/hooks/cdpfleet -H 'content-type: application/json' \
-H 'x-cdpfleet-event: job.done' -H "x-cdpfleet-delivery: test-$t" -H "x-cdpfleet-signature: t=$t,v1=$sig" -d "$body"
The receiver answers 204 and logs the event once, however often you send the same delivery id, and 401 to a request without a valid signature.
Retries
- Any
2xxis delivered. Anything else — another status, a redirect (not followed), no answer within 15 s, a connection error — is retried after 10 s, 1 min, 5 min, 30 min and 2 h: 6 attempts over about 2 h 36 min, then the delivery is dropped. - Pending retries are stored, so they carry on through a restart on our side.
- Each attempt is signed when it is sent, with the time then and the secret then, so a retry passes the 5-minute check.
- A retry can arrive after newer deliveries — a
job.pageafter its job'sjob.done. Don't rely on arrival order; for a job's state askGET /v1/jobs/{id}.
De-duplicate
An attempt can reach you and still count as failed — your endpoint was slow, or the answer was lost — and then you get it again. Store each x-cdpfleet-delivery you've handled for a few hours (the whole schedule fits in 3) and skip repeats. Answer first and do the work after, so a slow handler doesn't turn into a retry.
Rotate the secret
POST /v1/me/webhook-secret/rotate returns a new secret and replaces the old one at once: every attempt from then on, retries included, is signed with the new one. Answer a request whose signature doesn't verify with an error such as 401, not 2xx: then the deliveries sent while you switch are retried, and pass once your receiver has the new secret.
The URL
It must be https and resolve to a public address; a private or local one is refused when you create the job or monitor (400 invalid_option) and checked again on every attempt. The URL can carry its own path and query string if you want to tell endpoints apart.