Temporal Probes

A temporal probe verifies a delayed or asynchronous side-effect: “do X — and within N seconds Y happens”. It closes the gap ordinary tests can’t cover — anything that lands later, out-of-band:

• an async worker processes an order after the API already answered 202;
• a webhook arrives at your endpoint some seconds after the triggering event;
• eventual consistency — a write becomes visible on another read path;
• a cron / scheduled job produces its result on its own schedule;
• a retry with backoff eventually succeeds;
• a queue consumer picks up a message and applies a change.

Instead of a brittle fixed sleep, a probe polls until the effect is observed — and reports the actual latency (how long the effect took to land), which is a useful metric in itself.

How a probe is built

A probe has three stages:

1. Trigger (optional) — the action that starts the async process. http fires a request (e.g. POST /orders); omit the trigger to just watch for an effect already in flight (cron jobs, external events).
2. Wait — the poll strategy: intervalMs (default 1000), timeoutMs (default 30000, maximum 30 minutes), optional backoff (linear / exponential), optional minStableChecks (require N consecutive passes — protects against a flapping value).
3. Check — what confirms the effect:
   • http — poll an endpoint until the response matches (expectStatus, bodyContains, jsonPath + equals);
   • store — a Global/Chain store key holds the expected value (or exists);
   • mock_hit — a mock received a matching request — the webhook-delivery check.

The result is a report: verdict fired / timeout / error, latencyMs (time-to-fire), the number of attempts, the last observed value, and a full attempt-by-attempt trace. If the check URL answers 401 or 403 on every attempt, the verdict is error and says access was refused — nothing was measured, so it is not reported as a slow effect. Give the check the headers it needs to authenticate.

In the UI

Temporal Probes in the sidebar → build the three stages in the form, Run now, read the verdict and the trace timeline. Saved probes keep their run history.

Via the API

Run an ad-hoc probe in one call (with a timeout ≤ 60s the finished report comes back inline):

curl -X POST "http://localhost:5770/api/v1/namespaces/my-namespace/temporal-probes/run" \
  -H "X-API-Key: $TOKEN" -H "Content-Type: application/json" -d '{
  "trigger": {"kind": "http", "http": {"method": "POST", "url": "https://api.example.com/orders", "body": "{\"item\":\"42\"}"}},
  "wait": {"intervalMs": 1000, "timeoutMs": 30000},
  "check": {"kind": "http", "http": {"url": "https://api.example.com/orders/last", "jsonPath": "$.status", "equals": "processed"}}
}'

With a longer timeout the call returns 202 and a running run — poll GET /temporal-probes/runs/{runId} until the verdict is terminal.

Saved probes: POST /temporal-probes (body {name, description, config}), GET /temporal-probes, POST /temporal-probes/{id}/run, GET /temporal-probes/{id}/runs.

From an AI agent

The MCP tool temporal_probe_run does the whole cycle in a single call — trigger, poll, report. temporal_probe_create / temporal_probe_run_saved / temporal_probe_get_run manage saved probes and async runs.

In a Test Plan

Add a temporal_probe item referencing a saved probe: the plan step passes when the probe fires and fails (with the full trace in the step report) on timeout — so a plan can do functional step → wait for the async effect → next step.

Verifying webhook delivery

1. Create a mock that plays the webhook receiver.
2. Trigger the action that should send the webhook.
3. Probe check mock_hit with the mock’s ID (optionally method, pathContains, minCount) — the probe fires the moment the mock receives the callback, and latencyMs tells you how long delivery took.

Limits

A probe is a bounded poll: timeout is capped at 30 minutes, the interval floor is 250 ms, and the attempt count is capped — a probe can never turn into an endless retry loop.