Docs Reporting CI Results (External Runs)

Reporting CI Test Results into Mockarty (External Runs)

Send results from any test framework — go test, pytest, JUnit, Playwright, your
own runner — into Mockarty TCM. Each reported run becomes a test-case run with
steps, attachments, and history, visible in the same reports as runs executed
by Mockarty itself.

There are two ways to report:

Mode When to use
Single-shot — one POST with the whole finished run The run is already complete (e.g. you parse a JUnit XML at the end of the job)
Streaming lifecycle — create → stream steps → finish Long CI jobs where you want steps to arrive as they complete, safe retries, and file attachments
Allure results archive — upload the whole allure-results folder Your framework already writes Allure results and you want one upload at the end of the job

Both are namespace-scoped and authenticated with your API token
(Authorization: Bearer <token>).
The token needs test_case:write to report results, upload Allure archives,
or change a streaming run. Reading a streaming run or downloading its
attachment needs test_case:read in the same namespace.

Single-shot ingest

curl -X POST "http://localhost:5770/api/v1/namespaces/<ns>/tcm/external-runs" \
  -H "Authorization: Bearer $MOCKARTY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "caseName": "checkout smoke",
    "framework": "pytest",
    "status": "passed",
    "idempotencyKey": "job-4211/result-17",
    "steps": [
      {"name": "login", "status": "passed"},
      {"name": "add to cart", "status": "passed"}
    ]
  }'

A batch variant accepts multiple runs at once: POST .../tcm/external-runs/batch.
idempotencyKey is optional but recommended for CI. Repeating the same
payload with the same key returns the original runId, including after a
server restart; changed data with the same key returns 409. Scope the key
to one result attempt and use a new key for a genuine rerun. Without a key,
each accepted POST creates another attempt.

Streaming lifecycle

The lifecycle surface lives under .../tcm/external-runs/lifecycle and is what
the Go SDK’s externalruns package uses.

1. Create the run

curl -X POST "http://localhost:5770/api/v1/namespaces/<ns>/tcm/external-runs/lifecycle" \
  -H "Authorization: Bearer $MOCKARTY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name": "nightly regression", "framework": "go-test", "external_id": "ci-run-4211"}'

The response contains the run id used by all following calls and a monotonic
revision (also returned as the ETag header).
external_id makes creation idempotent: if your CI job retries, re-creating
with the same external_id returns the existing run instead of a duplicate.

2. Stream steps as they finish

curl -X POST ".../tcm/external-runs/lifecycle/<runId>/steps" \
  -H "Authorization: Bearer $MOCKARTY_API_TOKEN" \
  -H 'If-Match: "<revision>"' \
  -H "Content-Type: application/json" \
  -d '{"steps": [
        {"step_key": "auth-01", "name": "login", "status": "passed", "duration_ms": 420},
        {"step_key": "cart-01", "name": "add to cart", "status": "failed",
         "message": "expected 200, got 500"}
      ]}'

Each step carries a step_key you choose. Re-sending a step with the same
step_key updates it instead of duplicating — retried batches are safe.
Nested steps use parent_key. Send the latest quoted revision in If-Match
to fence a delayed CI worker; a stale revision is rejected with HTTP 409. The
header is optional for compatibility with clients released before this fence.

3. Attach files (optional)

curl -X POST ".../tcm/external-runs/lifecycle/<runId>/attachments" \
  -H "Authorization: Bearer $MOCKARTY_API_TOKEN" \
  -H 'If-Match: "<revision>"' \
  -F "file=@screenshot.png"

One file per request, up to 25 MiB each.

4. Finish

curl -X POST ".../tcm/external-runs/lifecycle/<runId>/finish" \
  -H "Authorization: Bearer $MOCKARTY_API_TOKEN" \
  -H 'If-Match: "<revision>"' \
  -H "Content-Type: application/json" \
  -d '{}'

An empty body is fine — the run status is inferred from the steps (any failed →
failed, any broken → broken, otherwise passed), or pass {"status": "failed", "summary": "..."} to set it explicitly. On finish the run is ingested into
TCM and the response carries the resolved case/run ids; the run then appears in
Test Runs and case history like any other run. Use the new revision returned by
each mutation for the next If-Match; the Go, Python, and Java SDKs expose
*AtRevision lifecycle methods that do this explicitly.

Inspecting runs

# One run with steps + attachments
curl ".../tcm/external-runs/lifecycle/<runId>" -H "Authorization: Bearer $MOCKARTY_API_TOKEN"

# List (filters: suite_id, framework, status, external_id; cursor pagination)
curl ".../tcm/external-runs/lifecycle?status=failed&limit=20" \
  -H "Authorization: Bearer $MOCKARTY_API_TOKEN"

Allure results archive

If your framework already writes an allure-results directory, zip it and send
it in one request. Each *-result.json becomes a test-case run; cases that
don’t exist yet are created automatically and filed into folders derived from
the Allure suite / feature labels.

cd allure-results && zip -qr ../allure-results.zip . && cd ..

curl -X POST "http://localhost:5770/api/v1/namespaces/<ns>/tcm/allure-results" \
  -H "Authorization: Bearer $MOCKARTY_API_TOKEN" \
  -H "Content-Type: application/octet-stream" \
  --data-binary @allure-results.zip

Multipart works too — -F "archive=@allure-results.zip".

Optional query parameters:

Parameter Effect
placeInTree=false Keep newly created cases at the namespace root instead of building the suite/feature folder tree
launchId=<runId> Attach the archive’s environment.properties / categories.json / executor.json to that launch, so its report shows them

Reading the answer

The status code tells you what actually landed — gate your pipeline on it:

Status outcome Meaning
200 ok Every result in the archive was stored
200 metadata_only The archive held no results, only launch metadata — and that was stored
207 partial Some results were rejected, or requested launch metadata could not be stored. The upload is incomplete — read errors
422 rejected No results were stored. Either the archive had no *-result.json files or every result was rejected, including results that reference a missing attachment
400 — The upload wasn’t a readable zip, or the namespace is missing
429 — Too many uploads at once — retry after the Retry-After delay

The body is the same object for all of them:

{
  "namespace": "qa",
  "outcome": "partial",
  "message": "stored 8 of 10 result(s); 2 rejected — the run history is incomplete, see `errors`",
  "results": 10,
  "ingested": 8,
  "created": 3,
  "skipped": 1,
  "failed": 1,
  "placed": 3,
  "attachments": 12,
  "errors": [
    "unparseable result JSON: unexpected end of JSON input",
    "checkout.test_pay: case name is required"
  ]
}
  • ingested is the number of results that were stored — the one counter to
    gate on. results is how many the archive contained, so ingested / results
    is the honest “N of M landed” ratio.
  • created — how many cases were newly created, placed — how many were filed
    into a folder, attachments — files found in the archive.
  • skipped counts result files that could not be read, failed those that were
    read but rejected. errors names each one and why (capped at 50 reasons, and
    it says so when it truncates).

The most common cause of 422 is uploading the rendered allure-report
directory instead of the raw allure-results one the adapter writes.

Go SDK

The externalruns package wraps the lifecycle endpoints:

import "github.com/mockarty/mockarty-go/externalruns"

runs, _ := externalruns.NewClient("http://localhost:5770", "<ns>", apiToken)
run, _ := runs.CreateRun(ctx, externalruns.CreateRunRequest{
    Name: "nightly regression", Framework: "go-test",
})
defer runs.FinishRun(ctx, run.ID, externalruns.FinishRunRequest{})

See SDK protocol clients for a full example that
records per-RPC steps automatically.

Good to know

  • Retries are safe end-to-end: external_id dedups run creation, step_key
    dedups steps.
  • Under heavy parallel CI load, finish may answer 429 with a Retry-After
    header — retry after the indicated delay.
  • Migrating from TestIT, Allure, Zephyr, TestRail or QASE? See the
    TestIT integration guide — the importers feed the
    same ingest underneath.
Previous
Autonomous Missions