Health & the Assurance Report

Everything assurance measures rolls up here: one health verdict per system, and one artifact that proves it.

One number, five components

health = min(liveness, behaviour, reliability, cost, conformance)

Worst-of, not an average. A system that is cheap, fast and dead is not 80% healthy — averaging is how a dashboard reports a comfortable number for a system nobody can call. Every component is shown alongside the total, because a score nobody can decompose is a score nobody trusts.

States: healthy · learning · degraded · drifting · stalled · silent · suspended · unknown.

Only healthy is green

learning and unknown render amber, never green. They are the two states where the platform has not measured something, and an optimistic UI rounds exactly those up. A system with too little history, or one whose call chain could only be partly observed, has no score at all rather than a flattering one — and the chain-integrity percentage sits next to the number so you know what it is worth.

Conformance: declared vs observed

intended_clients and intended_use are declarations. Conformance compares them against what the gateway actually saw:

• An undeclared client that called the system — on a high-risk system this is escalated, because it is the finding a risk owner reads first.
• A declared client that never called — reported only once there is enough traffic to distinguish “absent” from “quiet this week”.
• A granted resource never used — a least-privilege finding: blast radius nobody is getting value from.

A system with no traffic has no conformance score. It has not demonstrated conformance; it has demonstrated nothing.

The period verdict

Health says now; the report’s verdict says over the period, worst-first:

Status	Means
violations	At least one signed envelope term was breached
insufficient_evidence	No runs, or too few to evaluate any declared term — never rounded up to assured
assured_with_exceptions	Within its envelope, with governed refusals, open drift or unevaluable terms on record
assured	“The system remained within its approved operating envelope during the reporting period.”

A violation observed through thin evidence is still a violation — thin evidence weakens an assured claim, never a breach. The verdict sits above a per-term contract compliance table, an exceptions section (guardrail blocks, policy denials, approval escalations, automatic responses — counted, never netted away) and a changes section (contract versions, attributed model and dependency changes, current config drift).

The Assurance Report

One artifact per AI System answering all four guarantees with evidence: identity and declared intent, contract history with approvers and hashes, drift events with their causes and the responses taken, run outcomes and cost per completed run, liveness record, replay results, and conformance.

The engineer opens it at 09:00; the auditor exports it in December. The same artifact serves both.

The report also lists the system’s external evidence — impact assessments, evaluation, red-team and pen-test reports — as verifiable pointers (uri + sha256, assessor, independence, validity). When nothing is attached, or nothing attached is independently produced, the report says so in not_covered rather than letting platform-generated evidence pass as the whole story.

The section that makes the rest credible

Every report ends with what it does not cover — chain integrity below 100%, guardrails not re-evaluated during replay, clients that never traversed the gateway, behaviour detected but truth not evaluated. It is written from the system’s actual state, so it shrinks as coverage improves.

A report claiming total coverage is one an auditor stops trusting the moment they find a gap, and they always find a gap.

Snapshots are immutable. Capturing one freezes the JSON; recomputing it later would answer “what is true now” rather than “what was true then”, and evidence that changes when the system does is not evidence.

Snapshots are also attested: at capture time the report’s SHA-256 manifest is signed with the deployment audit key — the same Ed25519 key that signs audit-chain checkpoints. Downloading a snapshot yields a portable AI System Assurance Certificate (spec: ai.brutor/ai-system-assurance-report, schema at GET /v1/admin/standards/ai-system-assurance-report/schema) that a recipient verifies offline with sha256sum and any Ed25519 library — no API call back to the platform. Signing is best-effort and honest: a deployment without an audit key produces certificates that say signed: false with the reason, never an implied attestation.

Set assurance_capture_interval_hours on the tenant to capture automatically — continuous post-market monitoring evidence produced by a machine rather than by someone remembering to click. It is off by default: a job nobody switched on should not silently accumulate a snapshot of every system every day.

ISO 42001

Captured reports and ready baselines feed the AIMS control coverage automatically: A.6.2.4 (performance evaluation — behavioural baselines exist, so “performing as expected” is measurable) and A.6.2.7 (post-market monitoring — dated evidence keeps being produced). See Compliance.

Related

• How assurance works — the lifecycle this page is the verdict of
• Assurance API — health, reports and capture endpoints
• Mission Control — the fleet board the verdict drives