4 · Observe
Your system is live. This stage is about the views that answer the daily questions — what is it doing, what does it cost, what got blocked — and all of them share one property: they filter to your AI System. You are never reading a firehose of everyone’s traffic and guessing which lines are yours.
Mission Control, scoped to your system
Section titled “Mission Control, scoped to your system”Mission Control is the operations dashboard (Admin UI). Pick your system in the AI Systems selector at the top and every tab re-scopes: overview counters, cost, economics, governance activity, agents, reliability. Pick a timespan (24h → all time) and, if you import external telemetry, whether you’re looking at governed traffic, observed traffic, or both.
The tabs you’ll live in:
| Tab | The question it answers |
|---|---|
| Overview | Is it up, is it erroring, what’s the request volume right now? |
| Cost | Total spend, attributed by provider, model, group and user — with a residual line for agent/unattributed traffic so per-user spend reconciles to the total. If the system has a budget, the burn-down shows consumption against it and projects the overrun before it happens. |
| LLM Economics | The efficiency view: cost per 1,000 requests and request volume per month, side by side. When you enable smart routing or the semantic cache in Govern, this is where the unit-cost line visibly bends. |
| Governance | Guardrail blocks, policy denials, approvals — governance actually firing, not just configured. |
| Agents | A2A traffic, agent authorization decisions, dry-run would-blocks from your grant rollout. |
| Caching / Reliability | Cache hit rates and savings; breaker state, fallbacks, latency percentiles. |
The system’s own Usage view
Section titled “The system’s own Usage view”The AI System’s group page (Resource Groups → your system → Usage) is the self-serve version — the view you check without leaving the group you’re configuring. Totals for requests, cost, tokens, MCP/agent/skill calls and error rate, activity charts that adapt their grain to the period (daily bars up to a month, weekly to a quarter, monthly beyond), and per-model / per-tool / per-capability breakdowns.
Cost per request is not cost per task
Section titled “Cost per request is not cost per task”Every dashboard above meters requests. But you built an agent: one job is many requests. The run ledger — fed by the run header you wired in Develop — attributes cost to completed tasks:
“Refund handling costs $0.14 per resolved ticket” is a business number. “gpt-5.2 averaged 5,700 tokens per request” is not.
Runs live under AI Estate → AI Systems → your system → Runs, each expanding into its step-by-step timeline. Cost-per-run belongs to Assure as a drift signal too — a creeping step count shows up there before it shows up on the invoice.
Logs — from symptom to evidence in one id
Section titled “Logs — from symptom to evidence in one id”Every request through the gateway writes a proxy-log row: identity, group, model/tool, tokens, cost, latency, and the governance verdict (allowed / blocked / redacted / parked / denied) with the reason. Filter by your system’s group, or — the debugging move — send x-correlation-id on your agent’s requests and jump straight from a user complaint to the exact rows.
curl http://localhost:8100/v1/proxy/llm/chat/completions \ -H "Authorization: Bearer $KEY" -H "x-correlation-id: ticket-8841" ...Governance events (who approved that refund, which admin changed a grant) land in the audit trail alongside. → Logs & audit trail
Alerts — the system tells you
Section titled “Alerts — the system tells you”Configure usage alerts on the system’s group — spend thresholds, error-rate spikes, volume anomalies — with an acknowledgement workflow. From Assure, liveness and drift findings arrive through the same channel, so “the copilot went quiet on Saturday” is a notification, not a Monday discovery.
Your agent can also self-observe: governed responses carry X-Usage-Warning headers as budgets approach (Daily budget at 85%: $4.25 / $5.00) — back off before the hard 429.
Beyond the built-ins
Section titled “Beyond the built-ins”The gateway exports OpenTelemetry — traces, metrics and logs into the Prometheus/Grafana/Loki stack you already run (Metrics & Grafana). And if part of your AI estate does not route through the gateway yet, Traffic Data Import pulls vendor admin telemetry in as observed traffic — visible in the same Mission Control views, clearly labeled as observed rather than governed.
What you have at the end of this stage
Section titled “What you have at the end of this stage”You can answer “what is it doing and what does it cost” in seconds, scoped to your system, at request and task granularity — and the system pages you when something needs you. The last stage makes the deeper promise: proving it behaves, and proving it to someone else.