Skip to content
Brutor AI Docs

Architecture

Brutor is built as two planes with strictly separated responsibilities, sharing one PostgreSQL database.

The two planes

Section titled “The two planes”
                        CONFIG PLANE                    DATA PLANE
                 ┌────────────────────────┐    ┌───────────────────────────┐
  Admin Console  │   Control Plane        │    │   Core Proxy              │   User Portal
  (React :3002) ─┤   Python / FastAPI     │    │   Rust / Axum             ├─ (React :3001)
                 │   :5050                │    │   :8100                   │
                 │                        │    │                           │
                 │  /v1/admin/*           │    │  /v1/proxy/llm/*          │
                 │  tenants, users, keys, │    │  /v1/proxy/mcp/*  /vmcp   │
                 │  models, MCP servers,  │    │  /v1/proxy/oauth/*        │
                 │  guardrails, budgets   │    │  Agent Skills (via MCP)   │
                 │                        │    │  /a2a/*   /v1/messages    │
                 │  Alembic migrations    │    │  /v1/portal/* (end users) │
                 └───────────┬────────────┘    └──────┬──────────┬─────────┘
                     writes  │                  reads │          │ proxies
                             ▼                        ▼          ▼
                 ┌────────────────────────────────────────┐   ┌──────────────────┐
                 │        PostgreSQL (shared)             │   │  LLM providers,  │
                 │  config + usage ledger + logs          │   │  MCP servers,    │
                 │  LISTEN/NOTIFY → cache invalidation    │   │  A2A agents      │
                 └────────────────────────────────────────┘   └──────────────────┘
                       Redis (cache, rate limits)   Qdrant (semantic cache, KB vectors)
  • The Control Plane (brutor-gateway-control) handles every configuration write. It owns the database schema, runs Alembic migrations on startup, and backs the Admin Console only.
  • The Core Proxy (brutor-gateway-core) handles every runtime request — LLM, MCP, A2A, OAuth and Agent Skills proxying — and also serves the end-user Portal API (/v1/portal/*, including portal login, which issues the portal JWT). It reads configuration from the shared database and is strictly read-only with respect to config; it never alters the schema.

Both planes must share the same JWT_SECRET and ENCRYPTION_KEY: the portal JWT is minted by the proxy and validated on both sides, and encrypted secrets (provider API keys, OAuth tokens) written by the control plane are decrypted by the proxy.

Request flow through the Core Proxy

Section titled “Request flow through the Core Proxy”

A chat completion (POST /v1/proxy/llm/chat/completions) passes through these stages:

client ──► 1. auth waterfall        API key ▸ gateway JWT ▸ OAuth bearer ▸ anonymous
           2. tenant middleware     resolve tenant_id (token claim or X-Tenant-ID)
           3. group resolution      X-Resource-Group header ▸ primary group ▸ first membership
           4. preflight             model access check · budgets/quotas · rate limits ·
                                    input guardrails · param governance (clamps) ·
                                    semantic-cache lookup
           5. provider call        translated to the provider's wire format
           6. postflight            output guardrails · usage metering (tokens, cost) ·
                                    proxy log + audit row · cache store
       ◄── 7. response              OpenAI-compatible body + X-Correlation-Id,
                                    usage-warning headers when near a limit

Blocked requests short-circuit before the provider is called: guardrail violations return 403 with code: "guardrail_blocked", exceeded limits return 429 with a Retry-After header. Telemetry (proxy logs, metrics, usage rows) is written asynchronously through batched background writers so logging never sits on the request path.

Configuration read path and cache invalidation

Section titled “Configuration read path and cache invalidation”

The proxy does not query PostgreSQL on every request. Configuration — models, resource groups, guardrail configs, limits — is held in in-process caches. When an operator changes configuration through the Control Plane, the change is written to PostgreSQL and a notification is published; every Core Proxy instance subscribes via PostgreSQL LISTEN/NOTIFY, so caches are invalidated across a multi-instance cluster natively, with no separate message bus required. The same mechanism drives event-driven A2A streaming (pg_notify wakes SSE subscribers instead of polling).

Redis backs the runtime hot path: rate-limit and quota counters, semantic-cache bookkeeping and short-lived state. Setting REDIS_DISABLED=true falls back to in-memory equivalents, which is only correct for a single proxy instance.

Multi-tenant isolation

Section titled “Multi-tenant isolation”

Isolation is enforced in depth, not by convention:

  • Tenant middleware on both planes. Every API requires tenant context — resolved from the JWT claim or API key, or the X-Tenant-ID header — before any handler runs.
  • Row-level scoping. Every configuration and telemetry table carries a tenant_id column, and every query filters on it.
  • Per-tenant vector collections. Knowledge-base vectors live in per-tenant Qdrant collections (brutor_kb_{tenant_id}), and every Qdrant call is authenticated with a short-lived (300 s) tenant-scoped JWT minted per request.
  • Per-tenant Redis prefixes. Counters and cache keys are namespaced by tenant.

Where each API lives

Section titled “Where each API lives”
API Served by Base path Used by
LLM proxy (OpenAI-compatible) Core Proxy :8100 /v1/proxy/llm/* SDKs, agents, apps
Anthropic Messages (native) Core Proxy :8100 /v1/messages Anthropic SDK, Claude Code
MCP gateway Core Proxy :8100 /v1/proxy/mcp/{server_id} MCP clients, agents
Virtual MCP aggregator Core Proxy :8100 /v1/proxy/vmcp/{virtual_server_id} MCP clients
OAuth proxy Core Proxy :8100 /v1/proxy/oauth/* MCP OAuth flows
A2A v1.0 Core Proxy :8100 /a2a/{tenant_id}/{card_name} Agent-to-agent traffic
Agent Skills Core Proxy :8100 /v1/proxy/mcp/system-agent-skill-server-{tenant} MCP clients, agents
Batch & files Core Proxy :8100 /v1/proxy/batch, /v1/proxy/files Batch jobs, multimodal uploads
Portal API (end-user runtime) Core Proxy :8100 /v1/portal/* User Portal, custom portals
Admin API (all config writes) Control Plane :5050 /v1/admin/*, /v1/admin-users/* Admin Console, setup scripts

Services and ports

Section titled “Services and ports”

The full stack as shipped in the trial bundle:

Service Port(s) Purpose
Core Proxy (Rust) 8100 Runtime proxy + Portal API
Control Plane (Python) 5050 Admin API, migrations, seeding
Admin Console 3002 Management UI
User Portal 3001 End-user chat UI
KB Uploader internal :8200 Document extraction, chunking, embedding
KB Connector Sync internal :8201 Confluence / Notion / GDrive / Slack / GitHub / Jira / SharePoint sync
Skill Runner internal :8210 Sandboxed skill execution (JWT-authed, read-only rootfs)
mcp-app-map internal :3001 Demo MCP server (Leaflet map, MCP-Apps UI)
PostgreSQL 16 5432 Shared database
Redis 7.4 6379 Cache, rate limits, counters
Qdrant 1.18 6333 (REST) / 6334 (gRPC) Vectors — semantic cache + knowledge bases
MinIO 9000 (API) / 9001 (console) S3-compatible media storage
Ollama (profile local-embedding) 11434 Local embeddings (nomic-embed-text)
LocalStack + Vault (profile kms) 4566 / 8200 KMS emulation / Vault Transit
Presidio analyzer (profile presidio) 3030 PII NER engine for guardrails
Anthropic Enterprise mock (profile anthropic-mock) 8085 Traffic Data Import demo source

For development, the brutor-gateway-core repo’s Docker Compose adds a full observability stack — OpenTelemetry Collector (4317/4318), Prometheus (9090), Loki (3100) and Grafana (3010). See the observability tour.

Section titled “Related pages”