LLM Proxy API

All LLM traffic goes through the Core Proxy (:8100) — the Rust runtime plane. The surface is OpenAI-compatible, so existing OpenAI SDKs work by pointing base_url at the gateway.

Base URL	http://localhost:8100/v1/proxy/llm
Native Anthropic	http://localhost:8100/v1 (i.e. POST /v1/messages, no /proxy/ prefix)
Auth	API key (X-API-Key: sk_brutor_api_... or Authorization: Bearer sk_brutor_api_...) or Gateway JWT (X-Gateway-Authorization: Bearer <jwt>)
Tenant	From the JWT / API key, or explicit X-Tenant-ID: <tenant> header
Group scoping	X-Resource-Group header → user’s primary group → first membership
Errors	{"error": "<message>"} with an HTTP status code (see Errors)

For a guided walkthrough see LLM API guide and First LLM Call.

Endpoint overview

Endpoint	Method	Path	Purpose	Auth
Chat Completions	POST	/v1/proxy/llm/chat/completions	OpenAI-style chat; SSE streaming with "stream": true	API key or JWT
Responses API	POST	/v1/proxy/llm/responses	OpenAI Responses-style structured input/output	API key or JWT
Legacy Completions	POST	/v1/proxy/llm/completions	Legacy text completion	API key or JWT
Embeddings	POST	/v1/proxy/llm/embeddings	Vector embeddings	API key or JWT
Models (list)	GET	/v1/proxy/llm/models	Models the caller may use	API key or JWT
Models (detail)	GET	/v1/proxy/llm/models/{model_id}	Single model detail	API key or JWT
Images	POST	/v1/proxy/llm/images/generations	Image generation (DALL-E 3, SDXL, fal, Replicate, Stability)	API key or JWT
Audio TTS	POST	/v1/proxy/llm/audio/speech	Text-to-speech (OpenAI, ElevenLabs, Google)	API key or JWT
Audio STT	POST	/v1/proxy/llm/audio/transcriptions	Speech-to-text (Whisper, Deepgram, AssemblyAI)	API key or JWT
Video (submit)	POST	/v1/proxy/llm/video/generations	Async video generation — returns a job	API key or JWT
Video (poll)	GET	/v1/proxy/llm/video/jobs/{job_id}	Poll a video job (DELETE cancels)	API key or JWT
Health	GET	/v1/proxy/llm/health	LLM subsystem health	Public
Anthropic Messages	POST	/v1/messages	Native Anthropic Messages API (Anthropic SDK, Claude Code)	API key or JWT
Anthropic token count	POST	/v1/messages/count_tokens	Native Anthropic token counting	API key or JWT
Gemini native	POST	/v1beta/models/{model}:generateContent	Native Gemini surface (:streamGenerateContent for SSE)	API key or JWT
Batch	*	/v1/proxy/batch/jobs...	Async batch jobs (see Batch)	API key or JWT
Files	*	/v1/proxy/files...	Multimodal file upload/retrieval (see Files)	API key or JWT

Authentication

The proxy resolves credentials in this priority order:

1. API key — X-API-Key: sk_brutor_api_... or Authorization: Bearer sk_brutor_api_.... Format: sk_brutor_api_ + 43 URL-safe characters; stored SHA256-hashed. Scopes: global (shared, group-bound), user, group; keys may carry allowed_skill_ids.
2. Gateway JWT — X-Gateway-Authorization: Bearer <jwt> (HS256). The portal JWT from POST /v1/portal/auth/login works here; portal-style calls with plain Authorization: Bearer <jwt> are auto-promoted.
3. OAuth passthrough bearer.
4. Anonymous — public endpoints only (/health).

See Users and API Keys for issuing keys.

Model addressing: model vs model_id

Every request body accepts either field:

Field	Meaning	Example
model	Provider model name as configured in the gateway	"gpt-4o", "claude-sonnet-4-5"
model_id	Brutor model UUID (from GET /v1/proxy/llm/models or GET /v1/portal/models)	"3f9c2d1e-..."

model_id is unambiguous when the same provider name is registered more than once (e.g. per-team keys) and is what the shipped User Portal uses.

Chat Completions

POST /v1/proxy/llm/chat/completions

Request body

Field	Type	Required	Description
model	string	one of model/model_id	Provider model name
model_id	string (UUID)	one of model/model_id	Brutor model UUID
messages	array	yes	{role, content, name?, tool_calls?, tool_call_id?}; content may be a string or multimodal content array
temperature	number	no	Sampling temperature (may be clamped by param governance)
max_tokens	integer	no	Output token cap
stream	boolean	no	true → SSE streaming
top_p	number	no	Nucleus sampling
n	integer	no	Number of choices
stop	string | array	no	Stop sequence(s)
presence_penalty	number	no
frequency_penalty	number	no
tools	array	no	OpenAI tool definitions
tool_choice	string | object	no
(any other field)	—	no	Provider-specific params are passed through verbatim

Non-streaming

curl

curl http://localhost:8100/v1/proxy/llm/chat/completions \
  -H "Authorization: Bearer sk_brutor_api_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Hello"}],
    "max_tokens": 200
  }'

Python

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8100/v1/proxy/llm",
    api_key="sk_brutor_api_...",
)
resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}],
    max_tokens=200,
)
print(resp.choices[0].message.content)

Node.js

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'http://localhost:8100/v1/proxy/llm',
  apiKey: 'sk_brutor_api_...',
});
const resp = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Hello' }],
  max_tokens: 200,
});
console.log(resp.choices[0].message.content);

Response (200):

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1751457600,
  "model": "gpt-4o",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "Hello! How can I help?" },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 8,
    "completion_tokens": 9,
    "total_tokens": 17
  }
}

Streaming (SSE)

Set "stream": true. The response is text/event-stream with one JSON chunk per data: frame, terminated by data: [DONE]:

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant"},"index":0}]}

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"delta":{"content":"Hello"},"index":0}]}

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"delta":{"content":"!"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"delta":{},"index":0,"finish_reason":"stop"}]}

data: [DONE]

Chunk shape: {id, object: "chat.completion.chunk", choices: [{delta: {role?, content?}, index, finish_reason?}]}.

Output guardrails on streams

When a streaming output guardrail is configured with redaction, the gateway buffers and releases the stream; block-only checks can interrupt an in-flight stream. See Guardrails.

Responses API

POST /v1/proxy/llm/responses

Field	Type	Required	Description
model / model_id	string	one of the two	Model addressing as above
input	string | array	yes	Responses-style input
max_output_tokens	integer	no
stream	boolean	no	SSE streaming
(any other field)	—	no	Passed through to the provider

curl http://localhost:8100/v1/proxy/llm/responses \
  -H "Authorization: Bearer sk_brutor_api_..." \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o", "input": "Summarize the Brutor architecture in one line."}'

Legacy Completions

POST /v1/proxy/llm/completions

Field	Type	Required	Description
model / model_id	string	one of the two
prompt	string	yes
max_tokens	integer	no
temperature	number	no
stream	boolean	no	SSE streaming
(any other field)	—	no	Passed through

Embeddings

POST /v1/proxy/llm/embeddings

Field	Type	Required	Description
model / model_id	string	one of the two	Must be an embedding-mode model
input	string | array	yes	Text(s) to embed
encoding_format	string	no
dimensions	integer	no

curl http://localhost:8100/v1/proxy/llm/embeddings \
  -H "Authorization: Bearer sk_brutor_api_..." \
  -H "Content-Type: application/json" \
  -d '{"model": "text-embedding-3-small", "input": "hello world"}'

Response (OpenAI shape):

{
  "object": "list",
  "data": [
    { "object": "embedding", "embedding": [0.0123, -0.0456], "index": 0 }
  ],
  "model": "text-embedding-3-small",
  "usage": { "prompt_tokens": 2, "completion_tokens": 0, "total_tokens": 2 }
}

Models

Method	Path	Purpose
GET	/v1/proxy/llm/models	List models the calling identity is entitled to (resource-group scoped)
GET	/v1/proxy/llm/models/{model_id}	Detail for one model by Brutor UUID

Tip

End-user portal clients should use GET /v1/portal/models instead — it returns richer portal metadata (cost, capabilities, icons). See Portal API.

Native Anthropic Messages API

POST /v1/messages — mounted at the root /v1 (no /proxy/llm prefix) so the Anthropic SDK and Claude Code can point straight at the gateway. Same auth headers as the rest of the proxy. Token counting: POST /v1/messages/count_tokens.

curl

curl http://localhost:8100/v1/messages \
  -H "Authorization: Bearer sk_brutor_api_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4-5",
    "max_tokens": 256,
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Anthropic SDK

import anthropic

client = anthropic.Anthropic(
    base_url="http://localhost:8100",
    api_key="sk_brutor_api_...",
)
msg = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=256,
    messages=[{"role": "user", "content": "Hello"}],
)
print(msg.content[0].text)

The gateway also exposes a native Gemini surface: POST /v1beta/models/{model}:generateContent and POST /v1beta/models/{model}:streamGenerateContent, so Google SDK clients can target the gateway without body rewriting.

See Coding Agents for pointing Claude Code and other coding agents at the gateway.

Images

POST /v1/proxy/llm/images/generations — OpenAI Images-compatible. Providers: DALL-E 3 / gpt-image-1, SDXL, fal, Replicate, Stability.

Request:

Field	Type	Required	Description
model	string	yes
prompt	string	yes
n	integer	no (default 1)
size	string	no (default 1024x1024)
quality	string	no	DALL-E 3 / gpt-image-1 only; validated against the model’s supported_image_qualities
response_format	string	no (default url)	url or b64_json
style	string	no	OpenAI vivid | natural, forwarded verbatim
chat_id	string	no	Opaque chat tag for delete-fanout
display_name	string	no	User-facing label

Response:

{
  "created": 1751457600,
  "data": [
    { "url": "https://.../signed-url", "b64_json": null }
  ],
  "brutor_cost_usd": 0.04,
  "brutor_model_id": "3f9c2d1e-..."
}

Generated media is persisted via the Files API; url is a signed URL.

Audio — Text-to-Speech

POST /v1/proxy/llm/audio/speech — providers: OpenAI, ElevenLabs, Google. Returns raw audio bytes.

Field	Type	Required	Description
model	string	yes
voice	string	yes
input	string	yes	Text to speak
response_format	string	no (default mp3)
speed	number	no (default 1.0)
persist	boolean	no (default false)	Persist audio via Files API; file id returned in the X-Brutor-File-Id response header
chat_id	string	no	Chat tag for the persisted row

Audio — Transcription

POST /v1/proxy/llm/audio/transcriptions — multipart/form-data; providers: Whisper, Deepgram, AssemblyAI.

Part	Type	Required	Description
file	binary	yes	Audio file
model	text	yes
language	text	no
prompt	text	no
temperature	text (number)	no
response_format	text	no
persist	text (bool)	no	Persist the source audio via Files API
chat_id	text	no

curl http://localhost:8100/v1/proxy/llm/audio/transcriptions \
  -H "Authorization: Bearer sk_brutor_api_..." \
  -F file=@meeting.mp3 \
  -F model=whisper-1

Video (async jobs)

Video generation is asynchronous: submit a job, then poll it.

Method	Path	Purpose
POST	/v1/proxy/llm/video/generations	Submit a generation job
GET	/v1/proxy/llm/video/jobs/{job_id}	Poll job status
DELETE	/v1/proxy/llm/video/jobs/{job_id}	Cancel a job

Submit request:

Field	Type	Required	Description
model	string	yes
prompt	string	yes
resolution	string	no	Token from the model’s supported_video_resolutions whitelist (e.g. "720p")
duration_seconds	integer	no	≤ model max_video_duration_seconds; in supported_video_durations when set
chat_id	string	no	Chat tag for delete-fanout

Job response (submit and poll):

{
  "job_id": "vjob_abc123",
  "status": "succeeded",
  "output_file_id": "file_xyz",
  "cost_usd": 0.35,
  "brutor_model_id": "3f9c2d1e-...",
  "created_at": "2026-07-02T10:00:00Z",
  "updated_at": "2026-07-02T10:02:14Z"
}

output_file_id is set when status is succeeded; render it via GET /v1/proxy/files/{file_id}/url. error is set on failure.

Batch API

Base: /v1/proxy/batch. Async batch jobs over the same governed LLM surface.

Method	Path	Purpose
POST	/v1/proxy/batch/jobs	Create a batch job
GET	/v1/proxy/batch/jobs	List jobs (?status&limit&offset or ?page&page_size)
GET	/v1/proxy/batch/jobs/{batch_job_id}	Job status/detail
POST	/v1/proxy/batch/jobs/{batch_job_id}/cancel	Cancel
GET	/v1/proxy/batch/jobs/{batch_job_id}/results	Per-request results

Create body: {model | model_id, endpoint (default "/v1/chat/completions"), completion_window (default "24h"), requests: [{custom_id, body}], description?, metadata?} → {id, status, total_requests, endpoint, completion_window, created_at}.

Files API

Base: /v1/proxy/files. Multimodal file storage backing images/audio/video (S3/MinIO). Max upload 256 MB (per-modality limits apply below that).

Method	Path	Purpose
POST	/v1/proxy/files	Multipart upload — parts: file, kind (required); chat_id, model_id, display_name (optional)
GET	/v1/proxy/files/{file_id}	Metadata: {id, tenant_id, kind, mime_type, size_bytes, content_hash, chat_id, model_id, display_name, created_at, expires_at, deleted_at, access_count}
GET	/v1/proxy/files/{file_id}/url	Signed URL: {url, expires_in_secs, file_id, mime_type}
GET	/v1/proxy/files/{file_id}/content	Raw bytes
DELETE	/v1/proxy/files/{file_id}	Delete

Errors

All errors are JSON: {"error": "<message>"}.

Status	Meaning
400	Validation error (bad body, unknown model mode, unsupported param)
401	Missing/invalid credentials (WWW-Authenticate: Bearer)
403	Authenticated but forbidden — model not in the caller’s resource group, inactive user, blocked by policy
404	Unknown model / job / file
413	Upload exceeds size cap
429	Rate limit / quota exceeded (RPM/TPM/budget caps)
502	Upstream provider error (proxied)
5xx	Gateway error

Budget caps

Dollar-budget caps block requests only when the limit is configured with hard_stop: true; otherwise they warn. Token and request caps always block. See Limits.

Response headers

Header	Meaning
X-Usage-Warning	Set when approaching a configured limit, e.g. Daily budget at 85%: $4.25 / $5.00
X-Brutor-File-Id	TTS only, when persist: true — id of the persisted audio file

Approval retry: when a request is blocked pending human approval, resolve the approval (see Portal API — approvals) and retry the original request with the one-time X-Approval-Token: <token> header. Walkthrough: Approvals.