Budgets, Quotas & Rate Limits

Every limit in Brutor hangs off a resource group, optionally refined per bound model or MCP server. Limits are always restrictive down the tree: a child group can tighten what an ancestor set, never loosen it — the effective value is the minimum across the ancestor chain.

Enforcement is O(1): quotas and throughput are checked against Redis counters at admission time, not by scanning the usage ledger, so limits add no meaningful latency.

LLM limits on a resource group

Configured in the Admin UI on the group’s LLM Limits tab, or via one PATCH (the endpoint replaces the whole sub-document — send full desired state):

curl -X PATCH http://localhost:5050/v1/admin/resource-groups/{group_id}/llm-global-limits \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "llm_global_limits": {
      "budget": {
        "daily_limit_usd": 50.0,
        "monthly_limit_usd": 1000.0,
        "daily_warning_percent": 80,
        "monthly_warning_percent": 80,
        "hard_stop": true
      },
      "tokens":   { "daily_limit": 2000000, "monthly_limit": 40000000, "daily_warning_percent": 80 },
      "requests": { "daily_limit": 10000, "daily_warning_percent": 80 },
      "throughput": {
        "max_requests_per_minute": 120,
        "max_input_tokens_per_minute": 200000,
        "max_output_tokens_per_minute": 50000,
        "max_usd_per_minute": 0.50,
        "cooldown_seconds": 30
      },
      "concurrency": { "max_concurrent": 8 }
    }
  }'

The response echoes the stored config plus the resolved view: llm_global_limits (this group’s own), inherited_llm_limits (from ancestors), and effective_llm_limits (what actually enforces).

The four limit families

Family	Fields	Window	Blocks?
Budget ($)	daily_limit_usd, monthly_limit_usd, *_warning_percent, hard_stop	daily / monthly	Only if hard_stop: true — otherwise warn-only
Tokens	daily_limit, monthly_limit, *_warning_percent	daily / monthly	Always
Requests	daily_limit, monthly_limit, *_warning_percent	daily / monthly	Always
Throughput	max_requests_per_minute (RPM), max_input_tokens_per_minute (ITPM), max_output_tokens_per_minute (OTPM), max_usd_per_minute, request_burst, tokens_burst, usd_burst, cooldown_seconds	per-minute token buckets	Always
Concurrency	max_concurrent	simultaneous in-flight	Always

The dollar-budget rule

A $ budget blocks traffic only when hard_stop: true. Without it, exceeding the budget raises usage alerts and warning headers but requests keep flowing. Token and request quotas, throughput caps, and concurrency caps always block when exceeded. If you expect a spend cap to actually stop traffic, set hard_stop.

How throughput buckets behave

• RPM is checked before the request is admitted.
• ITPM / OTPM / $-per-minute are charged after the response, from actual prompt/completion tokens and billed cost — the pre-request gate only verifies those buckets aren’t already exhausted or in cooldown.
• When any bucket trips, cooldown_seconds imposes a penalty lockout on the whole scope regardless of bucket refill.
• *_burst fields set token-bucket capacity (defaults to the per-minute rate), letting you allow short spikes above the sustained rate.

*_warning_percent thresholds don’t block anything — they generate usage alerts and the warning response header below.

Per-model and routing-group caps

Refine limits for a single bound model on the binding row — effective is min(group-wide, per-model):

curl -X PATCH http://localhost:5050/v1/admin/resource-groups/{group_id}/llm-models/{model_id}/limits \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "limits": {
      "throughput": { "max_requests_per_minute": 30, "max_output_tokens_per_minute": 20000 }
    }
  }'

Two more places carry throughput caps, protecting the provider side rather than the consumer side:

• Model definition (/v1/admin/llms/{id}): rate_limit_rpm, rate_limit_tpm, max_concurrent_requests — the model’s own capacity ceiling across all groups.
• Routing group (/v1/admin/llm-router/routing-groups/{id}): rpm_limit, tpm_limit, max_parallel_requests — caps on the whole load-balanced pool.

Parameter governance (clamps)

Beyond volume, you can govern request parameters per group via llm-global-governance:

curl -X PATCH http://localhost:5050/v1/admin/resource-groups/{group_id}/llm-global-governance \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "governance": {
      "temperature":    { "max": 0.7, "enforce": "clamp" },
      "context_window": { "max_output_tokens": 4096, "enforce": "clamp" }
    }
  }'

enforce is "clamp" (silently cap the value) or "reject" (fail the request). context_window also supports max_input_tokens. The governance document additionally covers tool_use (allow/block tool lists, per-request/hour/day tool-call caps, approval_required mode — see Tool approvals) and system_prompt (mandatory prepend/append fragments, banned patterns).

Like limits, governance merges restrictively across the ancestor chain (min-wins), and per-model overrides exist at .../llm-models/{model_id}/governance.

MCP and skill limits

MCP tool traffic has its own limit document on the group — resource-exhaustion controls rather than token economics:

curl -X PATCH http://localhost:5050/v1/admin/resource-groups/{group_id}/mcp-global-limits \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "mcp_global_limits": {
      "frequency": {
        "max_calls_per_minute": 60,
        "max_calls_per_hour": 1000,
        "max_calls_per_day": 5000,
        "burst_limit": 10,
        "cooldown_seconds": 30
      },
      "execution": {
        "timeout_seconds": 30,
        "max_concurrency_per_user": 3,
        "max_concurrency_per_group": 10
      },
      "data": {
        "max_payload_size_mb": 5,
        "max_rows_per_query": 1000,
        "max_items_per_response": 200
      }
    }
  }'

Per-server refinements live at PATCH .../mcp-servers/{server_id}/limits. Skills have the parallel skill-global-limits document and per-skill rows at .../agent-skills/{skill_id}/limits.

What a limited client sees

When a quota or throughput cap blocks a request, the proxy returns 429 Too Many Requests with an error body describing which limit tripped. Two response headers are also stamped on responses as consumption approaches limits:

Header	Meaning
x-brutor-usage-utilization	Current utilization percentage of the tightest applicable limit
x-brutor-usage-warning	Human-readable warning once a *_warning_percent threshold is crossed, e.g. Daily budget at 80%: $80.00 / $100.00

Handle it like any provider rate limit:

curl

curl -s -D - http://localhost:8100/v1/proxy/llm/chat/completions \
  -H "Authorization: Bearer sk_brutor_api_..." \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "hi"}]}'
# HTTP/1.1 429 Too Many Requests
# x-brutor-usage-utilization: 100

Python

from openai import OpenAI, RateLimitError

client = OpenAI(base_url="http://localhost:8100/v1/proxy/llm",
                api_key="sk_brutor_api_...")
try:
    r = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "hi"}],
    )
except RateLimitError as e:
    # Group quota or throughput cap tripped — back off and retry later
    print("Rate limited by the gateway:", e)

Node.js

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:8100/v1/proxy/llm",
  apiKey: "sk_brutor_api_...",
});

try {
  await client.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: "hi" }],
  });
} catch (err) {
  if (err.status === 429) {
    // Gateway quota/throughput limit — inspect x-brutor-usage-* headers
  }
}

Related pages

• Usage alerts — the warning-threshold side of these limits
• Mission Control — budget burn-down and per-group consumption
• Resource groups — why limits merge restrictively down the tree