Budgets, Quotas & Rate Limits

Every limit in Brutor hangs off a resource group, optionally refined per bound model or MCP server. Limits are always restrictive down the tree: a child group can tighten what an ancestor set, never loosen it — the effective value is the minimum across the ancestor chain.

Enforcement is O(1): quotas and throughput are checked against Redis counters at admission time, not by scanning the usage ledger, so limits add no meaningful latency.

Run-level ceilings

Everything on this page is scoped to a resource group, a model or an MCP server, over a daily or monthly window. One control is scoped to a single run, and it lives with the operating envelope instead: max_cost_per_run_usd, max_tokens_per_run, max_llm_calls_per_run, max_delegation_depth_per_run and max_duration_seconds_per_run.

The distinction matters for agents. A rate limit caps velocity, so a runaway loop under its RPM simply runs all day at exactly the permitted rate; a group budget stops it only after it has spent the group’s whole daily allowance and started 429-ing every other system bound to that group. Run ceilings stop the run that caused the problem, at its own expense. Declare them on the AI System’s envelope, where they are also graded in the Assurance Report.

LLM limits on a resource group

Configured in the Admin UI on the group’s LLM Limits tab, or via one PATCH (the endpoint replaces the whole sub-document — send full desired state):

curl -X PATCH http://localhost:5050/v1/admin/resource-groups/{group_id}/llm-global-limits \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "llm_global_limits": {
      "budget": {
        "daily_limit_usd": 50.0,
        "monthly_limit_usd": 1000.0,
        "daily_warning_percent": 80,
        "monthly_warning_percent": 80,
        "hard_stop": true
      },
      "tokens":   { "daily_limit": 2000000, "monthly_limit": 40000000, "daily_warning_percent": 80 },
      "requests": { "daily_limit": 10000, "daily_warning_percent": 80 },
      "throughput": {
        "max_requests_per_minute": 120,
        "max_input_tokens_per_minute": 200000,
        "max_output_tokens_per_minute": 50000,
        "max_usd_per_minute": 0.50,
        "cooldown_seconds": 30
      },
      "concurrency": { "max_concurrent": 8 }
    }
  }'

The response echoes the stored config plus the resolved view: llm_global_limits (this group’s own), inherited_llm_limits (from ancestors), and effective_llm_limits (what actually enforces).

It also carries capacity: the budget this group draws on and how much of it is unclaimed. The same figures drive the sibling-sum check, so the remaining capacity the console shows is exactly what the API will accept. The budget holder is the nearest ancestor that sets limits, which is often not the immediate parent — estates commonly set a budget once at the organisation and leave the departments below it unset. parent_name names the holder, parent_is_immediate says whether it is the direct parent, siblings lists the other groups drawing on that budget (excluding this group’s own branch), and available is the holder’s limit minus what they have claimed. A limit above inherited_llm_limits is refused on PATCH, and so is one above available — the sum of this group’s value and everything else drawing on that budget may not exceed it.

The four limit families

Family	Fields	Window	Blocks?
Budget ($)	daily_limit_usd, monthly_limit_usd, *_warning_percent, hard_stop	daily / monthly	Only if hard_stop: true — otherwise warn-only
Tokens	daily_limit, monthly_limit, *_warning_percent	daily / monthly	Always
Requests	daily_limit, monthly_limit, *_warning_percent	daily / monthly	Always
Throughput	max_requests_per_minute (RPM), max_input_tokens_per_minute (ITPM), max_output_tokens_per_minute (OTPM), max_usd_per_minute, request_burst, tokens_burst, usd_burst, cooldown_seconds	per-minute token buckets	Always
Concurrency	max_concurrent	simultaneous in-flight	Always

The dollar-budget rule

A $ budget blocks traffic only when hard_stop: true. Without it, exceeding the budget raises usage alerts and warning headers but requests keep flowing. Token and request quotas, throughput caps, and concurrency caps always block when exceeded. If you expect a spend cap to actually stop traffic, set hard_stop.

How throughput buckets behave

• RPM is checked before the request is admitted.
• ITPM / OTPM / $-per-minute are charged after the response, from actual prompt/completion tokens and billed cost — the pre-request gate only verifies those buckets aren’t already exhausted or in cooldown.
• When any bucket trips, cooldown_seconds imposes a penalty lockout on the whole scope regardless of bucket refill.
• *_burst fields set token-bucket capacity (defaults to the per-minute rate), letting you allow short spikes above the sustained rate.

*_warning_percent thresholds don’t block anything — they generate usage alerts and the warning response header below.

Per-model and routing-group caps

Refine limits for a single bound model on the binding row — effective is min(group-wide, per-model):

curl -X PATCH http://localhost:5050/v1/admin/resource-groups/{group_id}/llm-models/{model_id}/limits \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "limits": {
      "throughput": { "max_requests_per_minute": 30, "max_output_tokens_per_minute": 20000 }
    }
  }'

Two more places carry throughput caps, protecting the provider side rather than the consumer side:

• Model definition (/v1/admin/llms/{id}): rate_limit_rpm, rate_limit_tpm, max_concurrent_requests — the model’s own capacity ceiling across all groups.
• Routing group (/v1/admin/llm-router/routing-groups/{id}): rpm_limit, tpm_limit, max_parallel_requests — caps on the whole load-balanced pool.

Parameter governance (clamps)

Beyond volume, you can govern request parameters per group via llm-global-governance:

curl -X PATCH http://localhost:5050/v1/admin/resource-groups/{group_id}/llm-global-governance \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "governance": {
      "temperature":    { "max": 0.7, "enforce": "clamp" },
      "context_window": { "max_output_tokens": 4096, "enforce": "clamp" }
    }
  }'

enforce is "clamp" (silently cap the value) or "reject" (fail the request). context_window also supports max_input_tokens. The governance document additionally covers tool_use (allow/block tool lists, per-request/hour/day tool-call caps, approval_required mode — see Tool approvals) and system_prompt (mandatory prepend/append fragments, banned patterns).

Like limits, governance merges restrictively across the ancestor chain (min-wins), and per-model overrides exist at .../llm-models/{model_id}/governance.

A child group may only tighten what it inherits. A PATCH that would relax an inherited value is refused with 400, naming each field:

{
  "detail": "Cannot set governance looser than the policy inherited from parent groups: Max Temperature: 1.4 is looser than the inherited maximum of 1.0"
}

This applies to the fields the merge actually constrains: numeric ceilings, enforce (a child may go clamp → reject, never the reverse), tool access, and allowed_tools (which merges by intersection, so naming a tool the ancestors do not allow grants nothing). Fields that merge by union — blocked_tools, banned patterns, mandatory prompt fragments — take any child value, because a child can only add to them.

The same rule governs the MCP, skill and media governance endpoints, and the per-model, per-server and per-skill overrides: a per-item value looser than the group’s effective ceiling is refused, because the restrictive merge would discard it.

Media governance is a {mode: {field: value}} document with per-field directions: whitelists (voice_whitelist, dim_whitelist, …) intersect, ceilings (max_input_characters, max_n, max_duration_seconds, …) take the minimum, speed_min is a floor and takes the maximum, and require_image_input is a requirement a child cannot drop. An empty whitelist means unconstrained, not “nothing allowed”.

MCP and skill limits

MCP tool traffic has its own limit document on the group — resource-exhaustion controls rather than token economics:

curl -X PATCH http://localhost:5050/v1/admin/resource-groups/{group_id}/mcp-global-limits \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "mcp_global_limits": {
      "frequency": {
        "max_calls_per_minute": 60,
        "max_calls_per_hour": 1000,
        "max_calls_per_day": 5000,
        "burst_limit": 10,
        "cooldown_seconds": 30
      },
      "execution": {
        "timeout_seconds": 30,
        "max_concurrency_per_user": 3,
        "max_concurrency_per_group": 10
      },
      "data": {
        "max_payload_size_mb": 5,
        "max_rows_per_query": 1000,
        "max_items_per_response": 200
      }
    }
  }'

Per-server refinements live at PATCH .../mcp-servers/{server_id}/limits. Skills have the parallel skill-global-limits document and per-skill rows at .../agent-skills/{skill_id}/limits.

What a limited client sees

When a quota or throughput cap blocks a request, the proxy returns 429 Too Many Requests with an error body describing which limit tripped. Two response headers are also stamped on responses as consumption approaches limits:

Header	Meaning
x-brutor-usage-utilization	Current utilization percentage of the tightest applicable limit
x-brutor-usage-warning	Human-readable warning once a *_warning_percent threshold is crossed, e.g. Daily budget at 80%: $80.00 / $100.00

Handle it like any provider rate limit:

curl

curl -s -D - http://localhost:8100/v1/proxy/llm/chat/completions \
  -H "Authorization: Bearer sk_brutor_api_..." \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "hi"}]}'
# HTTP/1.1 429 Too Many Requests
# x-brutor-usage-utilization: 100

Python

from openai import OpenAI, RateLimitError

client = OpenAI(base_url="http://localhost:8100/v1/proxy/llm",
                api_key="sk_brutor_api_...")
try:
    r = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "hi"}],
    )
except RateLimitError as e:
    # Group quota or throughput cap tripped — back off and retry later
    print("Rate limited by the gateway:", e)

Node.js

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:8100/v1/proxy/llm",
  apiKey: "sk_brutor_api_...",
});

try {
  await client.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: "hi" }],
  });
} catch (err) {
  if (err.status === 429) {
    // Gateway quota/throughput limit — inspect x-brutor-usage-* headers
  }
}

Related pages

• Usage alerts — the warning-threshold side of these limits
• Mission Control — budget burn-down and per-group consumption
• Resource groups — why limits merge restrictively down the tree