Skip to main content

Quotas & Budgets

To prevent API abuse, maintain Quality of Service (QoS), and control LLM consumption costs across teams, Taimoe Enterprise AI Gateway provides comprehensive Quotas & Budgets governance.

This page consists of three main features: Rate Limits & Current Usage, Spend This Period, and Team Budgets.

Rate Limits & Current Usage​

The gateway uses a 1-minute fixed window to track API request rates and token consumption, polling usage counters automatically every 20 seconds.

Rate limits are enforced across three hierarchy levels:

1. Teams​

  • Description: Defines team-wide ceilings for Requests Per Minute (RPM) and Tokens Per Minute (TPM). Applies to all Virtual Keys and Agent calls owned by the team.

2. Agents​

  • Description: Caps usage per Agent within the team's total limit, preventing any single Agent from exhausting team resources. Supports RPM and TPM limits.

3. Virtual Keys​

  • Description: Caps request rates (RPM) per Virtual Key. Token consumption is bounded by team and agent limits.

Editing Rate Limits​

  1. Click the Edit icon next to any Team, Agent, or Virtual Key row.
  2. In the dialog, enter the desired Requests per minute and Tokens per minute.
  3. Set value to 0 for Unlimited.
  4. Click "Save".

Usage Indicators​

Usage progress bars indicate current consumption relative to the configured limit:

  • Blue / Default: Usage below 80%.
  • Yellow (Warning): Usage reached 80% or higher.
  • Red (Error): Usage reached or exceeded 100%. Subsequent requests will be throttled with HTTP 429 RateLimitExceeded.

Spend This Period​

The gateway calculates real-time Month-to-Date (MTD) spend and token metrics based on request logs and model price schedules.

  • Overview Cards: Displays total Month-to-Date Total Spend (USD) and Total Requests.
  • Unpriced Traffic Warning: If traffic is recorded for unpriced models ($0 rate), a warning banner prompts administrators to configure rates under Models → Pricing to ensure accurate budget accounting.
  • Multi-dimensional Breakdown Tables:
    • By Team: Spend and token counts aggregated per team.
    • By Agent: Spend aggregated per agent.
    • By Model: Spend and token counts aggregated per model alias.

Team Budgets​

Team Budgets enforce hard spending caps (in USD) per team. Once a team reaches its budget cap, subsequent gateway requests are refused until the budget period rolls over or the cap is increased.

Configuring Team Budgets​

  1. In the Team Budgets table, click the Edit icon next to the target team.
  2. Enter the Budget (USD) limit (set to 0 to disable budget capping).
  3. Select the Reset Period:
    • Monthly: Resets spend at the start of each month.
    • Yearly: Resets spend annually.
    • Never: Accumulates spend indefinitely without automatic resets.
  4. Click "Save".

Budget Status Badges​

  • No cap set: Budget capping is disabled.
  • Near cap (Yellow Badge): Team spend is approaching its cap limit.
  • Blocked (Red Badge): Team spend has met or exceeded its cap. Gateway requests are currently refused.

Resetting Accrued Spend​

If a team hits its budget limit during an urgent workload, administrators can click the Reset icon on the team row to reset accrued spend to $0 and immediately restore API service.