Skip to main content

AI Gateway Request Lifecycle

The AI Gateway is the governed path for model calls. It is a model proxy, not the endpoint that executes an entire Agent workflow.

AI Gateway authentication, governance, routing, retry, and telemetry lifecycle

1. Authenticate​

The Gateway accepts a Virtual Key through Authorization: Bearer ... or x-api-key. The stored key is hashed, can be deactivated, and identifies the owning Team. Missing, unknown, or inactive keys are rejected before provider routing.

2. Apply limits and authorization​

The current request path checks Virtual Key and Team request/token windows, then the Team spending budget. It also evaluates whether the key may use the selected model alias. Rejected requests receive a stable request ID and are written to gateway audit data when the audit backend is available.

3. Resolve the upstream model​

The model name in the request is a Taimoe Model Alias. The alias resolves to a Provider and its upstream model identifier. Provider configuration supplies the endpoint and credential strategy used for the outbound call.

4. Send and retry​

Upstream retry is available. A Virtual Key can inherit platform defaults or override retry count, wall-clock budget, and status codes. A value of 0 retries means fail fast; null means inherit.

Governance checks run once for the logical request, not once per retry attempt. The retry summary is attached to the same audit record and request ID. Authentication failures may refresh provider credentials once before another attempt; deterministic client errors are not retried by default.

5. Observe and audit​

The Gateway returns X-Request-ID so callers can correlate a response with Requests and Audit Logs. Successful calls can contribute token usage, cost, conversation turns, and telemetry. Failed or limited calls retain the same correlation identifier.

Governance boundary​

Only requests sent through the Gateway receive this lifecycle. An Agent Runtime that calls a provider directly with its own credentials bypasses Gateway authentication, routing, retry, limits, and gateway audit. Preventing that path requires provider IAM and network controls in addition to Taimoe configuration.

Current protocol surface​

Gemini and Vertex AI compatible generateContent and streamGenerateContent routes are available. Broader provider compatibility should be treated as adapter-specific rather than assuming every provider exposes an identical protocol.