Deployment Architecture & Engineering Foundations

Supporting section: technical debt cleanup, K8s deployment, CI/CD pipelines, and environment management. Primary Audience: Platform / DevOps Engineers

---

1. Design Goals

The Taimoe Enterprise AI Gateway is designed for financial-grade platform operations, satisfying:

• Cloud-Agnostic / Hybrid Cloud: Unified K8s manifests deployable on GKE, on-premises K8s, or multi-cloud clusters.
• Reproducibility: Environment provisioning fully automated via reproducible scripts.
• Observability: Complete visibility into platform health, logs, and metrics.
• Zero-Downtime Upgrades: Rolling update support without service interruption.

---

2. Deployment Architecture

2.1 Core Components

┌─────────────────────────────────────────────────────────┐
│                        K8s Cluster                      │
│                                                         │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────┐  │
│  │  Frontend   │  │   Gateway   │  │   Worker        │  │
│  │  (Nginx)    │  │  (FastAPI)  │  │ (Async Tasks,   │  │
│  │             │  │             │  │  AuditLog flush)│  │
│  └─────────────┘  └─────┬───────┘  └────────┬────────┘  │
│         │               │                    │          │
│         └───────────────┴────────┬───────────┘          │
│                                  │                      │
│                  ┌───────────────┴───────────────┐      │
│                  │                               │      │
│            ┌─────▼─────┐                  ┌──────▼─────┐│
│            │ PostgreSQL│                  │   Redis    ││
│            │  (StatefulSet                │  (Rate     ││
│            │   or RDS) │                  │   Limit)   ││
│            └───────────┘                  └────────────┘│
└─────────────────────────────────────────────────────────┘
                    │
        ┌───────────┴───────────┐
        │                       │
   ┌────▼─────────┐      ┌──────▼─────────┐
   │ ClickHouse   │      │  LLM Providers │
   │ (Analytics)  │      │  (OpenAI /     │
   │              │      │   Vertex AI /  │
   │              │      │   Local LLM)   │
   └──────────────┘      └────────────────┘

2.2 K8s Manifest Manifest Status

Existing manifests under k8s/:

• backend.yaml (FastAPI Gateway Engine)
• postgres.yaml
• redis.yaml

Manifest Additions:

• frontend.yaml (Nginx + Static assets)
• worker.yaml (Background worker tasks: audit flush, billing sync)
• clickhouse.yaml (Traffic analytics database)
• ingress.yaml (Edge routing + TLS termination)
• configmap.yaml (Environment configurations)
• hpa.yaml (Horizontal Pod Autoscaler, auto-scaling Gateway pods based on CPU / API concurrency)

2.3 Environment Tiers

Environment	Deployment Target	Purpose
local	docker-compose	Local developer environment
dev	GKE dev cluster	Integration testing, feature demos
staging	GKE staging cluster	User Acceptance Testing (UAT), performance testing
prod	GKE prod / On-prem K8s	Production workloads

Each environment is managed via a dedicated kustomize overlay or Helm values file.

---

3. Hybrid / Multi-Cloud Strategy

3.1 Abstracting Cloud-Specific Dependencies

Dependency	GCP Default	On-Premises / Multi-Cloud Alternative
Secret Management	Secret Manager	K8s Secrets / HashiCorp Vault
Object Storage	GCS (Payload Archive)	AWS S3 / MinIO
Observability	Cloud Logging / Monitoring	Loki / Prometheus / Grafana
Database (prod)	Cloud SQL	PostgreSQL StatefulSet
LLM Provider	Vertex AI	OpenAI / Anthropic / Self-hosted (vLLM)

Implementation Pattern: Backend adapters encapsulate cloud-specific variations:

• infra/secrets.py interface with implementations: GcpSecretManager / K8sSecret / VaultClient
• infra/object_store.py interface with implementations: GcsStore / S3Store / MinioStore
• Environment variable TAIMOE_CLOUD_PROFILE=gcp|onprem|aws toggles runtime implementations.

---

4. CI/CD Pipeline

4.1 CI (GitHub Actions / GitLab CI)

on: [push, pull_request]
jobs:
  gateway:
    - lint (ruff)
    - typecheck (mypy)
    - test (pytest + coverage)
    - build docker image
    - push to registry (only on main/tag)
  frontend:
    - lint (eslint)
    - build (vite build)
    - bundle size check

4.2 CD

• main branch pushes deploy automatically to dev
• Release tags v*.*.* trigger automated deployments to staging
• Manual approval gate required for prod deployments
• Deployment Tooling: Argo CD (GitOps) or kubectl apply + kustomize

4.3 Database Migrations

• Tooling: Alembic (SQLModel compatible)
• Migrations shipped inside backend container images
• Deployment Flow: pre-deploy Job executes migrations prior to rolling update of Gateway pods
• Rollback Strategy: All migrations require corresponding downgrade() methods

---

5. Local Developer Experience

5.1 One-Command Startup

# One-command onboarding
git clone ...
cd taimoe-ai-gateway
make up           # docker-compose up postgres + redis + clickhouse + gateway + frontend
make seed         # seed test data (org / project / user / virtual key)
make logs         # inspect service logs
open http://localhost:5173

---

6. Meta-Observability

Platform self-monitoring signals:

Signal	Tooling
Gateway logs	stdout → Fluent Bit → Loki / Cloud Logging
Gateway metrics	Prometheus /metrics endpoint (FastAPI middleware)
Gateway tracing	OpenTelemetry → Tempo / Cloud Trace
Database metrics	postgres_exporter / clickhouse_exporter

---

7. Security Hardening

• Mandatory HTTPS on all external ingress endpoints (cert-manager + Let's Encrypt)
• Gateway container runs non-root with read-only root filesystems
• Zero secrets stored in Git (enforced via K8s Secrets / Sealed Secrets)
• Database connections enforce SSL
• Automated image vulnerability scanning (Trivy)
• Virtual Keys stored strictly as one-way hashes in database tables.

---

Status: CURRENT — Aligned with platform deployment architecture.