AI Model Platform

Multi-provider LLM gateway to manage providers, routing, prompts, RAG, policies, quotas, and usage.

ⓘ Legacy /ai-gateway redirects here. Use ?tab= to open a tab directly (overview, providers, models, routing, features, prompts, rag, policies, usage).

Architecture

▥ Feature / Assistant / Intelligence / Playground
↓
AiGatewayService.complete() / embed() / ragQuery()
↓
♢

Security scan + redact

Secrets, PII, injection, moderation

↝

Route select

Rules → feature binding → default / cheapest approved

♆

Provider adapter

OpenAI-compatible and vendor APIs

▣

Policy + quota enforcement

Allow/deny, quota, rate limiting

▤

AiRequestLog

Tokens, cost, latency, status

♢ Credentials at rest: AES-256-GCM via AiCredentialCryptoService. Prefer platform secret refs (apiKeySecretRef) over inline keys.
♧ First visit can POST .../bootstrap to seed the provider/model catalog for this organization.

Platform tabs

◔
Overview

Provider/model counts, analytics and playground

⌂
Providers

Catalog instances, health check, credentials

◇
Models

Register models, approve / unapprove

↯
Routing

Priority rules, failover, cost, latency, geography

⊞
Features

Bind features to preferred models (defaults)

▤
Prompts

Org prompt templates (version, test)

▤
Documents

Knowledge bases, ingest, RAG query

♢
Policy & Quotas

Policies and spend / rate quotas

▥
Usage

Request volume, cost, latency analytics

♙ Playground calls POST .../complete with a feature key, task, and privacy hint.

Provider catalog (gateway)

Kinds in the engine catalog include commercial, self-hosted, and enterprise providers.

▥

Commercial

OpenAI, Azure OpenAI, Anthropic, Google Gemini, AWS Bedrock, Cohere, Mistral, xAI, Together, Fireworks

⌂

Self-hosted

Ollama, vLLM, Hugging Face TGI, NVIDIA NIM, LM Studio, Open WebUI, llama.cpp

♢

Enterprise

Azure AI Foundry, Databricks Mosaic, OpenShift AI, IBM watsonx, Vertex AI, SageMaker

♧ Administration → Integrations → AI Providers currently exposes configure cards for OpenAI, Google Gemini, AWS Bedrock, Ollama, Vertex AI, and Azure AI Foundry. Other kinds can still be created via Model Platform / API.
Categories: COMMERCIAL SELF_HOSTED ENTERPRISE
Modalities: CHAT COMPLETION EMBEDDING MULTIMODAL

Feature bindings (defaults)

These defaults are used when no explicit model is selected and routing rules do not match.

Feature key Label Preferred model slug
◉ explain-terraform-plans Explain Terraform Plans gpt-5
✣ iac-generation IaC Generation claude-sonnet
∞ kubernetes-troubleshooting Kubernetes Troubleshooting llama-3-3
✦ cost-optimization Cost Optimization gemini-2-flash
♧ compliance-analysis Compliance Analysis bedrock-claude
∞ air-gapped-deployment Air-Gapped Deployment llama-3-3
♧ Routing order: explicit model id (Assistant) → enabled route rule (priority) → cost/latency preference → org default / cheapest approved non-embedding model. Failover model ids are used when the primary call fails.
♢

Security scan

Shared engine: apps/api/src/ai-gateway/engine/security.ts

  • Secrets (AWS keys, OpenAI sk-, GitHub PAT, generic api_key/password/token)
  • PII (email, SSN, card, phone) → redacted placeholders
  • Prompt injection / jailbreak patterns
  • Moderation (violence/malware phrasing)
Injection and moderation are BLOCKED by default. Statuses: SUCCESS, FAILED, BLOCKED, RATE_LIMITED, FAILOVER.
♧

Governance

Prompt lifecycle: DRAFT → PENDING_APPROVAL → APPROVED | REJECTED | ARCHIVED

Policy actions: ALLOW, DENY, REQUIRE_APPROVAL, REDACT, REROUTE

Memory scopes: SESSION, PROJECT, ORGANIZATION, KNOWLEDGE