AI gateway

Plan requirement: all plans can monitor. Protect and higher can enforce request policy. Business and Enterprise can inspect compatible responses.

The AI gateway is for internal services that call model providers through an OpenAI-compatible API, Anthropic Messages, or Gemini generateContent. It is separate from the Next.js control plane and should be deployed as its own service.

Configure an application

1. Provision an AI gateway key under Sensors & keys. Browser, desktop, and MCP keys are rejected by the gateway.
2. Deploy the gateway with its Orthalon ingest/policy configuration and provider allowlist.
3. Store the sensor credential in the calling application's server-side secret store. Never include it in frontend JavaScript, a mobile application, or a public repository.
4. Change the application's model base URL from the provider URL to the gateway URL.
5. Keep the original model/provider authorization header expected by the configured upstream.
6. Send a harmless request and verify the provider response.
7. Send invented detector test data in Monitor mode and confirm an Orthalon event.
8. On Protect or higher, validate one redaction or block rule before production.

The gateway does not keep an endpoint policy cache. Each incoming model request loads the current validated policy for its organization and uses one stable snapshot through the corresponding response. A newly published policy applies to the next request without restarting the gateway or configuring an endpoint notification client.

For a request that requires approval, the gateway returns 403 with a requestId and does not call the provider. A reviewer approves the immutable content fingerprint, detector set, provider domain, and model restriction in Orthalon. The calling service then retries the identical request. The gateway atomically consumes the bounded grant and forwards it once (or within the approved use/expiry limit). Changed content creates a different approval; raw request text is never stored with the approval.

The operator must configure UPSTREAM_OPENAI_BASE_URL, UPSTREAM_ANTHROPIC_BASE_URL, and UPSTREAM_GEMINI_BASE_URL only when using an approved compatible provider endpoint instead of the vendor default. Upstream URLs must use HTTPS in production and cannot contain credentials, query strings, or fragments. Orthalon does not follow upstream redirects. Keep provider keys in the calling service; do not put them in an upstream URL or in Orthalon settings.

Applications can send x-orthalon-agent-id and x-orthalon-agent-name when a request belongs to a known internal agent. Orthalon records the agent and model as organization assets without storing the prompt.

Security behavior

The gateway must handle request content in memory to proxy it. Orthalon does not persist raw prompts by default; stored telemetry is limited to detector metadata, decisions, and placeholder-only excerpts. Configure strict upstream allowlists, request-size limits, timeouts, and safe logging.

Redaction placeholders are unique to one request. Compatible text and streaming responses restore only placeholders created for that request, including placeholders split across server-sent event chunks. Tool and function arguments and results are scanned recursively instead of being treated as an unprotected side channel.

The complete bounded request is scanned in overlapping chunks, so content after the first 32,000 characters is not silently skipped. One policy snapshot is used for the request and its response stream. JSON, text, and server-sent-event responses are inspectable. For enforcing return-path policy, supported response streams are held within the configured size bound until their cross-frame content can be approved, redacted, or blocked safely. On Business and Enterprise, supported provider error bodies are inspected under the same rule; an unexpected binary response fails closed rather than bypassing return-path policy.

Send the Orthalon credential only in x-orthalon-key; keep the provider credential in the provider's normal Authorization or x-api-key header. Rotate the gateway credential in the caller's secret store before expiry. A leaked Orthalon key alone cannot spend the provider account because it is not a provider credential.

Coverage boundary

Only calls sent through the gateway are covered. Direct provider calls, browser sessions, consumer AI websites, and clients that ignore the configured base URL need a different Orthalon layer. OpenAI Chat/Responses, Anthropic Messages, and Gemini content generation have native handling; other provider-specific APIs must be acceptance-tested before a production rollout.

Provider-native image, audio, video, and file references cannot be content- inspected by this JSON gateway. Enforcing plans reject those request parts. Scan supported readable files with endpoint file protection first, then send the approved text or a provider reference outside claims of gateway content inspection. Native Gemini routing accepts only generateContent and streamGenerateContent; API keys in query parameters are rejected.