Version: 2026.10

AI Chat Setup

Istari AI Chat is an assistant inside the web app that answers questions about the resources, systems, and jobs a user can see. It is part of the istari-platform Helm chart and is off until an administrator configures a model. This page covers what an IT administrator must deploy and configure before organization administrators and users can turn it on.

Audience: IT administrators and platform operators. For the end-user feature, see Istari AI Chat. This page reflects the October 2026 release (Registry Service 12.0, Frontend 9.0). If you are upgrading from the September 2026 release, read Upgrading from the September 2026 release first.

Your LLM, your responsibility

AI Chat sends user prompts and platform data to the LLM provider you configure. Review the LLM integration disclaimer and your provider's data-retention terms before enabling it for users.

How it works

The registry service (istari-fileservice) calls the LLM through LiteLLM. A connection names the provider family (bedrock, azure, openai, anthropic, vertex_ai, and so on), how to reach it, and how to authenticate; one or more models hang off each connection.
To answer questions about live platform data, the registry service calls the Istari MCP Service (istari-mcp) with the requesting user's own token. Without MCP, the assistant can only chat; it cannot look anything up.
The frontend shows the chat entry points (header button and sidebar item) to each user who turns Istari AI Chat on under Application → Experimental Features; see Istari AI Chat. If no model is configured, the first message answers with the toast Istari AI is not configured on this server. and the header button hides for the rest of the session.
Users can also enter a personal LLM key in the chat panel (Set an LLM key). It stays in the browser session and travels with each message to the registry service, which forwards it to the provider; the platform never stores it. The key replaces the connection's credential for that request only; the model and endpoint stay as configured, so a user cannot redirect traffic to another provider.

There are two ways to configure connections. They can be combined:

Mode	Who configures it	Change takes effect	Use when
Connections in the web app	Organization administrators, under Admin → LLMs and AI Instructions → LLMs; IT provides the identity	On the next chat message	Administrators manage providers and keys themselves; model changes without a redeploy
Env-declared connections	IT administrator, in the registry service secret or Helm values	After a registry service restart	Infrastructure-as-code deployments; keys held in your own variables; connections the web app must not change; locked catalogs

Env-declared connections are an overlay on the web-app catalog: they appear on the LLMs page marked Managed by infrastructure, are usable in chat alongside connections created in the web app, and are rebuilt from the environment on every request. New chats start on the catalog's Default model; when no web-app model is enabled, on the top-priority env-declared model. With FILE_SERVICE_AI__CONNECTIONS_LOCKED: "true" the environment is the whole catalog: connections created in the web app are ignored and every write from the admin page and API is rejected. If no model is configured at all, POST /api/v3/ai/chat/stream returns 404 and the web app shows the toast above.

Upgrading from the September 2026 release

The October 2026 release replaces the September configuration surface. The single-model variables are ignored from the first start; the host-slot variables are read once more, by the upgrade migration, and never afterward:

Single-model variables are ignored. FILE_SERVICE_AI__MODEL, __API_KEY, __BASE_URL, __API_VERSION, __EXTRA_PARAMS, and __PROVIDER no longer exist as settings. The registry service starts without complaint and treats them as unset. Move that configuration into an env-declared connection or let an organization administrator create the connection in the web app, then delete the variables.
Host slots become connections. The upgrade migration turns every endpoint an administrator created on the September Endpoints and Models page into a connection and repoints its models. For each slot FILE_SERVICE_AI_NN_HOST / _AUTHN pair the migration reads the live _AUTHN value once:
- Non-empty _AUTHN: a key connection, with the credential sealed in the database under the FILE_SERVICE_REMOTE_MASTER_SECRET the migration runs with, so that variable must hold its final value before the upgrade (see Option A).
- Empty _AUTHN: an identity connection (iam for Bedrock, Azure, and Vertex; none for an OpenAI-compatible gateway).
- Missing _AUTHN: a connection without a credential, and a warning is printed. An administrator then pastes the key on the LLMs page, or, for a Bedrock or Vertex connection (cloud identity only in the web app), you grant the pod identity; for Bedrock an env-declared connection with auth_mode: key also works.
- FILE_SERVICE_AI__CONNECTIONS_LOCKED: "true": the migrated connections are ignored like every other web-app connection, so redeclare the ones you need from the environment.
Keep the slot variables in place for the upgrade, confirm the connections on the LLMs page, and remove the variables afterward; nothing reads them once the migration has run.
The Host picker is gone. The LLMs page no longer lists provisioned hosts; administrators enter the provider, endpoint, and credential on the connection itself.
The blocking POST /api/v3/ai/chat endpoint is removed. POST /api/v3/ai/chat/stream is the only chat path, and the Istari Python Client no longer has a chat call.
FILE_SERVICE_AI__MCP_SERVER_URL and the operational variables in the reference are unchanged.

Prerequisites

Istari Platform installed per Istari Platform Installation, October 2026 release or later.
MCP Service enabled and reachable at https://mcp.<customer_istari_fqdn>/mcp/. Follow Optional: MCP Secret and Scenario 1: MCP Service enabled. Required for the assistant to read platform data.
An LLM provider you are licensed to use, and a credential for it: an API key; for AWS Bedrock and Google Vertex AI, either a cloud identity grant to the registry service pods or a static credential of that cloud; for Azure OpenAI / AI Foundry, the resource key or a role assignment. See Credentials by provider, Bedrock IAM permissions, and Azure roles.
A stable FILE_SERVICE_REMOTE_MASTER_SECRET on the registry service, at least 32 characters, in place before the upgrade. Keys pasted in the web app and credentials migrated from September host slots are sealed with it. If the Secure Connection Service is deployed the variable already exists; keep that value. See Option A.
Network egress from the registry service pods to the provider hostname on port 443, and to your own mcp.<customer_istari_fqdn>. See Network requirements.
An organization administrator account to create connections in the web app. See Granting admin access.

Option A: Connections in the web app

Organization administrators manage connections and models at Admin → LLMs and AI Instructions → LLMs (/admin/ai?tab=models; the page heading is Connections and Models). Each connection has a provider, an authentication mode, and the fields that provider needs; credentials are pasted once, sealed, and never shown again. Changes apply on the next chat message with no restart. This page covers what IT must provide for that to work; the screens themselves are described in AI Models (Admin Guide).

1. Have the sealing secret in place before the upgrade and before anyone pastes a key. A pasted credential, like a credential migrated from a September host slot, is encrypted with the registry service's FILE_SERVICE_REMOTE_MASTER_SECRET, the same variable the Secure Connection Service shares with the registry service. If the variable is already set, keep its value. Otherwise set it in the registry service secret to a random value of at least 32 characters (the registry service refuses to start with a shorter one) and never change it afterwards: a connection sealed under a different value cannot be decrypted, chat on that connection fails with a 503, and an administrator must re-enter the key.

2. Provide the identity where the provider needs one. The web app offers each provider with the authentication modes the registry service supports for it:

Provider in the web app	Authentication offered	What IT provides
AWS Bedrock	Cloud identity only	AWS credentials the pod can find: EKS Pod Identity or IRSA on the pod role, or static keys in the pod environment. See Credentials by provider
Azure AI Foundry / OpenAI	Cloud identity or key	Key: nothing. Identity: a role assignment and a pod identity, see Azure roles
Google Vertex AI	Cloud identity only	GKE Workload Identity, or `GOOGLE_APPLICATION_CREDENTIALS` pointing at a mounted service-account key
Anthropic, OpenAI, Together AI, Groq, OpenRouter, Mistral, Fireworks AI, DeepSeek, xAI, Perplexity	Key	Nothing; the administrator pastes the provider key
OpenAI / OpenAI-compatible and Ollama (self-hosted gateways)	Key or no credential	Network reachability to the gateway; No credential is for a gateway on a trusted network
Other LiteLLM provider…	Key or no credential	The LiteLLM provider id, typed by the administrator

3. Open the egress for the provider hostname per Network requirements.

With those in place the administrator adds the connection, adds models (typed, or picked from the provider's catalog with Add models), tests it with Test (a real one-token call with the configured credential or identity), and marks one model as Default.

Option B: Declare connections from the environment

FILE_SERVICE_AI__CONNECTIONS declares connections from the registry service environment, for infrastructure-as-code deployments. Add the variables to the istari-fileservice secret you created in Registry Service Secret, or to fileservice.env in istari-values.yaml as in Scenario 7: Custom Environment Variables. Keep the secrets themselves in the secret, not in values files.

Each connection authenticates with a provider key held in its own variable (auth_mode: key), with the pod's cloud identity (auth_mode: iam, for Bedrock, Azure, and Vertex), or with nothing (auth_mode: none, for an OpenAI-compatible gateway on a trusted network). See Credentials by provider.

Declare the connections

FILE_SERVICE_AI__CONNECTIONS is a map keyed by connection name; each connection holds a map of models keyed by model name. Set it either way:

Split across variables, one per field. Connection and model names are lowercased.
One JSON object in FILE_SERVICE_AI__CONNECTIONS, which keeps the names as written.

Keep secrets out of this configuration. A key-mode connection names the variable holding its secret in credential_env; the secret itself goes in that variable, in the registry service secret. The connection stores the variable name, never the value.

istari-fileservice-secret.yaml (excerpt)
stringData:
  # Bedrock with the pod's AWS identity (Pod Identity, IRSA, or static AWS keys in the pod environment);
  # see Bedrock IAM permissions for the policy
  FILE_SERVICE_AI__CONNECTIONS__BEDROCK__PROVIDER: "bedrock"
  FILE_SERVICE_AI__CONNECTIONS__BEDROCK__AUTH_MODE: "iam"
  FILE_SERVICE_AI__CONNECTIONS__BEDROCK__REGION: "us-east-1"
  FILE_SERVICE_AI__CONNECTIONS__BEDROCK__MODELS__CLAUDE__MODEL_ID: "us.anthropic.claude-opus-4-8"
  FILE_SERVICE_AI__CONNECTIONS__BEDROCK__MODELS__CLAUDE__PRIORITY: "1"
  FILE_SERVICE_AI__CONNECTIONS__BEDROCK__MODELS__CLAUDE__MAX_TOKENS: "8192"

  # Azure OpenAI / AI Foundry with the resource's API key held in AZURE_OPENAI_KEY_PROD;
  # set AUTH_MODE "iam" and drop CREDENTIAL_ENV for the pod's identity, see Azure roles
  FILE_SERVICE_AI__CONNECTIONS__AZURE__PROVIDER: "azure"
  FILE_SERVICE_AI__CONNECTIONS__AZURE__AUTH_MODE: "key"
  FILE_SERVICE_AI__CONNECTIONS__AZURE__CREDENTIAL_ENV: "AZURE_OPENAI_KEY_PROD"
  FILE_SERVICE_AI__CONNECTIONS__AZURE__BASE_URL: "https://<resource>.openai.azure.com"
  FILE_SERVICE_AI__CONNECTIONS__AZURE__EXTRA_PARAMS: '{"api_version": "2024-10-21"}'
  FILE_SERVICE_AI__CONNECTIONS__AZURE__MODELS__GPT4O__MODEL_ID: "<deployment-name>"
  FILE_SERVICE_AI__CONNECTIONS__AZURE__MODELS__GPT4O__PRIORITY: "2"

  # OpenAI with a key held in OPENAI_API_KEY_PROD
  FILE_SERVICE_AI__CONNECTIONS__OPENAI__PROVIDER: "openai"
  FILE_SERVICE_AI__CONNECTIONS__OPENAI__AUTH_MODE: "key"
  FILE_SERVICE_AI__CONNECTIONS__OPENAI__CREDENTIAL_ENV: "OPENAI_API_KEY_PROD"
  FILE_SERVICE_AI__CONNECTIONS__OPENAI__MODELS__GPT4O__MODEL_ID: "gpt-4o"

  AZURE_OPENAI_KEY_PROD: "..."
  OPENAI_API_KEY_PROD: "sk-..."

  FILE_SERVICE_AI__MCP_SERVER_URL: "https://mcp.<customer_istari_fqdn>/mcp/"

The same three connections as one JSON value:

istari-fileservice-secret.yaml (excerpt)
stringData:
  FILE_SERVICE_AI__CONNECTIONS: |
    {
      "Bedrock": {"provider": "bedrock", "auth_mode": "iam", "region": "us-east-1",
                  "models": {"Claude": {"model_id": "us.anthropic.claude-opus-4-8", "priority": 1, "max_tokens": 8192}}},
      "Azure":   {"provider": "azure", "auth_mode": "key", "credential_env": "AZURE_OPENAI_KEY_PROD", "base_url": "https://<resource>.openai.azure.com",
                  "extra_params": {"api_version": "2024-10-21"},
                  "models": {"GPT-4o": {"model_id": "<deployment-name>", "priority": 2}}},
      "OpenAI":  {"provider": "openai", "auth_mode": "key", "credential_env": "OPENAI_API_KEY_PROD",
                  "models": {"GPT-4o": {"model_id": "gpt-4o"}}}
    }
  AZURE_OPENAI_KEY_PROD: "..."
  OPENAI_API_KEY_PROD: "sk-..."

Restart the registry service after changing these variables:

kubectl rollout restart deployment/istari-fileservice
kubectl rollout status deployment/istari-fileservice

A connection that fails validation (for example an Azure iam connection whose base_url is not an Azure host, or auth_mode: iam on a provider without an identity chain) is reported in the registry logs and skipped; the other connections keep working, and the skipped one is missing from the LLMs page. A key connection whose credential_env variable is unset still appears on the LLMs page, marked API key missing; chat skips it with the same log line until the variable is set.

Connection fields

Field	Required	Description
`provider`	Yes	LiteLLM provider family: `bedrock`, `azure`, `vertex_ai`, `openai`, `anthropic`, `gemini`, `openrouter`, and so on. Model ids are prefixed with it automatically (`gpt-4o` on an `azure` connection becomes `azure/gpt-4o`)
`auth_mode`	No (`iam`)	`iam`: the pod's cloud identity authenticates; valid for `bedrock`, `azure`, and `vertex_ai` only. `key`: a secret named by `credential_env`. `none`: no credential at all, only for an OpenAI-compatible gateway on a trusted network
`base_url`	Provider-dependent	Endpoint. Required for `azure` (`https://<resource>.openai.azure.com` or `https://<resource>.services.ai.azure.com`) and for OpenAI-compatible gateways. For an Azure `iam` connection it must be `https` and end in an allow-listed Azure suffix; see Azure roles
`region`	Bedrock, Vertex	The region whose endpoint to call and sign for (`us-east-1`, `us-gov-west-1`, `us-central1`). A bare region token; anything else is rejected
`project`	Vertex	GCP project id
`credential_env`	`key` mode	Name of the variable that holds the secret. The value is read when the connection is used, never stored. (`credential` holds the secret inline; for local development only)
`extra_params`	No	JSON object of provider parameters passed through on every call, for example `{"api_version": "2024-10-21"}` for Azure. Credential-like keys are rejected
`data_residency`	No	`us`, `eu`, `apac`, `us-gov`, or `global`. Limits model discovery to that tier; unset lists region-pinned tiers and hides `global`
`content_policy`	No	JSON object binding a provider content-safety policy. Bedrock: `{"id": "<guardrail-id>", "version": "<version>"}`
`cost_center`	No	Free-text tag forwarded as call metadata for cost attribution
`enabled`	No (`true`)	`false` keeps the connection declared but unused
`models`	Yes	Map of models keyed by model name; see below

Model fields:

Field	Required	Description
`model_id`	Yes	The provider's model id: a Bedrock model or inference-profile id, an Azure deployment name, an OpenAI model name. The provider prefix is added for you
`priority`	Recommended	Lower is tried first. A map has no order, so set it on every model to control the default and failover order
`max_tokens`	Recommended	Output-token cap. Set it for Bedrock: an unset cap reserves the model's maximum against your tokens-per-minute quota on every call, a common cause of throttling
`user_selectable`	No (`false`)	Lets users pick this model in the chat panel
`enabled`	No (`true`)	`false` hides the model
`path`	No	URL path appended to `base_url`, usually `/v1` for OpenAI-compatible gateways
`max_iterations`	No	Per-model override of `FILE_SERVICE_AI__MAX_ITERATIONS`
`disable_system_prompt`	No (`false`)	Fold the system prompt into the first user turn for this model
`disable_native_tools`	No (`false`)	Describe tools in the prompt instead of the provider's `tools` API for this model

Credentials by provider

A static credential is the supported setup for this release; cloud identity is available where noted and can be adopted later without changing the connection's models.

Provider	Static credential	Cloud identity
AWS Bedrock	An env-declared connection with `auth_mode: key` and a Bedrock API key in `credential_env`, generated for an IAM user that holds the policy under Bedrock IAM permissions, including `bedrock:CallWithBearerToken`; it is sent as a bearer token and is valid only in the region it was generated in. The web app does not offer key mode for Bedrock. Alternatively `auth_mode: iam` with that user's access keys in the pod environment	`auth_mode: iam` with EKS Pod Identity or IRSA and the same policy on the pod role; works from the web app and from the environment
Azure OpenAI / AI Foundry	`auth_mode: key` with the resource's API key, pasted in the web app or held in `credential_env`. Required on Azure Government and Azure China in this release	`auth_mode: iam` with AKS Workload Identity, or a service principal via `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_CLIENT_SECRET`; see Azure roles
Google Vertex AI	`auth_mode: iam` with `GOOGLE_APPLICATION_CREDENTIALS` pointing at a mounted service-account key (Vertex has no API-key mode)	`auth_mode: iam` with GKE Workload Identity
Anthropic, OpenAI, Gemini API, OpenRouter, gateways	`auth_mode: key` with the provider key, pasted in the web app or held in `credential_env`	None; these providers have no identity option

Two Bedrock details:

AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY in the pod environment are read by every AWS SDK client in the registry service, including the object-store client. If the registry service reaches S3 through pod identity, do not add static AWS keys for Bedrock; use a Bedrock API key on an env-declared key connection, or grant Bedrock to the pod role.
Add models (catalog discovery) for Bedrock always uses the AWS credential chain, not the Bedrock API key. On a key connection without AWS credentials in the pod, type the model ids instead; chat and Test work with the key.

If you use cloud identity, grant it to the registry service pods, not to the node. Each SDK resolves credentials in its own order: the AWS SDK tries static environment keys, then Pod Identity or IRSA, then the EC2 instance role; the Azure SDK (DefaultAzureCredential) tries environment variables, then Workload Identity, then the VM's managed identity; the Google SDK tries GOOGLE_APPLICATION_CREDENTIALS, then the pod's Workload Identity, then the node's metadata server. In each case the node-level identity is shared by every pod on the node, so keep LLM permissions off it; a broken pod identity then fails closed instead of silently authenticating as the node.

Frontend settings

AI chat is an experimental feature (ai-chat) that each user turns on under Application → Experimental Features. It is available on every deployment; you control its default in the istari-frontend secret (see Frontend Service Secret) or frontend.env:

Variable	Effect
`VITE_EXPERIMENTAL_DEFAULT_ON`	Comma-separated feature ids that start on for users who have never touched the toggle. Add `ai-chat` to turn AI chat on by default.
`VITE_EXPERIMENTAL_LOCKED`	Feature ids pinned to the environment default; the toggle is disabled. Combine with `DEFAULT_ON` to force on, or use alone to force off.

Restart the frontend deployment after changing these:

kubectl rollout restart deployment/istari-frontend
kubectl rollout status deployment/istari-frontend

The LLMs admin page needs no frontend setting; it is available to organization administrators on every deployment.

Environment variable reference

Registry service (istari-fileservice):

Variable	Required	Default	Description
`FILE_SERVICE_AI__CONNECTIONS`	Option B	unset	Env-declared connections, as one JSON object or split per field; see Option B
`FILE_SERVICE_AI__CONNECTIONS_LOCKED`	No	`false`	`true` makes env-declared connections the whole catalog and rejects every write from the admin page and API
`FILE_SERVICE_AI__AZURE_IAM_HOST_SUFFIXES`	Azure IAM	commercial	JSON list of host suffixes an Azure `iam` connection's `base_url` may use. Default `[".openai.azure.com", ".cognitiveservices.azure.com", ".services.ai.azure.com"]`; setting it replaces the default, so repeat the ones you keep. See Azure roles
`FILE_SERVICE_REMOTE_MASTER_SECRET`	Option A, upgrade	built-in	Seals credentials pasted in the web app and migrated from September host slots (shared with the Secure Connection Service). At least 32 characters; set once, before the upgrade, and never change it. See Option A
`FILE_SERVICE_AI__MCP_SERVER_URL`	Strongly recommended	unset	`https://mcp.<customer_istari_fqdn>/mcp/`. Without it the assistant cannot read platform data
`FILE_SERVICE_AI__SYSTEM_PROMPT_DELIVERY`	No	`system_role`	`user_turn` for gateways that drop `system` messages. Seed value; an administrator's choice on the AI Instructions tab wins once set
`FILE_SERVICE_AI__TOOL_MODE`	No	`native`	`prompt` for gateways that do not forward the `tools` API. Seed value, as above; `disable_native_tools` sets it per model
`FILE_SERVICE_AI__MAX_ITERATIONS`	No	`10`	Maximum tool-call steps per reply before the assistant answers with what it has
`FILE_SERVICE_AI__MAX_IDENTICAL_TOOL_CALLS`	No	`3`	Identical tool calls tolerated before a loop is declared
`FILE_SERVICE_AI__REQUEST_TIMEOUT`	No	`120`	Seconds allowed for one model call before it is retried and then fails visibly; `0` disables the bound. Override per model with `request_timeout` in Extra parameters (`extra_params.request_timeout` in Option B)
`FILE_SERVICE_AI__CARRY_TOOL_CONTEXT`	No	`true`	Default for new models' Tool carry setting (replaying earlier tool results into later turns); administrators override it per model
`FILE_SERVICE_AI__DEBUG`	No	`false`	Verbose agent logging. Turn on only while troubleshooting

Variables read by libraries inside the registry service (LiteLLM, the cloud SDKs, the HTTP clients), not by the registry service itself:

Variable	Read by	Notes
`HTTPS_PROXY`, `NO_PROXY`, `AIOHTTP_TRUST_ENV`	HTTP clients / LiteLLM	See Proxies
`SSL_CERT_FILE`, `AWS_CA_BUNDLE`	LiteLLM / AWS SDK	See TLS inspection and private CAs
`AWS_REGION_NAME`, `AWS_REGION`	LiteLLM	Bedrock fallback region when a connection has no `region`. Set `region` on the connection instead
`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`	AWS SDK	Static AWS credentials for a Bedrock `iam` connection. Also used by the object-store client; see Credentials by provider
`AWS_BEARER_TOKEN_BEDROCK`	LiteLLM	A Bedrock API key applied to every Bedrock `iam` connection that has no credential of its own. Prefer a `key` connection with `credential_env`
`AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_FEDERATED_TOKEN_FILE`	Azure SDK	Injected by AKS Workload Identity for an Azure `iam` connection. Do not set them by hand on AKS
`AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_CLIENT_SECRET`	Azure SDK	Service principal for an Azure `iam` connection on a cluster without Workload Identity. Not needed with `auth_mode: key`
`GOOGLE_APPLICATION_CREDENTIALS`	Google SDK	Service-account key file for a Vertex `iam` connection without GKE Workload Identity

Frontend service (istari-frontend): VITE_EXPERIMENTAL_DEFAULT_ON, VITE_EXPERIMENTAL_LOCKED — see Frontend settings.

Network requirements

Egress from the registry service pods

Destination	Port	Purpose
Your LLM provider hostname (see Provider hostnames)	443	Chat completions (long-lived streaming responses) and, for Add models, the provider's model listing
`mcp.<customer_istari_fqdn>`	443	Tool calls to the MCP Service. This is your own FQDN, so in-cluster traffic must be able to reach it (hairpin through the load balancer, or split DNS)
`login.microsoftonline.com`	443	Entra token requests for an Azure `iam` connection
`sts.<region>.amazonaws.com`	443	Credential exchange for a Bedrock `iam` connection with IRSA; already in place if pod identity works for object storage
`169.254.170.23` (EKS Pod Identity agent)	80	Credential retrieval for a Bedrock `iam` connection with EKS Pod Identity; link-local on the node, so no cluster egress is involved
`oauth2.googleapis.com`	443	Token requests for a Vertex `iam` connection
GKE metadata server (`169.254.169.252`; `169.254.169.254` on some clusters)	988, 80	Credential retrieval for a Vertex `iam` connection with GKE Workload Identity; link-local, but a deny-all egress NetworkPolicy must allow it. Confirm the address and port for your cluster in Google's Workload Identity documentation

The registry service ships with LiteLLM's model catalog bundled and does not download it at startup, so no egress to GitHub or other LiteLLM hosts is needed.

Provider hostnames

Provider	Hostname
Anthropic	`api.anthropic.com`
OpenAI	`api.openai.com`
Azure OpenAI / AI Foundry	`<resource>.openai.azure.com` or `<resource>.services.ai.azure.com`
AWS Bedrock	`bedrock-runtime.<region>.amazonaws.com`; Add models also calls `bedrock.<region>.amazonaws.com`. In GovCloud the region is `us-gov-west-1` or `us-gov-east-1` and the ARN partition is `aws-us-gov`
Google Vertex AI	`<location>-aiplatform.googleapis.com`
OpenAI-compatible gateway	The gateway host from `base_url`

The MCP Service in turn needs its own egress, which depends on whether the API Gateway is enabled:

API Gateway enabled (apiGateway.enabled: true): MCP calls api.<customer_istari_fqdn> on port 443 at the /registry and /identity prefixes.
No API Gateway: MCP calls registry.<customer_istari_fqdn> and zitadel.<customer_istari_fqdn> directly on port 443.

In both cases the MCP Service itself stays at mcp.<customer_istari_fqdn>; the gateway has no /mcp prefix. These routes already exist if MCP works for external MCP clients.

Firewalls, WAFs, and SNI allow-lists

Allow the provider hostname by SNI/hostname, not by IP; provider IPs change.
Chat replies stream over Server-Sent Events from POST /api/v3/ai/chat/stream on the registry service (/registry/api/v3/ai/chat/stream behind the API Gateway). Anything between the browser and the registry (load balancer, WAF, reverse proxy) must not buffer responses and needs an idle timeout of several minutes.
The registry service disables HTTP redirects on LLM calls. A proxy that answers with a 30x redirect breaks the call rather than being followed.

Proxies

The registry service honors the standard HTTPS_PROXY and NO_PROXY variables for most providers. LiteLLM's default transport (used for Bedrock and a few other providers) additionally requires AIOHTTP_TRUST_ENV: "true". Put the cluster-internal hostnames the registry service talks to (PostgreSQL, SpiceDB, NATS, and mcp.<customer_istari_fqdn> if it resolves internally) in NO_PROXY.

note

Validate proxy settings in a staging environment first; proxy behavior differs by product and by provider adapter.

TLS inspection and private CAs

If outbound traffic passes through TLS inspection, or your provider gateway presents a certificate from a private CA, the registry service must trust that CA or every LLM call fails with CERTIFICATE_VERIFY_FAILED.

Preferred: add the CA to the Helm chart's trustedCertBundle as in Scenario 2: Self-Signed TLS Certs.
If AI calls still fail, point both SSL_CERT_FILE (used by LiteLLM) and AWS_CA_BUNDLE (used by the AWS SDK) at one PEM bundle that contains the public roots and your inspection CA. A bundle that contains only your private CA breaks object-store and other public TLS calls from the same pod.

Validate the deployment

Run these from a workstation with kubectl access to the platform namespace. Replace <...> placeholders.

1. The registry service sees the configuration

kubectl exec deployment/istari-fileservice -- sh -c 'env | grep -E "^FILE_SERVICE_AI" | sed "s/=.*/=<set>/"'

FILE_SERVICE_AI__MCP_SERVER_URL should be listed, and for Option B the FILE_SERVICE_AI__CONNECTIONS variables. Then open Admin → LLMs and AI Instructions → LLMs as an organization administrator: an env-declared connection shows there marked Managed by infrastructure with its models; a missing one failed validation (step 6 has the reason).

2. Outbound connectivity and TLS from the pod

Probe the provider you configured. Any HTTP status means the pod reached it over TLS; 000 means blocked egress or a TLS failure.

Provider	Test URL	Expected
Anthropic	`https://api.anthropic.com/v1/models`	`401`
OpenAI	`https://api.openai.com/v1/models`	`401`
Azure OpenAI	`https://<resource>.openai.azure.com/openai/models?api-version=2024-10-21`	`401`
AWS Bedrock	`https://bedrock-runtime.<region>.amazonaws.com/`	`403` or `404`
Google Vertex AI	`https://<location>-aiplatform.googleapis.com/`	`404`
OpenAI-compatible gateway	`https://<gateway-host>/v1/models`	`401`, or `200` if it allows anonymous listing

kubectl exec deployment/istari-fileservice -- curl -sS -o /dev/null -w '%{http_code}\n' "<test-url>"

3. Check for TLS inspection

kubectl exec deployment/istari-fileservice -- curl -sv -o /dev/null "https://<provider-host>" 2>&1 | grep -E 'issuer|SSL certificate'

The issuer should be a public CA. A corporate CA name means traffic is inspected; see TLS inspection and private CAs. SSL certificate problem means the inspecting CA is not trusted yet.

4. The MCP Service is reachable from the pod

kubectl exec deployment/istari-fileservice -- curl -sS -o /dev/null -w '%{http_code}\n' https://mcp.<customer_istari_fqdn>/mcp/

Expected: 401 (reachable; the request carried no token). 404 means the request reached your ingress but there is no route to the MCP Service at /mcp/. A 5xx means the MCP Service is reachable but failing; check the istari-mcp pod logs. 000 means the registry service pod cannot resolve or connect to the MCP FQDN.

5. The connection and the chat path work

On the LLMs page, click Test on the connection and pick a model: the registry service makes a one-token call with the connection's own credential or identity, so a pass proves authentication, model access, and egress together, and a failure shows the provider's error. This is the same resolution chat uses.

Then, in the web app, turn on Istari AI Chat under Application → Experimental Features, click AI chat in the header, and send Reply with the single word OK. A reply confirms the provider path. Then ask What is the name of the system I am looking at? from a system page; a real answer confirms the MCP path. The toast Istari AI is not configured on this server. means no enabled model exists (step 1).

6. Registry service logs

kubectl logs deployment/istari-fileservice --since=15m | grep -i -E 'litellm|ai_service|env AI connection|mcp|CERTIFICATE'

Most provider-side failures reach the user as a generic "Istari AI hit an error" message; the specific cause is only in these logs. A skipped env-declared connection logs Skipping misconfigured env AI connection '<name>': <reason>.

Troubleshooting

Symptom	Likely cause	What to do
Toast Istari AI is not configured on this server.; the AI chat button then hides	No enabled model on any connection; or env-declared connections set but the pod not restarted; or the only connections failed validation	Add or enable a model on the LLMs page, or declare one (Option B) and `kubectl rollout restart deployment/istari-fileservice`; check validation step 6 for skipped connections. Users refresh or sign in again to get the button back
Assistant chats but cannot see systems or files; replies contain text that looks like tool calls	`FILE_SERVICE_AI__MCP_SERVER_URL` unset, wrong path (must end in `/mcp/`), MCP not deployed, or the registry cannot reach `mcp.<fqdn>`	Set the URL, run validation step 4, check `istari-mcp` pod logs; fix hairpin/split DNS so pods reach your own FQDN
Registry logs show `CERTIFICATE_VERIFY_FAILED` / `unable to get local issuer certificate`	TLS inspection or a private CA on the provider path	Run validation step 3; add the CA via `trustedCertBundle`, or set `SSL_CERT_FILE` and `AWS_CA_BUNDLE` to a full bundle
Connection timeouts or `000` from validation step 2	Egress blocked (security group, NAT, firewall, WAF SNI list) or proxy not configured	Allow the provider FQDN on 443; set `HTTPS_PROXY`/`NO_PROXY`/`AIOHTTP_TRUST_ENV` if a proxy is mandatory
Provider returns `401`/`403` (in registry logs or from Test)	Wrong key for the provider (`sk-ant-` vs `sk-`), key for another resource or region, revoked key	Re-check the key and provider match; for Azure confirm the key belongs to the resource in `base_url`
Chat on one connection fails with `503`; logs say the stored credential can't be decrypted	`FILE_SERVICE_REMOTE_MASTER_SECRET` changed after the key was pasted	Restore the original value, or have an administrator re-enter the key on the LLMs page
Bedrock `AccessDeniedException`	The IAM user or pod role lacks `bedrock:InvokeModel*`; an API-key user lacks `bedrock:CallWithBearerToken`; model access not enabled in that region; marketplace subscription missing	Apply Bedrock IAM permissions; enable model access in the Bedrock console for that region; subscribe to the model in AWS Marketplace
Bedrock `ValidationException` / model not found	Wrong model id or region; cross-region models need the `us.`/`eu.` inference-profile id	Use the inference profile id (e.g. `us.anthropic.claude-opus-4-8`) and set the connection's Region (`region` in Option B) to a region that serves it
Bedrock Add models fails on a key connection	Catalog discovery uses the AWS credential chain, not the Bedrock API key	Type the model ids, or give the pod AWS credentials; see Credentials by provider
Azure `404` / `DeploymentNotFound`	The model id is the model name instead of the deployment name, or the API version is missing	Set the model id to the deployment name; set API version on the connection (`extra_params.api_version` in Option B)
Registry logs: `An Azure IAM connection's base_url must be an https Azure endpoint`	The Azure `iam` connection's `base_url` is not `https`, or its host suffix is not in the allow-list	Use the resource's `https` endpoint. For a commercial host outside the defaults, add its suffix to `FILE_SERVICE_AI__AZURE_IAM_HOST_SUFFIXES`, repeating the defaults you keep. Azure Government and China hosts need `auth_mode: key` in this release; allow-listing them does not enable IAM there
Azure `401` / `PermissionDenied` on an `iam` connection	Role not assigned on the resource, assignment still propagating, or the pod has no Azure identity	See Azure roles; wait a few minutes after assigning; check the pod environment has `AZURE_CLIENT_ID` and `AZURE_TENANT_ID` plus either `AZURE_FEDERATED_TOKEN_FILE` (Workload Identity) or `AZURE_CLIENT_SECRET` (service principal)
LLMs page shows a connection as Managed by infrastructure, or an edit returns `403`	The connection is env-declared (Option B), or `FILE_SERVICE_AI__CONNECTIONS_LOCKED` is `true`	Change it in the environment and restart the registry service
Env-declared connection missing from the LLMs page, or shows API key missing	It failed validation and was skipped, or its `credential_env` variable is unset in the pod	Read the registry logs (validation step 6) for the connection name and reason; common causes are an unset `credential_env` variable, a non-Azure `base_url` on an Azure `iam` connection, or `auth_mode: iam` on a provider without an identity chain
Provider `400` mentioning `temperature` or another unsupported parameter for a brand-new model	The platform's bundled LiteLLM predates the model. A `temperature` rejection is retried once without it, and a Bedrock Claude model LiteLLM does not know is sent through the Converse API with no temperature, so a `400` that persists is about another parameter	Override the parameter per model in Extra parameters (`extra_params` in Option B), pick a model the current release supports, or upgrade the platform
After upgrading, a connection migrated from a slot has no credential	The slot's `_AUTHN` variable was missing when the migration ran	An administrator pastes the key on the LLMs page. For Bedrock or Vertex (cloud identity only in the web app) grant the pod identity; for Bedrock an env-declared connection with `auth_mode: key` also works. When `FILE_SERVICE_AI__CONNECTIONS_LOCKED` is `true` the migrated connection is ignored; redeclare it from the environment (Option B) and restart
Replies stop mid-sentence; "The connection dropped before the reply finished"	A proxy or load balancer buffers or times out the SSE stream	Disable response buffering and raise idle timeouts on the path to `/api/v3/ai/chat/stream`
Provider `429`	Provider rate limit or quota	Raise the quota with the provider or choose a different model; on Bedrock set `max_tokens` on the model

Bedrock IAM permissions

Under review

This section is being finalized with Istari Digital's infrastructure team. Validate it against your account before relying on it.

The registry service calls Bedrock as an IAM principal: the IAM user whose Bedrock API key or access keys you configure, or the role the pods assume. Either way the principal needs the policy below, scoped to the models you configure.

1. Enable model access for the models you want in the Bedrock console for the region on the connection. Third-party models (Anthropic, Meta, and others) also require an active AWS Marketplace subscription in the account.

2. Create the principal and attach a policy. Either:

Attach the policy to the IAM role the registry service pods assume (EKS Pod Identity or IRSA) and use auth_mode: iam with no credential; or
Create an IAM user for the registry service, attach the policy to it, and generate a Bedrock API key for it. Store the key in its own registry service secret variable (for example BEDROCK_API_KEY_PROD) and declare an env-declared connection with auth_mode: key and credential_env: BEDROCK_API_KEY_PROD. credential_env holds the variable's name; the key goes in the variable. Generate the key in the region on the connection; a long-term key is valid only there. AWS positions long-term API keys as exploration-grade, so treat the key like any static credential: give it an expiry and rotate it.

The template below covers one configured model id, <model-id>, for example anthropic.claude-opus-4-8 or amazon.nova-pro-v1:0. Replace <partition> (aws, or aws-us-gov in GovCloud), <region> (the region on the connection), <account-id>, <geo>, and the destination regions, then remove the resources and statements that do not apply to how you invoke the model.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "BedrockInvokeModel",
      "Effect": "Allow",
      "Action": ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"],
      "Resource": [
        "arn:<partition>:bedrock:<region>::foundation-model/<model-id>",
        "arn:<partition>:bedrock:<region>:<account-id>:inference-profile/<geo>.<model-id>",
        "arn:<partition>:bedrock:<destination-region-1>::foundation-model/<model-id>",
        "arn:<partition>:bedrock:<destination-region-2>::foundation-model/<model-id>"
      ]
    },
    {
      "Sid": "BedrockApiKeyUse",
      "Effect": "Allow",
      "Action": ["bedrock:CallWithBearerToken"],
      "Resource": "*"
    },
    {
      "Sid": "BedrockMarketplaceViewSubscriptions",
      "Effect": "Allow",
      "Action": ["aws-marketplace:ViewSubscriptions"],
      "Resource": "*"
    }
  ]
}

Direct model id (<model-id>, no geographic prefix): only the first resource is needed, the foundation model in the region you call.
Cross-region inference profile (<geo>.<model-id>, where <geo> is us, eu, apac, and so on): needs the profile ARN in the region you call and a foundation-model ARN for every region the profile routes to. List them with:
```
aws bedrock get-inference-profile --region <region> --inference-profile-identifier <geo>.<model-id> --query 'models[].modelArn'
```
The template names one model. Repeat the resources for each model id you configure, or use a prefix wildcard where your policy allows, for example arn:<partition>:bedrock:*::foundation-model/anthropic.claude-* for every region and Claude model.
bedrock:CallWithBearerToken is what lets a Bedrock API key authenticate at all; without it the key is refused even when the model resources match. It takes no resource scope. Keep it on the IAM user behind an API key and drop the statement from a pod role used with auth_mode: iam.
aws-marketplace:ViewSubscriptions lets the invoking principal verify the account's subscription for marketplace-gated models (Anthropic and other third-party models). Subscribing itself is a one-time account action, not something the pod role needs.
Add models on the LLMs page additionally calls bedrock:ListFoundationModels and bedrock:ListInferenceProfiles with the pod's AWS credentials. Grant them to the pod role if administrators should browse the catalog; typing model ids needs neither.

3. Optional: keep traffic inside the VPC with interface endpoints for bedrock-runtime (and bedrock) with private DNS enabled. No registry service change is needed; the same hostname resolves to the endpoint.

Azure roles

The recommended Azure setup uses the resource's API key (auth_mode: key), which needs no role assignment. This section covers the optional identity-based setup for an Azure connection with auth_mode: iam, created in the web app or declared from the environment. The registry service requests a Microsoft Entra bearer token for the Cognitive Services scope from the pod's identity and sends it to the connection's base_url, for chat calls and for Add models (GET {base_url}/openai/deployments). With Workload Identity no Azure key or client secret is stored anywhere; the service-principal option below keeps a client secret in the registry service secret.

Commercial Azure only in this release

The registry service requests Entra tokens from the commercial Azure cloud. On Azure Government or Azure China, use auth_mode: key with the resource's API key.

1. Assign a role on the resource. Grant the identity the Cognitive Services OpenAI User built-in role, scoped to the Azure OpenAI or AI Foundry resource (not the subscription). It allows inference calls and reading the resource's deployments, and nothing else: no key access, no deployment management. For an AI Foundry resource (<resource>.services.ai.azure.com) Microsoft's Entra guidance names Cognitive Services User for inference; assign that one as well if calls return 401. Role assignments can take up to five minutes to take effect.

az role assignment create \
  --assignee-object-id <identity-principal-id> --assignee-principal-type ServicePrincipal \
  --role "Cognitive Services OpenAI User" \
  --scope /subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.CognitiveServices/accounts/<resource>

2. Give the registry service pods an identity. Choose one:

AKS Workload Identity (recommended). Enable the OIDC issuer and workload identity on the cluster, create a user-assigned managed identity, and add a federated identity credential for the registry service's Kubernetes service account (system:serviceaccount:<namespace>:<registry-service-account>). The service account needs the annotation azure.workload.identity/client-id: <client-id> and the pods the label azure.workload.identity/use: "true"; AKS then injects AZURE_CLIENT_ID, AZURE_TENANT_ID, and AZURE_FEDERATED_TOKEN_FILE into the pod. Assign the role in step 1 to this managed identity. Do not grant it to the AKS node pool's identity; see the note under Credentials by provider.
Service principal (any cluster). Create an app registration with a client secret, assign the role in step 1 to it, and set AZURE_CLIENT_ID, AZURE_TENANT_ID, and AZURE_CLIENT_SECRET in the registry service secret. Treat AZURE_CLIENT_SECRET as a static credential like an API key: rotate it in Entra before it expires, update the registry service secret, and restart the registry service deployment so the running pods pick up the new value (a changed Kubernetes Secret does not update AZURE_CLIENT_SECRET in running containers); validate chat before retiring the old secret. It is scoped by Azure RBAC and lets you disable key access on the resource.

3. Create the connection with provider Azure, authentication Cloud identity (auth_mode: iam), the resource endpoint as Endpoint URL (base_url), the API version (extra_params.api_version), and one model per deployment. The endpoint must use https and end in one of the suffixes in FILE_SERVICE_AI__AZURE_IAM_HOST_SUFFIXES (by default .openai.azure.com, .cognitiveservices.azure.com, .services.ai.azure.com), because the token on the wire is your identity's and must only ever go to an Azure endpoint. Connections with a pasted key are not restricted this way.

4. Optional: turn off key access on the resource (disableLocalAuth) once chat works, so the managed identity is the only way in.

Egress: the resource hostname and login.microsoftonline.com, both on 443.

Validate from the pod before involving the registry service. Sign in as the identity the pod will use: with Workload Identity, az login --service-principal -u "$AZURE_CLIENT_ID" -t "$AZURE_TENANT_ID" --federated-token "$(cat "$AZURE_FEDERATED_TOKEN_FILE")"; with a service principal, az login --service-principal -u "$AZURE_CLIENT_ID" -t "$AZURE_TENANT_ID" -p "$AZURE_CLIENT_SECRET". Do not use az login --identity, which can authenticate as the node. Then az account get-access-token --scope https://cognitiveservices.azure.com/.default must return a token, and a chat completion against one of your deployments must succeed:

curl -sS -X POST "<base_url>/openai/deployments/<deployment-name>/chat/completions?api-version=<api_version>" \
  -H "Authorization: Bearer <token>" -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Reply with the single word OK."}],"max_tokens":5}'

A 200 with a reply proves the role and the identity for chat. Add models additionally needs GET <base_url>/openai/deployments?api-version=<api_version> to answer 200; if your resource's API version does not serve that listing, chat still works and administrators type the deployment names instead.

Istari AI Chat — the end-user feature.
Istari Platform Installation — secrets, MCP Service, trusted certificate bundles, custom environment variables.
MCP quick start — the MCP Service and the LLM integration disclaimer.
AI Models (Admin Guide) and AI Steering (Admin Guide) — the organization administrator screens.
Granting admin access — organization administrator rights for the LLMs page.

How it works​

Upgrading from the September 2026 release​

Prerequisites​

Option A: Connections in the web app​

Option B: Declare connections from the environment​

Declare the connections​

Connection fields​

Credentials by provider​

Frontend settings​

Environment variable reference​

Network requirements​

Egress from the registry service pods​

Provider hostnames​

Firewalls, WAFs, and SNI allow-lists​

Proxies​

TLS inspection and private CAs​

Validate the deployment​

Troubleshooting​

Bedrock IAM permissions​

Azure roles​

Related​