Kubernetes Delivery

Technical reference. This page is for integrations, advanced configuration, or platform operations. For portal tasks, start with the user guide.

The Kubernetes manifests are the reference production-shaped deployment surface. They are intentionally stricter than local Docker Compose so deployment drift is caught before stronger readiness claims are made.

Supported Path

The supported launch path is Kustomize-first:

single-tenant for small dedicated environments
saas for hosted multi-tenant environments
byoc for customer-owned clusters
AKS via examples/values-aks.yaml (Workload Identity, Key Vault CSI, Application Gateway for Containers, TLS DSNs/URLs, logical-tier execution)

The manifests include the Orqestra namespace, first-party services, worker deployments, PgBouncer, the runtime API/executor split, retrieval service, NetworkPolicies, HPAs, PDBs, Secret references, and a separate migration Job.

Operators own cluster provisioning, image registry rewrites, OIDC, Vault or an external-secret source, managed PostgreSQL/Redis/object storage, the model gateway, evidence storage buckets, ingress/TLS, and Prometheus custom metrics. Rendered manifests consume Kubernetes Secrets; secret values must not be committed to manifests.

Required Invariants

The base and overlays enforce:

immutable @sha256: digest pins for every first-party Orqestra image (mutable tag-only references, and the all-zeros placeholder digest, fail the delivery gates closed)
non-root pod and container security contexts
dropped Linux capabilities
read-only root filesystems where supported
PodDisruptionBudgets for scaled services
multi-replica PgBouncer
HPA scale-down stabilization
custom autoscaling signals for I/O-bound services
namespace default-deny ingress with explicit service peer allowlists

Validation

Run the structural gate before changing manifests:


make check-kubernetes-support-contract
make check-kubernetes-overlays

The supply-chain probe suite also checks the same delivery invariants, including image tags, security contexts, PgBouncer availability, PDB coverage, and HPA behavior blocks. It also checks the rendered NetworkPolicy shape: every internal service with an ingress path must have an explicit allowlist, and workloads without an ingress path must remain default-denied.

Network Isolation

The base manifests deny ingress to every pod by default. They then allow only the required internal service paths: portal and service peers to the control-plane API, runtime roles to retrieval and LiteLLM, application services to PgBouncer, and PgBouncer to Postgres.

External ingress is environment-owned. Add a profile-specific NetworkPolicy for the ingress controller that selects its namespace and pods explicitly before exposing portal or API routes.

Runtime Role Split

The runtime-gateway image runs in one of three roles controlled by the ORQESTRA_RUNTIME_ROLE environment variable: api, executor, or combined.

In Kubernetes, the base manifests deploy two separate workloads from the same image:

runtime-gateway — runs with ORQESTRA_RUNTIME_ROLE=api. Handles HTTP admission, authentication, request validation, and status streaming. The runtime-gateway Service routes traffic exclusively to this Deployment.
runtime-executor — runs with ORQESTRA_RUNTIME_ROLE=executor. Dequeues execution envelopes, acquires resource leases, and runs SequentialExecutor.

Each role has independent HPA targets:

API scales on orqestra_http_server_in_flight_requests (request admission pressure).
Executor scales on orqestra_runtime_execution_queue_depth and orqestra_runtime_executor_active_executions (execution backlog pressure).

For local development, docker compose and make dev default to combined mode so a single container serves both roles.

Container assurance tier: the host-native executor prerequisite

The deployment topologies above cannot realize the container assurance tier as shipped. Realizing container requires the runtime-gateway executor process running host-natively on a host with a local Docker CLI and daemon — outside the shipped container image, sharing that host’s filesystem and local daemon. code_execution.py writes each workload’s source into the executing process’s own tempfile.TemporaryDirectory and bind-mounts it with -v {tmpdir}:/workspace:ro, which Docker resolves on the daemon’s filesystem. A containerized executor, a sidecar daemon, or a remote daemon passes the hello-world preflight and then fails every real execution. No shipped compose, Kubernetes, or Helm topology provides this — an operator who needs container must provision a host-native executor deployment themselves.

The binding axis is queued-vs-in-process, not the role: api, executor, and combined work together, but two populations must be host-native for container work — every consumer of a dispatch stream carrying such work, and every process serving in-process execution (inline /v1/execute and /v1/execute/code-mode, which execute on the serving process). “The executor role” alone is neither necessary nor sufficient; a non-host-native process must have those in-process routes disabled or unrouted.

Because the admission gate is unconditional, a release bound to an external runtime — which contains on the delegated runtime and never runs a host sandbox — is still denied on a process-tier host. Plan for a host-native Docker capability on every process that must execute container-declaring work.

To roll back from queued dispatch to inline execution, set ORQESTRA_EXECUTION_DISPATCH_MODE=inline on the API Deployment, wait for executor work to drain, then scale runtime-executor to zero.

Install

Publish first-party images and rewrite the selected overlay (and the base kustomization images: transform / Helm images.*.digest values) to the published @sha256: digests. Production renders fail closed on mutable first-party tags, so this digest-resolution step is mandatory, not optional.
Provision PostgreSQL, Redis, object storage, OIDC, Vault or external secrets, and the model gateway.
Create the required Kubernetes Secrets in the orqestra namespace.
Run the migration Job and wait for completion.
Apply one overlay with kubectl apply -k infrastructure/kubernetes/overlays/<profile>/.
Add environment-owned ingress/TLS and matching NetworkPolicies.

Helm

Helm packaging lives at infrastructure/helm/orqestra and consumes the same supported topology. Use it when operators need repeatable install and upgrade commands instead of direct Kustomize applies.


helm lint infrastructure/helm/orqestra
helm template orqestra infrastructure/helm/orqestra \
  --namespace orqestra \
  --set global.profile=single-tenant
helm upgrade --install orqestra infrastructure/helm/orqestra \
  --namespace orqestra \
  --create-namespace \
  --set global.profile=single-tenant

The chart references Kubernetes Secrets by name and does not render credential values. Keep Redis URLs, Postgres DSNs, object-store credentials, service auth tokens, and the capability-token signing key in Kubernetes Secrets or an external-secret controller. The migration Job is a Helm pre-install/pre-upgrade hook, so it runs before long-running workloads roll during install or upgrade. The Helm runtime-gateway API deployment sets ORQESTRA_EXECUTION_DISPATCH_MODE=queued; provide ORQESTRA_RUNTIME_DISPATCH_REDIS_URL in the referenced Secret so requests reach the runtime-executor deployment. The default chart values keep control-plane auth on OIDC, portal auth enabled, runtime secret resolution on Vault, and runtime readiness pointed at the LiteLLM-compatible model gateway. Set the OIDC issuer/JWKS/client values, portal redirect URL, Vault address, ORQESTRA_VAULT_HEALTHCHECK_SECRET_REF, ORQESTRA_VAULT_HEALTHCHECK_NAMESPACE, ORQESTRA_VAULT_HEALTHCHECK_WORKSPACE, ORQESTRA_VAULT_HEALTHCHECK_ENVIRONMENT, and ORQESTRA_LITELLM_BASE_URL before installing into a shared cluster.

The AKS example selects ORQESTRA_SECRET_BACKEND=azure_key_vault for the runtime gateway and executor. Set ORQESTRA_PLATFORM_KEY_VAULT_URL to the platform vault and inject the workload identity client ID, tenant ID, and federated token file into both workloads. The configured readiness reference must exist at the complete namespace/workspace/environment scope with matching Key Vault tags. The chart refuses a rendered AKS profile that points runtime workspace reads at the separate CSI vault.

When enabling chart-managed Ingress (ingress.enabled=true), production edges must terminate TLS: the chart fails closed (both values-schema and template render) unless ingress.tls names at least one TLS secret with its hosts. Provision that secret out of band — for example with cert-manager, as the examples/values-byoc-ingress.yaml example annotates — and ensure your ingress controller serves it. The only bypass is the explicit ingress.allowInsecureHttp=true escape hatch, which serves HTTP-only ingress for local/test clusters and must never be set in production. This requirement is separate from, and additive to, the ingress NetworkPolicy controller-selector requirement.

Upgrade

Render the old and new overlays, review image and trust-boundary changes, run the support-contract and overlay gates, apply the migration Job with the new image, then roll the selected overlay. Keep the prior image references available until health verification passes.

For Helm upgrades, review helm template output before helm upgrade --install. Run make check-helm-chart with the Kubernetes support-contract and overlay gates before promoting the rendered release.

Rollback

Re-apply the last known-good overlay image references or use kubectl rollout undo for workloads whose prior ReplicaSet still contains the expected image. Do not roll schema backward unless a tested down migration exists; prefer a forward-fix migration or restore from backup when data integrity requires it.

Backup

Back up PostgreSQL and the object-storage buckets that hold assets, evidence artifacts, WORM exports, and runtime checkpoints before upgrades that include migrations. Treat Redis as best-effort cache/event state unless the deployment explicitly enables persistence.

Restore

Restore PostgreSQL, restore object storage, recreate Kubernetes Secrets from Vault or external-secret sources, apply the selected overlay, then run health verification and spot-check release/evidence graphs.

Health

Use the render gate, kubectl wait on the long-running Deployments, and service health probes as the accepted local-cluster or equivalent smoke path. Probe control-plane on port 8000, runtime-gateway on 8001, and retrieval-service on 8002 with /healthz. Probe portal on 3000 with the root path.

Operating Notes

PgBouncer is part of the data-plane availability path. Worker replica and Celery concurrency changes must stay within the PgBouncer pool budget, and any environment-specific override should be reviewed alongside the HPA maximum for control-plane-worker.