- Infrastructure: Terraform provisions AKS, Postgres, Redis, Blob Storage, Key Vault, cert-manager, KEDA, and ingress.
- Application: Helm installs the LangSmith chart against the cluster.
Prerequisites
Required tools
Required Azure RBAC
The identity running Terraform needs the following roles on the subscription:Owner includes both. Contributor alone is insufficient because role assignments require User Access Administrator.
Authenticate
dns_label (Azure subdomain, no DNS setup needed) or a custom langsmith_domain.
Quickstart
For the fastest path from zero to a running LangSmith instance:Provision infrastructure
Terraform provisions the following Azure resources:Clone and configure
modules/azure/. Run make help for the full target list.
Generate terraform.tfvars with the interactive wizard:
Blob Storage is always required, regardless of tier. Trace payloads must go to Azure Blob, never to ClickHouse.
Bootstrap secrets
setup-env.sh writes infra/secrets.auto.tfvars (gitignored, chmod 600). Terraform picks this file up automatically; no shell exports needed.
- First run: prompts for PostgreSQL password, LangSmith license key, admin password, and admin email. Generates
api_key_salt,jwt_secret, and four Fernet encryption keys locally. - Subsequent runs: reads the six generated secrets (
api_key_salt,jwt_secret, and the four Fernet keys) from Azure Key Vault. Re-prompts for the PostgreSQL password, license key, admin password, and admin email unlessLANGSMITH_PG_PASSWORD,LANGSMITH_LICENSE_KEY,LANGSMITH_ADMIN_PASSWORD, andLANGSMITH_ADMIN_EMAILare set in the environment.
Preflight
terraform.tfvars and secrets.auto.tfvars presence, and terraform/kubectl/helm on PATH.
Apply
Provisioning the Azure cloud foundation takes 15 to 20 minutes on a clean subscription. Do not interrupt the apply.
Skip
make plan on a fresh deploy. kubernetes_manifest resources require a live cluster API during plan, which does not exist yet. make apply handles resource ordering in three internal stages: Azure infrastructure including AKS → Kubernetes bootstrap (namespace, secrets, cert-manager, KEDA) → ClusterIssuer and remaining manifests.Cluster credentials and Kubernetes Secrets
Aftermake apply completes, get cluster credentials and push secrets into the cluster:
make k8s-secrets reads 8 secrets from Key Vault and creates or updates langsmith-config-secret. Safe to re-run; uses --dry-run=client | kubectl apply to update in place.
Verify infrastructure
Deploy LangSmith
Use one of the two supported deployment paths:Helm path (recommended)
Generate Helm values
make init-values reads terraform output and terraform.tfvars and generates helm/values/values-overrides.yaml with all fields populated:
config.hostname, your FQDN (fromdns_labelorlangsmith_domain).config.initialOrgAdminEmail, the first org admin account.config.existingSecretName: langsmith-config-secret, secrets reference.config.blobStorage, storage account name + container + Workload Identity client ID.- Workload Identity annotations for 8 ServiceAccounts (backend, platform-backend, queue, ingest-queue, host-backend, listener, agent-builder-tool-server, agent-builder-trigger-server).
- Ingress + TLS block (cert-manager annotation, TLS secret name).
- Postgres and Redis external secret references (when
postgres_source = "external"/redis_source = "external").
helm/values/examples/ into helm/values/.
The admin email is read from
langsmith_admin_email in terraform.tfvars (set during make setup-env) and written into values-overrides.yaml automatically. No manual editing needed.Deploy
make deploy does the following:
- Validates
values-overrides.yamlexists. - Refreshes kubeconfig via
az aks get-credentials. - Annotates the LoadBalancer service with
service.beta.kubernetes.io/azure-dns-label-name, required for Azure to assign the DNS label to the public IP. - Creates the
letsencrypt-prodcert-managerClusterIssueriftls_certificate_source = "letsencrypt"(idempotent). - Runs preflight checks (tools, cluster connectivity, Helm repo).
- Verifies
langsmith-config-secretexists; auto-creates from Key Vault if it is missing. - Builds and logs the values chain.
- Auto-recovers any stuck
pending-upgradeHelm release before proceeding. - Runs
helm upgrade --install langsmith langchain/langsmith --timeout 20m. - Waits for core deployments to roll out.
- Annotates the
langsmith-ksaServiceAccount with the Workload Identity client ID. - Prints the access URL and login credentials location.
Why
--timeout 20m? The langsmith-backend-auth-bootstrap Job runs DB migrations and org initialization as a post-install hook. This takes up to 5 minutes on first install. Without a long timeout, Helm may report failure even though the install eventually succeeds.Terraform path
Use this path when you want the Helm release, Kubernetes Secrets, and Workload Identity ServiceAccount managed in Terraform state.app/terraform.tfvars:
Verify the deployment
https://<HOSTNAME> and log in with the admin email and password from Key Vault:
Values chain
make deploy applies Helm values files in this order (last file wins on conflicts):
helm/values/ are gitignored (generated or contain live secrets). Source templates live in helm/values/examples/ and are copied by make init-values.
Day-2 operations
Enable add-ons
Each add-on is gated by a flag ininfra/terraform.tfvars. Set the flag, re-run make init-values to regenerate values, then re-run make deploy.
LangSmith Deployment
Enables LangSmith Deployment, which lets you deploy and manage agents as API servers directly from the LangSmith UI. This adds three new pods.Scale the node pool first
Before enabling, bumpdefault_node_pool_min_count to at least 5. The operator spawns agent deployment pods on demand and needs node headroom:
Apply, regenerate values, deploy
make init-values appends the LangSmith Deployment add-on overlay (langsmith-values-agent-deploys.yaml) to the values chain. It automatically injects:
Verify
langsmith-host-backend, langsmith-listener, and langsmith-operator all Running. Total pod count: ~20 Running + 3 Completed jobs.
KEDA is already installed alongside infrastructure. With enable_deployments = true, the operator creates KEDA ScaledObject resources for each agent deployment’s worker queue. Worker pods scale down to zero when idle and scale up based on Redis queue depth.
Agent Builder
Provides visual AI-assisted creation and management of LangGraph agents from the LangSmith UI. Noterraform apply needed; run make init-values && make deploy.
Prerequisite: LangSmith Deployment enabled (enable_deployments = true). Enabling Agent Builder without it causes a preflight error.
Enable:
make init-values appends the Agent Builder add-on overlay (langsmith-values-agent-builder.yaml) to the values chain. The overlay enables the Agent Builder UI and supporting services, sets backend.agentBootstrap.enabled: true (the post-install job that registers Agent Builder as a LangSmith Deployment and creates the required ConfigMap), and sets conservative agent worker pod resources (1 to 2 CPU, 512 MiB to 1 GiB memory) instead of the chart’s default 4 to 8 GiB memory.
Verify:
make deploy, an Agent Builder section appears in the LangSmith UI navigation.
Both langsmith-agent-builder-tool-server and langsmith-agent-builder-trigger-server need Workload Identity to access Azure Blob Storage. Their federated credentials are pre-registered in modules/k8s-cluster/main.tf; no additional setup is needed.
Insights and Polly
Two features, both of which require LangSmith Deployment. They are independent of each other; enable either one without the other.- Insights: AI-powered trace analytics (Clio). Surfaces patterns and anomalies in LangSmith traces. Clio deploys as a dynamic LangGraph deployment through the operator on first UI invocation. Adds no new static pods.
- Polly: AI-powered evaluation and monitoring agent. Runs as a dynamic LangGraph deployment, operator-managed. The overlay enables Polly (top-level
polly.enabled: true); the operator manages its resources.
terraform apply needed; run make init-values && make deploy.
make init-values appends the add-on overlays based on clickhouse_source in terraform.tfvars:
clickhouse_source = "in-cluster", generates a minimal overlay (top-levelinsights.enabled: trueonly). The Helm chart manages ClickHouse internally.clickhouse_source = "external", generates a full overlay withclickhouse.external.enabled: trueand alangsmith-clickhousesecret reference. Create this secret with the ClickHouse host and credentials before deploying.
Add-on summary
Ingress controllers
Setingress_controller in terraform.tfvars before make apply. For the full TLS compatibility matrix, see INGRESS_CONTROLLERS.md in the Azure module repo.
DNS and TLS
dns_label gives you a free Azure subdomain, <label>.<region>.cloudapp.azure.com, with no domain registration or DNS zone needed. deploy.sh annotates the correct LoadBalancer service automatically.
Quickstart default (HTTP, zero setup):
make applycreates the Azure DNS zone and outputs 4 nameservers.- At your registrar, add NS records for the subdomain pointing to those 4 nameservers.
- Verify:
dig NS langsmith.mycompany.com @8.8.8.8. make deployissues the cert via DNS-01 automatically (Workload Identity writes the TXT record to Azure DNS).- Get the LB IP, add
ingress_ip = "<ip>"toterraform.tfvars, thenmake apply(creates the A record). make statusshows exactly what NS and A records to add at each stage.
Why NS records, not CNAME: cert-manager must write TXT records to the zone to prove ownership. That requires Azure DNS to be authoritative for the subdomain, and NS delegation grants that authority. A CNAME only aliases traffic and does not transfer DNS authority; the DNS-01 challenge fails.
Next steps
- Reference the Azure variables and the quick reference.
- Review the Azure architecture for module structure, traffic flow, and Workload Identity.
- When something breaks, check the Azure troubleshooting guide.
- Enable agent deployment in the UI with LangSmith Deployment.
Connect these docs to Claude, VSCode, and more via MCP for real-time answers.

