- Infrastructure: Terraform provisions VPC, EKS, RDS, ElastiCache, S3, the ALB, and IAM, then bootstraps the cluster add-ons.
- Application: Helm installs the LangSmith chart against the cluster.
Prerequisites
Required tools
Install on macOS:
PATH:
Required AWS IAM permissions
The IAM user or role running Terraform needsAdministratorAccess, or PowerUserAccess plus IAMFullAccess. The module creates IAM roles and policies for IRSA, so PowerUserAccess alone is not enough. For a scoped-down policy, refer to PERMISSIONS.md in the module.
Authenticate
Configure AWS credentials with the CLI:License key and domain
- LangSmith license key. Contact the LangChain sales team to request one. The setup script stores the key in AWS SSM Parameter Store, not in
tfvars. - Domain (optional). Set
langsmith_domainto serve LangSmith on your own hostname. With a domain and noacm_certificate_arn, Terraform creates a Route 53 hosted zone, requests an ACM certificate, and creates the ALB alias record. To use an existing hosted zone instead, setdns_create_zone = falseanddns_existing_zone_id. To bring your own certificate, setacm_certificate_arn. Leavelangsmith_domainempty to reach LangSmith on the ALB hostname. Without a domain, settls_certificate_source = "none"to serve plain HTTP. Anacm_certificate_arnalso works without a domain, butmake preflight-postthen fails its TLS check becauseacmexpectslangsmith_domain.
Cluster sizing reference
Two independent settings control capacity:- Infrastructure capacity sets instance types and node counts directly through the infra variables
eks_managed_node_groups,postgres_instance_type, andredis_instance_type. The module defaults are onem5.4xlargenode group (min 3, max 10),db.t3.largefor RDS, andcache.m6g.xlargefor ElastiCache. Theinfra/terraform.tfvars.dev,terraform.tfvars.minimum, andterraform.tfvars.productiontemplates set smaller or larger values. sizing_profileselects the Helm sizing overlay (pod resource requests and limits).init-values.shanddeploy.shread it; Terraform does not.
For production workloads, also plan to provision external LangChain Managed ClickHouse or a self-managed external ClickHouse cluster. In-cluster ClickHouse is supported for dev/POC only.
Quickstart
For the fastest path from zero to a running LangSmith instance, run these commands in order:make deploy prints the LangSmith URL when it finishes.
To chain infrastructure and application in one command, first run make quickstart and source infra/scripts/setup-env.sh in the same shell, then:
make quickdeploy first checks that the setup-env.sh exports are loaded and that infra/terraform.tfvars exists, and stops with the command to run if either is missing. It runs terraform init if infra/.terraform/ does not exist, then runs terraform apply, sources set-kubeconfig.sh, and runs init-values and deploy.sh. If terraform apply fails, fix the issue and re-run make quickdeploy. If a later step fails, the command prints the make target to retry.
The following sections cover each stage in detail.
Provision infrastructure
Terraform provisions the following AWS resources:
Application secrets live in SSM Parameter Store.
setup-env.sh writes them, not Terraform, and External Secrets Operator syncs them into the cluster.
Clone and configure
main branch and v0.17.* release tags deploy LangSmith Helm chart v17, which this guide describes.
To stay on chart v16, run
git checkout release/0.16 or check out a v0.16.* tag. Callouts in this guide mark where chart v16 differs.modules/aws/. Run make help for the full target list.
Generate terraform.tfvars with the interactive wizard:
infra/terraform.tfvars. When the file already exists, the wizard offers to update it with the existing values preselected, start fresh, or cancel, and it backs up the original file first. Some settings, such as the bring-your-own security group variables (*_existing_security_group_id), are not in the wizard; set them in terraform.tfvars by hand. When the wizard updates an existing file, it drops hand-written settings like these, so restore them from infra/terraform.tfvars.backup.
Prefer to edit by hand? Copy the example and fill in the required fields:
With a domain and no
acm_certificate_arn, Terraform requests an ACM certificate, and make apply waits until DNS validation succeeds. With dns_create_zone = true (the default), Terraform creates a new hosted zone for langsmith_domain, and validation succeeds only after you delegate that zone’s NS records from the parent domain. To validate in a zone that already resolves, set dns_create_zone = false and dns_existing_zone_id. To deploy before the delegation is in place, follow the staged deploy pattern in Auto-provisioned DNS.Load secrets into SSM Parameter Store
terraform.tfvars, derives the SSM path /langsmith/{name_prefix}-{environment}/, then for each secret either reuses an exported value, reads the existing SSM parameter, auto-generates one, or prompts you. You supply three values interactively: the license key, the admin password, and the admin email. The script must be sourced (not executed) because make cannot export environment variables back to the parent shell.
The script manages the following SSM parameters:
If you turn on
enable_sandboxes later, source setup-env.sh again before make apply so it creates the sandbox secrets.
Verify the secrets are present and the TF_VAR_* environment variables are exported:
Apply
make plan shows the proposed diff. Review the output before applying. make apply runs a single terraform apply across all modules; Terraform orders the resources by dependency. Pass ARGS="-auto-approve" to skip the confirmation prompt.
Configure kubectl
set-kubeconfig.sh writes a dedicated ~/.kube/langsmith-<cluster> file and exports KUBECONFIG, so it leaves ~/.kube/config untouched. make kubeconfig only prints this command.
All nodes should report Ready. The core add-ons should be Running: CoreDNS, kube-proxy, and the VPC CNI in kube-system, KEDA in keda, External Secrets Operator in external-secrets, and Envoy Gateway in envoy-gateway-system. cert-manager runs only when tls_certificate_source = "letsencrypt" or create_cert_manager_irsa = true.
Deploy LangSmith
make init-values reads sizing_profile and the enable_* flags from terraform.tfvars, writes helm/values/langsmith-values-overrides.yaml from the Terraform outputs, and copies the matching values files from helm/values/examples/ into helm/values/. It reuses the admin email from the existing overrides file, or takes it from LANGSMITH_ADMIN_EMAIL (set by setup-env.sh), or prompts for it. On re-runs it refreshes the Terraform outputs and copies an add-on values file only when it is missing, so your edits to existing files survive.
make deploy runs helm/scripts/deploy.sh, which:
- Resolves the chart version and confirms it is on the v17 line.
- Refreshes the kubeconfig with
aws eks update-kubeconfig. - Runs preflight checks (AWS credentials, cluster reachability, the
langchainHelm repo). - Applies the External Secrets Operator
ClusterSecretStoreandExternalSecretso the cluster reads secrets directly from SSM. - Runs
helm upgrade --installwith the layered values files and waits for the core pods. - Upgrades once more when no
langsmith_domainis set and the load balancer hostname changed, then prints the LangSmith URL.
Pin the chart version
deploy.sh installs the latest patch on the LangSmith Helm chart v17 line (~0.17.0) by default. To pin an exact patch, set langsmith_helm_chart_version = "0.17.x" in terraform.tfvars, or run CHART_VERSION=0.17.x make deploy. The environment variable wins over the tfvars setting. deploy.sh rejects versions off the v17 line, and Terraform validation rejects a langsmith_helm_chart_version off that line at plan time. deploy.sh also rejects values files that still use chart v15 keys (config.insights, config.polly, backend.agentBootstrap).
On chart v16, use a
v0.16.* module release or the release/0.16 branch. Those releases pin ~0.16.0 and accept only 0.16.x versions. To move an existing chart v16 deployment to v17, follow Upgrade from chart v16 to v17.Verify
Running, open the URL that make deploy printed. It uses langsmith_domain when set, or the ALB hostname. Run make status at any time for a summary of every layer and the next step to run.
Upgrade from chart v16 to v17
A deployment withenable_sandboxes = false and enable_smithdb = false only moves the chart pin. Sandboxes and SmithDB need the extra steps below. The module sets no Engine values, so if you enabled Engine in your own values overrides, also apply the Engine change in Upgrade from v16 to v17.
-
Check out a
v0.17.*tag, or pullmain. -
If
terraform.tfvarspinslangsmith_helm_chart_versionto a0.16.xpatch, clear it or set a0.17.xpatch, and rununset CHART_VERSIONif your shell sets it. Terraform validation rejects a0.16.xpin at plan time, anddeploy.shrejects a0.16.xvalue inCHART_VERSION. -
Run
make apply. From module releasev0.16.83or later, the v17 module changes no AWS resources, so Terraform reports no infrastructure changes. From an earlierv0.16.*release, reviewmake planbefore you apply. -
Run
make init-values. It rewriteslangsmith-values-overrides.yamlin the chart v17 sandbox schema. -
With
enable_smithdb = true, update the SmithDB keys in the values files thatmake init-valuesdoes not overwrite. Chart v17 fails to render the old keys:- In
helm/values/langsmith-values-smithdb.yaml, renamesmithdb.migration.deploymenttosmithdb.migration.job. - In
helm/values/langsmith-values-sizing-*.yaml, movesmithdb.query.autoscaling.*,smithdb.ingestion.autoscaling.*, andsmithdb.compactionWorker.autoscaling.*underautoscaling.hpa.*. Inlangsmith-values-sizing-production-large.yaml, rename thelocal-ssd-storagevolume tocache.
make init-valuesto copy the v17 versions instead. For the chart changes behind these keys, see Cache storage. Only theproduction-largesizing file defines acachevolume. With thedev,minimum, orproductionprofile, chart v17 gives each SmithDB cache a per-pod PVC fromsmithdb.cache.storageClassName, or from the cluster default StorageClass when that is unset. On chart v16, each cache was anemptyDir. To choose where the cache lives, setsmithdb.cache.storageClassName, or define a volume namedcachefor each SmithDB component that uses a disk. - In
-
Run
make deploy. With sandboxes running on chart v16, the first run stops withnamespace langsmith still mounts sandbox volumes through the chart 0.16 JuiceFS CSI driverand prints the commands that drain the old volumes. Run them, wait until onlyjuicefs-csi-*pods remain, then runmake deployagain. Running sandboxes stop during the drain. Their data stays in S3 and Redis, and chart v17 mounts the same JuiceFS volume. -
Verify the sandbox cutover:
MIGRATION-0.16-to-0.17.md guide in the Terraform repository.
Enable add-ons
Each add-on is gated by a flag ininfra/terraform.tfvars. Set the flag, run make apply, re-run make init-values to copy the matching values file, then re-run make deploy. make apply is required for add-ons that Terraform provisions: Fleet with external storage (the default), LangSmith Chat or Insights with external storage, sandboxes, and SmithDB. Sandboxes and SmithDB create AWS resources. External storage creates a database on the existing RDS instance and the matching Kubernetes secrets. For the rest, make apply makes no changes.
enable_agent_builder was removed in chart v16. Setting it to true fails validation at terraform plan, so delete it from existing tfvars files and use enable_fleet. enable_standalone_polly and enable_standalone_insights remain as legacy shorthand that enables the add-on with external storage (enable_polly = true with polly_storage = "external", and the same for Insights).
For details on agent deployment, see LangSmith Deployment.
Fleet
Fleet is the current form of the feature formerly called Agent Builder. It requires LangSmith Helm chart v15 or later and the Fleet entitlement in your license.
enable_fleet = true. Fleet installs the host-backend it needs, so enable_deployments is optional.
With fleet_storage = "external" (the default), Fleet requires external Postgres and Redis (postgres_source = "external" and redis_source = "external"). Terraform creates a dedicated langsmith_fleet database on RDS and wires the langsmith-fleet-postgres and langsmith-fleet-redis secrets to the existing RDS and ElastiCache instances. Set fleet_storage = "in-cluster" to use the PostgreSQL and Redis StatefulSets in the chart instead.
Fleet reuses the agent-builder-encryption-key SSM parameter, so deployments that migrate from Agent Builder keep the same key and data.
LangSmith Chat and Insights
enable_polly turns on LangSmith Chat (formerly Polly), and enable_insights turns on Insights. Neither requires enable_deployments. Both default to in-cluster storage. Set polly_storage = "external" or insights_storage = "external" to use dedicated databases on the shared RDS and ElastiCache instances; this requires postgres_source = "external" and redis_source = "external". Each add-on needs its entitlement in your license.
Sandboxes
enable_sandboxes = true adds a sandbox-host node group on bare-metal x86_64 instances with KVM (m5d.metal by default) and a dedicated ElastiCache instance for JuiceFS metadata. Sandboxes require LangSmith Helm chart v16 or later.
sandbox-host mounts JuiceFS itself, and a juicefs-format Job formats the volume. make init-values writes sandboxes.juicefs.existingSecretName pointing at the juicefs-csi-config Secret, sets sandboxes.juicefs.hostMount.cacheDirs to the local NVMe cache paths on sandbox-host nodes when there are any, and puts the IRSA role annotation on sandboxes.sandboxHost.serviceAccount.
On chart v16, the chart bundles the JuiceFS CSI driver instead.
make init-values from a v0.16.* module release writes sandboxes.juicefs.csi.existingSecretName and puts the IRSA role annotation on sandboxes.juicefs.csi.node.serviceAccount. The juicefs-csi-config Secret is the same on both lines.setup-env.sh again creates the JuiceFS Redis auth token and the sandbox callback signing key. The sandbox_* variables control the node count, instance types, and JuiceFS settings; see the variables reference.
SmithDB
enable_smithdb = true provisions the SmithDB dependencies: a metastore PostgreSQL instance, an S3 bucket, an IRSA role, and Karpenter node pools on local NVMe instances. make init-values then writes langsmith-values-smithdb.yaml and langsmith-values-smithdb-overrides.yaml, and make deploy stops if either file is missing.
Three gates move traffic to SmithDB after the services are healthy:
smithdb_ingestion_enabledroutes new writes to SmithDB.smithdb_migration_enabledmigrates historical ClickHouse data. It requires ingestion.smithdb_query_enabledserves queries from SmithDB. It requires ingestion.
enable_smithdb = true. Keep smithdb_karpenter_chart_version (default 1.6.3) compatible with eks_cluster_version per the Karpenter compatibility matrix. See the variables reference for the full list.
Optional: private EKS cluster with bastion
For deployments that must run a fully private EKS API endpoint, the modules ship a bastion host pattern:- First, run from your workstation with
create_bastion = trueandenable_public_eks_cluster = trueso the bastion can be created. - After the initial deployment, set
enable_public_eks_cluster = falseand re-apply. The EKS API endpoint becomes private only. - All subsequent Terraform work happens on the bastion. SSM into it, clone the repo, copy your
terraform.tfvarsand SSM secrets, then run the deployment from there.
The bastion runs in a private subnet with no public IP and reaches SSM through the NAT gateway. Setting
bastion_enable_ssh = true moves it to a public subnet with a public IP. Set bastion_existing_security_group_id to attach your own security group. The bastion comes preinstalled with kubectl, helm, terraform, git, and jq, with kubeconfig already configured for the EKS cluster. Install the Session Manager plugin for the AWS CLI on your workstation.Ingress, DNS, and TLS
The ALB is always the entry point. Withacm, it terminates TLS. Behind it, Envoy Gateway is the default ingress: the ALB forwards to the Envoy pods through a TargetGroupBinding, and the LangSmith chart creates HTTPRoute resources instead of an Ingress.
Terraform rejects a configuration that turns on more than one of these.
In ALB Ingress mode, the AWS Load Balancer Controller creates its own ALB, and the Terraform ALB goes unused. The Route 53 alias record that Terraform creates for
langsmith_domain still points at the Terraform ALB, so point the domain at the Ingress hostname instead (kubectl get ingress -n langsmith).
tls_certificate_source selects the certificate:
acm(default): An ACM certificate on the ALB, either created by Terraform forlangsmith_domainor supplied withacm_certificate_arn. Works with every ingress option.letsencrypt: cert-manager with Let’s Encrypt, solving an HTTP-01 challenge through the ALB Ingress class. Works only in ALB Ingress mode (enable_envoy_gateway = false). Requiresletsencrypt_email.none: Plain HTTP with no certificate.
create_cert_manager_irsa = true, cert_manager_hosted_zone_id, and letsencrypt_email. Terraform creates the DNS-01 issuer when langsmith_domain is set. create_cert_manager_irsa and tls_certificate_source = "letsencrypt" are mutually exclusive.
With langsmith_domain set and acm_certificate_arn empty, make apply creates the ACM certificate and the Route 53 alias record. make tls does the same outside Terraform: it requests and DNS-validates a certificate, writes the alias record, and writes the certificate ARN to terraform.tfvars.
Split dataplane namespace
Envoy Gateway supports routing to alanggraph-dataplane chart that runs in its own namespace. Apply the RBAC manifest once per dataplane namespace:
langsmith-agents namespace; edit it for a different namespace. It grants the langsmith-host-backend ServiceAccount read access to pods, pod logs, deployments, and ReplicaSets in the dataplane namespace. Without it, agent run logs do not stream in the LangSmith UI.
Next steps
- Reference the AWS variables and the quick reference.
- Review the AWS architecture for platform layers, IRSA, and module dependencies.
- When something breaks, check the AWS troubleshooting guide.
- Enable agent deployment in the UI with LangSmith Deployment.
Connect these docs to your agent of choice via MCP for real-time answers.

