Self-hosted LangSmith on AWS

When running LangSmith on Amazon Web Services (AWS), self-hosted mode deploys a complete LangSmith platform with observability functionality.

This page provides:

• Initial setup steps for deploying to EKS, configuring managed services, and setting up authentication.
• AWS-specific architecture patterns and reference diagrams.
• Service recommendations and best practices.
• AWS Well-Architected best practices for operational excellence, security, and reliability.

LangChain publishes production-ready Terraform modules for AWS that provision EKS, RDS, ElastiCache, S3, and networking in a single workflow. Start with the Deploy with Terraform overview to choose between the Terraform and Helm-only paths.

Initial setup

1. Deploy to Kubernetes

   Follow the Kubernetes installation guide. LangSmith is tested on Amazon Elastic Kubernetes Service (EKS).

   EKS-specific notes:

   • Ensure the EBS CSI Driver is installed for persistent storage
   • Use the ebs.csi.aws.com storage class provisioner
2. Configure external services

   For production deployments, connect to AWS managed services:

   Amazon S3

   Store trace data in S3

   Amazon RDS

   PostgreSQL database

   Amazon ElastiCache

   Redis or Valkey for caching

   ClickHouse Cloud

   Analytics database

3. Set up authentication

   Use IAM Roles for Service Accounts (IRSA) to authenticate LangSmith pods to AWS services without static credentials.

   Key pages:

   • S3 IRSA configuration
   • RDS IAM authentication
   • ElastiCache IAM authentication

After completing these initial setup steps, you can review the complete AWS architecture and best practices below.

Reference architecture

We recommend leveraging AWS’s managed services to provide a scalable, secure, and resilient platform. The following architecture applies to both self-hosted and hybrid and aligns with the AWS Well-Architected Framework:

• Ingress & networking: Requests enter via Amazon Application Load Balancer (ALB) within your VPC, secured using AWS WAF and IAM-based authentication.

• Frontend & backend services: Containers run on Amazon EKS, orchestrated behind the ALB, and route requests to other services within the cluster as necessary.

• Storage & databases:

  • Amazon RDS for PostgreSQL or Aurora: metadata, projects, users, and short-term and long-term memory for deployed agents. LangSmith supports PostgreSQL version 14 or higher.
  • Amazon ElastiCache (Redis or Valkey): caching and job queues. ElastiCache can be in single-instance or cluster mode. LangSmith requires Redis OSS version 5 or higher, or Valkey 8.
  • ClickHouse + Amazon EBS: analytics and trace storage.
    • We recommend using an externally managed ClickHouse solution unless security or compliance reasons prevent you from doing so.
    • ClickHouse is not required for hybrid deployments.
  • Amazon S3: object storage for trace artifacts and telemetry.

• LLM integration: Optionally proxy requests to Amazon Bedrock or Amazon SageMaker for LLM inference.

• Monitoring & observability: Integrate with Amazon CloudWatch

Compute options

LangSmith supports multiple compute options depending on your requirements:

Compute option	Description	Suitable for
Elastic Kubernetes Service (preferred)	Advanced scaling and multi-tenant support	Large enterprises
EC2-based	Full control, BYO-infra	Regulated or air-gapped environments

AWS Well-Architected best practices

This reference is designed to align with the six pillars of the AWS Well-Architected Framework:

Operational excellence

• Automate deployments with IaC (CloudFormation / Terraform).
• Use AWS Systems Manager Parameter Store for configuration.
• Configure your LangSmith instance to export telemetry data and continuously monitor via CloudWatch Logs.
• The preferred method to manage LangSmith deployments is to create a CI process that builds Agent Server images and pushes them to ECR. Create a test deployment for pull requests before deploying a new revision to staging or production upon PR merge.

Security

• Use IAM roles with least-privilege policies.
• Enable encryption at rest (RDS, S3, ClickHouse volumes) and in transit (TLS 1.2+).
• Integrate with AWS Secrets Manager for credentials.
• Use Amazon Cognito as an IDP in conjunction with LangSmith’s built-in authentication and authorization features to secure access to agents and their tools.

Reliability

• Replicate the LangSmith data plane across regions: Deploy identical data planes to Kubernetes clusters in different regions for LangSmith Deployment. Deploy RDS and ECS services across Multi-AZ.
• Implement auto-scaling for backend workers.
• Use Amazon Route 53 health checks and failover policies.

Performance efficiency

• Leverage EC2 instances for optimized compute.
• Use S3 Intelligent-Tiering for infrequently accessed trace data.

Cost optimization

• Right-size EKS clusters using Compute Savings Plans.
• Monitor cost KPIs using AWS Cost Explorer dashboards.

Sustainability

• Minimize idle workloads with on-demand compute.
• Store telemetry in low-latency, low-cost tiers.
• Enable auto-shutdown for non-prod environments.

Security and compliance

LangSmith can be configured for:

• PrivateLink-only access (no public internet exposure, besides egress necessary for billing).
• KMS-based encryption keys for S3, RDS, and EBS.
• Audit logging to CloudWatch and AWS CloudTrail.

Customers can deploy in GovCloud, ISO, or HIPAA regions as needed.

Monitoring and evals

Use LangSmith to:

• Capture traces from LLM apps running on Bedrock or SageMaker.
• Evaluate model outputs via LangSmith datasets.
• Track latency, token usage, and success rates.

Integrate with:

• AWS CloudWatch dashboards.
• OpenTelemetry and Prometheus exporters.

---

Connect these docs to Claude, VSCode, and more via MCP for real-time answers.

Edit this page on GitHub or file an issue.