Skip to main content

LangSmith

Integrate agentgateway with LangSmith for LLM debugging and monitoring

LangSmith is LangChain’s platform for debugging, testing, evaluating, and monitoring LLM applications.

Features​

  • Trace logging - Detailed request/response logging
  • Debugging - Step-through debugging of LLM calls
  • Evaluation - Automated testing and evaluation
  • Monitoring - Production monitoring and alerting
  • Datasets - Build and manage evaluation datasets

Setup​

  1. Sign up at smith.langchain.com
  2. Create a project and get your API key

Configuration​

LangSmith accepts OpenTelemetry traces directly. Configure agentgateway to export traces directly to LangSmith:

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
config:
  tracing:
    otlpEndpoint: https://api.smith.langchain.com/otel
    randomSampling: true

gateways:
  default:
    port: 3000
routes:
- backends:
  - ai:
      name: openai
      provider:
        openAI:
          model: gpt-4o-mini
  policies:
    backendAuth:
      key: "$OPENAI_API_KEY"

Authentication​

LangSmith requires an API key for authentication. Set the OTEL_EXPORTER_OTLP_HEADERS environment variable with your LangSmith API key:

# Set the x-api-key header for LangSmith authentication
export OTEL_EXPORTER_OTLP_HEADERS="x-api-key=your-langsmith-api-key"

# Also set the protocol to HTTP/protobuf (LangSmith requires HTTP, not gRPC)
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf

Docker Compose example​

Agentgateway exports traces directly to LangSmith without needing an OTel Collector:

version: '3'
services:
  agentgateway:
    image: cr.agentgateway.dev/agentgateway:latest
    ports:
      - "3000:3000"
    volumes:
      - ./config.yaml:/config.yaml:ro
    command: ["-f", "/config.yaml"]
    environment:
      - OTEL_EXPORTER_OTLP_HEADERS=x-api-key=${LANGSMITH_API_KEY}
      - OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf

Learn more​