> ## Documentation Index
> Fetch the complete documentation index at: https://docs.langchain.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Manage evaluators

> View and manage evaluators at the workspace level in LangSmith.

[Evaluators](/langsmith/evaluation-concepts#evaluators) in LangSmith are [workspace-level](/langsmith/administration-overview#workspaces) resources. You can attach a single evaluator to multiple [tracing projects](/langsmith/observability-concepts#projects) and [datasets](/langsmith/evaluation-concepts#datasets), so you can apply consistent evaluation logic across your work without recreating it each time.

<Tip>
  The [LangSmith Engine](/langsmith/engine) suggests custom evaluators for detected issues and can deploy them with one click.
</Tip>

## View evaluators

In the [LangSmith UI](https://smith.langchain.com?utm_source=docs\&utm_medium=cta\&utm_campaign=langsmith-signup\&utm_content=langsmith-evaluators), select **Evaluators** in the left sidebar to view all evaluators in your workspace.

The evaluators table shows the following columns:

| Column                            | Description                                                                                                                                                                     |
| --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Name                              | The evaluator name                                                                                                                                                              |
| Type                              | **LLM as a judge** or **Code**. Composite score evaluators are scoped to individual tracing projects and datasets and do not appear here.                                       |
| Feedback Key                      | The feedback key the evaluator produces                                                                                                                                         |
| Projects & Datasets               | Tracing projects and datasets this evaluator is attached to                                                                                                                     |
| Evaluator Trace Count (this week) | Number of traces this evaluator ran on in the past week. Only shown when spend tracking is enabled; **–** for Code evaluators or evaluators with no attached rules.             |
| Spend (this week)                 | Estimated USD spend for this evaluator in the past week. Only shown when spend tracking is enabled; **–** for Code evaluators or evaluators with no attached rules.             |
| Spend Status                      | Whether the evaluator is **Under limits**, **Unlimited**, or has hit one or more configured spend limits. Only shown when spend tracking is enabled; **–** for Code evaluators. |
| Created By                        | The workspace member who created the evaluator                                                                                                                                  |
| Updated At                        | When the evaluator was last modified                                                                                                                                            |
| Created At                        | When the evaluator was created                                                                                                                                                  |

## Create an evaluator

You can create an evaluator in the [LangSmith UI](https://smith.langchain.com?utm_source=docs\&utm_medium=cta\&utm_campaign=langsmith-signup\&utm_content=langsmith-evaluators) or programmatically with the [SDK](#create-an-evaluator-with-the-sdk). Evaluators created either way are workspace-level resources that appear in the **Evaluators** table.

### Create an evaluator in the UI

1. In the [LangSmith UI](https://smith.langchain.com?utm_source=docs\&utm_medium=cta\&utm_campaign=langsmith-signup\&utm_content=langsmith-evaluators), select **Evaluators** in the left sidebar.
2. Click **+ Evaluator** to open the new evaluator panel.
3. The panel lets you:
   * **Create from scratch**: Build a new [LLM-as-a-Judge](/langsmith/llm-as-judge) or [Code](/langsmith/online-evaluations-code) evaluator.
   * **Create from a template**: Start from a ready-made evaluator (also known as a prebuilt evaluator) for common evaluation patterns. A **Recommended** section surfaces popular templates first, followed by templates organized by the following categories:

     | Category          | Description                                          |
     | ----------------- | ---------------------------------------------------- |
     | Security          | Detect leaks, injections, and adversarial inputs.    |
     | Safety            | Evaluate content safety and moderation.              |
     | Quality           | Measure output quality and accuracy.                 |
     | Conversation      | Evaluate conversational quality and user experience. |
     | Trajectory        | Evaluate agent tool use and decision paths.          |
     | Image Evaluations | Evaluate image content quality and safety.           |
     | Voice Evaluation  | Evaluate voice and audio interaction quality.        |

You can also add an evaluator directly from a [tracing project](/langsmith/observability-concepts#projects) or [dataset](/langsmith/evaluation-concepts#datasets). In that flow, you can additionally **attach an existing evaluator** from your workspace, or create a [Composite](/langsmith/composite-evaluators-ui) evaluator. Refer to [Set up LLM-as-a-judge online evaluators](/langsmith/online-evaluations-llm-as-judge) and [Automatically run evaluators on experiments](/langsmith/bind-evaluator-to-dataset).

### Create an evaluator with the SDK

Use the LangSmith SDK to create evaluators programmatically. The SDK is available for [Python](/langsmith/smith-python-sdk) and [TypeScript](/langsmith/smith-js-ts-sdk). Evaluators created through the SDK appear in the **Evaluators** table alongside those created in the UI.

<Note>
  Managing evaluators through the SDK requires `langsmith>=0.9.8` (Python, PyPI) or `langsmith>=0.7.16` (TypeScript, npm).
</Note>

<CodeGroup>
  ```python Python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
  import asyncio

  from langsmith import Client


  async def main():
      client = Client()

      created = await client.evaluators.create(
          name="Correctness evaluator",
          type="code",
          code_evaluator={
              "code": "def perform_eval(run, example):\n    return {'score': 1}",
              "language": "python",
          },
      )
      print("Created evaluator:", created.evaluator.id)


  asyncio.run(main())
  ```

  ```typescript TypeScript theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
  import { Client } from "langsmith";

  const client = new Client();

  const created = await client.evaluators.create({
    name: "Correctness evaluator",
    type: "code",
    code_evaluator: {
      code: "def perform_eval(run, example):\n    return {'score': 1}",
      language: "python",
    },
  });
  console.log("Created evaluator:", created.evaluator?.id);
  ```
</CodeGroup>

To create an LLM-as-a-judge evaluator and to retrieve, update, list, or delete evaluators, refer to [Manage evaluators with the SDK](/langsmith/manage-evaluators-sdk).

## View evaluator details

Click any evaluator in the table to open its detail view. The detail view has four tabs:

* **Overview**: The evaluator's feedback configuration and prompt or code definition.
* **Traces**: Traces processed by this evaluator across all attached resources.
* **Logs**: Execution logs for this evaluator across all attached resources.
* **Projects & Datasets**: The tracing projects and datasets this evaluator is attached to, with each attachment's [weekly spend and limit](/langsmith/evaluator-spend).

## Edit an evaluator

Open an evaluator. In the **Overview** tab, click the **Edit evaluator** <Icon icon="pencil" /> icon to open the **Configure Evaluator** panel. Update the evaluator's configuration. Click **Save**.

Because the evaluator is shared, changes apply across all tracing projects and datasets it is attached to.

## Manage evaluator trace retention

When an online evaluator scores a trace, it attaches feedback to that trace, which [auto-upgrades the trace to extended retention](/langsmith/usage-and-billing#data-retention-auto-upgrades). Extended retention keeps the trace longer but costs more. When you set up an online evaluator on a [tracing project](/langsmith/observability-concepts#projects), you can opt out of this upgrade so that scored traces stay at the project's base retention.

This control is available only when the project's [default retention](/langsmith/billing#change-project-level-default-retention) is the [base tier](/langsmith/usage-and-billing#how-it-works). If the project defaults to extended retention ([set at the project or workspace level](/langsmith/data-purging-compliance#data-retention)), traces scored by the evaluator follow that default and the option is locked.

To opt out of extending retention for scored traces:

1. When you [create](#create-an-evaluator) or [edit](#edit-an-evaluator) an online evaluator, set the source to a [tracing project](/langsmith/observability-concepts#projects), rather than a [dataset](/langsmith/evaluation-concepts#datasets).
2. Expand the **Advanced** section in the evaluator configuration panel.
3. Clear **Extend trace retention**.

The change applies to traces scored after you save the evaluator. Existing scored traces keep their current retention tier.

## Delete an evaluator

You cannot delete an evaluator while it is attached to a tracing project or dataset. To delete an evaluator:

1. In the [LangSmith UI](https://smith.langchain.com?utm_source=docs\&utm_medium=cta\&utm_campaign=langsmith-signup\&utm_content=langsmith-evaluators), select **Evaluators** in the left sidebar.
2. Select the evaluator you want to delete.
3. Open the **Projects & Datasets** tab. For each attached tracing project and dataset, select **Detach** in the **Actions** menu at the right of the row.
4. Return to the **Evaluators** page and click **Delete** at the top of the page.

***

<div className="source-links">
  <Callout icon="terminal-2">
    [Connect these docs](/use-these-docs) to Claude, VSCode, and more via MCP for real-time answers.
  </Callout>

  <Callout icon="edit">
    [Edit this page on GitHub](https://github.com/langchain-ai/docs/edit/main/src/langsmith/evaluators.mdx) or [file an issue](https://github.com/langchain-ai/docs/issues/new/choose).
  </Callout>
</div>