# Add a custom evaluator

Scorable provides evaluators that fit most needs, but you can add custom evaluators for specific needs. In this guide, we will add a custom evaluator and tune its performance using demonstrations.

### Example: Weasel words

Consider a use case where you need to evaluate a text based on its number of weasel words or ambiguous phrases. Scorable provides the optimized ***Precision*** evaluator for this, but let's build something similar to go through the evaluator-building process.

1. **Navigate to the Evaluator Page:**
   * Go to the evaluator page and click on "New Evaluator."
2. **Name Your Evaluator:**
   * Type the name for the evaluator, for example, "Direct language."
3. **Define the Intent:**
   * Give the evaluator an intent, such as "Ensures the text does not contain weasel words."
4. **Create the Prompt:**
   * "Is the following text clear and has no weasel words"
5. **Add a placeholder (variable) for the text to evaluate:**
   * Click on the "Add Variable" button to add a placeholder for the text to evaluate.
     * E.g., "Is the following text clear and has no weasel words: {{response}}"
6. **Select the Model:**
   * Choose the model, such as **gpt-5.5**, for this evaluation.
7. **Save and Test the Evaluator:**
   * Click **Create evaluator** and [begin experimenting with it](/concepts-and-examples/cookbooks/evaluate-an-llm-response.md).

### Improve the custom evaluator performance

You can add demonstrations to the evaluator to tune its scores to match more closely to the desired behavior.

#### Example: Improve the Weasel words evaluator

Let's penalize using the word "probably"

1. **Go to the Weasel words evaluator and click Edit**
2. **Click Add under Demonstrations section**
3. **Add a demonstration**
   * Type to the Response field: "This solution will probably work for most users."
   * Score: 0,1
4. **Save the evaluator and try it out**

Note that adding more demonstrations, such as

* "The project will probably be completed on time."
* "We probably won't need to make any major changes."
* "He probably knows the answer to your question."
* "There will probably be a meeting tomorrow."
* "It will probably rain later today."

will further adjust the evaluator's behavior. Refer to the full evaluator [documentation](/concepts-and-examples/usage/evaluators.md) for more information.

Once you have demonstrations tuned, the next step is verifying the evaluator is actually reliable. See [Add a calibration set](/concepts-and-examples/cookbooks/add-a-custom-evaluator/add-a-calibration-set.md) — including how to use the **ladder algorithm** to generate calibration examples automatically instead of hand-crafting them.\\


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.scorable.ai/concepts-and-examples/cookbooks/add-a-custom-evaluator.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.