# Unit Testing in CI/CD

Integrating Scorable into your CI pipeline allows you to automatically evaluate the quality of your LLM outputs as part of your testing workflow. This ensures that regressions in response quality are caught early, before they reach production.

## How It Works

Scorable integrates with standard test frameworks, allowing you to use Judges and Evaluators as assertions in your unit and integration tests. When tests run in CI, Scorable evaluates your LLM responses and fails the test if quality thresholds aren't met.

## Supported Test Frameworks

Scorable can be integrated into any test framework. Here are guides for popular options:

* [**Evalite guide**](/integrations/evalite.md) - Modern LLM testing framework with built-in support for custom scorers
* [**Pytest guide**](/integrations/pytest.md) - Standard Python testing framework with fixtures for Scorable integration

### CLI Tool

For prompt testing and model comparison, you can use the Scorable CLI tool directly in your CI pipeline:

* [**Prompt Testing CLI**](/concepts-and-examples/cookbooks/find-the-best-prompt-and-model.md) - Compare prompts and models, evaluate outputs, and track metrics

The CLI is particularly useful for:

* Systematically testing multiple prompt variations
* Comparing different model outputs side-by-side
* Running batch evaluations with YAML configuration files
* Generating reports on speed, cost, and quality metrics

Example CI usage:

```bash
# Install CLI
curl -sSL https://scorable.ai/cli/install.sh | sh

# Run prompt tests
roots prompt-test run
```

## Best Practices

### 1. Use Tags for Tracking

Tag your evaluations with metadata like git commit hashes to track results over time:

```typescript
// TypeScript
tags: ["test", process.env.GIT_COMMIT || "local"]
```

```python
# Python
tags=["test", os.getenv("GIT_COMMIT", "local")]
```

### 2. Set Appropriate Thresholds

Start with lower thresholds and gradually increase them as you improve your prompts:

```python
# Start conservative
assert_scorable_quality(..., threshold=0.7)

# Increase as quality improves
assert_scorable_quality(..., threshold=0.85)
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.scorable.ai/ci.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.