> For the complete documentation index, see [llms.txt](https://docs.scorable.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.scorable.ai/integrations/coding-agents.md).

# Coding Agents

Scorable is designed to be operated by AI coding agents. Everything you can do in the web UI is also available through the `scorable` CLI and the REST API, so an agent can set up evaluations, run them, and act on the results without a human clicking through the product.

There are four building blocks:

1. The **Agent Skill** for one-prompt setup
2. **Project skills** installed via `scorable skills-add`
3. The **CLI** as the agent's day-to-day interface
4. The **MCP server** for direct tool access

## One-prompt setup with the Agent Skill

The fastest way to add Scorable evals to an application is to paste this into your coding agent:

```
Add Scorable evals by following https://scorable.ai/SKILL.md
```

The [Agent Skill](/skill.md) instructs the agent to analyze your codebase for LLM interaction points, install the CLI, generate a Judge that matches what your application does, integrate judge execution into your code, and verify the setup. The agent handles all steps itself; you only provide an API key if you do not want to use a temporary demo key.

## Install Scorable skills into your project

The CLI can install a set of reusable agent skills into your repository:

```bash
scorable skills-add
```

This runs `npx skills add root-signals/scorable-skills` and makes the skills available to agents such as Claude Code and Cursor working in that project. Afterwards, prompts like "Integrate scorable evaluators" or "Add OTEL tracing to my agent and auto-evaluate every trace with Scorable" resolve to concrete, tested instructions instead of the agent improvising. See [OTEL Trace Evaluation via CLI](/concepts-and-examples/cookbooks/otel-evaluation-via-cli.md) for a full walkthrough of the tracing skill.

## The CLI is agent-friendly by design

Agents and scripts can drive the entire platform through the [CLI](/concepts-and-examples/cookbooks/cli.md):

* **Non-interactive auth**: the API key is read from the `SCORABLE_API_KEY` environment variable, and `scorable auth demo-key` creates a free temporary key without leaving the terminal.
* **Project scoping**: pass `--project-id`, or set `SCORABLE_PROJECT_ID` once for the whole session.
* **Structured output**: every command prints JSON on success and exits non-zero on failure, so results are easy to parse with `jq` and to gate on in scripts and CI. See [Unit Testing in CI/CD](/ci.md) for a ready-made GitHub Actions example.
* **Full surface**: judges, evaluators, models, datasets, annotations, calibration runs, prompt tests, execution logs, and OTEL trace filters are all manageable from the command line.

For example, an agent can create and run a judge end-to-end:

```bash
scorable judge generate --intent "Evaluate that support answers are accurate and polite."
scorable judge execute <judge_id> \
  --request "What is the refund policy?" \
  --response "You can return items within 30 days."
```

## Evaluate your agent's own traces

Scorable can also evaluate the coding or production agent itself. Point any OpenTelemetry exporter at Scorable's OTLP endpoint and create a filter that automatically evaluates matching traces:

```bash
scorable otel-filter create --name "agent-quality" ...
scorable otel-trace list --since 1h
```

The CLI ships extractor manifests for common trace shapes, including one for Claude Code traces and one for OpenInference-instrumented agents. See [OTEL Trace Evaluation via CLI](/concepts-and-examples/cookbooks/otel-evaluation-via-cli.md) for details.

## MCP server

For agents that speak the Model Context Protocol, the [Scorable MCP server](https://github.com/root-signals/root-signals-mcp) exposes evaluators and judges as tools. It supports both stdio and SSE transports, so it works with MCP clients such as Claude Code, Claude Desktop, and Cursor. Use it when you want the agent to run evaluations as part of its own reasoning loop rather than through shell commands. Installation and configuration instructions are in the repository README.

## Machine-readable docs

These docs are available in agent-readable form at [docs.scorable.ai/llms.txt](https://docs.scorable.ai/llms.txt) and [docs.scorable.ai/llms-full.txt](https://docs.scorable.ai/llms-full.txt). Point your agent at those URLs when it needs broader context than the Agent Skill provides.