> For the complete documentation index, see [llms.txt](https://docs.scorable.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.scorable.ai/concepts-and-examples/cookbooks/cli.md).

# CLI

The `scorable` CLI is a powerful tool for interacting with the Scorable API, particularly for managing and executing Judges. This guide provides a brief overview of its capabilities.

## Installation

To install the CLI, run the following command:

```bash
curl -sSL https://scorable.ai/cli/install.sh | sh
```

## Authentication

The CLI requires an API key to be set as an environment variable.

```bash
export SCORABLE_API_KEY=$MY_SCORABLE_API_KEY
```

## Judge Management

The `judge` command is the primary entry point for all judge-related operations.

### Generating a Judge

Generate a complete judge from a plain-language description. Scorable picks the right evaluators automatically.

```bash
scorable judge generate \
  --intent "I am building a customer support chatbot. Evaluate that responses are helpful, accurate, and follow our refund policy."
```

Attach a policy document to make the generated evaluators check compliance against it:

```bash
# Upload a PDF and generate in one step
scorable judge generate \
  --intent "Evaluate responses against the attached policy." \
  --file ./refund-policy.pdf

# Reuse a previously uploaded file by ID
scorable judge generate \
  --intent "Evaluate responses against the attached policy." \
  --file-id <file_uuid>
```

**Key options:** `--file` (path to PDF/PNG/JPG — uploads and attaches), `--file-id` (UUID of an already-uploaded file), `--stage`, `--extra-contexts` (JSON), `--reasoning-effort`, `--judge-id` (regenerate an existing judge), `--overwrite`

### Creating a Judge

You can create a new judge using the `create` subcommand.

```bash
scorable judge create --name "My New Judge" --intent "To evaluate the quality of LLM responses."
```

**Arguments:**

* `--name`: The name of the judge (required).
* `--intent`: The intent or purpose of the judge (required).
* `--stage`: The stage of the judge.
* `--evaluator-references`: A JSON string of evaluator references. Example: `[{"id": "<faithfulness_uuid>"}, {"id": "<truthfulness_uuid>"}]`

### Listing Judges

To see a list of all available judges, use the `list` subcommand.

```bash
scorable judge list
```

You can filter the list using various options like `--search` and `--name`.

### Running a Judge

The `execute` subcommand allows you to run a judge with specific inputs.

```bash
scorable judge execute <judge_id> --request "What is the capital of France?" --response "Paris"
```

## Custom Model Management

Register a custom or self-hosted LLM, then reference it from evaluators and judges. See also [Connect a model](/concepts-and-examples/cookbooks/connect-a-model.md).

### Creating a model

```bash
# SaaS provider — inline key
scorable model create --name my-gpt --model gpt-5.5 --key sk-...

# Self-hosted endpoint
scorable model create \
  --name azure/gpt-5.5 \
  --model azure/gpt-5.5 \
  --url https://my-azure-openai.openai.azure.com \
  --key sk-...

# Pipe the key from stdin to keep it out of shell history
echo "$MY_PROVIDER_KEY" | scorable model create --name my-gpt --model gpt-5.5 --key -
```

**Key options:** `--name` (required), `--model`, `--url` (custom endpoint), `--key` (provider API key; `-` reads from stdin), `--max-token-count`, `--max-output-token-count`.

### Listing, inspecting, updating, deleting

```bash
scorable model list                                    # table: ID, Name, Provider, Visibility
scorable model get <model_id>
scorable model update <model_id> --max-output-token-count 4096   # PATCH — only sent fields change
scorable model delete <model_id>                       # prompts; pass --yes to skip
```