# CLI

The `scorable` CLI is a powerful tool for interacting with the Scorable API, particularly for managing and executing Judges. This guide provides a brief overview of its capabilities.

## Installation

To install the CLI, run the following command:

```bash
curl -sSL https://scorable.ai/cli/install.sh | sh
```

## Authentication

The CLI requires an API key to be set as an environment variable.

```bash
export SCORABLE_API_KEY=$MY_SCORABLE_API_KEY
```

## Judge Management

The `judge` command is the primary entry point for all judge-related operations.

### Generating a Judge

Generate a complete judge from a plain-language description. Scorable picks the right evaluators automatically.

```bash
scorable judge generate \
  --intent "I am building a customer support chatbot. Evaluate that responses are helpful, accurate, and follow our refund policy."
```

Attach a policy document to make the generated evaluators check compliance against it:

```bash
# Upload a PDF and generate in one step
scorable judge generate \
  --intent "Evaluate responses against the attached policy." \
  --file ./refund-policy.pdf

# Reuse a previously uploaded file by ID
scorable judge generate \
  --intent "Evaluate responses against the attached policy." \
  --file-id <file_uuid>
```

**Key options:** `--file` (path to PDF/PNG/JPG — uploads and attaches), `--file-id` (UUID of an already-uploaded file), `--stage`, `--extra-contexts` (JSON), `--reasoning-effort`, `--judge-id` (regenerate an existing judge), `--overwrite`

### Creating a Judge

You can create a new judge using the `create` subcommand.

```bash
scorable judge create --name "My New Judge" --intent "To evaluate the quality of LLM responses."
```

**Arguments:**

* `--name`: The name of the judge (required).
* `--intent`: The intent or purpose of the judge (required).
* `--stage`: The stage of the judge.
* `--evaluator-references`: A JSON string of evaluator references. Example: `[{"id": "<faithfulness_uuid>"}, {"id": "<truthfulness_uuid>"}]`

### Listing Judges

To see a list of all available judges, use the `list` subcommand.

```bash
scorable judge list
```

You can filter the list using various options like `--search` and `--name`.

### Running a Judge

The `execute` subcommand allows you to run a judge with specific inputs.

```bash
scorable judge execute <judge_id> --request "What is the capital of France?" --response "Paris"
```

## Custom Model Management

Register a custom or self-hosted LLM, then reference it from evaluators and judges. See also [Connect a model](/concepts-and-examples/cookbooks/connect-a-model.md).

### Creating a model

```bash
# SaaS provider — inline key
scorable model create --name my-gpt --model gpt-5.5 --key sk-...

# Self-hosted endpoint
scorable model create \
  --name azure/gpt-5.5 \
  --model azure/gpt-5.5 \
  --url https://my-azure-openai.openai.azure.com \
  --key sk-...

# Pipe the key from stdin to keep it out of shell history
echo "$MY_PROVIDER_KEY" | scorable model create --name my-gpt --model gpt-5.5 --key -
```

**Key options:** `--name` (required), `--model`, `--url` (custom endpoint), `--key` (provider API key; `-` reads from stdin), `--max-token-count`, `--max-output-token-count`.

### Listing, inspecting, updating, deleting

```bash
scorable model list                                    # table: ID, Name, Provider, Visibility
scorable model get <model_id>
scorable model update <model_id> --max-output-token-count 4096   # PATCH — only sent fields change
scorable model delete <model_id>                       # prompts; pass --yes to skip
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.scorable.ai/concepts-and-examples/cookbooks/cli.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.