Projects

A Project is a workspace inside your organization. It groups related judges, evaluators, datasets, objectives, experiments, batch jobs, and execution logs under a single label so you can keep, for example, two different agents cleanly separated.

Projects are a filter and a scoping concept, not an access boundary. Every user in the organization can see and switch between every project; what changes between projects is which resources show up.

Default project

Every organization has exactly one default project, marked with a star in the project selector. It is created automatically the first time the organization needs one, and it cannot be deleted directly. To make a different project the default, open it in the selector and set it as default — the previous default is unset atomically.

Anything created without an explicit project (for example a judge generated from the CLI before you set a project, or an execution log produced by calling a public judge from another organization) is filed under your default project.

Switching projects in the UI

The project selector lives at the top of the side menu. Picking a project updates the URL with a projectId query parameter and remembers your choice in browser local storage. When you visit again, you land in the same project. Links you share from the app carry the projectId, so a teammate clicking your link sees the same view.

If you open a link to a resource that belongs to a different project than the one you currently have selected, you'll see a banner telling you so. Switch projects from the selector to view it.

Filtering via the REST API

All list endpoints that return project-scoped resources accept an optional project_id query parameter. Pass the UUID of a project to restrict results to that project. Omit it to get everything in your organization.

# Judges in a specific project
curl 'https://api.scorable.ai/v1/judges/?project_id=<project-uuid>' \
  -H 'authorization: Api-Key $MY_API_KEY'

# Execution logs scoped to a project, optionally combined with tag filters
curl 'https://api.scorable.ai/v1/execution-logs/?project_id=<project-uuid>&tags=staging' \
  -H 'authorization: Api-Key $MY_API_KEY'

When you execute a judge or evaluator that you own, the resulting execution log is automatically attached to the same project as the judge. To attach it to a different project instead, pass project_id in the request body:

curl 'https://api.scorable.ai/v1/judges/<judge-id>/execute/' \
  -H 'authorization: Api-Key $MY_API_KEY' \
  -H 'content-type: application/json' \
  --data-raw '{"response":"...","request":"...","project_id":"<project-uuid>"}'

For the OpenAI-compatible endpoints (/openai/chat/completions, /openai/responses), the same override is available as an X-Project-Id request header — the body has to stay compatible with the OpenAI wire format.

Python SDK

The Python SDK exposes projects as a first-class resource and accepts project_id on execution, list, create, and update methods.

from scorable import Scorable

client = Scorable(api_key="...")

# Manage projects
projects = client.projects.list()
prod = client.projects.create(name="Production", description="Live traffic")
client.projects.update(prod.id, is_default=True)  # promote to default

# Filter list endpoints by project
support_judges = client.judges.list(project_id=prod.id)

# Route an execution log to a specific project
client.judges.run(
    judge_id="<judge-id>",
    response="...",
    request="...",
    project_id=prod.id,
)

# Move an existing resource to another project
client.judges.update(judge_id="<judge-id>", project_id="<other-project-uuid>")

Response models expose project_id as Optional[str] — None for public resources owned by other organizations.

TypeScript SDK

import { Scorable } from '@root-signals/scorable';

const client = new Scorable({ apiKey: process.env.SCORABLE_API_KEY! });

// Manage projects
const projects = await client.projects.list();
const prod = await client.projects.create({ name: 'Production', isDefault: true });

// Filter list endpoints
const supportJudges = await client.judges.list({ projectId: prod.id });

// Route an execution log
await client.judges.execute(judgeId, {
  response: '...',
  request: '...',
  projectId: prod.id,
});

// Move a resource
await client.judges.update(judgeId, { projectId: '<other-project-uuid>' });

Response types expose project_id as string | null.

CLI

The CLI ships a project command group and accepts --project-id on every command that creates, executes, lists, or filters a project-scoped resource.

# Manage projects
scorable project list
scorable project create --name "Production" --is-default
scorable project set-default <project-id>
scorable project delete <project-id>

# Filter lists and route executions
scorable judge list --project-id <project-id>
scorable judge execute <judge-id> --project-id <project-id>
scorable execution-log list --project-id <project-id>

# Move a resource between projects
scorable judge update <judge-id> --project-id <other-project-id>

Setting a default project for your shell

To avoid passing --project-id on every command, set an env var or persist a per-machine default:

# Environment variable (great for CI)
export SCORABLE_PROJECT_ID=<project-id>

# Persistent default written to ~/.scorable/settings.json
scorable auth set-project <project-id>
scorable auth show-project    # see what's resolved and from where
scorable auth unset-project   # remove the saved default

Resolution order: --project-id flag (highest) → SCORABLE_PROJECT_ID env var → project_id in ~/.scorable/settings.json → omitted (backend resolves to org default). Pass --project-id "" to explicitly opt out of an inherited default for a single command.

Creating, renaming, deleting

You can create and rename projects from the selector. Project names must be unique within your organization.

Few rules:

• You cannot delete your only remaining project. Create another first.

• You cannot delete the default project while other projects exist. Promote another project to default first, then delete the old one.

• You cannot delete a project that still has judges, evaluators, or datasets attached. Move or delete those first.