Concepts

This section covers the core Scorable concepts in depth. For a short glossary of the main terms, see the Concepts overview.

• Projects: workspaces that group related resources inside your organization.

• Models: the LLMs that power evaluators, including your own connected models.

• Objectives: reusable definitions of what you want to evaluate.

• Evaluators: individual scoring rubrics, both built-in and custom.

• Datasets & Annotations: labeled data for calibrating and steering evaluators.

• Judges: bundles of evaluators that capture your definition of quality for a use case.

• Prompt Testing: compare prompts and models against datasets from the CLI.

• Monitoring & Insights: observe evaluation results in production.

• Issues: structured, searchable records of recurring failure patterns.

• Execution, Auditability and Versioning: logs, retention, and reproducibility.

• Access Controls & Roles: user roles and permissions.

• Lifecycle Management: evolving your evaluation stack over time.