Self-hosting

See Architecture for a high-level overview of the Scorable system.

Scorable Installation Guide

This document walks you through deploying Scorable to your own Kubernetes cluster with our Helm chart.

Self-hosting is available on the Scale plan. Contact Scorable at [e-mail]

---

Prerequisites

Required infrastructure

Component	Minimum	Notes
Kubernetes	v1.30+	Tested on AKS, GKE, and EKS
Helm	3.14+
PostgreSQL	16, with the pgvector extension available	A managed service (Azure Database for PostgreSQL, AWS RDS, GCP Cloud SQL) or an in-cluster Postgres both work.
Redis	7.x, TLS-capable	A managed service or in-cluster Redis.
Container registry credentials	Scorable provides on onboarding	Used to pull the Scorable container images.

Optional but recommended

• Ingress controller (Traefik, NGINX, etc.) — required for hostname-based access.

• cert-manager — for automatic TLS certificate lifecycle. Works with any ACME issuer (Let's Encrypt, internal CA, etc.) or a CA issuer if you have your own root.

• Sealed Secrets or External Secrets Operator (ESO) — for production-grade secrets management. If you skip both, the chart accepts a plain Kubernetes Secret you create out-of-band.

• Object storage — Azure Blob Storage or AWS S3 for media uploads and Django static files. Without object storage, uploads and static assets do not persist across pod restarts.

---

Installation Steps

1. Configure values

Create my-values.yaml based on the example below. The most commonly customised values are:

• domain — the hostname under which Scorable will be served.

• hosts.postgres / hosts.redis — DNS names of your data services.

• frontend.{authUrl, apiBaseUrl, ...} and api.{frontendUrl, apiBaseUrl} — your application's externally-visible URLs.

• useAzureStorage / useS3 and the corresponding storage settings.

• imagePullSecret — name of the Kubernetes Secret holding your registry credentials.

2. Install the chart

The Helm chart and container images are both hosted on GitHub Container Registry. Scorable provides a GitHub PAT with read:packages scope on onboarding — the same token authenticates Helm and your in-cluster image-pull Secret.

# 1. Log in to ghcr.io.
echo "<gh-pat>" | helm registry login ghcr.io --username <gh-user> --password-stdin

# 2. Install
helm upgrade --install scorable oci://ghcr.io/root-signals/charts/scorable \
  --version <version> \
  --namespace scorable --create-namespace \
  -f my-values.yaml \
  --atomic --timeout 10m

--atomic rolls back the install if any pod fails to come up within the timeout.

3. Wait for pods

kubectl -n scorable get pods -w

All deployments (api, frontend, evals, taskiq, pgdog) should reach Running 1/1. The api container runs database migrations on startup.

---

Database

Create the database and application role once, as your Postgres admin user, on the cluster that hosts.postgres points to.

CREATE ROLE scorable
  LOGIN
  PASSWORD 'REPLACE_WITH_STRONG_PASSWORD';

CREATE DATABASE scorable
  OWNER scorable
  ENCODING 'UTF8';

\c scorable

GRANT ALL ON SCHEMA public TO scorable;
CREATE SCHEMA llm_proxy AUTHORIZATION scorable;

If you change postgresDb or postgresUser in your values file, adjust the SQL above accordingly.

Required PostgreSQL extensions

The application uses several Postgres extensions. On most managed Postgres offerings, only the superuser can create extensions, so create these once as the Postgres admin before installing the chart:

-- Run as Postgres admin, against the `scorable` database
CREATE EXTENSION IF NOT EXISTS vector;
CREATE EXTENSION IF NOT EXISTS pg_trgm;
CREATE EXTENSION IF NOT EXISTS unaccent;
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
CREATE EXTENSION IF NOT EXISTS citext;
CREATE EXTENSION IF NOT EXISTS btree_gin;
CREATE EXTENSION IF NOT EXISTS btree_gist;

Some managed services additionally require extensions to be allow-listed at the server level before they can be created. Consult your provider's documentation if CREATE EXTENSION returns a "not allowed" error.

Connection pooler (pgdog)

The chart ships pgdog as a subchart that pools connections in front of Postgres. If you override postgresDb or hosts.postgres at the top level of your values file, you must also override pgdog.databases[0].name and pgdog.databases[0].host to match — Helm cannot cross-reference between subchart blocks.

postgresDb: "scorable"         # ← top-level setting …
hosts:
  postgres: "pg.example.com"   # ← and host
pgdog:
  databases:                   # ← must match here
    - name: scorable
      host: pg.example.com
      port: 5432
      role: primary

---

Object storage

The chart supports Azure Blob Storage or AWS S3. Pick one; set useAzureStorage: true or useS3: true.

Two locations are needed: a publicly-readable one for Django static assets, and a private one for user uploads. Django rewrites static-file URLs to point directly at the object-storage endpoint, so the static-files location must permit anonymous reads — or be fronted by a CDN you trust to serve them.

For Azure: a single storage account with two containers (public-data set to blob-level anonymous access, customer-data private). For AWS: two S3 buckets (one with a static-website / public-read policy, one private). The chart authenticates to the private location with the credentials you supply in the secret.

---

Secrets management

The chart needs a Kubernetes Secret available in the deployment namespace by the time pods start. There are three supported patterns.

Option A — Plain Kubernetes Secret

Simplest, suitable for environments where Kubernetes RBAC is your sole secrets boundary.

useSealedSecrets: false
secretRef: "rs-secrets"

Create the Secret yourself before installing the chart:

kubectl -n scorable create secret generic rs-secrets \
  --from-literal=POSTGRES_PASSWORD='...' \
  --from-literal=REDIS_PASSWORD='...' \
  --from-literal=REDIS_URL='rediss://:...?ssl_cert_reqs=none' \
  --from-literal=AZURE_STORAGE_CONNECTION_STRING='...' \
  --from-literal=SECRET_KEY='...' \
  --from-literal=NEXTAUTH_SECRET='...' \
  --from-literal=LLM_PROXY_MASTER_KEY='...' \
  --from-literal=LLM_PROXY_SALT_KEY='...' \
  --from-literal=OPENAI_API_KEY='sk-...' \
  --from-literal=AZURE_OPENAI_API_KEY='...'

REDIS_URL is optional. If unset, api/workers build the URL from the fragment-style REDIS_HOST / REDIS_PORT / REDIS_PASSWORD / REDIS_SSL env vars (set by the chart from hosts.redis, ports.redis, redisSsl, and the REDIS_PASSWORD secret). For managed Redis with TLS (Azure Managed Redis, Azure Cache for Redis Premium, ElastiCache with in-transit encryption), prefer setting REDIS_URL directly to avoid the fragment-based URL builder.

SECRET_KEY, NEXTAUTH_SECRET, LLM_PROXY_MASTER_KEY, and LLM_PROXY_SALT_KEY must not be changed after first deploy — they encrypt at-rest data. Generate them once and store them durably.

Option B — Sealed Secrets

useSealedSecrets: true
secrets:
  POSTGRES_PASSWORD: "AgAB..."          # encrypted with kubeseal
  # ... all the other keys, each encrypted
  PGDOG_USERS_TOML: "AgAB..."           # encrypted pgdog users config

Encrypt each value:

echo -n "your-secret-value" | kubeseal --scope cluster-wide --raw --from-file=/dev/stdin

Option C — External Secrets Operator

If you have ESO installed and a ClusterSecretStore configured against your secret store (Azure Key Vault, AWS Secrets Manager, Vault, etc.):

useSealedSecrets: false
secretRef: "rs-secrets"

Create an ExternalSecret resource yourself that materialises rs-secrets from your upstream store. The chart doesn't generate the ExternalSecret — it expects the rs-secrets Secret to exist when pods start.

pgdog users secret

The chart's pgdog subchart needs a pgdog-users Secret with a users.toml payload. Provide its contents in secrets.PGDOG_USERS_TOML. With Sealed Secrets the value is encrypted in your values file. With plain Secret / ESO, the simplest approach is to pass the file at install time so the plaintext doesn't sit in your values file:

cat > /tmp/pgdog-users.toml <<EOF
[[users]]
name = "scorable"
database = "scorable"
password = "<same value as POSTGRES_PASSWORD>"
pool_size = 32
EOF

helm upgrade --install scorable ... --set-file secrets.PGDOG_USERS_TOML=/tmp/pgdog-users.toml
rm /tmp/pgdog-users.toml

---

Ingress and TLS

The chart emits Ingress resources for api and frontend when ingress.enabled: true is set on each. Annotations and the tls block are fully values-driven — supply whatever your ingress controller and TLS strategy require.

Example: Traefik + cert-manager + Let's Encrypt

domain: "scorable.example.com"

api:
  frontendUrl: "https://scorable.example.com"
  apiBaseUrl:  "https://api.scorable.example.com"
  ingress:
    enabled: true
    className: traefik
    annotations:
      cert-manager.io/cluster-issuer: "letsencrypt"
      external-dns.alpha.kubernetes.io/hostname: "api.scorable.example.com"
    tls:
      enabled: true

frontend:
  authUrl:           "https://scorable.example.com"
  authUrlInternal:   "http://127.0.0.1:3000"
  apiBaseUrl:        "https://api.scorable.example.com"
  apiBaseUrlServer:  "http://api:80"
  ingress:
    enabled: true
    className: traefik
    host: "scorable.example.com"
    annotations:
      cert-manager.io/cluster-issuer: "letsencrypt"
      external-dns.alpha.kubernetes.io/hostname: "scorable.example.com"
    tls:
      enabled: true

Notes:

• frontend.ingress.host defaults to .Values.domain (the apex). If you prefer the frontend at app.<domain> while the API stays at api.<domain>, set frontend.ingress.host: "app.<domain>" explicitly.

• api.ingress.host defaults to api.<.Values.domain>.

• The four frontend.{authUrl, authUrlInternal, apiBaseUrl, apiBaseUrlServer} and api.{frontendUrl, apiBaseUrl} values must match the URLs your users will type. The chart's defaults are http://localhost:*, intended for kubectl port-forward development.

Example: WAF or LB in front, chart ingress controller behind

Common enterprise pattern. A WAF/LB layer (e.g. Azure Application Gateway, AWS ALB) terminates TLS in front; the chart's ingress controller serves plain HTTP behind it.

api:
  ingress:
    enabled: true
    className: traefik
    annotations:
      external-dns.alpha.kubernetes.io/hostname: "api.scorable.example.com"
    tls:
      enabled: false       # WAF terminates TLS

frontend:
  ingress:
    enabled: true
    className: traefik
    annotations:
      external-dns.alpha.kubernetes.io/hostname: "scorable.example.com"
    tls:
      enabled: false

---

Example values file

A complete-ish example covering common knobs. Adjust to your environment.

domain: "scorable.example.com"
environment: "production"

# Storage — pick one (Azure)
useAzureStorage: true
useS3: false
azureStorageAccountName: "scorablestorageabc123"
azureStoragePublicDataContainer: "public-data"
azureStorageCustomerDataContainer: "customer-data"

# Secrets
useSealedSecrets: false
secretRef: "rs-secrets"
imagePullSecret: "scorable-image-pull-secret"

# Hosts
hosts:
  postgres: "pg.scorable.example.com"
  redis:    "redis.scorable.example.com"

postgresUser: "scorable"
postgresDb:   "scorable"
redisSsl:     true

# Connection pooler — must match postgresDb + hosts.postgres
pgdog:
  enabled: true
  databases:
    - name: scorable
      host: pg.scorable.example.com
      port: 5432
      role: primary

# Application URLs — override when using Ingress
frontend:
  authUrl:           "https://scorable.example.com"
  authUrlInternal:   "http://127.0.0.1:3000"
  apiBaseUrl:        "https://api.scorable.example.com"
  apiBaseUrlServer:  "http://api:80"
  ingress:
    enabled: true
    className: traefik
    host: "scorable.example.com"
    annotations:
      cert-manager.io/cluster-issuer: "letsencrypt"
    tls:
      enabled: true

api:
  frontendUrl: "https://scorable.example.com"
  apiBaseUrl:  "https://api.scorable.example.com"
  ingress:
    enabled: true
    className: traefik
    annotations:
      cert-manager.io/cluster-issuer: "letsencrypt"
    tls:
      enabled: true

# LLM proxy — enable when you have a real LLM provider configured
llmProxy:
  enabled: false

# RAG — optional
rag:
  enabled: false

# Optional: include additional resources alongside the chart
extraManifests: []

---

Accessing the application

Using Ingress (production)

With Ingress configured per the example above, the app is at https://<domain> and the API at https://api.<domain>. Create an admin user, then sign in.

Creating the first admin user

POD=$(kubectl -n scorable get pods -l app=api --no-headers -o custom-columns=:metadata.name | head -1)
kubectl -n scorable exec "$POD" -- \
  env DJANGO_SUPERUSER_USERNAME=admin \
      DJANGO_SUPERUSER_EMAIL=[e-mail] \
      DJANGO_SUPERUSER_PASSWORD='ChangeMe123!' \
  python manage.py createsuperuser --noinput

Then visit https://api.<domain>/admin/ to log into Django admin, or https://<domain> to log into the frontend.

Using port forwarding (development)

kubectl port-forward --namespace scorable service/frontend 3000:80 &
kubectl port-forward --namespace scorable service/api 8000:80 &

Open http://localhost:3000. With this access path, leave the chart's default frontend.authUrl / frontend.apiBaseUrl values as-is — they target localhost.

---

Capping model spend per user

Model calls go out through the bundled LLM proxy, which enforces a rolling 24-hour spend cap per user. Once a user reaches their cap, their model calls are rejected until the window rolls over.

The deployment default

Set the fallback cap in your values file:

api:
  llmProxyBudgetPerDayPerUser: 25   # USD per user per rolling 24h

Leave it unset and no meaningful cap is applied.

This value is written to a user's proxy key when that key is first created. Changing it later applies to users created after the change. Existing users keep the cap their key was created with. To move an existing user, use one of the two mechanisms below.

Letting users lower their own cap

Users can set their own daily limit under Settings → Account. This is a self-protection guardrail: it is most useful for capping the damage from a runaway job or an oversized backfill.

What a user may pick is bounded by the quota their organization resolves through, on deployments that run Scorable's plan model. Two fields on Quota, editable in the Django admin under Billing → Quotas:

Field on Quota	Meaning
llm_proxy_budget_per_day_per_user	The cap a user gets when they have not chosen one.
llm_proxy_budget_per_day_per_user_max	The highest cap a user may set for themselves.

When a quota leaves the maximum unset, users on it can only lower their cap, never raise it.

Changing either field re-pushes the cap to every existing key resolving through that quota, so a raise reaches current users rather than only new ones.

Quotas are ignored unless you opt in. The quota rows ship with values chosen for Scorable's hosted plans, and they exist on every install. A self-hosted deployment ignores them entirely unless you set:

api:
  llmProxyPlanBudgetsEnabled: "true"

Leave it unset, which is the default, and caps resolve from llmProxyBudgetPerDayPerUser alone, so users can lower their own cap but nothing else overrides your configured default. Turn it on only if you intend to run Scorable's plan model.

A cap a user can raise is not a cost control. Only set llm_proxy_budget_per_day_per_user_max above the default if you are comfortable with users spending up to that amount.

Overriding a single user

To move one user without touching everyone else, edit LLM proxy → Max budget per day on that user in the Django admin. This is the support escalation path, and it deliberately ignores the quota maximum, so you can lift a single user above what they could set themselves. The change reaches their existing key immediately.

The same panel shows the cap the proxy was last successfully told. If that disagrees with the configured cap, a push to the proxy failed and the user is still running under the older value.

---

Monitoring and Logging

Sentry

Set api.sentry.dsn, frontend.sentry.dsn, evals.sentry.dsn, etc. in your values file. The chart wires DSNs into pod environments.

Prometheus

The application exposes metrics at /metrics endpoints. Configure your Prometheus instance to scrape:

• api:80/metrics

• frontend:80/metrics

• evals:80/metrics

• pgdog:9090/metrics

For detailed assistance, contact our support team.

---

Updates

Scorable notifies customer contact persons when new versions of the Helm chart are released.

helm upgrade scorable oci://ghcr.io/root-signals/charts/scorable \
  --version <new-version> \
  --namespace scorable \
  -f my-values.yaml \
  --atomic --timeout 10m

Always read the release notes for breaking changes before upgrading.

---

Support

• Email: [e-mail]

• Slack (provided on onboarding)

Keep your Kubernetes cluster, managed-service versions, and Helm up to date. Back up your database and configurations regularly.