> For the complete documentation index, see [llms.txt](https://docs.scorable.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.scorable.ai/concepts-and-examples/cookbooks/use-a-judge.md).

# Use a Judge

Once you have created your Judge using the [Scorable tool](https://scorable.ai/), you can integrate it into your application through various methods, including SDKs, API, or CLI.

Since OpenAI's API serves as the lingua franca of LLMs, it's one of the most popular ways to use Judges. Scorable provides an easy integration method by simply changing your base URL to point to the Scorable OpenAI proxy.

{% hint style="info" %}
**Provider key required.** The OpenAI-compatible proxy endpoints (`/openai/chat/completions`, `/openai/responses`) make the upstream model call on your behalf and require a customer-managed provider key for the requested model's provider. Connect a key in **Organization Settings → Providers**; otherwise the request returns `403 byok_required`. The non-proxy execution endpoints (e.g. `judges.run`) are unaffected.
{% endhint %}

## 🔍 Run a Judge to evaluate the quality of returns policy claims

Let's walk through an example where we have a Judge that evaluates the quality of returns policy claims.

```python
# pip install openai
from openai import OpenAI

client = OpenAI(
    api_key="$MY_SCORABLE_API_KEY",
    base_url="https://api.scorable.ai/v1/judges/$MY_JUDGE_ID/openai/"
)
response = client.chat.completions.create(
    model="claude-sonnet-4",
    messages=[
        {"role": "user", "content": "I want to return my product"}
    ]
)
```

The response will include the Judge's evaluation results in the `model_extra` field:

```python
print(response.model_extra.get('evaluator_results'))

[
  {
    "name": "Returns Policy Claims",
    "score": 0.95,
    "justification": "The policy claims are clear and easy to understand..."
  }
  ...
]
```

### Background Execution

You can also run the Judge in the background and check the results later through the monitoring dashboard:

```python
client = OpenAI(
    api_key="$MY_SCORABLE_API_KEY",
    # 💡 Add ?async_judge=true to the base_url
    base_url="https://api.scorable.ai/v1/judges/$MY_JUDGE_ID/openai/?async_judge=true"
)
response = client.chat.completions.create(
    model="gpt-5.2",
    messages=[
        {"role": "user", "content": "I want to return my product"}
    ]
)
```

{% hint style="info" %}
Judges automatically run in the background when you stream responses.
{% endhint %}

<figure><img src="/files/OE0MqmCipouvuKUM0rey" alt=""><figcaption></figcaption></figure>

## ✨ Use the Judge to improve model responses automatically

By switching the base URL to the Scorable OpenAI proxy **refine** endpoint, you can use the Judge to improve the model responses automatically.

```python
client = OpenAI(
    api_key="$MY_SCORABLE_API_KEY",
    base_url="https://api.scorable.ai/v1/judges/$MY_JUDGE_ID/refine/openai/"
)
response = client.chat.completions.create(
    model="gpt-5.2",
    messages=[
        {"role": "user", "content": "I want to return my product"}
    ]
)
```

Here, based on the Judge's evaluation, the Scorable platform will ensure that the model response aligns with the safeguards you have configured in the Judge.

{% hint style="info" %}
The refine endpoint proxies the model call through Scorable and requires a customer-managed provider key for the requested model's provider. Connect one in **Organization Settings → Providers** to avoid a `403 byok_required` response.
{% endhint %}

## Summary

Integrating Judges into your application is straightforward — simply change the base URL in your existing OpenAI client configuration. Scorable supports all major LLMs and providers, and you can bring your own models if needed.