# Use a Judge

Once you have created your Judge using the [Scorable tool](https://scorable.ai/), you can integrate it into your application through various methods, including SDKs, API, or CLI.

Since OpenAI's API serves as the lingua franca of LLMs, it's one of the most popular ways to use Judges. Scorable provides an easy integration method by simply changing your base URL to point to the Scorable OpenAI proxy.

{% hint style="info" %}
**Provider key required.** The OpenAI-compatible proxy endpoints (`/openai/chat/completions`, `/openai/responses`) make the upstream model call on your behalf and require a customer-managed provider key for the requested model's provider. Connect a key in **Organization Settings → Providers**; otherwise the request returns `403 byok_required`. The non-proxy execution endpoints (e.g. `judges.run`) are unaffected.
{% endhint %}

## 🔍 Run a Judge to evaluate the quality of returns policy claims

Let's walk through an example where we have a Judge that evaluates the quality of returns policy claims.

```python
# pip install openai
from openai import OpenAI

client = OpenAI(
    api_key="$MY_SCORABLE_API_KEY",
    base_url="https://api.scorable.ai/v1/judges/$MY_JUDGE_ID/openai/"
)
response = client.chat.completions.create(
    model="claude-sonnet-4",
    messages=[
        {"role": "user", "content": "I want to return my product"}
    ]
)
```

The response will include the Judge's evaluation results in the `model_extra` field:

```python
print(response.model_extra.get('evaluator_results'))

[
  {
    "name": "Returns Policy Claims",
    "score": 0.95,
    "justification": "The policy claims are clear and easy to understand..."
  }
  ...
]
```

### Background Execution

You can also run the Judge in the background and check the results later through the monitoring dashboard:

```python
client = OpenAI(
    api_key="$MY_SCORABLE_API_KEY",
    # 💡 Add ?async_judge=true to the base_url
    base_url="https://api.scorable.ai/v1/judges/$MY_JUDGE_ID/openai/?async_judge=true"
)
response = client.chat.completions.create(
    model="gpt-5.2",
    messages=[
        {"role": "user", "content": "I want to return my product"}
    ]
)
```

{% hint style="info" %}
Judges automatically run in the background when you stream responses.
{% endhint %}

<figure><img src="/files/OE0MqmCipouvuKUM0rey" alt=""><figcaption></figcaption></figure>

## ✨ Use the Judge to improve model responses automatically

By switching the base URL to the Scorable OpenAI proxy **refine** endpoint, you can use the Judge to improve the model responses automatically.

```python
client = OpenAI(
    api_key="$MY_SCORABLE_API_KEY",
    base_url="https://api.scorable.ai/v1/judges/$MY_JUDGE_ID/refine/openai/"
)
response = client.chat.completions.create(
    model="gpt-5.2",
    messages=[
        {"role": "user", "content": "I want to return my product"}
    ]
)
```

Here, based on the Judge's evaluation, the Scorable platform will ensure that the model response aligns with the safeguards you have configured in the Judge.

{% hint style="info" %}
The refine endpoint proxies the model call through Scorable and requires a customer-managed provider key for the requested model's provider. Connect one in **Organization Settings → Providers** to avoid a `403 byok_required` response.
{% endhint %}

## Summary

Integrating Judges into your application is straightforward — simply change the base URL in your existing OpenAI client configuration. Scorable supports all major LLMs and providers, and you can bring your own models if needed.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.scorable.ai/concepts-and-examples/cookbooks/use-a-judge.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.