# RAG evaluation

Scorable provides evaluators for *Retrieval Augmented Generation (RAG)* use cases, where you can give the context as part of the evaluated content.

## Hallucination Detection

One of the most useful evaluators in RAG settings is **Faithfulness** which detects claims that can not be deducted from the context, i.e., *hallucinations* in RAG setup.

Here is an example of running a hallucination check using the Python SDK:

```python
from scorable import Scorable

client = Scorable()

request = "Is the number of pensioners working more than 100k in 2023?"
response = "Yes, 150000 pensioners were working in 2024."

# Chunks retreived from a RAG pipeline
retreived_document_1 = """
While the work undertaken by seniors is often irregular and part-time, more than 150,000 pensioners were employed in 2023, the centre's statistics reveal. The centre noted that pensioners have increasingly continued to work for some time now.
"""
retreived_document_2 = """
According to the pension centre's latest data, a total of around 1.3 million people in Finland were receiving old-age pensions, with average monthly payments of 1,948 euros.
"""

# Measures is the answer faithful to my contexts (knowledge-base/documents)
faithfulness_result = client.evaluators.Faithfulness(
    request=request,
    response=response,
    contexts=[retreived_document_1, retreived_document_2],
)

print(faithfulness_result.score)  # 0.0 as the response does not match the retrieved documents
print(faithfulness_result.justification)
```

Another such evaluator is the **Truthfulness** evaluator, which measures the factual consistency of the generated answer against the given context as well as general knowledge.

Here is an example of running the **Truthfulness** evaluator:

```python
result = client.evaluators.Truthfulness(
    request="What was the revenue in Q1/2023",
    response="The revenue in the last quarter was 5.2 M USD",
    contexts=[
        "Financial statement of 2023"
        "2023 revenue and expenses...",
    ],
)
print(result.score)
# 0.5
```

For other RAG evaluators, refer to our [Evaluator Portfolio](/quick-start/evaluator-portfolio.md) page.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.scorable.ai/concepts-and-examples/cookbooks/rag-evaluation.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.