# OpenTelemetry

Scorable accepts OpenTelemetry (OTEL) traces from any agent framework. Once traces arrive, Scorable shows a per-trace view of every LLM call, its inputs and outputs, latency, and span count — and can automatically evaluate traces against your configured evaluators and judges.

## Prerequisites

You need a Scorable API key. Find it under **Settings → API Keys** in the dashboard.

***

## Example: pydantic-ai

[pydantic-ai](https://ai.pydantic.dev/) has built-in OTEL support via `InstrumentationSettings`. Configure it to export to Scorable:

```python
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from pydantic_ai import Agent, InstrumentationSettings


def _build_tracer_provider() -> TracerProvider:
    exporter = OTLPSpanExporter(
        endpoint="https://api.scorable.ai/otel/v1/traces",
        headers={"Authorization": "Api-Key <your-api-key>"},
    )
    resource = Resource.create({"service.name": "my-agent"})
    provider = TracerProvider(resource=resource)
    provider.add_span_processor(BatchSpanProcessor(exporter))
    return provider


agent = Agent(
    model="openai:gpt-5.2",
    instrument=InstrumentationSettings(
        tracer_provider=_build_tracer_provider(),
    ),
)
```

Every `agent.run()` call now produces a trace visible in Scorable.

***

## Example: any other framework

Configure the OTEL SDK to point at Scorable's collector endpoint and set the `Authorization` header. The example below works with any framework that supports OTEL instrumentation (LangChain, LlamaIndex, raw `openai` SDK with `opentelemetry-instrumentation-openai`, etc.).

```python
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

exporter = OTLPSpanExporter(
    endpoint="https://api.scorable.ai/otel/v1/traces",
    headers={"Authorization": "Api-Key <your-api-key>"},
)

resource = Resource.create({"service.name": "my-agent"})
provider = TracerProvider(resource=resource)
provider.add_span_processor(BatchSpanProcessor(exporter))
trace.set_tracer_provider(provider)
```

Then instrument your framework as usual — Scorable receives whatever spans the framework emits.

### Instrumentation libraries

Any OpenTelemetry-compatible instrumentation library works with Scorable. Popular options for AI/LLM workloads:

| Library                                                                                                           | Frameworks covered                                                 |
| ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------ |
| [OpenLIT](https://docs.openlit.io)                                                                                | OpenAI, Anthropic, LangChain, LlamaIndex, CrewAI, Cohere, and more |
| [OpenLLMetry](https://www.traceloop.com/docs/openllmetry/introduction)                                            | OpenAI, Anthropic, LangChain, LlamaIndex, Haystack, and more       |
| [smolagents](https://huggingface.co/docs/smolagents/en/tutorials/inspect_runs)                                    | Hugging Face smolagents                                            |
| [CrewAI](https://docs.crewai.com/en/observability/opentelemetry)                                                  | CrewAI                                                             |
| [AutoGen](https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/framework/telemetry.html)         | AutoGen                                                            |
| [LlamaIndex](https://docs.llamaindex.ai/en/stable/module_guides/observability/)                                   | LlamaIndex                                                         |
| [Semantic Kernel](https://learn.microsoft.com/en-us/semantic-kernel/concepts/enterprise-readiness/observability/) | Semantic Kernel (Python, .NET, Java)                               |

### Environment variable alternative

If you prefer to configure the exporter through env vars rather than code:

```bash
OTEL_EXPORTER_OTLP_ENDPOINT=https://api.scorable.ai/otel/v1/traces
OTEL_EXPORTER_OTLP_HEADERS="Authorization=Api-Key <your-api-key>"
OTEL_SERVICE_NAME=my-agent
```

***

## Viewing traces

Traces appear in the **Traces** tab in the dashboard. Each row represents one agent run (one `trace_id`), showing the root span name, time, and total span count. Click a trace to see the full span tree.

***

## Automatic evaluation

You can configure Scorable to automatically evaluate incoming traces against an evaluator or judge. See **Settings → Trace Evaluation Filters** to set up filter criteria, sampling rate, and evaluation delay (to allow late-arriving spans before evaluation runs).

Evaluation uses the `gen_ai.input.messages` and `gen_ai.output.messages` span attributes, which pydantic-ai and most OTEL-instrumented LLM frameworks emit automatically.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.scorable.ai/integrations/opentelemetry.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.