# Roadmap

[Root Signals](https://rootsignals.ai/) builds with the philosophy of transparency with multiple [open source](https://github.com/root-signals) projects. This roadmap is a living document about what we're working on and what's next. Scorable is the world's most principled and powerful system for measuring the behaviour of LLM based applications, agents and workflows.

[Scorable](https://scorable.ai/) is the automated LLM Evaluation Engineer agent for co-managing this platform with you.

## Vision

Our vision is to ***create and auto-optimize the strongest automated knowledge process evaluation stack possible, with the least amount of effort and information from the user.***

* **Maximum Automated Information Extraction**
  * From user intent and/or provided example/instruction data, extract as much relevant information as possible.
* **Awareness of the information quality**
  * Engage the user with the smallest amount of maximally impactful questions.
* **Maximally Powerful Evaluation Stack Generation**
  * Build the most comprehensive and accurate evaluation capabilities possible, within the confines of data available.
* **Built for Agents**
  * Maximum compatibility with autonomous agents and workflows.
* **Maximum Integration Surface**
  * Seamless integration with all key AI frameworks.
* **EvalOps Principles for Long Term**
  * Follow Root [EvalOps](https://www.rootsignals.ai/post/evalops) Principles for evaluator lifecycle management.
* [Principled Evaluator Infrastructure](https://docs.scorable.ai/overview/principles)

{% hint style="info" %}
**All feedback is highly appreciated and often leads to immediate action.** Submit new [GitHub issues](https://github.com/root-signals/rs-sdk/issues) or vote on existing ones, so we can take quick action on what is important to you.
{% endhint %}

## 🚀 Recently Released

* ✅ Rehashing of Example-driven Evaluation
  * *Smoothly create the full judge from examples*
* ✅ Insights generation in monitoring view
* ✅ Full OpenTelemetry (OTEL) exports <https://docs.scorable.ai/integrations/opentelemetry>
* ✅ Advanced Judge visibility controls
* ✅ TypeScript SDK
* ✅ Command Line Interface
* ✅ Automated Policy Adherence Judges
  * Create judges from uploaded policy documents and intents
* ✅ GDPR awareness of models ([link](https://docs.scorable.ai/usage/usage/models#control-and-compliance))
  * Ability to filter out models not complying with GDPR
* ✅ Evaluator Calibration Data Synthesizer v1.0 ([link](https://docs.scorable.ai/usage/usage/evaluators#calibration))
  * In the evaluator drill-in view, expand your calibration dataset from 1 or more examples
* ✅ Evaluator version history and control to include all native Root Evaluators ([link](https://docs.scorable.ai/usage/usage/evaluators#version-control))
* ✅ Evaluator determinism benchmarks and standard deviations in reference datasets ([link](https://docs.scorable.ai/usage/usage/evaluators#determinism))
* ✅ Agent Evaluation MCP: stdio & SSE versions ([link](https://github.com/root-signals/root-signals-mcp))
* ✅ [Root Judge LLM](https://www.rootsignals.ai/root-judge-llm) 70B judge available for download and running in Scorable for free!
* ✅ Public Evaluation Reports - Generate HTML reports from any judge execution ([link](https://docs.scorable.ai/usage/usage/monitoring-and-insights))
* ✅ Unified *Experiments & Prompt Testing* framework to Replace Skill Tests ([link](https://docs.scorable.ai/usage/usage/prompt-testing))

## 🏗️ Next Up

* Agentic Classifier Generation 2.0
  * Create classifiers with the same robustness as metric evaluator stacks
* RAG evaluators auto-placement
* Sync Judge & Evaluator Definitions to GitHub
* CLI extensions

## 🗓️ Planned

* Expansions to OTEL support
* Community Evaluators Support Extensions
* GitHub Connector
* Unify the onboarding sample synthesis with the full synthesis for maximum variance
* Upgrades to contract & execution manifest ontology

## Feature Requests and Bug Reports:

🐛 **Bug Reports**: [GitHub Issues](https://github.com/root-signals/rs-sdk/issues)

📧 **Enterprise Features**: Contact <sales@scorable.ai>

***

*Last updated: 2025-06-30*