Roadmap

Root Signals builds with the philosophy of transparency with multiple open source projects. This roadmap is a living document about what we're working on and what's next. Scorable is the world's most principled and powerful system for measuring the behaviour of LLM based applications, agents and workflows.

Scorable is the automated LLM Evaluation Engineer agent for co-managing this platform with you.

Vision

Our vision is to create and auto-optimize the strongest automated knowledge process evaluation stack possible, with the least amount of effort and information from the user.

• Maximum Automated Information Extraction

  • From user intent and/or provided example/instruction data, extract as much relevant information as possible.

• Awareness of the information quality

  • Engage the user with the smallest amount of maximally impactful questions.

• Maximally Powerful Evaluation Stack Generation

  • Build the most comprehensive and accurate evaluation capabilities possible, within the confines of data available.

• Built for Agents

  • Maximum compatibility with autonomous agents and workflows.

• Maximum Integration Surface

  • Seamless integration with all key AI frameworks.

• EvalOps Principles for Long Term

  • Follow Root EvalOps Principles for evaluator lifecycle management.

• Principled Evaluator Infrastructure

All feedback is highly appreciated and often leads to immediate action. Submit new GitHub issues or vote on existing ones, so we can take quick action on what is important to you.

🚀 Recently Released

• ✅ Rehashing of Example-driven Evaluation

  • Smoothly create the full judge from examples

• ✅ Insights generation in monitoring view

• ✅ Full OpenTelemetry (OTEL) exports https://docs.scorable.ai/integrations/opentelemetry

• ✅ Advanced Judge visibility controls

• ✅ TypeScript SDK

• ✅ Command Line Interface

• ✅ Automated Policy Adherence Judges

  • Create judges from uploaded policy documents and intents

• ✅ GDPR awareness of models (link)

  • Ability to filter out models not complying with GDPR

• ✅ Evaluator Calibration Data Synthesizer v1.0 (link)

  • In the evaluator drill-in view, expand your calibration dataset from 1 or more examples

• ✅ Evaluator version history and control to include all native Root Evaluators (link)

• ✅ Evaluator determinism benchmarks and standard deviations in reference datasets (link)

• ✅ Agent Evaluation MCP: stdio & SSE versions (link)

• ✅ Root Judge LLM 70B judge available for download and running in Scorable for free!

• ✅ Public Evaluation Reports - Generate HTML reports from any judge execution (link)

• ✅ Unified Experiments & Prompt Testing framework to Replace Skill Tests (link)

🏗️ Next Up

• Agentic Classifier Generation 2.0

  • Create classifiers with the same robustness as metric evaluator stacks

• RAG evaluators auto-placement

• Sync Judge & Evaluator Definitions to GitHub

• CLI extensions

🗓️ Planned

• Expansions to OTEL support

• Community Evaluators Support Extensions

• GitHub Connector

• Unify the onboarding sample synthesis with the full synthesis for maximum variance

• Upgrades to contract & execution manifest ontology

Feature Requests and Bug Reports:

🐛 Bug Reports: GitHub Issues

📧 Enterprise Features: Contact [email protected]

---

Last updated: 2025-06-30