Roadmap
Root Signals builds with the philosophy of transparency with multiple open source projects. This roadmap is a living document about what we're working on and what's next. Scorable is the world's most principled and powerful system for measuring the behaviour of LLM based applications, agents and workflows.
Scorable is the automated LLM Evaluation Engineer agent for co-managing this platform with you.
Vision
Our vision is to create and auto-optimize the strongest automated knowledge process evaluation stack possible, with the least amount of effort and information from the user.
Maximum Automated Information Extraction
From user intent and/or provided example/instruction data, extract as much relevant information as possible.
Awareness of the information quality
Engage the user with the smallest amount of maximally impactful questions.
Maximally Powerful Evaluation Stack Generation
Build the most comprehensive and accurate evaluation capabilities possible, within the confines of data available.
Built for Agents
Maximum compatibility with autonomous agents and workflows.
Maximum Integration Surface
Seamless integration with all key AI frameworks.
EvalOps Principles for Long Term
Follow Root EvalOps Principles for evaluator lifecycle management.
All feedback is highly appreciated and often leads to immediate action. Submit new GitHub issues or vote on existing ones, so we can take quick action on what is important to you.
π Recently Released
β Rehashing of Example-driven Evaluation
Smoothly create the full judge from examples
β Insights generation in monitoring view
β Full OpenTelemetry (OTEL) exports https://docs.scorable.ai/integrations/opentelemetry
β Advanced Judge visibility controls
β TypeScript SDK
β Command Line Interface
β Automated Policy Adherence Judges
Create judges from uploaded policy documents and intents
β GDPR awareness of models (link)
Ability to filter out models not complying with GDPR
β Evaluator Calibration Data Synthesizer v1.0 (link)
In the evaluator drill-in view, expand your calibration dataset from 1 or more examples
β Evaluator version history and control to include all native Root Evaluators (link)
β Evaluator determinism benchmarks and standard deviations in reference datasets (link)
β Agent Evaluation MCP: stdio & SSE versions (link)
β Root Judge LLM 70B judge available for download and running in Scorable for free!
β Public Evaluation Reports - Generate HTML reports from any judge execution (link)
β Unified Experiments & Prompt Testing framework to Replace Skill Tests (link)
ποΈ Next Up
Agentic Classifier Generation 2.0
Create classifiers with the same robustness as metric evaluator stacks
RAG evaluators auto-placement
Sync Judge & Evaluator Definitions to GitHub
CLI extensions
ποΈ Planned
Expansions to OTEL support
Community Evaluators Support Extensions
GitHub Connector
Unify the onboarding sample synthesis with the full synthesis for maximum variance
Upgrades to contract & execution manifest ontology
Feature Requests and Bug Reports:
π Bug Reports: GitHub Issues
π§ Enterprise Features: Contact [email protected]
Last updated: 2025-06-30
Last updated