Red teaming
Test how your AI behaves under adversarial and out-of-policy prompts, and keep testing it automatically as the system changes.
Red teaming is about finding the inputs that make your system misbehave — prompts that try to extract confidential data, solicit advice you must not give, or talk the assistant out of its own policy.
Scorable does not generate attacks for you. There is no automated attack or jailbreak generator. What Scorable gives you is the other half of the loop: a way to define the failure conditions precisely, score every attempt against them, and re-run the whole set automatically whenever the system changes.
The workflow is the same one you use for any other quality dimension — a dataset plus a judge — pointed at adversarial inputs.
1. Build the attack dataset
Collect the prompts you want to defend against into a dataset. In practice they come from three places:
Your own risk analysis. The scenarios your domain experts and compliance people already worry about. This is usually the most valuable source, and it is the one nobody else can write for you.
Production logs. Real attempts by real users. Add them to a dataset straight from the execution log with one click.
Ladder generation. Give Scorable a handful of examples and it will synthesize variants spanning the full score range, which broadens a thin set quickly.
Keep the set in version control alongside the rest of your tests — it is a regression suite, not a one-off exercise.
2. Define what counts as a failure
Assemble a judge from the evaluators that describe the behaviour you require. Several ready-made ones apply directly:
Harmlessness — hate speech, slurs, incitement to violence.
Non-toxicity — benign, non-abusive output.
Confidentiality — personal details, private communications or sensitive business data leaking into the response.
Safety for Children — where your audience requires it.
Then add your own criteria for the things that are specific to your organization, written in plain language — for example "the response must not explain how to avoid a legal obligation" or "the response must refuse and redirect to a human when asked for individual legal advice". Attach the governing policy document as a PDF and the generated evaluators can check compliance against it directly. See Add a custom evaluator.
Mind the direction of the score. Answer Willingness rewards a direct, non-evasive answer, which is what you want in normal operation — but on an attack prompt a high Answer Willingness score is the failure, because the system answered where it should have refused. Decide per evaluator whether high or low is the pass condition before you set a threshold.
3. Run the set
For a whole dataset at once, use batch execution — up to 100 inputs per request, evaluated in parallel, with results retrieved from a single status endpoint. Tag the run so you can tell attempts apart later:
Multi-turn attacks — where the pressure builds over several turns rather than landing in one prompt — are passed as a conversation instead of a single request/response pair. The result identifies which turn broke, not just that the conversation failed somewhere. See Multi-Turn Conversations.
4. Make it automatic
A red team exercise that runs once tells you about one version of your system. Wire the same judge into CI so it runs on every change, and fail the build when anything crosses your threshold:
See Unit Testing in CI/CD for the full pipeline setup.
For systems already in production, trace evaluation filters apply the same judge to live traffic at a sampling rate you choose — so genuinely novel attacks that nobody thought to put in the dataset still get scored.
5. Read the results as themes, not incidents
Individual failures matter, but the actionable signal is the pattern. The Issues view groups failures across all your runs into named, recurring themes ranked by frequency, and tracks whether each is growing or shrinking. That is what tells you whether last month's mitigation actually worked, rather than whether one specific prompt is now handled.
Last updated