Trying Individual Validators

You select which validators to use (maybe Toxicity, Hallucination, Relevance, and Bias). You provide a dataset of test cases with prompts.

DisseqtAI sends each prompt to your AI and collects the response. Then it runs your selected validators on each response. Each validator returns a score (usually 0 to 1 or 0 to 100) indicating performance on that metric.

Click "Test Input" on any validator card to try it immediately. A modal opens where you can paste sample text, click "Run Validator," and see the score.

Reading Validator Results

After running an evaluation, you'll see results for each validator:

Average score across all test cases (maybe Toxicity average is 0.03, Relevance average is 0.82).

Score distribution showing how many responses fell into different ranges (80% scored above 0.7 on Relevance, 15% scored 0.4 to 0.7, 5% below 0.4).

Pass/fail rate based on thresholds you set (if you defined "passing" as Toxicity under 0.3, maybe 95% passed).

Individual scores for each test case so you can drill into specific failures.

Trends over time if you've run this test before, showing whether metrics are improving or degrading.

Setting Thresholds

Most validators return scores between 0 and 1. For Toxicity, lower is better (0 is clean, 1 is extremely toxic). For Relevance, higher is better (0 is off-topic, 1 is perfectly relevant).

Setting thresholds determines what counts as passing or failing. If you set Toxicity threshold at 0.3, anything 0.3 or above fails. If you set Relevance threshold at 0.6, anything 0.6 or below fails.