What you get
Scored on your mission profile
Not a generic benchmark. Questions come from the work your model will actually do, graded against a published rubric.
Where the model fails
By task and by confidence. Answers that are wrong and confident come first, because they read like right ones.
Every result traced
Each score links to the question, the source material, the model’s output, the expert’s reasoning and each grading decision.
See where your model is confidently wrong before you spend an expert hour.
A model that is wrong and unsure already hedges. A model that is wrong and confident gives a contracting officer a wrong answer in the tone of a right one. Move the line and watch the queue reorder.
Sample agreement across five runs of the same question.
How a Model Evaluation Sprint runs
A 30-minute call
Tell us the domain, how the model is used and what a wrong answer would cost.
We run your model
Each question five times, so confidence is how often the runs agree.
Experts grade it
Credentialed experts score each answer against the rubric, one dimension at a time.
You get the report
Results by task and by confidence, with a provenance report you can pass to your own customer.
Three rules for how we score.
Aegis Eval follows three rules and is tested against all three. A change that breaks one doesn’t ship.
No magic number
Each dimension gets its own result. We never combine them into one trust score, because how much each one matters depends on your mission, and that’s your call.
Unknown is a valid result
Anything we didn’t test is marked NOT EVALUATED. A run that couldn’t reach a judgment never counts as a pass.
Evidence over claims
Every score links back to the question, the source material, the model’s output, the expert’s reasoning and each grading decision, plus a sha256 fingerprint of the stored record, so any later change shows.
Talk to the
Your Roster team.
Tell us what you’re working on or what you’d like to know, and a person will write back.