Know who wrote
every answer
Other expert-data vendors say “passed our assessment.” Every Your Roster answer comes with the credentials of the expert who wrote it, and every expert is screened for conflicts before the first task.
- Credentials on record for every expert
- A standing cell of five to twenty, built to your rubric
- Scored answers traced to the expert who wrote them
A small, trusted pool
of operators
Procurement judgment, policy tradeoffs, cleared-program decisions. The reasoning that makes government work defensible lives in the people who did it, not in a public dataset.
- Each expert’s credentials reviewed
- Qualified problem by problem
- Conflict-screened before the first task
Domains
Federal, state and local government. Each expert credentialed and qualified problem by problem.
Federal Acquisition & Contracting
Contracting and procurement officers who wrote the justifications, ran the source selections and lived with the protests.
Defense Programs
Program managers from DoD and the services who carried cost, schedule and requirements through milestone decisions.
Federal Policy & Regulation
Policy analysts and congressional staff who know how a rule is drafted, scored and challenged.
Cleared IT & Cyber
IT specialists who ran authorizations, accreditation and cleared-program systems.
State & Local Procurement
Procurement officers, purchasing agents and finance directors at states, counties and cities who ran competitive bids, cooperative contracts and grant-funded purchases.
State & Local Policy & Regulation
Legislative analysts, agency rule writers and city and county attorneys who know how a statute becomes a rule and an ordinance becomes practice.
Public Safety & Emergency Management
Emergency managers, public safety leaders and grant administrators who ran response plans, mutual aid agreements and disaster recovery funding.
Health & Human Services
Medicaid, public health and social services staff who ran eligibility, program integrity and state plan rules.
Start with
a call
Experts grade your model’s answers and write out the reasoning it’s missing. Every grade and every trace becomes training data for reinforcement learning.
Start with a CallFour steps to a cell or a dataset
Design partners help shape the rubric and the makeup of their cell while both are still open.
- 01A 30-Minute CallTell us the domain, the kind of work and the rough scale.
- 02Experts Grade Your Model’s AnswersCredentialed experts grade each answer your model gives against the rubric, one dimension at a time. Each grade becomes a reward signal your model can learn from.
- 03Experts Write the ReasoningWhere an answer falls short, the expert writes out the right steps, each with the rule it rests on. That reasoning trace shows your model what good work looks like.
- 04A Standing Cell or a Sealed DatasetGrades and reasoning traces arrive as one sealed package, ready for your reinforcement learning pipeline. Every record is hashed and comes only from experts qualified for the problem.
No Magic Number
Each dimension gets its own result. We never combine them into one trust score, because how much each one matters depends on your mission, and that’s your call.
Unknown Is a Valid Result
Anything we didn’t test is marked Not Evaluated. A run that couldn’t reach a judgment never counts as a pass.
Evidence Over Claims
Every score links back to the question, the source material, the model’s output, the expert’s reasoning and each grading decision, plus a sha256 fingerprint of the stored record, so any later change shows.
Three rules for how we score. Aegis Eval, the scoring engine underneath Your Roster, is tested against all three, and a change that breaks one doesn’t ship.
Tell us where your
model is used.
We run it first and show you where it breaks.