Versalist uses the term training for instruction and skill changes. It does not mean model weight updates.
Evaluation sequence
- Define the challenge. Record inputs, constraints, required artifacts, and acceptance criteria.
- Run an Episode. Execute the skill and model. Record outputs and evaluation results.
- Judge the result. Apply tests, rubric criteria, baseline comparisons, or reviewer decisions.
- Record the reward. Store the score, result status, failure mode, and supporting evidence.
- Update the skill. Change an instruction only when the evidence supports the change.
Artifacts
Challenge
A challenge defines a repeatable task environment. Its fixed boundary makes different runs comparable.
Episode
An Episode records one skill and model attempt. It contains outputs and evaluation results. It can contain bounded trace metadata when capture is enabled.
Evaluation
An evaluation applies a defined method to an Episode result. Use deterministic tests when the task permits them.
Reward
A reward converts the evaluation into a structured result. It can contain a score, pass state, failure category, and evidence links.
Skill version
A skill version contains reusable agent instructions. Its provenance must identify the evidence that caused the change.
Compare runs
- Keep the challenge definition unchanged.
- Keep the evaluator unchanged.
- Change one agent variable.
- Run the candidate.
- Compare scores, failures, artifacts, and available bounded trace metadata.
- Accept or reject the change.
A score change can hide a regression. Review the result and available execution metadata.
Provider roles
Provider keys
Provider keys let Versalist send supported model calls through a user-managed account. Manage these keys in Integrations.
Custom model endpoints
This capability is planned. The current product does not expose a general endpoint adapter.
Compute adapters
This capability is planned. A catalog entry does not mean that a runtime adapter exists.
Related guides
- Challenges explains environment selection and creation.
- Skill bundles explains skill inspection and reuse.
- CLI explains local run and comparison records.