Run your first evidence-bound evaluation.

Requirements

The reference suite targets Linux with a working systemd --user manager, unprivileged user and mount namespaces, Python 3.11 or newer, Git, Go, and Node.js. An authenticated provider executable is required only for provider-backed runs.

Skivolve runs generated code. Read the security model before using proprietary fixtures, private holdouts, or valuable provider credentials.

Install from source

git clone https://github.com/Dhi13man/skivolve.git
cd skivolve
python3 -m venv .venv
. .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e ".[test]"

The runtime has one exact third-party Python dependency for RFC 8785 JSON canonicalization. Development, test, build, and fuzz dependencies are maintained in separate hash-locked requirement sets.

Validate without dispatch

skivolve --comparison original-vs-no-skill --dry-run

A dry run validates the manifest and selected cases, resolves required tools, binds source variants, checks provider protocol locks, and reports configured call and spend ceilings. Provider validation still requires the configured executable and local runtime prerequisites.

Run objective verifiers

skivolve \
  --comparison candidate-vs-original \
  --verifier-only \
  --output-dir /tmp/skivolve-verifier

This mode skips comparator judgment but still invokes the configured generation provider. It is suitable when the case-specific objective oracle is the intended decision authority. Provider-backed runs can consume metered API spend or subscription quota.

Understand the suite contract

A suite declares comparison arms, repetitions, task fixtures, bundle sources, verifier commands, execution limits, and—on current schemas—provider adapters, judgment mode, and artifact contracts. The included suite compares a frozen original bundle, the current candidate, and a no-skill control.

• Objective-only: the case verifier decides; no comparator is constructed.
• Judged: eligible ambiguity may use a selected, calibrated comparator profile.
• Artifact contract: each case selects a workspace diff, final text, or final JSON representation.
• Source authority: Git commits, paths, bytes, modes, and context inputs are fingerprinted and rechecked.

Read the authoritative suite contract, manifest schema, and case acceptance checklist.

Know what a result can claim

A public validation run is not a private holdout. A model-generated answer is not objective verification. A comparator decision is not source authority. Skivolve preserves those distinctions and fails closed when an artifact tries to borrow authority from another layer.

A release claim requires a separately stored private suite frozen before candidate evaluation, independent reviewers, a mode-0600 sealed plan, declared release comparisons, and a single consumed execution record. Judged claims additionally require fresh live comparator certification.