Proofs: AI work checked and signed

A model can draft a detection rule faster than anybody can check it. A proof is the check, made by a program rather than by the author, and written down so whoever receives the rule — a reviewer, CI, a customer — can rely on it without trusting whoever handed it over.

Three verdicts#

Verdict	Means
proven	every check passed on evidence that could have failed it
refuted	a check failed, and the proof names what
inconclusive	nothing failed, but nothing could have: a rule checked only against its own fixtures

A refutation is signed like a proof: what did not work is what the next attempt most needs to know.

Prove a detection rule#

aertun content keygen --out root.key     # once: a signing key in the store; move root.key off the machine
aertun proof detection --rule rule.json --corpus corpus.jsonl \
    --key .aertun/content/release.key --public-key proof-key.json \
    --out rule.proof.json --by you@example.com

It checks, in order:

1. valid — the rule is a rule; an invalid one is refuted and nothing else runs.
2. fixtures — its own must-match and must-not-match examples hold. Necessary, never enough.
3. corpus — over a labelled corpus (one JSON object a line, as aertun detect replay reads), it fires on every entry labelled as its own and on nothing else it reads. A false positive somebody recorded and nobody fixed still refutes it.

Recall and specificity are reported as 95% Wilson lower bounds, so a proof over one example reads as the weak thing it is. Without a corpus the best verdict is inconclusive. --model and --prompt-sha256 record which model wrote the rule and a digest of its prompt — never the prompt.

Verify one#

aertun proof verify --key proof-key.json --subject rule.json rule.proof.json

Both signatures must hold against a key you were handed, never one the document supplies; --subject checks the proof is about exactly that file. It exits non-zero on anything but proven, so it is a CI gate.

What a proof is#

An in-toto Statement in a DSSE envelope: the subject is the rule file by digest; the predicate (https://aertun.github.io/proof/detection/v1) names the claim in words, the verifier and its version, every material it read by digest, each check, and the verdict. It is signed twice, Ed25519 and ML-DSA-65, the second in the context aertun/proof/v1. The Ed25519 half is plain DSSE, so in-toto and Sigstore tooling check it with no help from Aertun. The signing key file must be readable by its owner alone.