What Release Gates Add to InvarLock Evaluation Artifacts
InvarLock's AIware '26 paper shows how InvarLock reports can feed artifact readiness, stop-depth analysis, and host-parity gates without widening the claim beyond release governance.
InvarLock reports carry the evidence; release gates make the policy layer explicit
Highlights
- InvarLock already centers its public claim on typed artifacts:
evaluation.report.json, runtime provenance, and report verification contracts. - Our ACM AIware '26 paper uses InvarLock v0.3.10 reports and documented contracts as its artifact layer, then derives release-policy outputs from them.
- The added layer is governance: artifact readiness, evidence-depth stopping, and host-parity admissibility without claiming downstream task correctness or universal thresholds.
InvarLock's public surface is not a leaderboard. It is an evidence system for baseline-vs-subject evaluation. The report carries the comparison, policy, provenance, and guard evidence that a later verifier or reviewer can inspect.
Our paper Artifact Readiness Gates with Saturation Stop Rules and Host-Parity Admissibility for FM Release Evaluation, published in the ACM AIware '26 proceedings on July 5, 2026, asks what can sit on top of that artifact surface.
The study covers 340 runs spanning seven edit families, four model families, ten seeds, and dual-host H100/H200 execution. The technical connection is concrete: the paper treats InvarLock v0.3.10 reports and documented contracts as typed evaluation artifacts. Its release-policy layer consumes those reports; it does not change the underlying model metrics.
That pinned version matters. Since v0.3.10, InvarLock's public report surface has evolved (current release: v0.12.1), but the report role remains the same: typed reports plus adjacent provenance support bounded review instead of raw-score claims.
If you consume InvarLock reports in CI or release review, the practical takeaway is straightforward: these gates are a pattern for policy above the fields already carried by the report and adjacent runtime manifest. They help decide which artifacts are admissible, when repeated evidence has stopped moving the decision, and when host-specific results need a parity check.
What InvarLock Already Carries
An InvarLock evaluation report is more than a score line.
The public report contract carries baseline and subject identity, dataset/window evidence, primary-metric fields, validation flags, policy metadata, guard outputs, and artifact paths. Container-backed evaluations also keep runtime provenance adjacent to the report so the result can be checked later with the same boundary in view.
That is the report layer the paper builds on. Before a release policy can ask whether a candidate should promote or block, it needs a typed artifact whose identity, metric, and provenance fields are explicit enough to consume.
What Artifact Readiness Adds
Artifact readiness is the first release gate because a malformed artifact should not enter policy analysis.
The paper's readiness gate checks whether report rows can be trusted enough to enter the policy layer: report parsing, model identity, baseline identity, runtime payload, and metric finiteness. Error-level issues block analysis. Warning-level issues remain visible instead of disappearing into a summary score.
In InvarLock terms, readiness is not a new quality verdict. It is a prerequisite over the evidence object: is the report bundle structured and coherent enough for the next decision layer?
Why Stop Rules Are A Policy Layer
InvarLock reports can tell a release process what happened on a configured run. The release process still needs a rule for how much repeated evidence is enough.
The paper separates evidence depth from evidence existence. In the 340-run matrix, additional seed repetitions did not change promote/block outcomes, while edit-family breadth remained decision-informative. That does not mean seed depth is never useful. It means the useful question is conditional: under this policy and workload, which additional runs still change release decisions?
That is the value of a saturation stop rule. It ties compute budget to observed decision movement instead of treating a fixed number of repeated runs as inherently more trustworthy.
Why Host Parity Belongs Beside The Score
Host parity is the third gate because release decisions can change near strict thresholds even when score differences are small.
The paper models H100/H200 admissibility as a separate promotion condition. A candidate can look acceptable on a single host and still be blocked when the matched host pair exceeds the parity tolerance or when one host fails the promote condition.
For InvarLock-style evidence, that matters because the report is already scoped to a configured runtime and policy surface. If a release decision depends on cross-host behavior, the parity rule should be explicit, versioned, and reported as its own policy surface.
Claim Map
The narrow claim is:
- InvarLock reports make release-policy inputs inspectable
- artifact readiness keeps malformed or mismatched report rows out of stronger conclusions
- stop-depth analysis asks whether extra runs still change promote/block outcomes
- host-parity admissibility keeps boundary promotions from depending on one host's result
Those are release-governance claims. They do not replace task-specific validation, safety review, or deployment approval.
Limitations
- Run-level evidence is limited to the paper's published matrix.
- Numeric thresholds, parity cutoffs, and boundary bands are workload-specific rather than universal constants.
- The paper evaluates one toolchain; it does not empirically validate cross-runner replication or downstream task correctness.
Sources
More in Evidence Note
Continue through nearby posts in the same reading thread.