Review and publish evidence

Evidence publication is a selection and review process, not a dump of operational diagnostics. The goal is enough public material to assess a bounded claim while keeping keys, payment identities, credentials, and unnecessary environment detail outside Git. The collector produces a sanitized local bundle; a human review determines which artifacts support publication.

Collection follows an actually observed, authorized owned-testnet run. It does not create one, repair it, or establish that an operator-supplied status is true. Evidence interpretation explains the claim boundary; format reference defines the structured records.

Plan without creating a directory

./scripts/collect-evidence.sh --plan \
  --scenario chaos-gate --status unavailable --run-id <new-run-id>

This validates scenario routing and prints a no-network plan. Choose pass, fail, blocked, or unavailable according to observations. unavailable is for evidence or identity genuinely not observable; it is not a way to conceal a failed result. An expected negative test can correctly have release status fail.

The default output root is /tmp/resilience-gate-evidence, outside the repository. Each scenario/run-ID directory is fresh and refuses overwrite. Pass metadata must identify source, rendered revision, application digest, and exact AnalysisRun. Staging chaos passes additionally require gate-runner, signer, and loadgen digests and the exact gate Job or Pod.

Bind collection to exact observed objects

After the relevant objects finish, an approved operator can collect:

./scripts/collect-evidence.sh --collect \
  --acknowledge-owned-testnet-lab \
  --config platform_setup_scripts/config.env \
  --scenario chaos-gate --status pass --run-id <new-run-id> \
  --analysis-run <actual-analysisrun-name> --gate-job <actual-gate-job> \
  --repository-revision <actual-source-freight-sha> \
  --chart-revision <actual-rendered-environment-sha> \
  --release-image-digest <actual-app-sha256-digest> \
  --gate-runner-image-digest <actual-gate-sha256-digest> \
  --signer-image-digest <actual-signer-sha256-digest> \
  --loadgen-image-digest <actual-k6-sha256-digest>

Every placeholder must come from that run. chart_revision records the observed rendered environment branch, not the chart-source commit carried in Freight. Keep both identities in the narrative when they differ. Do not infer application source from a platform checkout, or treat a mutable tag as the runtime identity.

The collector checks the configured exact context and reads narrow non-secret resources for the fixed scenario. It does not inspect Secret values, shell environments, or raw configuration and does not start, patch, or delete Kubernetes resources. For chaos passes it reads all three scorecards from the gate Pod; if a completed Pod no longer serves exec, it can recover emitted cards from the sanitized gate log. Missing cards block a complete pass collection.

--include /absolute/path/to/extract adds a small local extract after sanitization. Only regular files are accepted. An output root inside the repository needs the explicit --allow-repository-output flag, but normal practice remains collection outside Git followed by deliberate selection.

Review beyond schema validity

Use evidence_utils.py to validate metadata and sanitize supported artifacts. The sanitizer replaces risky fields and records redaction categories. Schema validation proves shape, allowed status, and scenario/namespace alignment; it does not prove execution, truth, complete public availability, or safe publication of every free-text string.

Review each selected artifact for credentials, Kubernetes Secret data, wallet keys, payment signatures and headers, authorization headers, cookies, transaction identifiers, and copied configuration. Also review encoded data, symlinks, unnecessary infrastructure details, and misleading captions. Retain only the relevant minimum. A tool’s redaction pass is not authorization to publish raw diagnostics.

For a passing gate, join all scorecards to the same release/run identity, compare the completed AnalysisRun and Job status, and verify cleanup. Preserve a failed predecessor alongside a new recovery record. Public metadata may index the full private collection; clearly state when only a selected subset is published instead of presenting every indexed file as downloadable.

Add a screenshot only with provenance

The reviewed gallery uses manifest.json, which records each file’s SHA-256 hash, capture timestamp, surface, evidence role, scenario, source/runtime identity, observed window, visible proof, and review status. Caption the observed window separately from capture time. A screenshot from October 2 cannot illustrate a claimed October 3 score unless its historical role is explicit.

Recheck hashes and run the maintained presentation validator after changing public assets:

python3 scripts/validate-docs.py

This website consumes the reviewed public manifest and source artifacts. It never reads the private archive or local operator config during build. The October 3 case honestly presents a public reviewed summary and screenshots while keeping its private raw scorecards unavailable to public readers.

Collection guards and log recovery are tested in test_evidence_tools.py. The canonical evidence policy remains the operating authority for retained bundles; new evidence must repeat its review rather than inheriting a historical scan result.