Keeping runtime and test evidence with a release

A firmware release is often justified by evidence SBOMFlow cannot read: a trace from a bench run, a core dump from a device in the field, a fuzzing campaign's report, a hardware-in-the-loop bench log. Each was produced by a tool you chose and examined by an engineer who drew a conclusion from it. Months later someone asks:

Which runtime and test evidence did we review for this release, what did the engineer conclude, is it still the same file, and where is the original?

This page walks the workflow that answers that question, end to end, with one worked example and two shorter ones. Every command is offline.

Three rules hold throughout:

• SBOMFlow keeps custody; it does not interpret. It records a file's name, size and SHA-256, what a person stated about it, and a named reviewer's conclusion. It never parses the file, never runs the tool that produced it, and never infers what the file shows.
• Your conclusion is the only conclusion. A preserved file is unreviewed until a named person records a decision, with a note saying what it covers and what it does not.
• None of it moves the release gate. Preserved artifacts never enter the gate, the evidence pack or the digest an approval binds to, and they are never mapped to a regulatory requirement. They are context a reviewer can find and verify, not a verdict.

The worked example: a bench trace and a field event#

The example uses file types from Percepio's tools because they show both shapes of the problem well: Percepio's public demonstration material for Detect shows an alert carrying a cc_coredump.dmp core dump and a dfm_trace.psfs trace (percepio/demos). SBOMFlow does not connect to either tool, and nothing on this page depends on what those files contain: any file from any tool follows the same path.

The files used here are the synthetic stand-ins shipped with SBOMFlow's source in examples/preserved-artifact-demo/, each labelled synthetic in its own first bytes. They are not real captures.

Two pieces of evidence belong to release 2.4.1 of a sensor node:

1. a trace an engineer captured on the bench, from boot to idle on a cold start;
2. one field event, which produced a trace and a core dump.

1. Preserve each file with its own record#

sbomflow ingest-preserved-artifact captures/boot-to-idle.psf \
  --release-id 2.4.1 --scenario "boot-to-idle, 30s, cold start" \
  --tool-name "Percepio Tracealyzer" --tool-version 4.11 \
  --as-of 2026-09-10 --operator fw-bench \
  --output preserved/boot-to-idle.preserved-artifact.json

sbomflow ingest-preserved-artifact captures/field-crash.psfs \
  --release-id 2.4.1 --scenario "field event 1, trace" \
  --tool-name "Percepio Detect" --as-of 2026-09-12 --operator field-support \
  --notes "SYNTHETIC demonstration capture. One alert carried this trace and a core dump; the core dump has its own record." \
  --output preserved/field-crash.preserved-artifact.json

sbomflow ingest-preserved-artifact captures/cc_coredump.dmp \
  --release-id 2.4.1 --scenario "field event 1, core dump" \
  --tool-name "Percepio Detect" --as-of 2026-09-12 --operator field-support \
  --notes "SYNTHETIC demonstration capture. One alert carried this core dump and a trace; the trace has its own record." \
  --output preserved/field-crash-coredump.preserved-artifact.json

Each record holds the file's digest and every value you typed, marked as a statement a person made — user_supplied — never as a measurement. Nothing is defaulted: an unstated tool version is listed as missing. .dmp is recorded as core_dump from the extension alone.

The field event gets two records on purpose. Two files have two digests, and a reviewer's conclusion about the core dump must not quietly cover a trace nobody opened.

Give every file its own --output. Sending a different file to a destination that already holds a record is refused with E019, and nothing is written, so a second capture can never silently replace the first.

2. Declare the records for the release#

In sbomflow.yaml:

preserved_artifacts:
  - record: preserved/boot-to-idle.preserved-artifact.json
    original: captures/boot-to-idle.psf
    firmware_image: firmware/zephyr.elf
  - record: preserved/field-crash.preserved-artifact.json
    original: captures/field-crash.psfs
    firmware_image: firmware/zephyr.elf
  - preserved/field-crash-coredump.preserved-artifact.json

A record is read only when it is declared; SBOMFlow never searches a directory for records. The full form names the original, so the run can re-hash it, and the firmware image the capture was stated to be taken against. The short form names the record alone — useful when the original lives in a separate device-data store rather than beside the release.

Keep the firmware image in a directory the scan hashes. build/ and dist/ are pruned and never hashed, so an image there is reported not_in_scanned_tree: a statement about the scan, not a finding that the image is foreign.

3. Run the audit#

sbomflow audit . --output evidence

evidence/preserved-artifacts.json has one row per artifact, with three checks, each saying exactly what it proves:

Every row is unreviewed. START-HERE counts the preserved artifacts awaiting a decision in their own unit, and the assessment report, the evidence bundle and the reviewer console (sbomflow review-view evidence) show the rows. The release gate is exactly what it would be with nothing declared.

Only the small custody records are copied into the run, under preserved-artifacts/. The original files are never copied into the run, the bundle, the evidence store or any sharing pack: a trace or a core dump can hold memory contents, device identifiers and source paths, and SBOMFlow does not keep it. Keeping the original is yours.

4. Examine the files, then record your conclusion#

Open each file in the tool that produced it and decide what it shows. Then:

sbomflow next-actions evidence

lists each unreviewed artifact with the exact command, and:

sbomflow review evidence \
  --accept preserved-artifact:sha256:<boot-trace-digest> \
  --reviewer fw-lead \
  --note "No deadline misses in the boot window; covers cold start only"

records it. --reject and --needs-more-evidence take the same form. For a preserved artifact, --reviewer and a non-empty --note are required for every status; without either, the decision is refused with E021 and nothing is written. Write the note as the conclusion and its limits — here, that the trace covers a cold start and nothing else.

The key is the file's digest, so the decision is bound to those bytes. A different file, even under the same name, is a different key and is unreviewed.

5. Re-run, validate and verify#

sbomflow audit . --output evidence --reviews evidence/reviews.json
sbomflow validate evidence
sbomflow verify-bundle evidence --layered --require-layers content

analyze and audit apply the reviews file they are given — --reviews, or reviews: in sbomflow.yaml. A re-run without either reads no decision at all, even though evidence/reviews.json exists, and every row stays unreviewed; nothing was lost, the run was simply not given the file. The boot trace's row now reads accepted, applied: true, with the reviewer and the note beside it. validate checks the index against its copies and the release record, and the bundle's content layer re-hashes every custody copy it lists.

If a supplied original later stops matching its record — one changed byte is enough — the row reads mismatch and the decision is shown but not applied, with the reason. A conclusion about other bytes never carries over silently. A decision whose artifact this run no longer declares is listed under undeclared_decisions and never applied, and a decision from a previous release is never carried into this one.

6. Compare releases, and store the record#

sbomflow compare-releases --previous <earlier-evidence> \
  --current evidence --output drift

drift/preserved-artifact-changes.json lists the artifacts added, removed, re-decided or re-checked since the previous release. Compared with an output written before this record existed, it says not compared, with the reason — never "no change".

sbomflow store add evidence keeps the release, custody copies included, and sbomflow store verify re-hashes them.

Two shorter examples#

The workflow does not change with the tool. Only the stated values do.

A fuzzing campaign's report. Preserve the report the fuzzing tool wrote, with the release, the campaign and the tool you ran, and declare it with its original. A reviewer's note might read "no new crashes in the parser target over the stated run; the network stack was not in scope". SBOMFlow records that a person concluded it, about those bytes. It does not read the report, count crashes or judge the campaign.

A hardware-in-the-loop bench log. Preserve the log file a bench run produced, with the release, the rig and the scenario, and declare it with the firmware image the rig was flashed with. in_release then says that image is a file of this release; whether the bench actually ran that image is what your note, and the bench's own records, have to establish.

What this is not#

• Not analysis. No trace, dump, report or log is parsed. The format hint comes from the file extension.
• Not a test result or coverage. A preserved file existing says nothing about what was tested, what was covered or whether anything passed.
• Not a security or conformity statement. Nothing here says a product is secure or meets any regulation, and SBOMFlow never makes a conformity claim.
• Not a connection to any tool. SBOMFlow never runs, drives, authenticates to or downloads from the tools that produced the files.
• Not a gate. No preserved artifact, missing decision or mismatch changes the release gate.

See also#

• The clean-machine installed journey — this workflow walked from an installed wheel, step by step, with every step asserted.
• Reviewer runbook — recording decisions.
• Evidence outputs — preserved-artifacts.json and the other artifacts.
• Observed vs reviewed — why the review boundary matters.