Keeping runtime evidence in custody

A release is often justified by files SBOMFlow cannot read: a trace from a bench run, a crash dump from a unit in the field, a network capture from the lab. A custody set keeps a record of each one — its name, size and SHA-256, what a person stated about it, and what a named reviewer concluded — in a directory you own, with a hash-chained audit log. You can keep one with no scan, no SBOM and no requirement regime, and join it to a release run later.

Every command on this page is offline. The custody commands are experimental. This page is the short path through one worked example; the full reference for every option and state is Keeping runtime and test evidence with a release.

What a custody record proves, and what it never proves#

A custody record proves:

• these bytes — the file had this SHA-256 and size when it was recorded, and custody verify can show it still does;
• what people stated about it — the release, the scenario, the tool — each recorded as stated, with its origin;
• kept, or found again — a local copy is kept only when you ask for one, and custody verify --originals DIR finds an original by its content, never by its name;
• reviewed by a named person — a decision binds a reviewer and a note to exactly those bytes.

It never proves:

• that a trace or dump is correct, complete or relevant to the release;
• that it shows or rules out exploitation of anything;
• what the file contains — SBOMFlow reads no event, sample or packet in it (at most the identity an open-format header declares, step 4 below), never runs the tool that wrote it, and derives no finding from it;
• that a stated tool, scenario or person is true — a statement is recorded, never checked.

Nothing in a custody set moves the release gate, sets a VEX status, enters the approval digest or maps to a regulatory requirement.

The worked example#

The files are synthetic stand-ins shipped in SBOMFlow's source under examples/runtime-evidence-custody/. Each one is labelled synthetic in its own bytes or in a sidecar file beside it. None is a recording.

Release 1.3.0 of a gateway has evidence from two tools and a lab capture:

Two of the files are called trace.bin. They are different files from different tools, and the example shows that neither can stand in for the other.

1. Start a set#

Work in a copy, so the example stays as it is:

cp -R examples/runtime-evidence-custody custody-walkthrough
cd custody-walkthrough
mkdir custody
sbomflow custody init custody/gateway-1.3.0 --set-id gateway-evidence --release-id 1.3.0

init creates the set in an empty or absent directory whose parent exists: a marker file, the index custody-set.json and the audit log audit-log.jsonl. A scan never walks a directory holding that marker.

2. Add each file with what you know about it#

sbomflow custody add custody/gateway-1.3.0 bench/trace.bin \
  --release-id 1.3.0 --scenario "boot to idle, cold start" \
  --tool-name "RTOS tracing tool" --artifact-kind runtime_trace
sbomflow custody add custody/gateway-1.3.0 field/unit-0042/trace.bin \
  --release-id 1.3.0 --scenario "field event, unit 0042" \
  --tool-name "fleet crash reporting" --artifact-kind runtime_trace \
  --capture-session field-unit-0042
sbomflow custody add custody/gateway-1.3.0 field/unit-0042/coredump.bin \
  --release-id 1.3.0 --scenario "field event, unit 0042" \
  --tool-name "fleet crash reporting" --artifact-kind core_dump \
  --capture-session field-unit-0042
sbomflow custody add custody/gateway-1.3.0 lab/uplink.pcapng \
  --release-id 1.3.0 --scenario "uplink during a firmware update" \
  --tool-name "network capture tool" --artifact-kind network_capture \
  --keep-original

• Each file gets its own record, digest and review key. The field event's trace and dump share a --capture-session, so a reviewer sees they came from one event; they are still two files with two decisions.
• --artifact-kind states what the file is. It is never read from the file name or the bytes.
• --keep-original keeps a copy of the capture inside the set. The other three originals stay where they are, and the set records only their digests.
• Adding the same file with the same statements again changes nothing.

3. Two files called trace.bin#

sbomflow custody list custody/gateway-1.3.0
sbomflow custody verify custody/gateway-1.3.0 --originals .

custody list prints one row per artifact, keyed by the first 12 characters of its SHA-256, and groups the artifacts that state a capture session:

  Digest        Name           Size       Records  Original  Decision
  99d6b97cce0b  trace.bin      209 bytes  1        not kept  none recorded
  9fc4b7e2c070  coredump.bin   120 bytes  1        not kept  none recorded
  bd0ea054dcca  trace.bin      196 bytes  1        not kept  none recorded
  d372a41b500b  uplink.pcapng  308 bytes  1        kept      none recorded

  Capture sessions (a view grouping stated sessions; never a decision target):
    Session          Artifacts    Decisions
    field-unit-0042  2 artifacts  2 not reviewed yet

Every custody screen ends with a Next: line naming the command to run next; here it is custody show for the first artifact with no decision. custody list --capture-session field-unit-0042 lists only that session's two artifacts, and its first line says so: 2 of 4 artifacts, the ones that state capture session field-unit-0042.

A set keys every artifact by its digest, never by its name, so the bench trace and the field trace are two rows with two review keys, and their records are stored under records/<sha256>.json. custody verify --originals . looks for each original by size and then SHA-256, never by file name, so each trace.bin is found as itself: renaming or moving one, or keeping both in one directory under other names, changes nothing about which row it verifies.

4. What an open-format file declares about itself#

sbomflow custody show custody/gateway-1.3.0 d372a41b500b --json

lab/uplink.pcapng is in an open format with a published specification, so SBOMFlow reads the identity its header declares and nothing past it. The row's original.file_declared_identity holds, each value with origin extracted:

Without --json, custody show prints the same block under Declared by the file, apart from what people stated (Stated by you): each value with a plain label and the file's own field name beside it, such as Capture host OS synthetic-os 1.0 [shb_os], then the build ID and version context in words.

These values describe the machine and program that wrote the capture, not the device under test, and SBOMFlow did not verify them. No packet is read. The row names the specification it read under; custody verify --json carries the same block for an original it found. A file in a format with no published specification is hashed and kept, and nothing is read from it.

Beside it, original.build_id_match reads not_established with the reason original_declares_no_build_id: a network capture declares no GNU build id, so there is nothing to compare. A perf.data that does declare one reads the reason custody_set_is_not_a_release here instead, because a custody set on its own has no release whose images a build id could be compared with; it is compared when a release run declares it (see Runtime and test evidence). An original whose format declares a version (a DFM alert's firmware version, a Memfault-format dump's software version) also carries original.declared_version_context, which reads not_comparable with the reason custody_set_is_not_a_release: a set has no release version to set the declared version beside.

5. Record a decision#

After examining the bench trace, record what you concluded about exactly those bytes, and its limits:

sbomflow custody review custody/gateway-1.3.0 \
  --accept preserved-artifact:sha256:99d6b97cce0b2096b770f8ca22d955d3583ca31688be28b90dd8506af1dec1cf \
  --reviewer lead@example.org \
  --note "bench trace examined; covers boot to idle on a cold start only"

A decision takes the full review key, a reviewer and a non-empty note. The field trace with the same file name is untouched by it.

6. Join the set to a release run#

The example's sbomflow.yaml declares the whole set, and the firmware image every row of the set is declared against:

requirement_regime: none

preserved_artifacts:
  - set: custody/gateway-1.3.0
    firmware_image: build/zephyr/zephyr.elf

sbomflow audit . --output evidence

evidence/preserved-artifacts.json has one row per artifact in the set, each naming the set it came from. The image sits under build/, which the scan prunes and never hashes, so every row reads firmware_image: not_in_scanned_tree and the run warns preserved_artifact_firmware_image_not_in_scanned_tree. That is a statement about the scan, not a finding that the image is foreign to the release.

Give the run the build's statement about what it produced:

sbomflow audit . --output evidence-provenance --build-provenance build-provenance.json
sbomflow review evidence-provenance
sbomflow custody compare custody/gateway-1.3.0 --run evidence-provenance

The image's SHA-256 is a subject of that statement, so every row now reads named_by_build_provenance. Its meaning quotes the statement's signature state: here not_applicable_plain_statement, because the statement is unsigned. It is a statement by the build system, not a file this run scanned, and it is corroborated only when a signed statement verifies against a public key you supply.

The capture the set keeps is verified by the run, and its row's run_checks.original_verification.file_declared_identity is the block custody show printed in step 4, read by the same reader. The three originals the set does not keep read not_supplied and carry no such block.

The bench trace's decision is applied in the run, because the set verifies and declares the run's release. The other three rows read unreviewed. custody compare --run names anything the set and the run record differently; here it reports no change.

7. Hand the set over#

sbomflow custody export custody/gateway-1.3.0 --output handover/gateway-1.3.0 --zip
sbomflow custody verify handover/gateway-1.3.0
sbomflow custody verify handover/gateway-1.3.0 --originals .

The export carries the index, the records, the decisions and the audit log, and never an original — not even the capture the set kept.

To hand one customer only the field event, select it:

sbomflow custody export custody/gateway-1.3.0 --output handover/unit-0042 \
  --capture-session field-unit-0042 --recipient customer-a
sbomflow custody verify handover/unit-0042

That package carries the two rows stating the session and nothing about the other two: their names, digests, records and statements are in no file of it, and its manifest says how many rows were left out, never which. custody verify reads its audit chain as partial_export, not valid, because the events about the rows left out were omitted and the chain cannot be recomputed from the package. custody compare against the set lists the rows it left out as not in the selection and not compared, never as removed. The recipient label is a statement; the selection decides what travels. A recipient who holds the files checks them by content with --originals. An unsigned export is checked only against itself; sign it with --key and --key-id when the recipient needs to know who packaged it.

8. Record a CI job's output directory#

A CI job, a test rig or a lab run usually leaves a directory behind: a test report, a trace, a dump per unit, its own log. The example has one under build/ci-output/nightly-1187/, and an intake manifest beside it that names what the job was expected to leave there:

{
  "schema_version": "sbomflow-custody-intake-v1",
  "defaults": {
    "release_id": "1.3.0",
    "scenario": "nightly hardware-in-the-loop run 1187",
    "capture_session": "ci-nightly-1187"
  },
  "entries": [
    {
      "directory": "nightly-1187",
      "names": ["test-report.xml", "boot-trace.bin"],
      "patterns": ["*.dmp"]
    }
  ]
}

Record it in a set of its own, in one command:

sbomflow custody init custody/ci-nightly-1187 --set-id ci-nightly-1187 --release-id 1.3.0
sbomflow custody add custody/ci-nightly-1187 --intake-manifest build/ci-output/intake.json

Intake of 1 manifest entry (4 files): 4 added, 0 second record, 0 unchanged
  (already in the set; nothing was written for those)

  Entry  Outcome  File, then its review key
  1      added    nightly-1187/boot-trace.bin
                  preserved-artifact:sha256:023ae70d03c5e8dccc5cd5433683be3c949bc9a25f191b6b32df5f42ec32cfcd
  1      added    nightly-1187/test-report.xml
                  preserved-artifact:sha256:e10fbd30edd26b6f148d74f4c51458e3c45f48d82b7e810f3fe7d849965d4f1a
  1      added    nightly-1187/unit-a.dmp
                  preserved-artifact:sha256:7232a103abbef328701af38f2a158a27f3579753799e39ec4b0f163acfd68b23
  1      added    nightly-1187/unit-b.dmp
                  preserved-artifact:sha256:01d4c2fc8d742ce3d55bdad5a4bca8255531a8c29cb5a20512897d4cd0d9b9d0

  Directory nightly-1187 (entry 1): 5 listed, 4 taken (2 named, pattern *.dmp
    matched 2), 1 not taken
    not taken: job.log (not named by the manifest)

Nothing is discovered. The manifest names what enters; the directory only supplies it. The job's log is in the directory and is not in the set, because the entry does not name it: it is reported as not taken and counted, never admitted. This works the same for any CI system, rig, trace tool or dump tool, because SBOMFlow reads only the list and the directory's entry names, never the files' contents.

• One directory, listed once. directory is relative to the manifest and is located like a path entry: a .., an absolute path, or a symlink on the way or at the end is refused. Only its direct entries count: it is never recursed, and a name or a pattern holding a / (or **) is refused.
• Names and patterns. names are matched exactly, so [x].bin names a file called that. patterns are shell-style (*, ?, [...]), case-sensitive, and match the direct entries' names only. The output and each admission event record every pattern with the number of entries it matched.
• What is not taken, and why. Every direct entry the entry does not take is listed with its reason and counted: not_named (a regular file nobody named), symlink (never followed), directory (never walked), not_regular_file (a device, FIFO or socket) or unreadable. Taken plus not taken always equals listed. A symlink or special file the manifest does not name never stops the intake; it is only counted.
• Refused whole. An expected name that is missing, a pattern that matches nothing, or a named or matched entry that is a symlink, a directory or a special file refuses the whole intake before any file is hashed, and nothing is written. The refusal lists every such problem across every directory entry, with their count, so one run shows everything to fix.
• Bounded, and the same every time. A directory holding more than 10 000 entries is refused rather than read in part, and all entries together take at most 10 000 files. Names are sorted, so two intakes of the same directory print and write the same bytes whatever order the filesystem lists it in.
• Statements apply to every file the entry takes. To state a different artifact_kind per file, write one entry per kind for the same directory; each entry then reports the files the others take as not taken by it.
• Large intakes stay linear, and a stopped one is completed by re-running it. An intake reads the set and its audit log once, appends one audit event per record, and writes the index once at the end, so its work grows in proportion to the files it admits and the set it admits them into. The bytes written are exactly those of adding the same files one by one. A decision recorded with custody review while an intake runs is chained after the event before it, never beside it. An intake that stops with a refusal first writes the index rows of every file it admitted; one killed outright leaves audit events the index does not list yet, which custody verify names index_behind_audit_log, and re-running the same command completes them.

Standalone, or joined to a run#

In both cases the gate, VEX, the approval digest and any requirement mapping are unaffected.

See also#

• Keeping runtime and test evidence with a release — every option, state and refusal.
• Trace and crash evidence in the release record — traces and crash dumps across two releases, and what SBOMFlow reads from each format.
• Goals — the custody goal and the other jobs SBOMFlow does.
• Use SBOMFlow without CRA assessment.
• Observed vs reviewed.