Observed vs reviewed

The most important boundary in SBOMFlow: machine observation is never human review.

What the engine observes#

The deterministic engine can observe facts: a component exists, a file is present, an advisory matches a version, a requirement has no attached evidence. These are observations. An observed status is not a judgement that anything is acceptable.

Each advisory finding also states how it was matched, so you can decide what to verify first. A purl-exact match (the advisory was resolved for the exact package URL) is stronger evidence than a name-version-heuristic match (the component's name and version strings matched, but no package coordinate took part). An osv-ecosystem-name match is as strong as purl-exact — the OSV server still evaluated the exact version — but the coordinate was an ecosystem + package name rather than a full package URL, because OSV serves that ecosystem (Bioconductor is the first) without a package-URL type; the label says so instead of claiming a purl match the query never made. A purl-version-range match carries the range engine's own verdict, which drops to low when a version could not be ordered or a distribution rebuild was compared against an upstream range.

This is context for prioritising your review, not a decision. A low-confidence finding is never dropped, never downgraded, and never suppressed — it is exactly the finding most worth a human look. Nothing about the match basis changes the release gate or the VEX status.

Why audit and review print different numbers#

They count different things, and each says which.

analyze and audit count observed rows — one line per observation the scan recorded, in evidence-pack.json. review counts distinct evidence decisions — one line per decision key a reviewer must rule on, in review-queue.json. Several observations of the same requirement, from different sources, share one decision key, so a single ruling settles all of them. That is why the review queue's number is usually the smaller one.

Neither number is a verdict, and neither contradicts the other. evidence gaps (requirements with no evidence at all), requirements missing evidence and artifacts are three further units again — a count is only comparable with another count in the same unit.

The review row states what is missing first#

When a run recorded requirements with no evidence at all, the review row opens with that count:

  review    7 requirements missing evidence (cra-coverage.json) · 6 observed
            rows (evidence-pack.json) await human review, carrying 3 distinct
            evidence decisions (review-queue.json) — the number `sbomflow
            review` shows

The row is one wrapped paragraph, and a cropped view — a CI log excerpt, a chat paste, a terminal narrower than the row — keeps only its first line. Stating the absence first means that line cannot report how much evidence is waiting while leaving out the requirements that have none. A run that recorded no such absence keeps the original order, because leading with a zero would put the reassuring number where the qualifier belongs. Order is the only thing that changes: no count, artifact, exit code or decision moves with it.

The gate verdict stays on the first screenful#

Rows that qualify the run — where a source was not consulted, which waivers were refused — print below the gate row. The reason is arithmetic: a run that declares no product name or version already puts gate on the last line the reader's screen holds, so any row added above the verdict costs them the verdict, however short it is. The rule is about placement, never length — nothing is shortened to fit, since losing the caveat and the verdict is the worse trade.

One of those rows is preserved. When the run's sbomflow.yaml declares preserved artifacts (preserved_artifacts:) and any of them has no applied review decision, the summary counts them and names the command that lists them:

  preserved 3 preserved artifacts await a review decision
            (preserved-artifacts.json) — `sbomflow review out` lists them

A run that declared none, or whose declared artifacts all carry an applied decision, prints no preserved row.

What only a human can decide#

• Whether a vulnerability is not_affected or fixed (a VEX determination).
• Whether an evidence item is accepted.
• Whether a release may proceed.

These come only from review files and approvals, never from the engine. A not_affected claim without a valid justification is downgraded.

The justification vocabulary is CISA's, and it is closed: a label from another schema (CycloneDX's code_not_reachable, for instance) or a sentence of English is not one of the five, so the claim is downgraded to under_investigation and the finding stays visible. That downgrade is not silent. The run records vex_not_affected_invalid_justification in scan-warnings.json, naming the review and what it recorded; the review queue carries the reason on the row; and next-actions counts such findings apart from the ones nobody ruled on, because the fix is to correct a decision that already exists rather than to make a second one. The reason is written on every surface that shows the finding — explain --finding, explain --gate, the review-view console and assessment-report.html — all quoting one sentence, so a refused decision never reads like a finding nobody opened. Its sibling vex_suppressing_review_unattributed covers the other half of the same boundary: a suppressing decision with no attributed reviewer.

Build-applicability evidence is the same kind of context. With build_applicability declared, a kernel finding can say not_observed_in_supplied_build: no file its CVE record names was among the sources the supplied build compiled. That is a fact about the build that was read, not a VEX status. The engine never turns it into not_affected; a reviewer may cite it, by digest, in a not_affected + vulnerable_code_not_present decision.

Your own analysis of a finding can be attached to the run, hashed and carried with the evidence, without crossing that line: see Supplying your own reachability evidence for the file shape. It is recorded as unreviewed context — a decision still comes only from a review file.

AI is assistance, never authority#

AI, where used at all, is optional assistance at the edges. It is never the source of evidence, a VEX decision, or a release decision.

Two reviewers, one decision file#

Reviewer decisions live in flat, key-addressed JSON (reviews.json, vulnerability_reviews.json, waivers.json). SBOMFlow refuses a concurrent write that would silently clobber someone else's decision — so when two people (or a laptop copy and the CI copy) both recorded decisions, combine them with a domain-aware merge rather than a line-based git merge, which can interleave one reviewer's status with another's justification:

# 1. Merge by decision key. Non-overlapping keys are unioned in and identical
#    decisions deduplicate.
sbomflow review-merge --ours alice/vulnerability_reviews.json --theirs bob/vulnerability_reviews.json --output merged/vulnerability_reviews.json

# 2. If both reviewers decided the SAME key DIFFERENTLY, nothing is guessed: the
#    key is omitted from the merged file, written to merge-conflicts.json, and the
#    command exits 1. Read the side-by-side report:
cat merged/merge-conflicts.json

# 3. A human chooses each conflicting key explicitly, then re-runs.
sbomflow review-merge --ours alice/vulnerability_reviews.json --theirs bob/vulnerability_reviews.json --output merged/vulnerability_reviews.json --resolve <finding-id>=theirs

Pass --base (the file both reviewers started from) for a three-way merge: a side that changed relative to the base wins over a side that did not, including deletions, so only a genuinely divergent decision is a conflict. Any parse failure, or an attempt to merge decisions recorded against different products, aborts the whole merge — there are no partial merges. The merged file records both inputs and their SHA-256 so the merge itself is auditable.

Claiming an item before you decide#

Each evidence item, finding, or gap in the review queue moves through a small lifecycle: open (untouched) → in_review (someone is looking at it) → decided (a decision is recorded — accepted/rejected/needs_more_evidence for evidence, a VEX status for a finding). decided is reached only by recording an actual decision, exactly as before; the state itself carries zero evidence authority.

The open → in_review step is new: claim an item so a second reviewer sees it is already being looked at, instead of silently duplicating the work.

sbomflow review ./evidence --start secure_boot_configured --as security --reviewer sec@example.com

Claiming an item that is already claimed is refused unless you pass --force (which reclaims it and records the prior claimant alongside the new one — both claim events stay in the append-only audit log). Claiming an item that already has a recorded decision is also refused: decisions are immutable for the release, so a claim can never move one back to open/in_review — the next release starts fresh. The review queue (and the reviewer console) show the claimed role next to an in-review item.

The reviewer console (sbomflow review-view) opens with a Reviewer workload table: one row per role that has claimed or decided something (plus an "unassigned" row for everything nobody has touched), with open/in-review/decided counts and how long the oldest in-review item has sat claimed. Rows are roles, never reviewer names — this is a workload snapshot, not per-person analytics.

Every review-view render re-reads reviews.json, vulnerability_reviews.json, and review-claims.json from the output directory at render time — it never shows a decision or claim as pending after sbomflow review has already recorded it, even if analyze has not been re-run since.

Bulk triage, one row at a time#

A 200-finding queue needs multi-select convenience, but a blanket "accept all" would destroy the point of per-finding review. review --bulk gives you the speed without the blanket authority: it consumes a pre-reviewed .csv or .json worksheet — one row per finding, each with its own status and justification — and records every row as its own independent decision with its own audit event, exactly as if you had run --triage that many times.

A worksheet is a plain array of rows:

[{"finding_key": "CVE-2024-…", "status": "fixed", "reviewer": "you@example.com"}]

Then run sbomflow review ./evidence --bulk worksheet.json --reviewed-at <timestamp>.

finding_key is the vulnerability id — the same value --triage takes. Columns/keys: finding_key, status, justification, reviewer, note, evidence_ref (a per-row reviewer/note overrides the --reviewer/CLI default; justification is required for not_affected exactly as --triage requires it). A row targeting an unknown id, or missing finding_key/status, is listed and skipped — valid rows still apply, and the exit code reflects whether anything failed. There is no "accept all": every row states its own status, and the justification-quality lint (copy-pasted rationale reused across findings) still fires on the result exactly as it would for individually-recorded decisions.

Why this matters#

A security or audit buyer trusts a tool that is honest about what it knows. Keeping observation and review distinct is both an ethics rule and the reason the evidence is credible.