Skip to content

ADR-0007: Probe contract frozen via byte-for-byte snapshot test against localv2.md §4

Status: Accepted Date: 2026-05-11 Tags: contract · stability · drift-detection · invariant Related: production ADR-0007

Context

production ADR-0007 commits to preserving the Probe ABC from ../../../localv2.md §4 byte-for-byte from POC to service. The reversibility cost on that ADR is High — changing the contract retroactively requires rewriting every probe and every consumer.

../critique.md §6.3 flags the shared blind spot: all three lens designs say "copy localv2.md §4 verbatim and freeze," but ../../../localv2.md is itself a living document (currently v2; a v3 is plausible). None of the lens designs has a story for what happens when §4 itself updates. Best-practices proposed a snapshot test comparing against a frozen string — but that test compares against a previous version of itself, not against the source document. Drift between the spec document and the implemented ABC could occur silently.

The contract freeze needs an enforcement loop, not just a copy-paste convention.

Options considered

  • Documentation freeze only (the lens-design default). "Copy verbatim and don't edit." No CI check; relies on contributor discipline.
  • String-equality snapshot test against the implementation file. Compares the rendered Probe class signature to a stored snapshot. Catches code drift but not spec drift — if localv2.md updates and code follows along, the snapshot also follows along; nothing rings the bell.
  • Fingerprint hash of localv2.md §4's body + structural snapshot of the implementation (synth). The test stores two things: (a) a normalized fingerprint hash of localv2.md §4's body at Phase 0 close, and (b) a structural snapshot of Probe class (field names, types, defaults, MRO). When localv2.md §4 updates, the fingerprint changes and CI fails. When the code drifts, the structural snapshot changes and CI fails. Either drift triggers an explicit ADR-amendment PR using the adr-amendment.md template.

Decision

tests/unit/test_probe_contract.py snapshots two artifacts to tests/snapshots/probe_contract.v1.json:

  1. A fingerprint hash — SHA-256 of a normalized representation of localv2.md §4's body (whitespace-collapsed, no trailing newlines), generated by scripts/regen_probe_contract_snapshot.py.
  2. A structural signature of codegenie.probes.base.Probe — class attribute names with their types, defaults, decorators, MRO.

Drift in either artifact fails CI. Resolution policy: localv2.md is the source of truth; the implementation must conform. A drift between code and localv2.md is always resolved by changing code, never by editing localv2.md to match. The adr-amendment.md issue template exists in Phase 0 for exactly this purpose.

Tradeoffs

Gain Cost
production ADR-0007 goes from aspirational to tested — POC contributors and service contributors share the same enforced contract Editing localv2.md §4 becomes a two-PR workflow: doc PR + amendment PR; the friction is the point but it must be communicated
Both kinds of drift surface — spec drift and code drift — and they surface with the same CI failure shape One more snapshot artifact to maintain; one more regen script to keep working
The "change code to match doc" policy resolves the directionality question once: drift never silently propagates from code back into the spec Editing the §4 body for cosmetic reasons (typos, formatting) triggers the same CI failure as a semantic change — readers must learn to read the diff
The fingerprint normalization (whitespace-collapsed, no trailing newlines) makes the test stable against trivial doc-editor noise The normalization rules must be auditable; the regen script lives in-repo for that reason
The adr-amendment.md template is exercised in Phase 0 (when §4 is first canonicalized), so the workflow is real, not theoretical Some friction tax on every localv2.md edit; mitigated by batching edits and resolving via a single amendment

Consequences

  • src/codegenie/probes/base.py is byte-for-byte localv2.md §4. No "small improvements." No Pydantic wrapping of the contract itself (the Pydantic trust boundary is internal to the coordinator — see ADR-0010).
  • The regen script scripts/regen_probe_contract_snapshot.py ships in Phase 0. The normalization algorithm is in code, not prose — auditable.
  • The tests/snapshots/probe_contract.v1.json snapshot has version v1 in its filename. A v2 (breaking) bump would create probe_contract.v2.json and migrate; the old snapshot is preserved for historical replay.
  • ADR-amendment workflow: when localv2.md §4 is edited intentionally, the contributor (a) edits localv2.md, (b) runs the regen script, (c) updates the implementation if structural drift surfaces, (d) opens a PR using .github/ISSUE_TEMPLATE/adr-amendment.md referencing this ADR.
  • Phase 1 inherits and may not change without ADR amendment: the Probe ABC, RepoSnapshot, Task, ProbeContext, ProbeOutput dataclasses, the @register_probe decorator and Registry shape.

Reversibility

Medium. Disabling the snapshot test is a one-line change to tests/unit/test_probe_contract.py (@pytest.mark.skip) and a deliberate pyproject.toml exclusion. The cost is low mechanically and High politically — it directly undermines production ADR-0007. The fingerprint algorithm itself can be evolved by bumping to probe_contract.v2.json; that's expected over the project's lifetime.

Evidence / sources

  • ../final-design.md §2.3 (Contract freeze policy — three bullets)
  • ../final-design.md §L4 row 3 (Shared blind spot: ABC frozen against moving localv2.md)
  • ../final-design.md §11 exit criterion #10 (Probe ABC snapshot test passes)
  • ../critique.md §6.3 (Shared blind spot — localv2.md §4 is itself moving)
  • ../phase-arch-design.md §Goals (Goal #2 — contract preserved byte-for-byte)
  • ../phase-arch-design.md §Open questions deferred to implementation (Q3 — fingerprint normalization specifics)
  • production ADR-0007 — the commitment this snapshot enforces
  • ../../../localv2.md §4 — the source of truth