Skip to content

S6-01 — E2E kill/resume closeout

Status: HARDENED Validated: 2026-05-26 — see _validation/S6-01-e2e-kill-resume-closeout.md. Depends on: S1-01-sut-contract-types.md — VulnRemediationCase, VulnRemediationResult, SutDigest, VulnRemediationSut, _FROZEN_FORBID; S1-02-ledger-state-union.md — VulnLedgerState seven-variant closed sum type, FailedUnrecoverableReason, HumanReviewReason; S2-01-semantic-checkpoints.md — CheckpointStore Protocol + the SQLite/in-memory adapters; S2-02-replay-verification.md — hydrate_or_fail is the SOLE integrity site (tampered-checkpoint Scenario 4 fails closed here, not in the adapter); S3-01-plugin-local-subgraph.md — build_subgraph(deps) -> StateGraph[SubgraphState] (uncompiled); S3-02-transition-table-tests.md — every conditional edge already exercised at the unit level; S4-01-hitl-interrupt-and-resume.md — HitlInterrupt, ResumeInput, resume_or_reject, stale-resume-token rejection; S5-01-stable-sut-adapter.md — LocalVulnRemediationSut, SutConfig, _project_terminal_state_to_result, from_plugin, sole-importer fence (this story is the consumer of that adapter; it does not re-import the private subgraph builder).

Goal: Land Phase 6's end-to-end closeout package — five integration fixtures + one workflow-scope determinism property + one Phase-6.5-consumer isolation simulation + one docs/contract closeout sweep — proving each row of final-design.md §"Exit criteria mapping" and each scenario of phase-arch-design.md §"Scenarios" is enforced by a runtime test, not a doc claim. The four canonical scenarios (Clean completion / Retry-then-recovery / Kill-mid-run-then-resume / HITL-interrupt-then-resume) each ship as a pytest-asyncio integration test under tests/integration/workflows/ that drives the LocalVulnRemediationSut adapter S5-01 ships through the plugin's build_local_sut(config) factory against the fixture cohort node_typescript_helm + node_yarn_berry_pnp + node_pnpm_native named verbatim by docs/roadmap.md §"Phase 6". The fifth integration test covers phase-arch-design.md §"Scenarios" row 4 — tampered checkpoint — and asserts terminal_state="failed_unrecoverable" with failure_modes=("phase6.failed_checkpoint_integrity", ...) after directly mutating one row of checkpoints.sqlite between two run_case calls on the same workflow_id. The workflow-scope replay-determinism property at tests/property/test_workflow_replay_determinism.py is a Hypothesis property drawing (repo_snapshot_sha, cassette_id, embedding_model_digest) triples and asserting ≥ 50 independent run_case(case) calls produce byte-identical VulnRemediationResult payloads (under model_dump_json(by_alias=True)) modulo a closed allowlist of timestamp + workflow_id fields. The Phase-6.5-consumer isolation simulation lives at tests/integration/test_phase6_consumer_isolation.py — it constructs a synthetic in-process Phase-6.5 consumer that imports only the four ADR-0001 names (VulnRemediationSut, VulnRemediationCase, VulnRemediationResult, SutDigest) plus the plugin's build_local_sut factory, drives one clean-completion case end-to-end, and asserts an AST walk over the synthetic consumer's source confirms it imports zero names from codegenie.workflows.vuln_ledger, codegenie.workflows.local_sut, codegenie.workflows._chain, codegenie.workflows._replay, codegenie.workflows._frozen, codegenie.workflows.checkpoints, codegenie.workflows.sqlite_checkpoints, codegenie.workflows.in_memory_checkpoints, codegenie.workflows.replay, codegenie.workflows.errors, or any plugins.vulnerability_remediation__node__npm.subgraph.* module — the structural enforcement of final-design.md §"Relationship to Phase 6.5"'s "may NOT depend on" four-bullet list. The cross-cutting tests/e2e/scenarios.yaml addition extends the Phase-3 harness (precedent: tests/fixtures/adversarial/*/.codegenie/scenarios.yaml) with three new rows — one per fixture in the Phase-6 cohort — each asserting terminal_state reached + replay-byte-equality across two independent runs. The docs/contract closeout (i) sweeps every cross-doc reference to build_vuln_loop / vuln_loop / build_subgraph and pins each to the ADR-0001 four-name surface or the plugin's build_local_sut, (ii) extends tests/unit/test_phase6_docs.py with assertions that the roadmap row + the mkdocs nav entry + every per-phase README link resolve to docs/phases/06-sherpa-vuln-loop/, (iii) confirms the contract snapshot tests/golden/phase6-contract/snapshot.json is byte-equal to its S5-01 post-state (no further additive amendments), (iv) appends a ## Phase-6 closeout (2026-05-26) entry to docs/phases/06-sherpa-vuln-loop/stories/_attempts/_lessons.md capturing the three load-bearing lessons (per-mode dispatch, sanitization-aware fold, sole-importer fence), and (v) flips docs/phases/06-sherpa-vuln-loop/stories/README.md's Definition-of-done bullets from prose to - [x] checkbox state.

The story is the final consumer of S1-01..S5-01's contracts; it does NOT introduce a new public name in codegenie.workflows.__all__ (still byte-equal at the post-S1-02 value), does NOT widen ALLOWED_BINARIES, does NOT amend any ADR, and does NOT touch the subgraph builder. Every load-bearing assertion is a runtime test or a fence walk; every fence walk follows the AST-driven precedent set by S3-01's no-peer-call fence and S5-01's sole-importer fence; every integration test is driven via the Phase-6.5-shaped contract (the await asyncio.wait_for(sut.run_case(case), timeout=...) envelope) so that the exact code path Phase-6.5 will later drive is the code path Phase-6 closes out on.

References

  • final-design.md §"Exit criteria mapping" all four rows (each row maps to one AC in this story); §"Decisions of record" item 3 ("Checkpoint on semantic boundaries … Resume verifies the prior chain head before replay" — drives AC-3 kill/resume + AC-5 tampered-checkpoint) + item 5 ("Typed interruption … 'Paused' is not a boolean side channel" — drives AC-4 HITL-interrupt-resume); §"Main workflow" steps 5–7 verbatim (validation + routing + return — the e2e tests drive all three transitively); §"Relationship to Phase 6.5" verbatim "may NOT depend on: the concrete graph builder; node names; checkpoint backend internals; plugin-local file layout" (drives AC-7 four-bullet AST walk over the synthetic consumer); §"Non-goals" item 1 ("No second task class") — drives Anti-refactor #1 (no generalized e2e harness across task classes — S6-01 is one task class closeout).
  • phase-arch-design.md §"Scenarios" #1 verbatim ("Recipe applies, gate passes, ledger records Completed, VulnRemediationResult.terminal_state == 'completed'" — drives AC-1); §"Scenarios" #2 verbatim ("Gate fails with retryable evidence, planner re-enters with prior-attempt context, second patch passes, chain shows two gate attempts" — drives AC-2); §"Scenarios" #3 verbatim ("Gate fails twice, graph emits AwaitingHumanReview, process exits cleanly, resume input is validated, approved transition continues from the latest verified checkpoint" — drives AC-4); §"Scenarios" #4 verbatim ("Replay verification fails before hydration, graph returns FailedUnrecoverable(reason='checkpoint_integrity'), no patch work resumes" — drives AC-5); §"Failure modes" row 1 (checkpoint chain mismatch — AC-5) + row 4 (stale human resume token — AC-4 sub-assertion) + row 5 (planner/gate exception — AC-2 retry path under exception-trapped node failure); §"Testing strategy" "Integration tests: kill/resume, retry recovery, HITL interrupt/resume" (drives AC-1..AC-4 module placement under tests/integration/workflows/); §"Cross-cutting test-architecture additions" verbatim (drives AC-8 scenarios.yaml rows + AC-6 replay-determinism property).
  • ADRs/0001-stable-vuln-remediation-sut-contract.md §Consequences ("Phase 6.5 imports the contract only … Contract changes require ADR amendment and downstream review" — drives AC-7 four-bullet consumer-isolation AST walk + AC-9 contract-snapshot byte-equality assertion).
  • ADRs/0002-plugin-local-subgraph-topology.md — drives AC-7 sub-assertion: synthetic consumer must NOT import any plugins.vulnerability_remediation__node__npm.subgraph.* module.
  • ADRs/0003-checkpointed-ledger-replay-boundary.md — drives AC-3 (kill mid-run; resume verifies prior chain head before hydration; same final state) and AC-5 (tampered checkpoint fails closed before hydration).
  • High-level-impl.md §"Step 6 — End-to-end verification" all six bullets (clean-completion + retry-recover + kill-resume + HITL-interrupt-resume + docs-contract-snapshot-closeout + cross-cutting test-architecture additions — every bullet is one AC).
  • stories/README.md §"Definition of done" all five bullets (drives AC-11 closeout checklist; the story's last edit flips each bullet from prose to - [x]).
  • docs/roadmap.md §"Phase 6" — "fixture cohort node_typescript_helm + node_yarn_berry_pnp + node_pnpm_native" verbatim (drives AC-1..AC-4 fixture cohort + AC-8 scenarios.yaml row count); "for any (repo_snapshot, cassette_id, embedding_model_digest) triple, the pipeline produces byte-identical outputs across N independent runs" (drives AC-6 property triple + N≥50).
  • S5-01 (HARDENED) — LocalVulnRemediationSut, build_local_sut(config) factory, the _project_terminal_state_to_result projection table, the sole-importer fence (this story is local_sut.py's second consumer; the existing sole-importer fence at tests/fence/test_subgraph_builder_sole_importer.py continues to assert exactly one importer — the e2e tests import the factory from the plugin's api.py, not the builder itself).
  • S4-01 (HARDENED) — HitlInterrupt, ResumeInput, resume_or_reject; drives AC-4 stale-resume-token + AC-4 approved-resume continuation from the latest verified checkpoint.
  • S2-02 (HARDENED) — hydrate_or_fail is the SOLE integrity site; drives AC-5 (the tampered-checkpoint test mutates a row directly via SQLite, calls run_case(case_with_workflow_id=...) in replay execution mode, asserts the ReplayVerdict.ChainMismatch is what causes the FailedUnrecoverable, NOT a re-implementation in LocalVulnRemediationSut).
  • Phase-4 S6-07 (HARDENED) — Determinism cassette-replay property — the precedent this story extends from FallbackTier-scope to workflow-scope; AC-6 mirrors S6-07's N≥50 + byte-identical-modulo-allowlist structure.
  • Phase-3 tests/fixtures/adversarial/*/.codegenie/scenarios.yaml — the schema this story extends additively at tests/e2e/scenarios.yaml; AC-8 mirrors the scenarios: [- name: ..., command: [...]] shape.
  • tests/unit/test_phase6_docs.py (existing) — the test this story extends additively with AC-10 docs-link resolution assertions.
  • tests/integration/test_phase6_sut_contract_snapshot.py (existing — extended by S5-01) — the snapshot this story asserts is byte-equal to its S5-01 post-state (AC-9 closeout — no further additive amendments in this story).
  • tests/fence/test_phase6_no_graph_imports_from_phase65.py (existing — placeholder, pytest.skip until Phase 6.5 ships) — AC-7 ships the complementary test_phase6_consumer_isolation.py so the Phase-6.5-may-not-import closure is enforced from this side without waiting for Phase 6.5 to land.
  • CLAUDE.md §"Functional core / imperative shell" — drives AC-6 (the determinism property's assertion-of-byte-equality is pure; the test harness around it is the imperative shell); §"Honest confidence" + §"Facts, not judgments" — drives AC-1..AC-5 (every integration test asserts facts — terminal_state byte-value, failure_modes tuple, gate_summary counts — never qualitative "the workflow worked"); §"Extension by addition" — drives Anti-refactor #1 (no generalized e2e harness across task classes; the second task class earns its own closeout story).

Acceptance criteria

Scenario 1 — Clean completion (recipe applies, gate passes, terminal completed)

  • [ ] AC-1 — tests/integration/workflows/test_phase6_clean_completion.py drives await sut.run_case(case) against a fixture from the cohort {node_typescript_helm, node_yarn_berry_pnp, node_pnpm_native} (parametrized — one test row per fixture; three rows total) and asserts terminal_state="completed" AND patch_digest is not None AND gate_summary.all_passed is True AND failure_modes == () AND the checkpoints.sqlite chain contains exactly the expected semantic-boundary checkpoints (PlanReady, PatchApplied, Completed — final-design.md §"Decisions of record" item 3). The test uses @pytest.mark.parametrize("fixture_name", ["node_typescript_helm", "node_yarn_berry_pnp", "node_pnpm_native"]). The SUT is constructed via build_local_sut(SutConfig(plugin_id=..., cassette_id=..., ...)) — the test does NOT import any name from plugins.vulnerability_remediation__node__npm.subgraph.* (a meta-assertion at the bottom of the file assert "subgraph" not in <source_of_this_module> enforces). Mutation thinking: a stub that returns a hardcoded VulnRemediationResult(terminal_state="completed", patch_digest=BlobDigest("blake3:" + "0"*64), ...) would pass terminal_state + patch_digest is not None but fail the chain-content assertion (no actual checkpoints written); the chain-content assertion is the mutation-resistant anchor. Why three fixtures, not one: the roadmap explicitly names three; a single fixture would silently coast on package-manager-specific assumptions (npm vs. yarn-berry-pnp vs. pnpm-native node-modules vs. content-addressed lockfile shapes); the three together exercise the package-manager dispatch the plugin's transforms adapters carry.

Scenario 2 — Retry then recovery

  • [ ] AC-2 — tests/integration/workflows/test_phase6_retry_recovery.py drives a fixture (one parametrized row per cohort fixture — three rows) where the first gate attempt fails with a retryable signal (injected via a test-double GateRunner that returns RetryableFailure on call 1, Passed on call 2) and asserts the final result is terminal_state="completed" AND gate_summary.attempt_count == 2 AND the checkpoint chain contains the planner-replan transition (PatchApplied → GateFailedRetryable → NeedsPlan → PlanReady → PatchApplied → Completed). The test-double GateRunner is constructed via the Phase-5 gate-runner Protocol (which Phase 5 ships); the SUT's SutConfig.gate_runner_factory is the dependency-injection seam (an additive SutConfig field would erode S5-01's anti-refactor #1 — instead, the test-double swap happens via the plugin's Plugin.gate_runner adapter slot, mirroring how Phase 4 swaps the LeafLLMPort for cassette replay). Mutation thinking: a stub that swallows the first failure and silently re-emits Passed on call 1 would land attempt_count == 1 — the explicit == 2 check is the mutation anchor. The chain-walk assertion catches a different mutation: a planner that "remembers" the prior attempt by mutating the existing PlanReady row instead of inserting a new one would corrupt the chain head — the explicit five-transition walk catches it.

Scenario 3 — Kill mid-run then resume

  • [ ] AC-3 — tests/integration/workflows/test_phase6_kill_resume.py drives a clean-completion case under a forced cancellation between PlanReady and PatchApplied, then a second run_case call with execution_mode="replay" and the same workflow_id, and asserts (i) the first call propagates asyncio.CancelledError (via asyncio.wait_for(sut.run_case(case), timeout=0.05) against a graph stub that sleeps inside the patch-apply node), (ii) the per-run cancelled marker file exists (S5-01 AC-11), (iii) the second call returns terminal_state="completed" AND the result's evidence_references + gate_summary + patch_digest are byte-identical to a never-killed counterfactual run (metamorphic-test pair — kill-and-resume === never-killed at the result-byte level, modulo workflow_id because that's preserved on resume, and modulo timestamps). The byte-equality assertion uses result.model_dump_json(by_alias=True) with a canonical-keys serialization and a _BYTE_EQUAL_MODULO = frozenset({"_timestamp_fields"}) allowlist module-level constant (defined in the test module, not pushed into the production code). Mutation thinking: an adapter that "helpfully" mints a fresh workflow_id on the replay-mode call would silently break resume — the chain would be empty, hydrate_or_fail would return Hydrated(kind="empty_workflow"), the graph would start from scratch and the byte-equality would still pass (because both runs reach the same Completed) BUT the chain-head walk would show only the resumed-run checkpoints (no PlanReady from the first call). The explicit chain-walk sub-assertion asserts the chain head on the second call matches the chain head left by the first call's last successful pre-cancellation checkpoint (PlanReady). The metamorphic property — killed-and-resumed result === never-killed result at the byte level — is the load-bearing invariant Phase-9 (Temporal) inherits.

Scenario 4 — HITL interrupt then resume

  • [ ] AC-4 — tests/integration/workflows/test_phase6_hitl_interrupt_resume.py drives a case where the gate fails twice (test-double GateRunner returns RetryableFailure on calls 1 + 2, then Passed on call 3 if reached), asserts (i) the first run_case returns terminal_state="awaiting_human_review" with failure_modes containing "phase6.hitl_trust_outcome_failed" AND the handoff_path evidence reference resolves to a file under the per-run directory, (ii) a stale ResumeInput (constructed against a different workflow_id) is rejected by resume_or_reject with ResumeRejected(reason="stale_token") (phase-arch-design.md §"Failure modes" row 4), (iii) a valid ResumeInput approves continuation, the second run_case(execution_mode="replay", workflow_id=...) returns terminal_state="completed", and the chain walk shows the approved-resume transition originates from the latest verified checkpoint (the AwaitingHumanReview row), NOT from genesis. The resume_or_reject validation seam is S4-01's ResumeRejected discriminated union — the test imports the four ADR-0001 names + HitlInterrupt + ResumeInput + resume_or_reject (all already in codegenie.workflows.__all__ per S4-01's AC-12 extension). Mutation thinking: an adapter that accepts a stale ResumeInput without validation would silently advance an unrelated workflow's chain — the explicit ResumeRejected(reason="stale_token") assertion catches; the chain-origin sub-assertion catches a different mutation where the resume restarts from genesis instead of from the verified checkpoint (replay-byte-equality would still hold for the result but the chain would carry zero pre-resume rows).

Scenario 5 — Tampered checkpoint (final-design.md §"Decisions of record" item 3 + phase-arch-design.md §"Scenarios" #4)

  • [ ] AC-5 — tests/integration/workflows/test_phase6_tampered_checkpoint.py drives a clean-completion case to PlanReady, directly mutates one column of one checkpoints.sqlite row via raw sqlite3.connect(...).execute(...) (the test bypasses CheckpointStore deliberately — the tamper has to be substrate-level, not Protocol-level), then drives a second run_case(execution_mode="replay", workflow_id=...) and asserts (i) the result is terminal_state="failed_unrecoverable" AND failure_modes == ("phase6.failed_checkpoint_integrity",) AND patch_digest is None, (ii) no patch work was attempted (the test-double TransformPort records zero apply() calls), (iii) the integrity decision happened in hydrate_or_fail — NOT in LocalVulnRemediationSut (assert by patching codegenie.workflows.replay.hydrate_or_fail with a Mock(wraps=...) and confirming it was called exactly once; assert by AST-walking local_sut.py confirming no ChainMismatch / TornWrite / EmptyWorkflow literal appears — already covered by S5-01 AST fence but pin again for closeout). Mutation thinking: an adapter that "helpfully" recovers from a tampered chain by truncating and replaying from genesis would silently produce a Completed result for a tampered run — the bench harness would never see the integrity failure. The apply() call-count == 0 sub-assertion catches the mutation directly; the hydrate_or_fail-was-called sub-assertion catches the converse mutation (adapter re-implements integrity check inline, leaving hydrate_or_fail unused — the SOLE-site discipline breaks silently).

Scenario 6 — Workflow-scope replay-determinism property (extends Phase 4 S6-07 from FallbackTier-scope to workflow-scope)

  • [ ] AC-6 — tests/property/test_workflow_replay_determinism.py is a Hypothesis property under @settings(max_examples=N, deadline=None, suppress_health_check=[HealthCheck.too_slow]) (N resolved at module import as int(os.environ.get("PHASE6_DETERMINISM_EXAMPLES", "1")) in the default test run; the bench-marked variant under @pytest.mark.bench uses N = 50 per docs/roadmap.md §"Phase 6" "N independent runs"). The property draws (fixture_name, cassette_id, embedding_model_digest) triples from st.sampled_from(...) over {node_typescript_helm, node_yarn_berry_pnp, node_pnpm_native} × the cassette-pin closed set × the embedding-model-digest closed set, drives 50 independent run_case(case) calls (in a single test invocation; concurrent via asyncio.gather), and asserts pairwise byte-identical result.model_dump_json(by_alias=True) modulo the allowlist _BYTE_EQUAL_MODULO = frozenset({"_timestamp_fields", "workflow_id"}) — the same allowlist AC-3 uses. The pairwise comparison uses set(...) over the canonical-byte serializations after stripping the allowlist fields — a set of size 1 over 50 runs is the byte-identical-invariant. Mutation thinking: an adapter that folds a time.time() reading into evidence_references would silently break determinism — the property would shrink to a two-example counter-example showing the offending field; the AST fence S5-01 AC-10 already forbids time.time in digest() but NOT in the adapter's shell — the property is the behavioural backup. The property's failure surface is exactly the Phase-6.5 nightly-bench's flake mode: if determinism breaks at this layer, every downstream eval is silently poisoned (the cached BenchScore is keyed on sut_digest + case_digest + cassette_digest; if the SUT produces non-deterministic outputs for the same key, the cache hits a stale-but-valid score and the operator never sees the regression). Why concurrent asyncio.gather, not sequential: sequential runs can hide a stateful side channel (e.g., a module-level cache that silently makes run-N depend on run-1's output); gather exercises the cancellation-free concurrent code path Phase-6.5's bench harness will run in production.

Scenario 7 — Phase-6.5-consumer isolation (the four-bullet "may NOT depend on" enforced from this side)

  • [ ] AC-7 — tests/integration/test_phase6_consumer_isolation.py constructs a synthetic in-process consumer (a synthetic_consumer.py source file under tests/golden/phase6/synthetic_consumer.py) that imports ONLY the four ADR-0001 names + build_local_sut from the plugin's api.py + HitlInterrupt, ResumeInput, resume_or_reject, drives one clean-completion case, asserts the result is the expected shape, AND asserts an AST walk over the synthetic consumer's source confirms zero imports from each of the four final-design.md §"Relationship to Phase 6.5" "may NOT" bullets: (i) codegenie.workflows.vuln_ledger.* (graph-internal state); (ii) plugins.vulnerability_remediation__node__npm.subgraph.* (concrete graph builder + node names); (iii) codegenie.workflows.sqlite_checkpoints / codegenie.workflows.in_memory_checkpoints / codegenie.workflows.checkpoints (checkpoint backend internals); (iv) any plugins.vulnerability_remediation__node__npm.subgraph._* private module (plugin-local file layout). The walk also rejects any import of a module under codegenie.workflows.* that does NOT appear in the post-S4-01 __all__ — the four ADR-0001 names PLUS the S4-01 HITL trio are the only legal harness-facing names. Mutation thinking: a future-cohabitating developer who "just needs to peek at the chain head" would silently import codegenie.workflows._chain and break the contract boundary — the AST walk catches at PR time; the placeholder fence at tests/fence/test_phase6_no_graph_imports_from_phase65.py skips when Phase 6.5 has not yet landed (today), so this story's complementary test_phase6_consumer_isolation.py is what enforces the closure during the Phase-6/Phase-6.5 gap. Why a synthetic source file, not a from __future__ runtime test: an AST walk over a file surface is mutation-resistant (a developer who adds an import survives byte-equal source); a runtime import test is mutation-resistant only against the specific names the test happens to import.

Cross-cutting test-architecture additions (docs/roadmap.md §"Test architecture evolution" Phase 6 rows)

  • [ ] AC-8 — tests/e2e/scenarios.yaml is created (the file does not exist today; the precedent schema lives at tests/fixtures/adversarial/*/.codegenie/scenarios.yaml) carrying exactly three rows — one per fixture in the Phase-6 cohort — each row asserting terminal_state reached + replay-byte-equality of two independent runs. Schema (additive over the existing tests/fixtures/adversarial/*/.codegenie/scenarios.yaml shape — name: <str>, command: [<argv>], plus new keys phase: 6, fixture: <fixture-name>, expected_terminal_state: <ledger-terminal-state>, expected_byte_equal_modulo: [<field-list>]): 

    # Phase 6 closeout — full state-machine slice
scenarios:
  - name: phase6_clean_completion_typescript_helm
    phase: 6
    fixture: node_typescript_helm
    expected_terminal_state: completed
    expected_byte_equal_modulo: [_timestamp_fields, workflow_id]
  - name: phase6_clean_completion_yarn_berry_pnp
    phase: 6
    fixture: node_yarn_berry_pnp
    expected_terminal_state: completed
    expected_byte_equal_modulo: [_timestamp_fields, workflow_id]
  - name: phase6_clean_completion_pnpm_native
    phase: 6
    fixture: node_pnpm_native
    expected_terminal_state: completed
    expected_byte_equal_modulo: [_timestamp_fields, workflow_id]
    A test at tests/integration/test_e2e_scenarios_yaml.py loads the file, asserts the schema validates against a Phase6ScenariosFile Pydantic model with extra="forbid" (defined in the test module, not pushed into production), and drives each row via the same build_local_sut(...) + await sut.run_case(...) envelope AC-1 uses. Mutation thinking: a schema with extra="allow" would let a future developer slip an expected_secret_leak: ... row through; extra="forbid" catches at validate time. Why a YAML file + a separate test, not three more parametrized rows in AC-1: the YAML row IS the cross-cutting test-architecture surface — Phase 7's migration plugin closeout (and every future plugin closeout) extends this file additively with phase: 7 rows; the phase: 6 rows are the precedent the future rows pattern-match against.

  • [ ] AC-9 — Contract snapshot at tests/golden/phase6-contract/snapshot.json is byte-equal to its S5-01 post-state — no further additive amendments by this story (the story ships zero new public types). A test asserts (snapshot_after_S5_01_sha256, snapshot_after_S6_01_sha256) are equal. The post-S5-01 SHA256 is captured as a single-line tests/golden/phase6-contract/.s5_01_post_sha256 artifact written by S5-01's executor (the artifact's content is the bytes of hashlib.sha256(snapshot_bytes).hexdigest() after S5-01's PHASE6_CONTRACT_GOLDEN_REWRITE=1 regeneration; if S5-01 has not landed at the time S6-01 executes, this AC blocks per Rule 12 — fail loud). Mutation thinking: an executor who silently regenerates the snapshot via PHASE6_CONTRACT_GOLDEN_REWRITE=1 to "absorb" a non-existent change would slip a contract amendment in without a corresponding ADR amendment; the SHA256 cross-check from the prior story's checkpoint catches it.

  • [ ] AC-10 — tests/unit/test_phase6_docs.py is extended additively with three new tests: (i) test_phase6_roadmap_row_intact() — asserts the row at docs/roadmap.md § "Phase 6" carries the ✅ glyph + the link target phases/06-sherpa-vuln-loop/; (ii) test_phase6_mkdocs_nav_intact() — asserts mkdocs.yml carries a nav entry phases/06-sherpa-vuln-loop/README.md (and the README exists); (iii) test_phase6_internal_cross_links_resolve() — walks every Markdown file under docs/phases/06-sherpa-vuln-loop/ and asserts every relative link target resolves to an existing file (mutation-resistant against a renamed ADR file or a typoed link). Mutation thinking: a future doc-renaming PR that accidentally orphans ADRs/0003-checkpointed-ledger-replay-boundary.md would silently break the design package's discoverability; the cross-link walk catches at PR time.

Closeout sweep

  • [ ] AC-11 — docs/phases/06-sherpa-vuln-loop/stories/README.md's "Definition of done" section is updated additively: every bullet ("Story acceptance criteria are green", "New public types are covered by mypy-strict and serialization tests", "No graph node directly calls another node", "Resume paths are replay-verified", "Any change to VulnRemediationSut updates ADR-0001 and the Phase 6.5 contract tests") flips from prose to - [x] checkbox state with an inline reference to the test or fence that enforces it. A unit test at tests/unit/test_phase6_docs.py::test_phase6_definition_of_done_marked_done() asserts the README contains exactly five - [x] lines under the "Definition of done" heading and zero bare - bullets. Mutation thinking: a doc-only PR that "tidies" the checkbox state back to prose would silently un-document the closeout — the ==5 and 0 assertion catches.

  • [ ] AC-12 — docs/phases/06-sherpa-vuln-loop/stories/_attempts/_lessons.md gains a ## From S6-01 (2026-05-26) section capturing 3–5 cross-story lessons surfaced during the e2e closeout (lessons to be captured by the executor — examples the Notes-for-implementer surfaces: (a) per-fixture parametrization is mutation-resistant where a single-fixture test silently coasts; (b) the metamorphic kill-and-resume === never-killed property is what makes Phase-9 Temporal substrate-swappable; (c) the apply() call-count == 0 sub-assertion for the tampered-checkpoint scenario is the only mutation-resistant defense against an adapter that "helpfully" recovers; (d) AST-walking a file-surface synthetic consumer is mutation-resistant where a runtime-import test is mutation-resistant only against the specific names the test happens to import; (e) the workflow-scope determinism property's failure surface is exactly the Phase-6.5 bench-cache-poisoning mode). A unit test at tests/unit/test_phase6_docs.py::test_phase6_lessons_has_s6_01_section() asserts the section exists. Mutation thinking: if the executor ships the e2e tests but skips the lesson capture, the next phase's story authors lose the load-bearing context — the assertion catches at unit-test time.

Closeout discipline (make check, type-, lint-, fence-conformance)

  • [ ] AC-13 — make check passes (lint + typecheck + test + fence); mypy --strict src/ reports zero errors over all touched files; make lint-imports (import-linter) passes with zero new contract violations. No ADR amendments are required (this story ships zero new public types, zero new ALLOWED_BINARIES entries, zero new Literal widenings). A meta-assertion at tests/unit/test_phase6_docs.py::test_phase6_s6_01_no_adr_amendments() walks docs/phases/06-sherpa-vuln-loop/ADRs/*.md and asserts every ADR's Status: field reads Accepted (no Amended line referencing S6-01). Mutation thinking: the closeout discipline is the structural defense — a silent ADR amendment slipped in via S6-01 (e.g., "amend ADR-0001 to allow Phase-6.5 import of _chain") would violate the contract-boundary discipline ADR-0001 §Consequences pins. The walk catches.

Anti-refactor (Rule 2 — what this story explicitly does NOT do)

The following are NOT added by this story; the rule-of-three for each remains unmet. Surfacing here so the executor under closeout-pressure does not silently add them:

  1. No generalized BaseE2ETestCase / PhaseClosureHarness / WorkflowFixtureBuilder ABC. Phase 7's migration-plugin closeout will be the second e2e closeout; Phase 10+ will earn the rule-of-three. Until then, pytest-asyncio fixtures composed via the canonical pytest dependency-injection idiom (function-scoped async def fixture_node_typescript_helm(tmp_path) -> Path etc.) are sufficient. The five integration tests share fixtures, not a base class.
  2. No scenarios.yaml registry / @register_scenario decorator. The YAML file IS the registry (data, not prompts; CLAUDE.md "Organizational uniqueness as data, not prompts"). A decorator-based registry would couple the schema to import-time side effects; the YAML file is parsed once at test-collection time.
  3. No new public name in codegenie.workflows.__all__. The allowlist is byte-equal-unchanged from its post-S4-01 value. The integration tests import from the existing 14-name surface + HitlInterrupt / ResumeInput / resume_or_reject (already added by S4-01) + the plugin's build_local_sut factory.
  4. No new module under src/codegenie/workflows/. All new code in this story is under tests/. The closeout asserts the existing surface is sufficient; introducing a _e2e_helpers.py module would be premature abstraction (only one Phase-6 e2e closeout exists; the second is Phase-7's migration plugin and is structurally different).
  5. No LangGraph-version-pinning hardening, no aiosqlite-version-pinning hardening, no Hypothesis profile changes in pyproject.toml. The existing pins are sufficient; deeper compatibility hardening is deferred to phase-specific stability work, not closeout.
  6. No coverage ratchet bump. S6-01 is a closeout — the --cov-fail-under=85 gate in pyproject.toml already governs; ratcheting in a closeout invites a separate dependency: the executor would have to verify the bump does not regress unrelated coverage paths. Deferred to a focused coverage-discipline story.

Files to touch

  • tests/integration/workflows/test_phase6_clean_completion.py (new) — AC-1 (parametrized over three fixtures).
  • tests/integration/workflows/test_phase6_retry_recovery.py (new) — AC-2 (parametrized over three fixtures; test-double GateRunner swapped via the plugin's adapter slot).
  • tests/integration/workflows/test_phase6_kill_resume.py (new) — AC-3 (metamorphic kill-and-resume === never-killed; chain-walk sub-assertion).
  • tests/integration/workflows/test_phase6_hitl_interrupt_resume.py (new) — AC-4 (stale + valid resume; chain-origin sub-assertion).
  • tests/integration/workflows/test_phase6_tampered_checkpoint.py (new) — AC-5 (direct SQLite tamper; apply() call-count == 0; hydrate_or_fail call-count == 1).
  • tests/property/test_workflow_replay_determinism.py (new) — AC-6 (Hypothesis property; concurrent asyncio.gather; _BYTE_EQUAL_MODULO allowlist).
  • tests/integration/test_phase6_consumer_isolation.py (new) — AC-7 (AST walk over tests/golden/phase6/synthetic_consumer.py).
  • tests/golden/phase6/synthetic_consumer.py (new) — AC-7 (the synthetic Phase-6.5-shaped consumer source the AST walk inspects).
  • tests/e2e/scenarios.yaml (new) — AC-8 (three rows; extra="forbid" schema).
  • tests/integration/test_e2e_scenarios_yaml.py (new) — AC-8 (loader + driver).
  • tests/golden/phase6-contract/.s5_01_post_sha256 (new — created by S5-01's executor; this story READS it) — AC-9 (cross-check anchor; this story does NOT write the file).
  • tests/integration/test_phase6_contract_snapshot_byte_equal_s5_01.py (new) — AC-9 (the SHA256 cross-check assertion).
  • tests/unit/test_phase6_docs.py — extend additively with test_phase6_roadmap_row_intact, test_phase6_mkdocs_nav_intact, test_phase6_internal_cross_links_resolve, test_phase6_definition_of_done_marked_done, test_phase6_lessons_has_s6_01_section, test_phase6_s6_01_no_adr_amendments (AC-10, AC-11, AC-12, AC-13).
  • tests/conftest.py OR tests/integration/workflows/conftest.py (new) — shared fixtures (fixture_node_typescript_helm, fixture_node_yarn_berry_pnp, fixture_node_pnpm_native, cassette_pin_default, embedding_model_digest_default, build_local_sut_for_test) — composed via canonical pytest dependency injection; no base class (Anti-refactor #1).
  • docs/phases/06-sherpa-vuln-loop/stories/README.md — flip each "Definition of done" bullet from prose to - [x] with a test-or-fence reference inline (AC-11).
  • docs/phases/06-sherpa-vuln-loop/stories/_attempts/_lessons.md — append a ## From S6-01 (2026-05-26) section with 3–5 lessons (AC-12).
  • docs/phases/06-sherpa-vuln-loop/stories/_attempts/S6-01-e2e-kill-resume-closeout.md (new — append-only attempt log) — created by the executor.

TDD plan

Red. Write the failing tests in this dependency-respecting order; each step asserts the failure mode is meaningful before any production code lands (most of this story is test code — there is little new production code, mostly composition of existing adapters):

  1. AC-1 — Clean completion. Write test_phase6_clean_completion.py with the three parametrized rows; assert terminal_state="completed" + patch_digest is not None + gate_summary.all_passed + checkpoint chain content. Fails: no build_local_sut factory yet (S5-01 not landed), OR fails on cassette pinning (the fixture's cassette file does not exist). Either failure is meaningful.
  2. AC-2 — Retry recovery. Write test_phase6_retry_recovery.py with the test-double GateRunner; assert attempt_count == 2 + the five-transition chain walk. Fails: planner does not re-enter; chain walk shows only 3 transitions.
  3. AC-3 — Kill / resume. Write test_phase6_kill_resume.py with the metamorphic counterfactual. Fails: asyncio.wait_for(..., timeout=0.05) does not cancel cleanly, OR the resumed run does not produce byte-equal output.
  4. AC-4 — HITL interrupt / resume. Write test_phase6_hitl_interrupt_resume.py with stale + valid resume. Fails: resume_or_reject does not reject the stale token, OR the resume restarts from genesis.
  5. AC-5 — Tampered checkpoint. Write test_phase6_tampered_checkpoint.py with the direct SQLite tamper. Fails: the adapter recovers from the tamper, OR apply() is called despite the integrity failure.
  6. AC-6 — Workflow-scope replay-determinism property. Write test_workflow_replay_determinism.py with N=1 in the default run (full N=50 under @pytest.mark.bench). Fails: 50 concurrent runs produce non-identical outputs.
  7. AC-7 — Consumer isolation. Write tests/golden/phase6/synthetic_consumer.py importing the legal four-name surface + S4-01 trio + plugin factory; write tests/integration/test_phase6_consumer_isolation.py with the AST walk + drive-one-case. Fails: synthetic consumer has nothing to import, OR AST walk discovers an illegal import.
  8. AC-8 — scenarios.yaml. Write the YAML + the loader test. Fails: file does not exist; or schema validation rejects.
  9. AC-9 — Contract snapshot byte-equal. Write the SHA256 cross-check. Fails: the .s5_01_post_sha256 anchor file does not exist (S5-01 has not landed yet).
  10. AC-10, AC-11, AC-12, AC-13 — Docs + closeout tests. Extend test_phase6_docs.py additively. Fails: the roadmap row glyph is missing, the mkdocs nav entry is missing, the cross-link walk discovers an orphan, the Definition-of-done bullets are still prose, the _lessons.md section is missing, or an ADR carries an unexpected Amended line.

Green. Implement the minimum that makes all red tests pass, in this order. The story ships mostly test code (this is a closeout); the few non-test edits are:

  1. Compose the integration tests using the canonical pytest fixtures (tmp_path, caplog, the new shared fixtures under tests/integration/workflows/conftest.py). The SUT is constructed via build_local_sut(SutConfig(...)). The test-double GateRunner (AC-2) is constructed via the Phase-5 gate-runner Protocol and wired into the plugin's adapter slot per the existing plugin-adapter idiom.
  2. Land the scenarios.yaml + loader test. Three rows; extra="forbid" schema.
  3. Land the workflow-scope determinism property. Hypothesis @settings(max_examples=...)-driven; concurrent via asyncio.gather; _BYTE_EQUAL_MODULO allowlist module-level.
  4. Land the synthetic consumer file + AST walk. AST walk follows the precedent at tests/fence/test_phase6_no_graph_imports_from_phase65.py (already exists).
  5. Land the contract-snapshot cross-check. Reads .s5_01_post_sha256; asserts equality.
  6. Extend test_phase6_docs.py additively with the six new tests.
  7. Flip the Definition-of-done bullets in stories/README.md to - [x] with inline test references.
  8. Append the _lessons.md section with the 3–5 cross-story lessons.

Refactor. Cleanup only — no new behaviour: - Confirm every integration test imports the SUT via build_local_sut(...) only — no integration test imports LocalVulnRemediationSut directly (S5-01 anti-refactor #5). - Confirm _BYTE_EQUAL_MODULO is defined once (in AC-3's module) and re-used by AC-6 via module-import, not re-declared (DRY across exactly two consumers — rule-of-three not yet met, but explicit re-use is cheaper than duplication for this invariant). - Confirm the conftest fixtures carry no port references (Anti-refactor #4 god-object precedent inherited from S5-01). - Confirm tests/e2e/scenarios.yaml has exactly three rows (no more, no fewer) — phase-7 will add phase: 7 rows additively, not here.

Anti-refactor. See the Anti-refactor block in §Acceptance criteria — do NOT introduce BaseE2ETestCase, a @register_scenario decorator, a _e2e_helpers.py module under src/codegenie/workflows/, or any addition to codegenie.workflows.__all__.

Out of scope

  • The Phase-7 migration-plugin closeout — its own e2e closeout story; Phase-7's row addition to tests/e2e/scenarios.yaml is additive at that time.
  • The Phase-6.5 nightly-bench job — Phase-6.5 S4-01 + S4-02 own.
  • The Phase-6.5 harness-side import-fence test (tests/integration/test_phase65_harness_imports.py) — Phase-6.5 S1 owns; the codegenie-side enforcement is AC-7 + the existing tests/fence/test_phase6_no_graph_imports_from_phase65.py placeholder (which un-skips when Phase 6.5's harness directory lands).
  • The Phase-9 TemporalVulnRemediationSut substrate-swap conformance — Phase-9 S4-05 §G5 owns; the metamorphic kill-and-resume === never-killed property (AC-3) is the substrate this assertion later compares against.
  • LangGraph-version-compatibility matrix work — deferred to a focused stability story; no new pins in pyproject.toml.
  • Coverage ratchet bump (--cov-fail-under=85 → 90 or higher) — deferred to a focused coverage-discipline story (Anti-refactor #6).
  • A generalized cross-task-class e2e harness — deferred until Phase-7 + Phase-10 earn the rule-of-three (Anti-refactor #1).

Notes for the implementer

  • Why three fixtures and not one. The roadmap explicitly names node_typescript_helm + node_yarn_berry_pnp + node_pnpm_native. A single fixture would silently coast on package-manager-specific assumptions (npm vs. yarn-berry-pnp vs. pnpm-native). The three together exercise the package-manager dispatch the plugin's transforms adapters carry, which is exactly the kind of dispatch a closeout must close-pin. Parametrizing — @pytest.mark.parametrize("fixture_name", ["node_typescript_helm", "node_yarn_berry_pnp", "node_pnpm_native"]) — keeps the test bodies single-source while exercising three independent code paths.

  • Why the metamorphic kill-and-resume === never-killed property is load-bearing. Phase-9's Temporal substrate later asserts byte-identical results across Local + Temporal SUTs under arbitrary worker failures (worker dies mid-step; Temporal replays from the last completed activity). The Phase-6 metamorphic property (AC-3) is exactly the substrate that assertion compares against — if the Local SUT doesn't produce byte-identical results across kill-and-resume cycles, no amount of Phase-9 Temporal scaffolding can rescue it. The property is the load-bearing invariant Phase-9's TemporalVulnRemediationSut inherits structurally.

  • Why the tampered-checkpoint test mutates SQLite directly, not via the CheckpointStore Protocol. The tamper has to be substrate-level — a future malicious mutation against a checkpoint file in production would happen via direct file-system write, not via the Protocol. The Protocol's sanitization is at write time; the integrity check is at hydration time. Mutating via the Protocol would write a sanitized tampered row that the chain head would correctly track — the integrity check would not fire. The direct SQLite write skips sanitization, producing a row whose stored next_head does not match the recomputed-from-bytes fold — exactly the ChainMismatch arm of S2-02's ReplayVerdict.

  • Why concurrent asyncio.gather for the determinism property, not sequential. Sequential runs can hide a stateful side channel (e.g., a module-level cache that silently makes run-N depend on run-1's output). gather exercises the cancellation-free concurrent code path Phase-6.5's bench harness will run in production. The property's failure surface is exactly the Phase-6.5 nightly-bench's flake mode.

  • Why AST-walking the synthetic consumer file, not a runtime import test. A runtime import test catches only the specific names the test happens to import; an AST walk over the file surface catches every import, including imports that would be syntactically valid but never resolved at test time (e.g., conditional imports under if TYPE_CHECKING:). The walk follows the precedent set by tests/fence/test_phase6_no_graph_imports_from_phase65.py (already exists; placeholder-skips when Phase 6.5 has not landed).

  • Why the contract-snapshot cross-check reads a SHA256 anchor file written by S5-01. S5-01 ships the additive contract-snapshot extension (adding concrete_implementers.LocalVulnRemediationSut). S6-01's job is to close-pin that S5-01-post state — to assert no further additive amendments land between S5-01 and S6-01. The SHA256 anchor file is the cheapest possible cross-check: one file, 64 hex digits, byte-equal-or-fail. An executor adding a public method to LocalVulnRemediationSut between S5-01 and S6-01 would regenerate the snapshot, change the SHA256, and break this story's AC-9 — surfacing the contract amendment for ADR review.

  • Why extra="forbid" on the scenarios.yaml schema. The YAML file is the cross-cutting test-architecture surface (Phase 7+ will add phase: 7 rows additively). A extra="allow" schema would let a future developer slip an expected_secret_leak: true row through; extra="forbid" makes every new field require an explicit schema amendment. The schema lives in the test module, not in production code — it's a test invariant, not a production invariant.

  • Why the Definition of done bullets flip to - [x] with inline test references. The bullets in stories/README.md are prose today; flipping them to checkboxes with (enforced by tests/integration/workflows/test_phase6_clean_completion.py)-style inline references makes the closeout visibly enforced (a future reader sees the checkmark + the test path and can verify the assertion in one click). The unit test (AC-11) asserts the count of - [x] lines is exactly five — mutation-resistant against a "tidying" PR that reverts the checkbox state.

  • Why no new public name in codegenie.workflows.__all__. S6-01 is the closeout — it ships zero new public types. The closeout's job is to compose existing surfaces and assert the closure. Adding a public name during closeout would slip a contract amendment under the radar (the contract snapshot would change; AC-9 would catch it; but the discipline is to refuse the addition in the first place). The Anti-refactor block makes the refusal explicit so a deadline-pressured executor does not silently add a "convenience" name.