ADR 005 — Controlled replay and isolated rebuild

Status: controlled replay and isolated rebuild implemented · updated 3 October 2026

Context

Re-reading old input into intact state is different from reconstructing empty state. Restarting output versions against an existing consumer view can cause valid rebuilt output to be ignored or stale data to reappear.

Decision

Limit replay to a retained already-consumed interval in intact compatible state. Run it under a valid partition fence, use a separate scan cursor, and keep the authoritative source frontier monotonic. Do not release quarantined records or change interpretation during replay.

Rebuild from a sufficient full source history into a fresh namespace with new state, outbox, output topic, and view. Include namespace in output IDs. Verify the new namespace independently and leave the original namespace unchanged.

The implemented commands default to dry runs and require --execute --quiesce for execution. Replay checks the scanned records while holding the partition row lock, then commits only a new ownership epoch; it never repairs state. Rebuild namespaces read the same retained source incarnation and share its producer session guard, while keeping processing and output lineage separate. The owned rebuild relay drains and joins before bounded verification. See the operator guide for preconditions, failure reports, and drills.

Alternatives

Rewinding the authoritative frontier conflates source progress with an administrative scan. Empty reconstruction from an arbitrary middle offset omits facts. Reusing the old output namespace requires a version-generation and consumer-cutover protocol that v1 does not implement.

Consequences

The supported recovery commands have narrow, explainable preconditions. Rebuilds can have different publication histories and versions while producing equal business values. Retention loss, incompatible configuration, or topic recreation prevents casual resume/replay.

Checkpoint restore, live alias replacement, consumer cutover, and production backup recovery remain documented extensions. No command silently attempts them.

Review trigger

Revisit when a real cutover or bounded recovery objective exists. Specify snapshot completeness, output generations, and consumer behavior before sharing an output namespace across rebuilt state.