tracking: read the ledger's version, not just write it; clear owned on any block; channel-correct prune recovery

src/core/tracking/CLAUDE.md

@@ -21,21 +21,33 @@ non-destructive side of the question being asked — over-protection (prune skip
 reclaimable file, or refuses to run; resample adds instead of replacing) is an
 accepted residual; under-protection is a data-loss bug.
 
-**No silent gaps.** A system-created file is tracked from the instant it exists.
-`ReaSamplerSession::recordCreated` is the only writer, called at the same point the
-`Sample` is added, and it reads lineage off that `Sample`'s own provenance — so the
-recipe fingerprint and the lineage record come from one act and cannot disagree.
+**No silent gaps — in memory at creation, on disk at the next save.**
+`ReaSamplerSession::recordCreated` is the only writer, called in the same
+straight-line block as the bank add with no I/O or early return between, so a
+system-created file is tracked *in memory* the instant it exists. The record reaches
+the `.rpp` only at the following `saveToActiveProject()`; a crash in that window, or a
+whole session running with a degraded ledger (below), leaves the file untracked —
+foreign, therefore never reclaimed, which is the safe direction. That residual is
+accepted and stated here rather than papered over by a stronger claim.
 
 **Lineage is never backfilled.** `OriginLedger::record` on a path already present is
 `AlreadyPresent` and leaves the stored record untouched. A record says what was known
 at birth or says nothing at all; nothing may later rewrite history from a guess.
+Consequence, and the reason both blobs can carry a parent id: `recordCreated` *seeds*
+the record from the `Sample`'s own `provenance.parentSampleId` (one source, one act),
+but the `Sample` is mutable via `updateSampleInPlace` and the record is not — so where
+the two ever diverge, **the ledger record is authoritative for lineage** and the
+`Sample`'s copy is only the recipe facet's view of it.
 
-**Never-recorded and unreadable are different absences.** `LedgerStatus` keeps them
+**Never-recorded and degraded are different absences.** `LedgerStatus` keeps them
 apart, and only `loadLedger` can tell them apart (an empty stored value is not valid
 JSON, so the parser alone cannot). `Fresh` — a new project or a bank predating the
-ledger — blocks nothing and yields definite answers. `Unreadable` blocks every
-destructive answer AND suppresses the next write, so a corrupt blob survives for
-recovery instead of being replaced by a ledger missing every earlier file.
+ledger — blocks nothing and yields definite answers. `Unreadable` (corrupt) and
+`FutureVersion` (a `"v"` newer than this build reads) are the degraded pair: each
+blocks every destructive answer AND suppresses the next write, so the stored blob
+survives for recovery instead of being replaced by a ledger missing every earlier
+file. `ledgerDegraded()` is the one test for both treatments — they coincide today, so
+the two axes are deliberately not split.
 
 **Consumers cannot disagree.** Both answers come out of one `TrackingState`. The two
 universes differ deliberately — prune protects bank-referenced paths, ledger-owned
@@ -54,9 +66,9 @@ birth record is written for every system-created file, ambiguous parentage or no
 - `origin_ledger` — the record family: `OriginRecord` (project-relative path, `OriginKind`,
   the sample id minted at birth, the parent sample id) and the insertion-ordered,
   path-keyed, deduplicated `OriginLedger` that holds them, with its JSON codec and the
-  `loadLedger` three-way `Fresh` / `Loaded` / `Unreadable` classification. Persisted
-  under the FOREVER-STABLE `owned_files` ext-state key; the legacy path-only shape
-  (`{"owned":[...]}`) lifts in as `Unknown`-kind records with no lineage.
+  `loadLedger` `Fresh` / `Loaded` / `Unreadable` / `FutureVersion` classification.
+  Persisted under the FOREVER-STABLE `owned_files` ext-state key; the legacy path-only
+  shape (`{"owned":[...]}`) lifts in as `Unknown`-kind records with no lineage.
 - `tracking_authority` — the one decision surface: `pruneProtection` (the `owned` and
   held-path inputs prune's set algebra consumes, plus the blocked/blockers verdict)
   and `tiedUsageExists` (`Yes` / `No` / `Indeterminate`, per capture, excluding the
@@ -66,7 +78,11 @@ birth record is written for every system-created file, ambiguous parentage or no
 
 - `OriginKind` values are PERSISTED INTEGERS — never renumber, only append. An
   unrecognized value degrades to `Unknown` rather than failing the parse: a vocabulary
-  gap must not halt the prune.
+  gap must not halt the prune. This degrade-don't-halt rule is about *field
+  vocabulary* only — an unrecognized **document version** (`"v"`) does the opposite
+  and blocks, because a changed record shape makes the whole ledger untrustworthy.
+- `OriginRecord.sampleId` is NOT unique: a recapture writes a new file under the same
+  bank id, so two records can share a `sampleId`. `relativePath` is the only key.
 - A `unioned` usage record can never be excluded as "my own" — it carries more than one
   incarnation's holds, so attributing it to a single owner could hide a sibling's tie.
 - The set algebra prune runs on `(owned ∩ present) − referenced` is `core/reclaim`'s,

src/core/tracking/origin_ledger.cpp

@@ -1,8 +1,9 @@
 #include "core/tracking/origin_ledger.h"
 
-#include <cctype>
+#include <utility>
 
 #include "core/json/json.h"
+#include "core/util/relative_path.h"
 
 // Version ladder for the stored blob, under the FOREVER-STABLE "owned_files" key:
 //
@@ -13,21 +14,20 @@
 // v1 blobs lift to v2 records with kind Unknown and empty ids — a pre-existing bank
 // keeps every protection it had (the paths are still owned) and gains no invented
 // lineage. Both shapes parse; only v2 is written.
+//
+// "v" is READ and validated, not just written: a v3 record shape parsed by these v2
+// rules would yield a plausible-but-partial ledger, and the next save would overwrite
+// the v3 blob with that truncation. A version above kLedgerVersion is therefore its
+// own degraded status, never a Loaded ledger.
 
 namespace reasampler::tracking {
 
 namespace {
 
-// Any leading '/' or '\' (POSIX root / UNC), or a leading <alpha>: (Windows drive,
-// incl. drive-relative "C:foo"). Must match bank_model's rejection exactly — the
-// ledger holds the same kind of path as Sample.relativePath.
-bool isAbsolutePath(const std::string& p) {
-    if (p.empty()) return false;
-    if (p[0] == '/' || p[0] == '\\') return true;
-    if (p.size() >= 2 && std::isalpha(static_cast<unsigned char>(p[0])) && p[1] == ':')
-        return true;
-    return false;
-}
+using util::isAbsolutePath;
+
+// The version serialize() writes, and the highest this build can read.
+constexpr int kLedgerVersion = 2;
 
 // An unrecognized persisted integer degrades to Unknown rather than failing the
 // parse: a newer channel's kind must not make the whole ledger unreadable, which
@@ -75,7 +75,7 @@ std::vector<std::string> OriginLedger::ownedPaths() const {
 }
 
 std::string OriginLedger::serialize() const {
-    std::string out = "{\"v\":2,\"records\":[";
+    std::string out = "{\"v\":" + json::numToStr(kLedgerVersion) + ",\"records\":[";
     for (std::size_t i = 0; i < records_.size(); ++i) {
         if (i) out += ',';
         const OriginRecord& r = records_[i];
@@ -139,14 +139,18 @@ bool parseRecordArray(json::Reader& r, OriginLedger& out) {
     }
 }
 
-bool parseLedger(json::Reader& r, OriginLedger& out) {
+// `versionOut` stays 0 when no "v" key was present — the legacy path-only shape, or
+// an empty object. Read regardless of key order, so it is validated after the close.
+bool parseLedger(json::Reader& r, OriginLedger& out, int& versionOut) {
     if (!r.consume('{')) return false;
     r.skipWs();
     if (r.consume('}')) return true;
     for (;;) {
         std::string key;
         if (!r.parseKey(key)) return false;
-        if (key == "records") {
+        if (key == "v") {
+            if (!r.parseInt(versionOut)) return false;
+        } else if (key == "records") {
             if (!parseRecordArray(r, out)) return false;
         } else if (key == "owned") {
             std::vector<std::string> paths;
@@ -162,25 +166,41 @@ bool parseLedger(json::Reader& r, OriginLedger& out) {
     }
 }
 
+enum class ParseOutcome { Ok, Malformed, FutureVersion };
+
+// The one parse; the two public entry points differ only in how much of the outcome
+// they can express. `out` is written only on Ok — never a partial ledger.
+ParseOutcome parseStored(const std::string& text, OriginLedger& out) {
+    OriginLedger parsed;
+    int version = 0;
+    ::reasampler::json::Reader r(text);
+    if (!parseLedger(r, parsed, version)) return ParseOutcome::Malformed;
+    // Trailing garbage means the blob is not what it claims; accepting it would turn
+    // a detectably-corrupt value into a silently-partial ledger.
+    r.skipWs();
+    if (!r.eof()) return ParseOutcome::Malformed;
+    if (version < 0) return ParseOutcome::Malformed;
+    if (version > kLedgerVersion) return ParseOutcome::FutureVersion;
+    out = std::move(parsed);
+    return ParseOutcome::Ok;
+}
+
 }  // namespace
 
 std::optional<OriginLedger> OriginLedger::deserialize(const std::string& json) {
     OriginLedger ledger;
-    ::reasampler::json::Reader r(json);
-    if (!parseLedger(r, ledger)) return std::nullopt;
+    if (parseStored(json, ledger) != ParseOutcome::Ok) return std::nullopt;
     return ledger;
 }
 
 LedgerLoad loadLedger(const std::string& stored) {
     LedgerLoad load;
     if (stored.empty()) return load;  // absent key -> Fresh, not an error
-    std::optional<OriginLedger> parsed = OriginLedger::deserialize(stored);
-    if (!parsed) {
-        load.status = LedgerStatus::Unreadable;
-        return load;
+    switch (parseStored(stored, load.ledger)) {
+        case ParseOutcome::Ok:            load.status = LedgerStatus::Loaded; break;
+        case ParseOutcome::Malformed:     load.status = LedgerStatus::Unreadable; break;
+        case ParseOutcome::FutureVersion: load.status = LedgerStatus::FutureVersion; break;
     }
-    load.status = LedgerStatus::Loaded;
-    load.ledger = std::move(*parsed);
     return load;
 }
 

src/core/tracking/origin_ledger.h

@@ -2,10 +2,7 @@
 // origin_ledger — the one persisted record family behind file tracking: for every
 // file the system itself created, what act created it and what it derives from.
 // Supersedes the path-only owned manifest (same ext-state key, widened shape).
-//
-// Lineage is written at birth and never backfilled — record() on a path already
-// present is a no-op, so a record says what was known when the file appeared or
-// says nothing at all. Nothing here decides anything; tracking_authority does.
+// Nothing here decides anything; tracking_authority does.
 
 #include <cstddef>
 #include <optional>
@@ -32,7 +29,11 @@ enum class OriginKind {
 struct OriginRecord {
     std::string relativePath;
     OriginKind  kind = OriginKind::Unknown;
-    std::string sampleId;        // bank id minted at birth; "" when never recorded
+    // The bank id as of birth; "" when never recorded. NOT unique across records: a
+    // recapture writes a new file under the SAME bank id, so the ledger legitimately
+    // holds two records sharing a sampleId with different paths. `relativePath` is
+    // the only key; a by-sampleId lookup is ambiguous by construction.
+    std::string sampleId;
     std::string parentSampleId;  // the capture this derives from; "" = root / none
 
     bool operator==(const OriginRecord& o) const;
@@ -77,7 +78,8 @@ public:
     bool operator==(const OriginLedger& o) const { return records_ == o.records_; }
 
     // Lossless round-trip: deserialize(serialize(x)) == x. std::nullopt on malformed
-    // input — the caller must treat that as unreadable, never as empty.
+    // input AND on a blob whose "v" is newer than this build can read — the caller
+    // must treat either as unreadable, never as empty. loadLedger tells them apart.
     std::string serialize() const;
     static std::optional<OriginLedger> deserialize(const std::string& json);
 
@@ -85,11 +87,19 @@ private:
     std::vector<OriginRecord> records_;
 };
 
-// The three states of a stored ledger, kept apart because never-recorded and
-// unreadable demand opposite treatment: `Fresh` is a legitimate empty (a new
-// project, or a bank predating the ledger) and blocks nothing; `Unreadable` is a
-// present-but-corrupt blob and must block every destructive answer.
-enum class LedgerStatus { Fresh, Loaded, Unreadable };
+// The states of a stored ledger. `Fresh` (absent key) and `Loaded` are the two
+// usable ones. The other two are distinguished only so the operator gets the right
+// recovery advice — clearing a corrupt blob is repair, clearing a newer build's blob
+// is destruction.
+enum class LedgerStatus { Fresh, Loaded, Unreadable, FutureVersion };
+
+// The one degraded-state test, deliberately serving BOTH axes — "may I answer a
+// destructive question?" and "may I overwrite the stored blob?" — because today
+// every degraded status answers no to both. Split them only when a status needs one
+// without the other.
+inline bool ledgerDegraded(LedgerStatus s) {
+    return s == LedgerStatus::Unreadable || s == LedgerStatus::FutureVersion;
+}
 
 struct LedgerLoad {
     LedgerStatus status = LedgerStatus::Fresh;

src/core/tracking/tracking_authority.cpp

@@ -14,20 +14,22 @@ ProtectionAnswer pruneProtection(const TrackingState& state) {
         answer.unreadableUsageKeys = state.usage.offendingKeys;
     }
 
-    if (state.ledgerStatus == LedgerStatus::Unreadable) {
+    if (ledgerDegraded(state.ledgerStatus)) {
         answer.blocked = true;
-        answer.ledgerUnreadable = true;
-        return answer;  // ownedPaths left empty -> (owned ∩ present) is empty
+        answer.ledgerUnreadable = state.ledgerStatus == LedgerStatus::Unreadable;
+        answer.ledgerFutureVersion = state.ledgerStatus == LedgerStatus::FutureVersion;
     }
 
-    answer.ownedPaths = state.ledger.ownedPaths();
+    // Populated only on a clean answer: on ANY block an empty `owned` makes
+    // (owned ∩ present) empty, so even a caller that ignored `blocked` deletes nothing.
+    if (!answer.blocked) answer.ownedPaths = state.ledger.ownedPaths();
     return answer;
 }
 
 Answer tiedUsageExists(const TrackingState& state, const std::string& capturePath,
                        const std::string& ownUsageKey) {
     if (capturePath.empty()) return Answer::Indeterminate;
-    if (state.ledgerStatus == LedgerStatus::Unreadable) return Answer::Indeterminate;
+    if (ledgerDegraded(state.ledgerStatus)) return Answer::Indeterminate;
     if (state.usage.abortPrune) return Answer::Indeterminate;
 
     for (const wire::CountedUsage& counted : state.usage.counted) {

src/core/tracking/tracking_authority.h

@@ -26,12 +26,13 @@ struct TrackingState {
 // blocker fields say what to tell the operator; the caller composes the message
 // (the pure core does not know ext-state key spellings).
 //
-// Both path lists stay populated on a block as belt-and-braces: heldPaths carries
-// the usage fold's protect-all set, and ownedPaths is left EMPTY on an unreadable
-// ledger, so a caller that ignored `blocked` still computes an empty orphan set.
+// Belt-and-braces on ANY block, whichever side blocked: heldPaths still carries the
+// usage fold's protect-all set, and ownedPaths is left EMPTY — so a caller that
+// ignored `blocked` still computes an empty orphan set rather than deleting.
 struct ProtectionAnswer {
     bool blocked = false;
-    bool ledgerUnreadable = false;
+    bool ledgerUnreadable = false;    // corrupt blob — repairable by clearing the key
+    bool ledgerFutureVersion = false; // written by a newer build — must NOT be cleared
     std::vector<std::string> unreadableUsageKeys;
     std::vector<std::string> heldPaths;   // union into prune's `referenced`
     std::vector<std::string> ownedPaths;  // prune's `owned`
@@ -46,13 +47,12 @@ enum class Answer { No, Yes, Indeterminate };
 
 // Does a usage tied to `capturePath` exist, other than the asking instance's own?
 //
-//   * unreadable ledger, or any unreadable usage record -> Indeterminate.
+//   * degraded ledger, or any unreadable usage record -> Indeterminate.
 //   * otherwise Yes iff some counting live record holds the path.
 //
 // `ownUsageKey` is the asking instance's own "rsusage_<guid>" key, excluded from the
 // scan so a bake does not see itself; empty counts every holder. A `unioned` record
-// is NEVER excluded — it carries more than one incarnation's holds, so attributing
-// it to a single owner could hide a sibling's tie.
+// is never excluded.
 //
 // Never-recorded (no ledger record for the path) is a definite answer, not an
 // abstention: a pre-existing capture nothing holds answers No.