tracking: one ledger, one authority — prune protection and replace-vs-add answered from the same records, fail-safe on unreadable state

src/core/tracking/CLAUDE.md

@@ -0,0 +1,76 @@
+# src/core/tracking — the consolidated file-tracking system
+
+## Scope
+
+The one system behind every question about a file the tool created: who created it,
+what it derives from, who is using it now. Pure (REAPER-free, unit-tested outside the
+DAW). Two safety-critical consumers read it and no others: prune's protected set and
+the resample's replace-vs-add decision.
+
+This is the territory's **authority**, not one of three mechanisms. Two codecs feed it
+from where codecs belong — the persisted ledger's own JSON (here) and the
+instrument→extension usage wire (`core/wire/sample_usage`) — but every *decision* is
+made in `tracking_authority`, from one `TrackingState`.
+
+## Invariants
+
+**Safety-critical, overriding.** This territory gates the system's only file-deletion
+authority (prune) and its only capture-replacement act (resample). A tracking error
+loses a user's audio. Every failure, ambiguity, or unreadable input resolves to the
+non-destructive side of the question being asked — over-protection (prune skips a
+reclaimable file, or refuses to run; resample adds instead of replacing) is an
+accepted residual; under-protection is a data-loss bug.
+
+**No silent gaps.** A system-created file is tracked from the instant it exists.
+`ReaSamplerSession::recordCreated` is the only writer, called at the same point the
+`Sample` is added, and it reads lineage off that `Sample`'s own provenance — so the
+recipe fingerprint and the lineage record come from one act and cannot disagree.
+
+**Lineage is never backfilled.** `OriginLedger::record` on a path already present is
+`AlreadyPresent` and leaves the stored record untouched. A record says what was known
+at birth or says nothing at all; nothing may later rewrite history from a guess.
+
+**Never-recorded and unreadable are different absences.** `LedgerStatus` keeps them
+apart, and only `loadLedger` can tell them apart (an empty stored value is not valid
+JSON, so the parser alone cannot). `Fresh` — a new project or a bank predating the
+ledger — blocks nothing and yields definite answers. `Unreadable` blocks every
+destructive answer AND suppresses the next write, so a corrupt blob survives for
+recovery instead of being replaced by a ledger missing every earlier file.
+
+**Consumers cannot disagree.** Both answers come out of one `TrackingState`. The two
+universes differ deliberately — prune protects bank-referenced paths, ledger-owned
+paths, and every live hold; the tie query counts only live holds other than the
+asker's own — but the replace-vs-add universe is a **strict subset** of the
+prune-protection universe, proven in `test_tracking_authority`.
+
+**Conservatism is asymmetric by facet.** The recipe fingerprint (`core/model/provenance`)
+keeps its record-nothing-when-ambiguous stance: an ambiguous parent is no parent. The
+lineage half has no record-nothing option — replace-vs-add must be computable — so a
+birth record is written for every system-created file, ambiguous parentage or not
+(the parent is simply empty).
+
+## Modules
+
+- `origin_ledger` — the record family: `OriginRecord` (project-relative path, `OriginKind`,
+  the sample id minted at birth, the parent sample id) and the insertion-ordered,
+  path-keyed, deduplicated `OriginLedger` that holds them, with its JSON codec and the
+  `loadLedger` three-way `Fresh` / `Loaded` / `Unreadable` classification. Persisted
+  under the FOREVER-STABLE `owned_files` ext-state key; the legacy path-only shape
+  (`{"owned":[...]}`) lifts in as `Unknown`-kind records with no lineage.
+- `tracking_authority` — the one decision surface: `pruneProtection` (the `owned` and
+  held-path inputs prune's set algebra consumes, plus the blocked/blockers verdict)
+  and `tiedUsageExists` (`Yes` / `No` / `Indeterminate`, per capture, excluding the
+  asker's own usage key). Both read one borrowed `TrackingState`.
+
+## Gotchas
+
+- `OriginKind` values are PERSISTED INTEGERS — never renumber, only append. An
+  unrecognized value degrades to `Unknown` rather than failing the parse: a vocabulary
+  gap must not halt the prune.
+- A `unioned` usage record can never be excluded as "my own" — it carries more than one
+  incarnation's holds, so attributing it to a single owner could hide a sibling's tie.
+- The set algebra prune runs on `(owned ∩ present) − referenced` is `core/reclaim`'s,
+  not this directory's; this directory supplies two of its three inputs.
+- `sample_usage` deliberately stays in `core/wire` — it is a wire format, and its
+  instrument-side writer needs it there. Consolidation is of the *decisions*, not of
+  the codecs.

src/core/tracking/CMakeLists.txt

@@ -0,0 +1,10 @@
+reasampler_pure_library(origin_ledger SOURCES origin_ledger.cpp LINK PRIVATE json)
+reasampler_test(origin_ledger LINK origin_ledger json)
+
+# prune_reconcile + bank_book are linked into the authority's test, not the library:
+# the test proves end-to-end that a tied usage can never reach the orphan set, which
+# is the "consumers cannot disagree" property stated at the pure layer.
+reasampler_pure_library(tracking_authority
+    SOURCES tracking_authority.cpp
+    LINK PUBLIC origin_ledger sample_usage)
+reasampler_test(tracking_authority LINK tracking_authority prune_reconcile bank_book)

src/core/tracking/origin_ledger.cpp

@@ -0,0 +1,187 @@
+#include "core/tracking/origin_ledger.h"
+
+#include <cctype>
+
+#include "core/json/json.h"
+
+// Version ladder for the stored blob, under the FOREVER-STABLE "owned_files" key:
+//
+//   v1 (legacy, path-only)  {"owned":["reasampler_bank/a.wav"]}
+//   v2 (current)            {"v":2,"records":[{"path":"...","kind":1,
+//                                              "sample":"id","parent":"pid"}]}
+//
+// v1 blobs lift to v2 records with kind Unknown and empty ids — a pre-existing bank
+// keeps every protection it had (the paths are still owned) and gains no invented
+// lineage. Both shapes parse; only v2 is written.
+
+namespace reasampler::tracking {
+
+namespace {
+
+// Any leading '/' or '\' (POSIX root / UNC), or a leading <alpha>: (Windows drive,
+// incl. drive-relative "C:foo"). Must match bank_model's rejection exactly — the
+// ledger holds the same kind of path as Sample.relativePath.
+bool isAbsolutePath(const std::string& p) {
+    if (p.empty()) return false;
+    if (p[0] == '/' || p[0] == '\\') return true;
+    if (p.size() >= 2 && std::isalpha(static_cast<unsigned char>(p[0])) && p[1] == ':')
+        return true;
+    return false;
+}
+
+// An unrecognized persisted integer degrades to Unknown rather than failing the
+// parse: a newer channel's kind must not make the whole ledger unreadable, which
+// would halt prune on nothing worse than a vocabulary gap.
+OriginKind kindFromInt(int v) {
+    switch (v) {
+        case 1: return OriginKind::Capture;
+        case 2: return OriginKind::Ingest;
+        case 3: return OriginKind::Recapture;
+        case 4: return OriginKind::Resample;
+        default: return OriginKind::Unknown;
+    }
+}
+
+}  // namespace
+
+bool OriginRecord::operator==(const OriginRecord& o) const {
+    return relativePath == o.relativePath && kind == o.kind &&
+           sampleId == o.sampleId && parentSampleId == o.parentSampleId;
+}
+
+RecordResult OriginLedger::record(const OriginRecord& rec) {
+    if (rec.relativePath.empty()) return RecordResult::RejectedEmptyPath;
+    if (isAbsolutePath(rec.relativePath)) return RecordResult::RejectedAbsolutePath;
+    if (contains(rec.relativePath)) return RecordResult::AlreadyPresent;
+    records_.push_back(rec);
+    return RecordResult::Recorded;
+}
+
+const OriginRecord* OriginLedger::find(const std::string& relativePath) const {
+    for (const OriginRecord& r : records_)
+        if (r.relativePath == relativePath) return &r;
+    return nullptr;
+}
+
+bool OriginLedger::contains(const std::string& relativePath) const {
+    return find(relativePath) != nullptr;
+}
+
+std::vector<std::string> OriginLedger::ownedPaths() const {
+    std::vector<std::string> out;
+    out.reserve(records_.size());
+    for (const OriginRecord& r : records_) out.push_back(r.relativePath);
+    return out;
+}
+
+std::string OriginLedger::serialize() const {
+    std::string out = "{\"v\":2,\"records\":[";
+    for (std::size_t i = 0; i < records_.size(); ++i) {
+        if (i) out += ',';
+        const OriginRecord& r = records_[i];
+        out += "{\"path\":";
+        json::writeEscaped(out, r.relativePath);
+        out += ",\"kind\":" + json::numToStr(static_cast<int>(r.kind));
+        out += ",\"sample\":";
+        json::writeEscaped(out, r.sampleId);
+        out += ",\"parent\":";
+        json::writeEscaped(out, r.parentSampleId);
+        out += '}';
+    }
+    out += "]}";
+    return out;
+}
+
+namespace {
+
+bool parseRecord(json::Reader& r, OriginRecord& out) {
+    if (!r.consume('{')) return false;
+    r.skipWs();
+    if (r.consume('}')) return true;
+    for (;;) {
+        std::string key;
+        if (!r.parseKey(key)) return false;
+        if (key == "path") {
+            if (!r.parseString(out.relativePath)) return false;
+        } else if (key == "kind") {
+            int k = 0;
+            if (!r.parseInt(k)) return false;
+            out.kind = kindFromInt(k);
+        } else if (key == "sample") {
+            if (!r.parseString(out.sampleId)) return false;
+        } else if (key == "parent") {
+            if (!r.parseString(out.parentSampleId)) return false;
+        } else if (!r.skipValue()) {
+            return false;
+        }
+        r.skipWs();
+        if (r.consume(',')) continue;
+        if (r.consume('}')) return true;
+        return false;
+    }
+}
+
+bool parseRecordArray(json::Reader& r, OriginLedger& out) {
+    if (!r.consume('[')) return false;
+    r.skipWs();
+    if (r.consume(']')) return true;
+    for (;;) {
+        OriginRecord rec;
+        if (!parseRecord(r, rec)) return false;
+        // Through record() so the persisted invariants (dedup, reject empty/absolute)
+        // are re-asserted on load: a hand-edited or corrupt blob cannot smuggle an
+        // absolute or duplicate path in.
+        out.record(rec);
+        r.skipWs();
+        if (r.consume(',')) continue;
+        if (r.consume(']')) return true;
+        return false;
+    }
+}
+
+bool parseLedger(json::Reader& r, OriginLedger& out) {
+    if (!r.consume('{')) return false;
+    r.skipWs();
+    if (r.consume('}')) return true;
+    for (;;) {
+        std::string key;
+        if (!r.parseKey(key)) return false;
+        if (key == "records") {
+            if (!parseRecordArray(r, out)) return false;
+        } else if (key == "owned") {
+            std::vector<std::string> paths;
+            if (!r.parseStringArray(paths)) return false;
+            for (const std::string& p : paths) out.record(OriginRecord{p});
+        } else if (!r.skipValue()) {
+            return false;
+        }
+        r.skipWs();
+        if (r.consume(',')) continue;
+        if (r.consume('}')) return true;
+        return false;
+    }
+}
+
+}  // namespace
+
+std::optional<OriginLedger> OriginLedger::deserialize(const std::string& json) {
+    OriginLedger ledger;
+    ::reasampler::json::Reader r(json);
+    if (!parseLedger(r, ledger)) return std::nullopt;
+    return ledger;
+}
+
+LedgerLoad loadLedger(const std::string& stored) {
+    LedgerLoad load;
+    if (stored.empty()) return load;  // absent key -> Fresh, not an error
+    std::optional<OriginLedger> parsed = OriginLedger::deserialize(stored);
+    if (!parsed) {
+        load.status = LedgerStatus::Unreadable;
+        return load;
+    }
+    load.status = LedgerStatus::Loaded;
+    load.ledger = std::move(*parsed);
+    return load;
+}
+
+}  // namespace reasampler::tracking

src/core/tracking/origin_ledger.h

@@ -0,0 +1,104 @@
+#pragma once
+// origin_ledger — the one persisted record family behind file tracking: for every
+// file the system itself created, what act created it and what it derives from.
+// Supersedes the path-only owned manifest (same ext-state key, widened shape).
+//
+// Lineage is written at birth and never backfilled — record() on a path already
+// present is a no-op, so a record says what was known when the file appeared or
+// says nothing at all. Nothing here decides anything; tracking_authority does.
+
+#include <cstddef>
+#include <optional>
+#include <string>
+#include <vector>
+
+namespace reasampler::tracking {
+
+// The act that created the file. `Unknown` is the never-recorded case — a record
+// lifted from a legacy path-only manifest, or one whose creator did not know.
+// PERSISTED AS INTEGERS: never renumber an existing value, only append.
+enum class OriginKind {
+    Unknown   = 0,
+    Capture   = 1,
+    Ingest    = 2,
+    Recapture = 3,  // regenerated in place from its recorded source recipe
+    Resample  = 4,  // baked from an instrument's own processing chain
+};
+
+// One system-created file's birth record. `relativePath` is the key and is ALWAYS
+// project-relative (the same invariant as Sample.relativePath); an absolute path is
+// rejected rather than relativized, because the pure model has no project root and
+// guessing one could point at the wrong file.
+struct OriginRecord {
+    std::string relativePath;
+    OriginKind  kind = OriginKind::Unknown;
+    std::string sampleId;        // bank id minted at birth; "" when never recorded
+    std::string parentSampleId;  // the capture this derives from; "" = root / none
+
+    bool operator==(const OriginRecord& o) const;
+    bool operator!=(const OriginRecord& o) const { return !(*this == o); }
+};
+
+// Outcome of a record(). The op reports what happened rather than silently mutating
+// on a bad request (mirrors BankModel::AddResult).
+enum class RecordResult {
+    Recorded,
+    AlreadyPresent,        // dedup no-op; the existing record is NOT overwritten
+    RejectedEmptyPath,
+    RejectedAbsolutePath,
+};
+
+// Insertion-ordered, path-keyed, deduplicated. Insertion order is preserved so
+// serialize() round-trips byte-identically (deterministic ext-state).
+//
+// NOT a mirror of the bank index: removing or moving an index entry leaves the
+// record alone. The ledger tracks files *created*; prune reconciles it against the
+// index later.
+class OriginLedger {
+public:
+    OriginLedger() = default;
+
+    // A path already present is AlreadyPresent and leaves the stored record
+    // untouched — the no-backfill rule, enforced here rather than at call sites.
+    RecordResult record(const OriginRecord& rec);
+
+    // nullptr when the path was never recorded. Distinguishing that from an
+    // unreadable ledger is the loader's job (see LedgerStatus).
+    const OriginRecord* find(const std::string& relativePath) const;
+    bool contains(const std::string& relativePath) const;
+
+    // Prune's `owned` input, in insertion order.
+    std::vector<std::string> ownedPaths() const;
+
+    const std::vector<OriginRecord>& records() const { return records_; }
+    std::size_t size() const { return records_.size(); }
+    bool empty() const { return records_.empty(); }
+
+    bool operator==(const OriginLedger& o) const { return records_ == o.records_; }
+
+    // Lossless round-trip: deserialize(serialize(x)) == x. std::nullopt on malformed
+    // input — the caller must treat that as unreadable, never as empty.
+    std::string serialize() const;
+    static std::optional<OriginLedger> deserialize(const std::string& json);
+
+private:
+    std::vector<OriginRecord> records_;
+};
+
+// The three states of a stored ledger, kept apart because never-recorded and
+// unreadable demand opposite treatment: `Fresh` is a legitimate empty (a new
+// project, or a bank predating the ledger) and blocks nothing; `Unreadable` is a
+// present-but-corrupt blob and must block every destructive answer.
+enum class LedgerStatus { Fresh, Loaded, Unreadable };
+
+struct LedgerLoad {
+    LedgerStatus status = LedgerStatus::Fresh;
+    OriginLedger ledger;  // empty unless status == Loaded
+};
+
+// Classifies a raw stored value. An empty string is the absent key (Fresh), not an
+// error — an empty string is not valid JSON, so the two cases cannot be told apart
+// by the parser alone.
+LedgerLoad loadLedger(const std::string& stored);
+
+}  // namespace reasampler::tracking

src/core/tracking/tracking_authority.cpp

@@ -0,0 +1,43 @@
+#include "core/tracking/tracking_authority.h"
+
+namespace reasampler::tracking {
+
+ProtectionAnswer pruneProtection(const TrackingState& state) {
+    ProtectionAnswer answer;
+
+    // heldPaths is taken unconditionally: on an abort the usage fold returns its
+    // protect-all set, which is the widest (safest) answer available.
+    answer.heldPaths = state.usage.heldPaths;
+
+    if (state.usage.abortPrune) {
+        answer.blocked = true;
+        answer.unreadableUsageKeys = state.usage.offendingKeys;
+    }
+
+    if (state.ledgerStatus == LedgerStatus::Unreadable) {
+        answer.blocked = true;
+        answer.ledgerUnreadable = true;
+        return answer;  // ownedPaths left empty -> (owned ∩ present) is empty
+    }
+
+    answer.ownedPaths = state.ledger.ownedPaths();
+    return answer;
+}
+
+Answer tiedUsageExists(const TrackingState& state, const std::string& capturePath,
+                       const std::string& ownUsageKey) {
+    if (capturePath.empty()) return Answer::Indeterminate;
+    if (state.ledgerStatus == LedgerStatus::Unreadable) return Answer::Indeterminate;
+    if (state.usage.abortPrune) return Answer::Indeterminate;
+
+    for (const wire::CountedUsage& counted : state.usage.counted) {
+        const bool isOwn = !ownUsageKey.empty() && counted.key == ownUsageKey &&
+                           !counted.record.unioned;
+        if (isOwn) continue;
+        for (const wire::UsageHold& hold : counted.record.holds)
+            if (hold.relativePath == capturePath) return Answer::Yes;
+    }
+    return Answer::No;
+}
+
+}  // namespace reasampler::tracking

src/core/tracking/tracking_authority.h

@@ -0,0 +1,65 @@
+#pragma once
+// tracking_authority — the ONE place the two safety-critical consumers are answered:
+// prune's protected set and the resample's replace-vs-add decision. Both read the
+// same TrackingState, so they cannot drift apart; every unreadable or ambiguous
+// input resolves to the non-destructive side of its own question.
+
+#include <string>
+#include <vector>
+
+#include "core/tracking/origin_ledger.h"
+#include "core/wire/sample_usage.h"
+
+namespace reasampler::tracking {
+
+// A borrowed view of everything both consumers read, gathered once by the shell.
+// References, not values: the ledger can hold thousands of records and both answers
+// are computed from one gather. Never outlives the gather that built it.
+struct TrackingState {
+    LedgerStatus                 ledgerStatus;
+    const OriginLedger&          ledger;
+    const wire::UsageFoldResult& usage;
+};
+
+// Prune's answer. `blocked` means the protected set is unknowable and the prune must
+// HALT — deleting with degraded protection is the data-loss direction. The two
+// blocker fields say what to tell the operator; the caller composes the message
+// (the pure core does not know ext-state key spellings).
+//
+// Both path lists stay populated on a block as belt-and-braces: heldPaths carries
+// the usage fold's protect-all set, and ownedPaths is left EMPTY on an unreadable
+// ledger, so a caller that ignored `blocked` still computes an empty orphan set.
+struct ProtectionAnswer {
+    bool blocked = false;
+    bool ledgerUnreadable = false;
+    std::vector<std::string> unreadableUsageKeys;
+    std::vector<std::string> heldPaths;   // union into prune's `referenced`
+    std::vector<std::string> ownedPaths;  // prune's `owned`
+};
+
+ProtectionAnswer pruneProtection(const TrackingState& state);
+
+// The resample's answer, per capture. `Indeterminate` is not a failure to compute —
+// it is the recorded verdict that the state could not be read, and the caller must
+// treat it exactly as `Yes` (never take the replace branch).
+enum class Answer { No, Yes, Indeterminate };
+
+// Does a usage tied to `capturePath` exist, other than the asking instance's own?
+//
+//   * unreadable ledger, or any unreadable usage record -> Indeterminate.
+//   * otherwise Yes iff some counting live record holds the path.
+//
+// `ownUsageKey` is the asking instance's own "rsusage_<guid>" key, excluded from the
+// scan so a bake does not see itself; empty counts every holder. A `unioned` record
+// is NEVER excluded — it carries more than one incarnation's holds, so attributing
+// it to a single owner could hide a sibling's tie.
+//
+// Never-recorded (no ledger record for the path) is a definite answer, not an
+// abstention: a pre-existing capture nothing holds answers No.
+//
+// The universes cannot disagree: a Yes implies `capturePath` is in the same
+// pruneProtection(state).heldPaths, which is itself a subset of what prune protects.
+Answer tiedUsageExists(const TrackingState& state, const std::string& capturePath,
+                       const std::string& ownUsageKey);
+
+}  // namespace reasampler::tracking