diff --git a/docs/TODO.md b/docs/TODO.md index 126ab84..5f904dc 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -165,3 +165,17 @@ Forward-looking follow-ups. Deferred by decision, not oversight — each entry r **Priority / risk.** Not stated as a priority level; the source characterizes this as a "panel-polish detail." **Done looks like.** Not stated in the source beyond choosing one of the three placement options. + +## A realtime capture interrupted by a project switch leaves an untracked file behind + +**Context (found by the tracking-consolidation review, 2026-07-30).** `DriveRealtimeCapture` detects that the active project is no longer the one the in-flight capture belongs to, aborts the backend, and drops the handle. On a `Done` abort the backend has *already* moved the recorded WAV into the **original** project's bank folder (`capture_realtime_finalize`), so a file the tool created exists with no bank entry and no ledger record. + +**The wart.** This is the one hole in "no silent gaps": a system-created file that is never recorded. It is in the safe direction — an untracked file is foreign, so prune will never reclaim it — but it is permanent, and the bank folder grows by one orphan per interrupted record. + +**Intended fix.** Record the birth against the project the capture belongs to. Neither half is available at the switch point: `session`'s ledger and `saveToActiveProject` both target the *active* project, which is by definition the wrong one here. + +**The constraint the fix MUST handle.** Writing the record into the now-active project would attribute another project's file to it — a worse error than the gap, since prune would then consider deleting a file it does not own the folder for. Deleting the stranded file instead was considered and rejected: it is the user's just-recorded audio, and prune is the system's only deletion authority over bank-folder bytes (`shell/persist/CLAUDE.md`) — a shell self-cleanup exemption covers transient scratch, not a finished recording. The fix therefore needs a deferred write against a *named* project (or a re-entry into the original project on the next poll), not a change at the abort site. + +**Priority / risk.** Low / deferred. Mitigated in the meantime: the console message names the stranded file's project-relative path, so the operator can recover or remove it rather than discovering it later as an unexplained orphan. + +**Done looks like.** Switching projects mid-record leaves the recorded file with a ledger record in the project it belongs to, so a later prune of that project can reclaim it normally. diff --git a/src/core/json/CLAUDE.md b/src/core/json/CLAUDE.md index be2242f..d0e4b27 100644 --- a/src/core/json/CLAUDE.md +++ b/src/core/json/CLAUDE.md @@ -9,12 +9,12 @@ has — stay in the consumers; this module owns lexing/emitting only. ## Modules -- `json` (`core/json`) — the ONE hand-rolled JSON lexical layer (Q-W1): string/number/bool/null tokens, the scoped object `Writer`, and the bounds-checked `Reader` cursor, byte-compatible with the five pre-extraction per-module writers it replaced (`bank_model` / `bank_book` / `view_mode_model` / `owned_manifest` / `tail_control`). Domain grammars stay in the consumers; this owns lexing/emitting only. +- `json` (`core/json`) — the ONE hand-rolled JSON lexical layer (Q-W1): string/number/bool/null tokens, the scoped object `Writer`, and the bounds-checked `Reader` cursor, byte-compatible with the five pre-extraction per-module writers it replaced (`bank_model` / `bank_book` / `view_mode_model` / the retired `owned_manifest`, now `core/tracking/origin_ledger` / `tail_control`). Domain grammars stay in the consumers; this owns lexing/emitting only. ## Gotchas - Byte-compatible with the five pre-extraction per-module writers it replaced - (`bank_model` / `bank_book` / `view_mode_model` / `owned_manifest` / + (`bank_model` / `bank_book` / `view_mode_model` / the retired `owned_manifest` / `tail_control`) — a change here risks silently breaking round-trip compatibility with ext-state blobs already persisted by projects written before the Q-W1 extraction. diff --git a/src/core/json/json.h b/src/core/json/json.h index 54c292b..1b4c869 100644 --- a/src/core/json/json.h +++ b/src/core/json/json.h @@ -7,7 +7,7 @@ // unknown-value skipping, and the emit side (escaping, %.17g/%d/%lld rendering, // the scoped object writer). Domain grammars — which keys exist, what shape each // value takes — stay in the consumers (bank_model, bank_book, view_mode_model, -// owned_manifest, tail_control). +// origin_ledger, tail_control). // // Byte-compatibility contract (load-bearing): the emit helpers reproduce the prior // per-module writers EXACTLY — writeEscaped's escape set, %.17g for doubles diff --git a/src/core/model/bank_model.cpp b/src/core/model/bank_model.cpp index 93575f0..1de7b56 100644 --- a/src/core/model/bank_model.cpp +++ b/src/core/model/bank_model.cpp @@ -1,8 +1,7 @@ #include "core/model/bank_model.h" -#include - #include "core/json/json.h" +#include "core/util/relative_path.h" // bank_model implementation. JSON rides on the shared core/json lexical layer; // only the Sample/index DOMAIN grammar lives here. Doubles are emitted with 17 @@ -44,22 +43,7 @@ bool Sample::operator==(const Sample& o) const { provenance == o.provenance && createdTimestamp == o.createdTimestamp; } -// -- path invariant ------------------------------------------------------- - -// Rejects absolute paths rather than normalizing them: the pure model has no -// knowledge of the project root, so any "normalization" would be a guess that -// could point at the wrong file. Covers POSIX ("/x"), Windows drive ("C:\x", -// "C:/x", "C:foo" drive-relative), and UNC ("\\host\share") forms. Any leading -// : is rejected regardless of what follows — drive-relative paths -// ("C:foo.wav") resolve against the drive's current directory, not the project -// root, so they violate relative-paths-only just as much as "C:\foo.wav" does. -static bool isAbsolutePath(const std::string& p) { - if (p.empty()) return false; - if (p[0] == '/' || p[0] == '\\') return true; // POSIX root or UNC - if (p.size() >= 2 && std::isalpha(static_cast(p[0])) && p[1] == ':') - return true; // Windows drive (C:\, C:/, C:foo, C:) - return false; -} +using util::isAbsolutePath; // -- BankModel ------------------------------------------------------------ diff --git a/src/core/reclaim/CLAUDE.md b/src/core/reclaim/CLAUDE.md index b4ba092..2d91301 100644 --- a/src/core/reclaim/CLAUDE.md +++ b/src/core/reclaim/CLAUDE.md @@ -32,7 +32,7 @@ enumeration and the actual deletion live in `shell/persist` (`prune_fs`), not he ## Modules -- `prune_reconcile` — pure prune core: `pruneOrphans(present, referenced, owned)` computes `(owned ∩ present) − referenced`; the safety-critical "which files are orphans" decision, filesystem-free and hard-tested before any I/O exists. `mergeReferenced(bankRefs, heldPaths)` unions live instance holds into the referenced-set. Two of the three inputs (`owned`, and the held half of `referenced`) come from `core/tracking`'s authority, never assembled ad hoc by a shell. `PruneReport` carries the authority's verdict: `blockedByTracking` + `ledgerUnreadable` / `unreadableUsageKeys`. +- `prune_reconcile` — pure prune core: `pruneOrphans(present, referenced, owned)` computes `(owned ∩ present) − referenced`; the safety-critical "which files are orphans" decision, filesystem-free and hard-tested before any I/O exists. `mergeReferenced(bankRefs, heldPaths)` unions live instance holds into the referenced-set. Two of the three inputs (`owned`, and the held half of `referenced`) come from `core/tracking`'s authority, never assembled ad hoc by a shell. `PruneReport` carries the authority's verdict: `blockedByTracking` + `ledgerUnreadable` / `ledgerFutureVersion` / `unreadableUsageKeys`. ## Gotchas diff --git a/src/core/reclaim/prune_reconcile.h b/src/core/reclaim/prune_reconcile.h index 5cc6018..e3e4bf7 100644 --- a/src/core/reclaim/prune_reconcile.h +++ b/src/core/reclaim/prune_reconcile.h @@ -53,9 +53,11 @@ namespace reasampler::reclaim { // (count 0, empty list) and the prune must HALT — deleting with // degraded protection is the data-loss direction. Set by the scan // shell, never by buildPruneReport (which stays a pure tally). -// * ledgerUnreadable / unreadableUsageKeys — which side blocked, so the action can -// tell the operator what to recover. The key names are the exact -// "rsusage_" spellings. +// * ledgerUnreadable / ledgerFutureVersion / unreadableUsageKeys — which side +// blocked, so the action can tell the operator what to recover. The +// two ledger flags need opposite advice (a corrupt blob may be +// cleared; a newer build's blob must not be). The key names are the +// exact "rsusage_" spellings. struct PruneReport { std::size_t count = 0; std::uint64_t totalBytes = 0; @@ -63,6 +65,7 @@ struct PruneReport { bool truncated = false; bool blockedByTracking = false; bool ledgerUnreadable = false; + bool ledgerFutureVersion = false; std::vector unreadableUsageKeys; }; diff --git a/src/core/tracking/CLAUDE.md b/src/core/tracking/CLAUDE.md index 6947640..20f06be 100644 --- a/src/core/tracking/CLAUDE.md +++ b/src/core/tracking/CLAUDE.md @@ -21,21 +21,33 @@ non-destructive side of the question being asked — over-protection (prune skip reclaimable file, or refuses to run; resample adds instead of replacing) is an accepted residual; under-protection is a data-loss bug. -**No silent gaps.** A system-created file is tracked from the instant it exists. -`ReaSamplerSession::recordCreated` is the only writer, called at the same point the -`Sample` is added, and it reads lineage off that `Sample`'s own provenance — so the -recipe fingerprint and the lineage record come from one act and cannot disagree. +**No silent gaps — in memory at creation, on disk at the next save.** +`ReaSamplerSession::recordCreated` is the only writer, called in the same +straight-line block as the bank add with no I/O or early return between, so a +system-created file is tracked *in memory* the instant it exists. The record reaches +the `.rpp` only at the following `saveToActiveProject()`; a crash in that window, or a +whole session running with a degraded ledger (below), leaves the file untracked — +foreign, therefore never reclaimed, which is the safe direction. That residual is +accepted and stated here rather than papered over by a stronger claim. **Lineage is never backfilled.** `OriginLedger::record` on a path already present is `AlreadyPresent` and leaves the stored record untouched. A record says what was known at birth or says nothing at all; nothing may later rewrite history from a guess. +Consequence, and the reason both blobs can carry a parent id: `recordCreated` *seeds* +the record from the `Sample`'s own `provenance.parentSampleId` (one source, one act), +but the `Sample` is mutable via `updateSampleInPlace` and the record is not — so where +the two ever diverge, **the ledger record is authoritative for lineage** and the +`Sample`'s copy is only the recipe facet's view of it. -**Never-recorded and unreadable are different absences.** `LedgerStatus` keeps them +**Never-recorded and degraded are different absences.** `LedgerStatus` keeps them apart, and only `loadLedger` can tell them apart (an empty stored value is not valid JSON, so the parser alone cannot). `Fresh` — a new project or a bank predating the -ledger — blocks nothing and yields definite answers. `Unreadable` blocks every -destructive answer AND suppresses the next write, so a corrupt blob survives for -recovery instead of being replaced by a ledger missing every earlier file. +ledger — blocks nothing and yields definite answers. `Unreadable` (corrupt) and +`FutureVersion` (a `"v"` newer than this build reads) are the degraded pair: each +blocks every destructive answer AND suppresses the next write, so the stored blob +survives for recovery instead of being replaced by a ledger missing every earlier +file. `ledgerDegraded()` is the one test for both treatments — they coincide today, so +the two axes are deliberately not split. **Consumers cannot disagree.** Both answers come out of one `TrackingState`. The two universes differ deliberately — prune protects bank-referenced paths, ledger-owned @@ -54,9 +66,9 @@ birth record is written for every system-created file, ambiguous parentage or no - `origin_ledger` — the record family: `OriginRecord` (project-relative path, `OriginKind`, the sample id minted at birth, the parent sample id) and the insertion-ordered, path-keyed, deduplicated `OriginLedger` that holds them, with its JSON codec and the - `loadLedger` three-way `Fresh` / `Loaded` / `Unreadable` classification. Persisted - under the FOREVER-STABLE `owned_files` ext-state key; the legacy path-only shape - (`{"owned":[...]}`) lifts in as `Unknown`-kind records with no lineage. + `loadLedger` `Fresh` / `Loaded` / `Unreadable` / `FutureVersion` classification. + Persisted under the FOREVER-STABLE `owned_files` ext-state key; the legacy path-only + shape (`{"owned":[...]}`) lifts in as `Unknown`-kind records with no lineage. - `tracking_authority` — the one decision surface: `pruneProtection` (the `owned` and held-path inputs prune's set algebra consumes, plus the blocked/blockers verdict) and `tiedUsageExists` (`Yes` / `No` / `Indeterminate`, per capture, excluding the @@ -66,7 +78,11 @@ birth record is written for every system-created file, ambiguous parentage or no - `OriginKind` values are PERSISTED INTEGERS — never renumber, only append. An unrecognized value degrades to `Unknown` rather than failing the parse: a vocabulary - gap must not halt the prune. + gap must not halt the prune. This degrade-don't-halt rule is about *field + vocabulary* only — an unrecognized **document version** (`"v"`) does the opposite + and blocks, because a changed record shape makes the whole ledger untrustworthy. +- `OriginRecord.sampleId` is NOT unique: a recapture writes a new file under the same + bank id, so two records can share a `sampleId`. `relativePath` is the only key. - A `unioned` usage record can never be excluded as "my own" — it carries more than one incarnation's holds, so attributing it to a single owner could hide a sibling's tie. - The set algebra prune runs on `(owned ∩ present) − referenced` is `core/reclaim`'s, diff --git a/src/core/tracking/origin_ledger.cpp b/src/core/tracking/origin_ledger.cpp index 7fe971d..5e7067d 100644 --- a/src/core/tracking/origin_ledger.cpp +++ b/src/core/tracking/origin_ledger.cpp @@ -1,8 +1,9 @@ #include "core/tracking/origin_ledger.h" -#include +#include #include "core/json/json.h" +#include "core/util/relative_path.h" // Version ladder for the stored blob, under the FOREVER-STABLE "owned_files" key: // @@ -13,21 +14,20 @@ // v1 blobs lift to v2 records with kind Unknown and empty ids — a pre-existing bank // keeps every protection it had (the paths are still owned) and gains no invented // lineage. Both shapes parse; only v2 is written. +// +// "v" is READ and validated, not just written: a v3 record shape parsed by these v2 +// rules would yield a plausible-but-partial ledger, and the next save would overwrite +// the v3 blob with that truncation. A version above kLedgerVersion is therefore its +// own degraded status, never a Loaded ledger. namespace reasampler::tracking { namespace { -// Any leading '/' or '\' (POSIX root / UNC), or a leading : (Windows drive, -// incl. drive-relative "C:foo"). Must match bank_model's rejection exactly — the -// ledger holds the same kind of path as Sample.relativePath. -bool isAbsolutePath(const std::string& p) { - if (p.empty()) return false; - if (p[0] == '/' || p[0] == '\\') return true; - if (p.size() >= 2 && std::isalpha(static_cast(p[0])) && p[1] == ':') - return true; - return false; -} +using util::isAbsolutePath; + +// The version serialize() writes, and the highest this build can read. +constexpr int kLedgerVersion = 2; // An unrecognized persisted integer degrades to Unknown rather than failing the // parse: a newer channel's kind must not make the whole ledger unreadable, which @@ -75,7 +75,7 @@ std::vector OriginLedger::ownedPaths() const { } std::string OriginLedger::serialize() const { - std::string out = "{\"v\":2,\"records\":["; + std::string out = "{\"v\":" + json::numToStr(kLedgerVersion) + ",\"records\":["; for (std::size_t i = 0; i < records_.size(); ++i) { if (i) out += ','; const OriginRecord& r = records_[i]; @@ -139,14 +139,18 @@ bool parseRecordArray(json::Reader& r, OriginLedger& out) { } } -bool parseLedger(json::Reader& r, OriginLedger& out) { +// `versionOut` stays 0 when no "v" key was present — the legacy path-only shape, or +// an empty object. Read regardless of key order, so it is validated after the close. +bool parseLedger(json::Reader& r, OriginLedger& out, int& versionOut) { if (!r.consume('{')) return false; r.skipWs(); if (r.consume('}')) return true; for (;;) { std::string key; if (!r.parseKey(key)) return false; - if (key == "records") { + if (key == "v") { + if (!r.parseInt(versionOut)) return false; + } else if (key == "records") { if (!parseRecordArray(r, out)) return false; } else if (key == "owned") { std::vector paths; @@ -162,25 +166,41 @@ bool parseLedger(json::Reader& r, OriginLedger& out) { } } +enum class ParseOutcome { Ok, Malformed, FutureVersion }; + +// The one parse; the two public entry points differ only in how much of the outcome +// they can express. `out` is written only on Ok — never a partial ledger. +ParseOutcome parseStored(const std::string& text, OriginLedger& out) { + OriginLedger parsed; + int version = 0; + ::reasampler::json::Reader r(text); + if (!parseLedger(r, parsed, version)) return ParseOutcome::Malformed; + // Trailing garbage means the blob is not what it claims; accepting it would turn + // a detectably-corrupt value into a silently-partial ledger. + r.skipWs(); + if (!r.eof()) return ParseOutcome::Malformed; + if (version < 0) return ParseOutcome::Malformed; + if (version > kLedgerVersion) return ParseOutcome::FutureVersion; + out = std::move(parsed); + return ParseOutcome::Ok; +} + } // namespace std::optional OriginLedger::deserialize(const std::string& json) { OriginLedger ledger; - ::reasampler::json::Reader r(json); - if (!parseLedger(r, ledger)) return std::nullopt; + if (parseStored(json, ledger) != ParseOutcome::Ok) return std::nullopt; return ledger; } LedgerLoad loadLedger(const std::string& stored) { LedgerLoad load; if (stored.empty()) return load; // absent key -> Fresh, not an error - std::optional parsed = OriginLedger::deserialize(stored); - if (!parsed) { - load.status = LedgerStatus::Unreadable; - return load; + switch (parseStored(stored, load.ledger)) { + case ParseOutcome::Ok: load.status = LedgerStatus::Loaded; break; + case ParseOutcome::Malformed: load.status = LedgerStatus::Unreadable; break; + case ParseOutcome::FutureVersion: load.status = LedgerStatus::FutureVersion; break; } - load.status = LedgerStatus::Loaded; - load.ledger = std::move(*parsed); return load; } diff --git a/src/core/tracking/origin_ledger.h b/src/core/tracking/origin_ledger.h index e7e81b7..f04264e 100644 --- a/src/core/tracking/origin_ledger.h +++ b/src/core/tracking/origin_ledger.h @@ -2,10 +2,7 @@ // origin_ledger — the one persisted record family behind file tracking: for every // file the system itself created, what act created it and what it derives from. // Supersedes the path-only owned manifest (same ext-state key, widened shape). -// -// Lineage is written at birth and never backfilled — record() on a path already -// present is a no-op, so a record says what was known when the file appeared or -// says nothing at all. Nothing here decides anything; tracking_authority does. +// Nothing here decides anything; tracking_authority does. #include #include @@ -32,7 +29,11 @@ enum class OriginKind { struct OriginRecord { std::string relativePath; OriginKind kind = OriginKind::Unknown; - std::string sampleId; // bank id minted at birth; "" when never recorded + // The bank id as of birth; "" when never recorded. NOT unique across records: a + // recapture writes a new file under the SAME bank id, so the ledger legitimately + // holds two records sharing a sampleId with different paths. `relativePath` is + // the only key; a by-sampleId lookup is ambiguous by construction. + std::string sampleId; std::string parentSampleId; // the capture this derives from; "" = root / none bool operator==(const OriginRecord& o) const; @@ -77,7 +78,8 @@ public: bool operator==(const OriginLedger& o) const { return records_ == o.records_; } // Lossless round-trip: deserialize(serialize(x)) == x. std::nullopt on malformed - // input — the caller must treat that as unreadable, never as empty. + // input AND on a blob whose "v" is newer than this build can read — the caller + // must treat either as unreadable, never as empty. loadLedger tells them apart. std::string serialize() const; static std::optional deserialize(const std::string& json); @@ -85,11 +87,19 @@ private: std::vector records_; }; -// The three states of a stored ledger, kept apart because never-recorded and -// unreadable demand opposite treatment: `Fresh` is a legitimate empty (a new -// project, or a bank predating the ledger) and blocks nothing; `Unreadable` is a -// present-but-corrupt blob and must block every destructive answer. -enum class LedgerStatus { Fresh, Loaded, Unreadable }; +// The states of a stored ledger. `Fresh` (absent key) and `Loaded` are the two +// usable ones. The other two are distinguished only so the operator gets the right +// recovery advice — clearing a corrupt blob is repair, clearing a newer build's blob +// is destruction. +enum class LedgerStatus { Fresh, Loaded, Unreadable, FutureVersion }; + +// The one degraded-state test, deliberately serving BOTH axes — "may I answer a +// destructive question?" and "may I overwrite the stored blob?" — because today +// every degraded status answers no to both. Split them only when a status needs one +// without the other. +inline bool ledgerDegraded(LedgerStatus s) { + return s == LedgerStatus::Unreadable || s == LedgerStatus::FutureVersion; +} struct LedgerLoad { LedgerStatus status = LedgerStatus::Fresh; diff --git a/src/core/tracking/tracking_authority.cpp b/src/core/tracking/tracking_authority.cpp index 190780a..62843c4 100644 --- a/src/core/tracking/tracking_authority.cpp +++ b/src/core/tracking/tracking_authority.cpp @@ -14,20 +14,22 @@ ProtectionAnswer pruneProtection(const TrackingState& state) { answer.unreadableUsageKeys = state.usage.offendingKeys; } - if (state.ledgerStatus == LedgerStatus::Unreadable) { + if (ledgerDegraded(state.ledgerStatus)) { answer.blocked = true; - answer.ledgerUnreadable = true; - return answer; // ownedPaths left empty -> (owned ∩ present) is empty + answer.ledgerUnreadable = state.ledgerStatus == LedgerStatus::Unreadable; + answer.ledgerFutureVersion = state.ledgerStatus == LedgerStatus::FutureVersion; } - answer.ownedPaths = state.ledger.ownedPaths(); + // Populated only on a clean answer: on ANY block an empty `owned` makes + // (owned ∩ present) empty, so even a caller that ignored `blocked` deletes nothing. + if (!answer.blocked) answer.ownedPaths = state.ledger.ownedPaths(); return answer; } Answer tiedUsageExists(const TrackingState& state, const std::string& capturePath, const std::string& ownUsageKey) { if (capturePath.empty()) return Answer::Indeterminate; - if (state.ledgerStatus == LedgerStatus::Unreadable) return Answer::Indeterminate; + if (ledgerDegraded(state.ledgerStatus)) return Answer::Indeterminate; if (state.usage.abortPrune) return Answer::Indeterminate; for (const wire::CountedUsage& counted : state.usage.counted) { diff --git a/src/core/tracking/tracking_authority.h b/src/core/tracking/tracking_authority.h index d93fef6..f26177a 100644 --- a/src/core/tracking/tracking_authority.h +++ b/src/core/tracking/tracking_authority.h @@ -26,12 +26,13 @@ struct TrackingState { // blocker fields say what to tell the operator; the caller composes the message // (the pure core does not know ext-state key spellings). // -// Both path lists stay populated on a block as belt-and-braces: heldPaths carries -// the usage fold's protect-all set, and ownedPaths is left EMPTY on an unreadable -// ledger, so a caller that ignored `blocked` still computes an empty orphan set. +// Belt-and-braces on ANY block, whichever side blocked: heldPaths still carries the +// usage fold's protect-all set, and ownedPaths is left EMPTY — so a caller that +// ignored `blocked` still computes an empty orphan set rather than deleting. struct ProtectionAnswer { bool blocked = false; - bool ledgerUnreadable = false; + bool ledgerUnreadable = false; // corrupt blob — repairable by clearing the key + bool ledgerFutureVersion = false; // written by a newer build — must NOT be cleared std::vector unreadableUsageKeys; std::vector heldPaths; // union into prune's `referenced` std::vector ownedPaths; // prune's `owned` @@ -46,13 +47,12 @@ enum class Answer { No, Yes, Indeterminate }; // Does a usage tied to `capturePath` exist, other than the asking instance's own? // -// * unreadable ledger, or any unreadable usage record -> Indeterminate. +// * degraded ledger, or any unreadable usage record -> Indeterminate. // * otherwise Yes iff some counting live record holds the path. // // `ownUsageKey` is the asking instance's own "rsusage_" key, excluded from the // scan so a bake does not see itself; empty counts every holder. A `unioned` record -// is NEVER excluded — it carries more than one incarnation's holds, so attributing -// it to a single owner could hide a sibling's tie. +// is never excluded. // // Never-recorded (no ledger record for the path) is a definite answer, not an // abstention: a pre-existing capture nothing holds answers No. diff --git a/src/core/util/CLAUDE.md b/src/core/util/CLAUDE.md index 1a8ee9f..f979495 100644 --- a/src/core/util/CLAUDE.md +++ b/src/core/util/CLAUDE.md @@ -3,12 +3,13 @@ ## Scope Tiny, dependency-free pure helpers linked by both artifacts: whole-file byte -loading and unit-interval clamping. +loading, unit-interval clamping, and the absolute-path rejection test. ## Modules - `file_bytes` (`core/util`) — the ONE whole-file byte loader (Q-W1), linked by both artifacts; blocking I/O, off-audio-thread only. - `clamp01` (`core/util`, header-only) — the ONE unit-interval clamp (Q-W1), replacing four per-module static copies; NaN passes through unchanged rather than collapsing to a bound. +- `relative_path` (`core/util`, header-only) — the ONE absolute-path rejection test behind the relative-paths-only invariant, shared by `bank_model` (`Sample.relativePath`) and `core/tracking/origin_ledger` (`OriginRecord.relativePath`). The two must reject identically or a path one accepts could be smuggled past the other; that is why it is one function and not two. ## Gotchas diff --git a/src/core/util/relative_path.h b/src/core/util/relative_path.h new file mode 100644 index 0000000..a13329a --- /dev/null +++ b/src/core/util/relative_path.h @@ -0,0 +1,26 @@ +#pragma once +// relative_path — the ONE absolute-path rejection test behind the +// relative-paths-only invariant, shared by every persisted path family +// (Sample.relativePath, OriginRecord.relativePath). +// +// Rejects rather than normalizes: the pure core has no knowledge of the project +// root, so any "normalization" would be a guess that could point at the wrong file. + +#include +#include + +namespace reasampler::util { + +// True for POSIX root ("/x"), UNC ("\\host\share"), and any leading : — +// including drive-RELATIVE forms ("C:foo.wav"), which resolve against the drive's +// current directory rather than the project root and so violate the invariant just +// as much as "C:\foo.wav" does. +inline bool isAbsolutePath(const std::string& p) { + if (p.empty()) return false; + if (p[0] == '/' || p[0] == '\\') return true; + if (p.size() >= 2 && std::isalpha(static_cast(p[0])) && p[1] == ':') + return true; + return false; +} + +} // namespace reasampler::util diff --git a/src/core/wire/CLAUDE.md b/src/core/wire/CLAUDE.md index 775dcc5..d6b3038 100644 --- a/src/core/wire/CLAUDE.md +++ b/src/core/wire/CLAUDE.md @@ -57,8 +57,7 @@ This directory owns two cross-artifact contracts specifically: protected (identity-failure net — a matcher failure must never degrade toward delete). The guarantee: a capture held by any live instance can never be deleted; if the prune cannot determine with certainty which captures are held, it aborts - entirely (deletes nothing). Over-protection is the accepted residual; under-protection - is a data-loss bug. The fold reports `counted` — the live records still attributed to + entirely (deletes nothing). The fold reports `counted` — the live records still attributed to their `rsusage_*` keys — because the flattened path list cannot answer "who holds this"; `counted` is empty whenever `abortPrune` is set, since attribution is exactly what an unreadable record destroys. @@ -70,14 +69,10 @@ This directory owns two cross-artifact contracts specifically: Resolution always leans over-protect: same-nonce + not-unioned → clean replace; same-track foreign nonce or unioned → union; cross-track foreign nonce → remint under a fresh key. None of the three directions can under-protect. - - **Deferred follow-up (TODO.md, deliberately NOT absorbed by the tracking - consolidation):** `ownerNonce` is not persisted, so after save→reopen an instance - cannot recognize its own prior-session usage record — it unions and marks the record - `unioned` forever, so prune stops reclaiming captures the instance once held but no - longer uses (safe, but the bank folder grows unbounded). This is a *completeness* - wart, not a safety one; every candidate fix examined so far trades it for a new - under-protection window, which the consolidation's own safety mandate forbids. See - `docs/TODO.md` for the constraint and the rejected session-epoch candidate. + - **Deferred follow-up:** `ownerNonce` is not persisted, so after save→reopen an + instance cannot recognize its own prior-session usage record and unions forever. + Reasoning, constraints, and the rejected candidates live in `docs/TODO.md` + ("Persist ReaSampler 9000 instance identity…"). ## Modules diff --git a/src/core/wire/sample_usage.h b/src/core/wire/sample_usage.h index 9553deb..9252e93 100644 --- a/src/core/wire/sample_usage.h +++ b/src/core/wire/sample_usage.h @@ -17,11 +17,9 @@ // per-instance key, never banks/view/tail/assign; the bridge's write entry // point structurally accepts only "rsusage_"-prefixed keys. // -// THE SAFETY PROPERTY (overrides every other consideration): every failure, -// ambiguity, or uncertainty here must fail-safe toward PROTECT. Over-protection -// (prune skips a reclaimable file, or refuses to run) is an accepted residual; -// under-protection (deleting a file an instance may still be playing) is a -// data-loss bug. Three folds enforce this: +// Every failure, ambiguity, or uncertainty here fails safe toward PROTECT (the +// territory-wide asymmetry, stated in core/tracking/CLAUDE.md). Three folds enforce +// it here: // * sibling-collision -> UNION, never clean-replace over a foreign writer; // * zero-identified -> records exist but no instance was identified live -> // protect ALL records' paths (a matcher failure must diff --git a/src/shell/actions/ingest.cpp b/src/shell/actions/ingest.cpp index e4cbb79..e239c6e 100644 --- a/src/shell/actions/ingest.cpp +++ b/src/shell/actions/ingest.cpp @@ -273,8 +273,6 @@ ImportResult importFileIntoActiveBank(const std::string& absoluteSourcePath) { s.createdTimestamp = nowSec; const AddResult r = book.activeIndex().add(s); - // Record the birth regardless of outcome — the tool WROTE the file, so prune must - // attribute it even in the narrow Collapsed race below. g_session->recordCreated(s, tracking::OriginKind::Ingest); switch (r) { diff --git a/src/shell/actions/prune_action.cpp b/src/shell/actions/prune_action.cpp index dc2ec93..f163442 100644 --- a/src/shell/actions/prune_action.cpp +++ b/src/shell/actions/prune_action.cpp @@ -6,6 +6,7 @@ #include #include +#include "core/version/app_version.h" // extStateNamespace — the printed recovery line must be channel-correct #include "shell/persist/session.h" // ReaSamplerSession — pruneDryRun / pruneOrphanSet / pruneReclaim #define REAPERAPI_MINIMAL @@ -27,22 +28,36 @@ void doBankPruneFolder(ReaSamplerSession& session) { // set unknowable, so the prune HALTS outright rather than proceed with degraded // protection. Both blockers can fire at once; report each one that did. if (report.blockedByTracking) { + // Every printed recovery line names THIS build's ext-state namespace: a beta + // user handed the stable spelling would clear the stable channel's key in + // their project and still be blocked. + const std::string& ns = version::extStateNamespace(); std::string msg = "ReaSampler prune: ABORTED -- the file-tracking state could not be read. " "Nothing was deleted.\n"; if (report.ledgerUnreadable) { msg += "The stored file-tracking ledger is malformed. It has been left " "intact rather than overwritten, so it can be repaired or cleared:\n" - " reaper.SetProjExtState(0, \"reasampler\", \"owned_files\", \"\")\n" + " reaper.SetProjExtState(0, \"" + ns + "\", \"owned_files\", \"\")\n" "Clearing it makes every existing bank file un-reclaimable (they stop " - "being attributable to ReaSampler); no file is lost.\n"; + "being attributable to ReaSampler); no file is lost. Reopen the " + "project afterwards -- the block is held for the rest of this " + "session, and until then no new capture is persisted to the ledger " + "either.\n"; + } + if (report.ledgerFutureVersion) { + msg += "The stored file-tracking ledger was written by a NEWER version of " + "ReaSampler than this one, so its records cannot be read safely. It " + "has been left intact and will NOT be overwritten. Reopen the project " + "with that newer version -- do NOT clear this key from here, that " + "would discard tracking records this build cannot see.\n"; } if (!report.unreadableUsageKeys.empty()) { msg += "One or more instance usage records could not be read or decoded. " "If the owning instance is still loaded it will republish its record " "on the next poll tick, clearing the abort. If the instance no longer " "exists (the key is an orphaned corrupt record), clear it manually:\n" - " reaper.SetProjExtState(0, \"reasampler\", \"\", \"\")\n" + " reaper.SetProjExtState(0, \"" + ns + "\", \"\", \"\")\n" "Offending key(s):\n"; for (const std::string& key : report.unreadableUsageKeys) { msg += " " + key + "\n"; diff --git a/src/shell/capture/capture_batch.cpp b/src/shell/capture/capture_batch.cpp index 350c58e..ad3aaef 100644 --- a/src/shell/capture/capture_batch.cpp +++ b/src/shell/capture/capture_batch.cpp @@ -455,13 +455,13 @@ void RunRecaptureFromSource(ReaSamplerSession& session) // One batched undo point around the in-place mutation; index-only ext-state, // nothing placed on the timeline. Undo_BeginBlock2(nullptr); + // Before the index outcome is even known: the render above already wrote the file, + // and makeUniqueTag guarantees a NEW path, so this is a second record carrying the + // same sampleId as the original. The superseded file becomes an orphan for prune. + session.recordCreated(updated, tracking::OriginKind::Recapture); const bool changed = session.book().updateSampleInPlace(sampleId, updated); if (changed) { - // Record the regenerated file's birth; the superseded file becomes an orphan - // for prune to reclaim. A re-capture onto the same path is a dedup no-op, so - // the original birth record — not this one — stays authoritative. - session.recordCreated(updated, tracking::OriginKind::Recapture); // Regenerating the same id's audio is exactly why instances need the generation // bump — they'd otherwise keep playing stale audio until reload. Bumped inside // the undo block so undo rolls back the generation with the rest of the blob. diff --git a/src/shell/capture/capture_orchestrator.cpp b/src/shell/capture/capture_orchestrator.cpp index 35d9ea4..6dc67b6 100644 --- a/src/shell/capture/capture_orchestrator.cpp +++ b/src/shell/capture/capture_orchestrator.cpp @@ -250,9 +250,6 @@ CaptureResult captureAndIndexOne(ReaSamplerSession& session, // AddResult tells a fresh add from a hash-dedup collapse, so the assign path (S8) can // target the sample actually in the bank (the existing entry on a collapse). const model::AddResult addResult = session.bank().add(res.sample); - // Record the birth at the same point the Sample is added, regardless of the index - // AddResult — even a hash-collapse still WROTE a file the tool owns, and the ledger - // dedups a repeat path itself (prune reconciles ledger vs index later). session.recordCreated(res.sample, tracking::OriginKind::Capture); // Resolve the LANDED bank-index id into res.sample.id for the S8 assign path: the new diff --git a/src/shell/capture/realtime_lifecycle.cpp b/src/shell/capture/realtime_lifecycle.cpp index 094183c..fed3eb3 100644 --- a/src/shell/capture/realtime_lifecycle.cpp +++ b/src/shell/capture/realtime_lifecycle.cpp @@ -31,8 +31,6 @@ void CommitRealtimeResult(ReaSamplerSession& session, const CaptureResult& res) return; } session.bank().add(res.sample); - // Record the birth regardless of the index AddResult — even a hash-collapse still - // wrote a file the tool owns; the ledger dedups a repeat path itself. session.recordCreated(res.sample, tracking::OriginKind::Capture); // A capture add changes what a live instance could play, so bump the generation // before persisting to refresh instances. @@ -56,14 +54,28 @@ void DriveRealtimeCapture(ReaSamplerSession& session) { RealtimeTickResult r = g_rtBackend.abort(*g_rtCapture); // Log without persisting — we restored into the original project but must - // not persist into the now-active foreign one. A Failed abort surfaces + // not persist into the now-active foreign one: the ledger and bank we would + // have to write belong to the project we just left. A Failed abort surfaces // abort()'s own message, distinguishing a clean tab-switch abort from the // closed-project case (nothing restored because the pointers were already // freed). - if (r.status == RealtimeTickStatus::Done) - ShowConsoleMsg("ReaSampler realtime capture: project switched mid-record -- " - "captured audio restored into the original project; not " - "persisted to avoid crossing projects.\n"); + // + // The abort already moved the recorded WAV into the ORIGINAL project's bank + // folder, so it survives here untracked — deleting a user's just-recorded + // audio is the destructive direction and is not this shell's call, so the + // path is NAMED instead and the operator decides (docs/TODO.md). + if (r.status == RealtimeTickStatus::Done && r.result.status == CaptureStatus::Ok) + ShowConsoleMsg(("ReaSampler realtime capture: project switched mid-record -- " + "captured audio restored into the original project; not " + "persisted to avoid crossing projects. The recorded file was " + "left in the original project's bank folder as '" + + r.result.sample.relativePath + + "', untracked -- reopen that project and re-import it, or " + "delete it by hand.\n").c_str()); + else if (r.status == RealtimeTickStatus::Done) + ShowConsoleMsg(("ReaSampler realtime capture: project switched mid-record -- " + "the capture was aborted and produced no usable file: " + + r.result.message + "\n").c_str()); else ShowConsoleMsg(("ReaSampler realtime capture: project changed mid-record -- " + r.result.message + "\n").c_str()); diff --git a/src/shell/persist/CLAUDE.md b/src/shell/persist/CLAUDE.md index a7a29d3..9dbe517 100644 --- a/src/shell/persist/CLAUDE.md +++ b/src/shell/persist/CLAUDE.md @@ -34,17 +34,20 @@ REAPER/filesystem-facing half only, and it gathers rather than decides. record/input chains, containers recursively, take FX), and folds via the pure `sample_usage::foldUsageRecords`. Read-only at prune-scan time: `usage_scan` writes no ext-state. -- **A malformed tracking ledger is Unreadable, never "empty".** `ext_state_io` keeps +- **A ledger this build cannot read is degraded, never "empty".** `ext_state_io` keeps the `LedgerStatus` alongside the ledger, and `saveToActiveProject` SKIPS the - `owned_files` write while it is `Unreadable` — replacing a corrupt blob would - destroy the only record of every file created before the corruption, silently - turning them into permanently unreclaimable foreign files. Captures made during - such a session are recorded in memory but not persisted; they degrade to foreign - (untouchable), which is the safe direction. + `owned_files` write while `tracking::ledgerDegraded` holds — replacing a blob we + could not read would destroy the only record of every file created before it, + silently turning them into permanently unreclaimable foreign files. The status is + written only by `loadFromProject`, so it is sticky until the project is reloaded: + captures made during such a session are recorded in memory but not persisted, and + degrade to foreign (untouchable), which is the safe direction. - **`PruneReport` carries `blockedByTracking` + `ledgerUnreadable` / - `unreadableUsageKeys`**; dry-run, orphan-set, and reclaim each independently abort - (delete nothing) on a block. `BANK_PRUNE_FOLDER` (in `shell/actions`) halts on the - flag and prints whichever blockers fired. + `ledgerFutureVersion` / `unreadableUsageKeys`**; dry-run, orphan-set, and reclaim + each independently abort (delete nothing) on a block. `BANK_PRUNE_FOLDER` (in + `shell/actions`) halts on the flag and prints whichever blockers fired, with + channel-correct recovery lines — a corrupt blob may be cleared, a newer build's + must not be. ## Modules diff --git a/src/shell/persist/ext_state_io.cpp b/src/shell/persist/ext_state_io.cpp index 3eb7c90..112cbce 100644 --- a/src/shell/persist/ext_state_io.cpp +++ b/src/shell/persist/ext_state_io.cpp @@ -161,11 +161,9 @@ bool ReaSamplerSession::saveToActiveProject() { SetProjExtState(static_cast(proj), projExtNamespace(), kProjExtTailKey, tailJson.c_str()); - // Written on every save so the ledger and the bank stay in lockstep on disk — - // EXCEPT over a blob we could not read: replacing it would destroy the only - // record of every file created before the corruption, silently turning them - // into permanently unreclaimable foreign files. - if (trackingStatus_ != tracking::LedgerStatus::Unreadable) { + // Never write over a blob this build could not read (see this directory's + // CLAUDE.md for why the suppression, not a rewrite, is the safe direction). + if (!tracking::ledgerDegraded(trackingStatus_)) { const std::string ledgerJson = tracking_.serialize(); SetProjExtState(static_cast(proj), projExtNamespace(), kProjExtOwnedKey, ledgerJson.c_str()); @@ -234,10 +232,8 @@ capture::TailSetting loadTailSetting(ReaProject* proj) { return *loaded; } -// Absent/empty key -> Fresh (a new project, or a bank predating the ledger). -// Malformed -> Unreadable, which halts the prune and suppresses the next write -// rather than degrading to an empty ledger that looks like "nothing was ever -// created". +// Absent/empty key -> Fresh (a new project, or a bank predating the ledger). Anything +// this build cannot read is a degraded status, never an empty ledger. tracking::LedgerLoad loadOriginLedger(ReaProject* proj) { if (!proj) return tracking::LedgerLoad{}; tracking::LedgerLoad load = tracking::loadLedger( @@ -246,6 +242,11 @@ tracking::LedgerLoad loadOriginLedger(ReaProject* proj) { ShowConsoleMsg("ReaSampler: the stored file-tracking ledger is malformed. Prune " "is halted for this project and the stored value is left intact " "for recovery.\n"); + } else if (load.status == tracking::LedgerStatus::FutureVersion) { + ShowConsoleMsg("ReaSampler: the stored file-tracking ledger was written by a " + "NEWER version of ReaSampler. Prune is halted for this project " + "and the stored value will not be overwritten -- reopen the " + "project with that version.\n"); } return load; } diff --git a/src/shell/persist/prune_fs.cpp b/src/shell/persist/prune_fs.cpp index 853bf54..00fa2ce 100644 --- a/src/shell/persist/prune_fs.cpp +++ b/src/shell/persist/prune_fs.cpp @@ -72,6 +72,7 @@ struct PruneScan { std::unordered_map sizeByRel; bool blocked = false; bool ledgerUnreadable = false; + bool ledgerFutureVersion = false; std::vector unreadableUsageKeys; }; @@ -134,6 +135,7 @@ PruneScan scanPruneOrphans(const BankBook& book, const tracking::OriginLedger& l // action tell the user what to recover. scan.blocked = true; scan.ledgerUnreadable = protection.ledgerUnreadable; + scan.ledgerFutureVersion = protection.ledgerFutureVersion; scan.unreadableUsageKeys = protection.unreadableUsageKeys; return scan; } @@ -211,6 +213,7 @@ reclaim::PruneReport ReaSamplerSession::pruneDryRun() const { // the prune refused to run. report.blockedByTracking = scan.blocked; report.ledgerUnreadable = scan.ledgerUnreadable; + report.ledgerFutureVersion = scan.ledgerFutureVersion; report.unreadableUsageKeys = scan.unreadableUsageKeys; return report; } diff --git a/src/shell/persist/session.cpp b/src/shell/persist/session.cpp index fa241de..8babf88 100644 --- a/src/shell/persist/session.cpp +++ b/src/shell/persist/session.cpp @@ -35,6 +35,7 @@ #define REAPERAPI_MINIMAL #define REAPERAPI_WANT_MarkProjectDirty #define REAPERAPI_WANT_SetProjExtState +#define REAPERAPI_WANT_ShowConsoleMsg #include "reaper_plugin_functions.h" namespace reasampler { @@ -56,7 +57,18 @@ void ReaSamplerSession::recordCreated(const model::Sample& sample, // rather than re-deriving keeps one source for the lineage fact. Absent // provenance is a root capture, not a gap. if (sample.provenance) rec.parentSampleId = sample.provenance->parentSampleId; - tracking_.record(rec); + + const tracking::RecordResult result = tracking_.record(rec); + // A rejection means a file exists that nothing attributes to us — invisible + // otherwise, and exactly the gap this ledger exists to close. AlreadyPresent is + // the normal dedup outcome, not a gap. + if (result == tracking::RecordResult::RejectedEmptyPath || + result == tracking::RecordResult::RejectedAbsolutePath) { + ShowConsoleMsg(("ReaSampler: could not record the origin of '" + + sample.relativePath + + "' -- the path is empty or absolute. The file is NOT tracked and " + "prune will treat it as a foreign file.\n").c_str()); + } } bool ReaSamplerSession::consumeLoadSignal() { diff --git a/src/shell/persist/session.h b/src/shell/persist/session.h index 605e510..b7aee90 100644 --- a/src/shell/persist/session.h +++ b/src/shell/persist/session.h @@ -71,17 +71,20 @@ public: capture::TailSetting& tail() { return tail_; } const capture::TailSetting& tail() const { return tail_; } - // Birth records for the files the system itself created. Read as a PAIR — - // the ledger alone cannot say whether an absent record means never-recorded - // or unreadable, and the two demand opposite treatment. This is the reach - // any consumer outside persist uses to build a tracking::TrackingState. - const tracking::OriginLedger& tracking() const { return tracking_; } - tracking::LedgerStatus trackingStatus() const { return trackingStatus_; } - - // Record a system-created file at the moment it exists — the ONLY way a - // birth record is written, so lineage can never be backfilled from a later - // guess. Lineage is read off the Sample's own provenance, the same act that - // stamped it, so the two cannot disagree. A repeat path is a no-op. + // Record a system-created file at the moment it exists — the ONLY way a birth + // record is written. Lineage is read off the Sample's own provenance, the same + // act that stamped it, so the two cannot disagree; where they later diverge the + // ledger record is authoritative (core/tracking/CLAUDE.md). + // + // Call it at EVERY creation site regardless of the bank's AddResult: a + // hash-collapse still wrote a file the tool owns, and an unrecorded file is a + // permanently unreclaimable foreign file. A rejected record is reported to the + // console — a file with no record is the gap this track exists to eliminate. + // + // A consumer outside persist that needs the ledger must take it WITH its + // LedgerStatus (a tracking::TrackingState); no accessor exposes one without the + // other, because an absent record and an unreadable ledger demand opposite + // treatment. void recordCreated(const model::Sample& sample, tracking::OriginKind kind); // The version that last wrote the active project: PreVersioning (no @@ -153,9 +156,8 @@ private: ViewModeModel view_; // reset to default on a project with no stored view_state capture::TailSetting tail_; // reset to default (None / 2s) with no stored tail key tracking::OriginLedger tracking_; // reset to empty/stored on EVERY load path, never inherited - // Unreadable is sticky for the project's session: it halts the prune AND - // suppresses the ledger write, so a corrupt blob survives for recovery - // instead of being silently replaced by a ledger missing every earlier file. + // Written only by loadFromProject, so a degraded status is sticky until the + // project is reloaded (see this directory's CLAUDE.md). tracking::LedgerStatus trackingStatus_ = tracking::LedgerStatus::Fresh; version::WritingVersion writingVersion_; // recovered per load; PreVersioning default std::int64_t bankGeneration_ = 0; // recovered per load (absent -> 0); monotonic diff --git a/tests/test_origin_ledger.cpp b/tests/test_origin_ledger.cpp index 078b77f..f14efd1 100644 --- a/tests/test_origin_ledger.cpp +++ b/tests/test_origin_ledger.cpp @@ -1,10 +1,10 @@ // Standalone tests for reasampler::tracking::OriginLedger — no REAPER, no framework. // // The record family behind file tracking. Covers: the relative-paths-only invariant, -// dedup, insertion order, the JSON round-trip (incl. a golden byte literal), the -// no-backfill rule, the legacy path-only lift, and the three-way Fresh / -// Loaded / Unreadable classification that keeps never-recorded apart from -// unreadable. +// exact-string ownership, dedup, insertion order, the JSON round-trip (incl. golden +// byte literals over every persisted enum value), the no-backfill rule, the legacy +// path-only lift, and the Fresh / Loaded / Unreadable / FutureVersion classification +// that keeps never-recorded apart from the two degraded states. #include "../src/core/tracking/origin_ledger.h" @@ -151,17 +151,49 @@ static void testRoundTripWithJsonMetacharacters() { } // Golden byte literal: pins the EXACT serialized bytes, so a format drift both -// writer and reader agree on still fails here. -static void testSerializeGoldenLiteral() { +// writer and reader agree on still fails here. EVERY OriginKind appears, because the +// integers are persisted — a swap of two values in both the enum and kindFromInt +// round-trips perfectly and would mislabel every existing project. This literal is +// the only thing standing in the way of that. +static void testSerializeGoldenLiteralPinsEveryPersistedKind() { OriginLedger l; - l.record(rec("reasampler_bank/a.wav", OriginKind::Capture, "S-a")); - l.record(rec("reasampler_bank/b.wav", OriginKind::Resample, "S-b", "S-a")); + l.record(rec("bank/unknown.wav", OriginKind::Unknown)); + l.record(rec("bank/capture.wav", OriginKind::Capture, "S-a")); + l.record(rec("bank/ingest.wav", OriginKind::Ingest, "S-b")); + l.record(rec("bank/recapture.wav", OriginKind::Recapture, "S-c", "S-a")); + l.record(rec("bank/resample.wav", OriginKind::Resample, "S-d", "S-a")); const std::string expected = "{\"v\":2,\"records\":[" - "{\"path\":\"reasampler_bank/a.wav\",\"kind\":1,\"sample\":\"S-a\",\"parent\":\"\"}," - "{\"path\":\"reasampler_bank/b.wav\",\"kind\":4,\"sample\":\"S-b\",\"parent\":\"S-a\"}" + "{\"path\":\"bank/unknown.wav\",\"kind\":0,\"sample\":\"\",\"parent\":\"\"}," + "{\"path\":\"bank/capture.wav\",\"kind\":1,\"sample\":\"S-a\",\"parent\":\"\"}," + "{\"path\":\"bank/ingest.wav\",\"kind\":2,\"sample\":\"S-b\",\"parent\":\"\"}," + "{\"path\":\"bank/recapture.wav\",\"kind\":3,\"sample\":\"S-c\",\"parent\":\"S-a\"}," + "{\"path\":\"bank/resample.wav\",\"kind\":4,\"sample\":\"S-d\",\"parent\":\"S-a\"}" "]}"; CHECK(l.serialize() == expected); + + // And the reader agrees with the writer on the same bytes, kind by kind. + auto back = OriginLedger::deserialize(expected); + CHECK(back.has_value()); + CHECK(back->find("bank/unknown.wav")->kind == OriginKind::Unknown); + CHECK(back->find("bank/capture.wav")->kind == OriginKind::Capture); + CHECK(back->find("bank/ingest.wav")->kind == OriginKind::Ingest); + CHECK(back->find("bank/recapture.wav")->kind == OriginKind::Recapture); + CHECK(back->find("bank/resample.wav")->kind == OriginKind::Resample); +} + +// contains() is an EXACT-string predicate, never a prefix or substring match — the +// prune's whole ownership attribution rests on it, and a looser match would attribute +// (and so expose to deletion) a file the system never created. +static void testContainsIsExactStringNotPrefixOrSubstring() { + OriginLedger l; + l.record(rec("reasampler_bank/a.wav", OriginKind::Capture)); + + CHECK(l.contains("reasampler_bank/a.wav")); + CHECK(!l.contains("reasampler_bank/a")); // prefix of the stored path + CHECK(!l.contains("a.wav")); // suffix of the stored path + CHECK(!l.contains("reasampler_bank/a.wave")); // stored path is a prefix of this + CHECK(!l.contains("REASAMPLER_BANK/A.WAV")); // no case folding } // --- malformed input --------------------------------------------------------- @@ -177,12 +209,57 @@ static void testMalformedParsesToNullopt() { // Unknown rather than failing the whole ledger — a vocabulary gap must not halt // the prune. auto tolerated = OriginLedger::deserialize( - "{\"v\":3,\"future\":{\"x\":1},\"records\":[{\"path\":\"a.wav\",\"kind\":99}]}"); + "{\"v\":2,\"future\":{\"x\":1},\"records\":[{\"path\":\"a.wav\",\"kind\":99}]}"); CHECK(tolerated.has_value()); CHECK(tolerated->size() == 1); CHECK(tolerated->find("a.wav")->kind == OriginKind::Unknown); } +// Trailing garbage is rejected outright: accepting it would turn a detectably-corrupt +// blob into a silently-partial ledger, and the records it dropped would then be lost +// on the next save. +static void testTrailingGarbageIsRejected() { + CHECK(!OriginLedger::deserialize("{\"v\":2,\"records\":[]}JUNK").has_value()); + CHECK(!OriginLedger::deserialize("{\"owned\":[\"a.wav\"]} {\"owned\":[]}").has_value()); + // ... but trailing whitespace alone is not garbage. + CHECK(OriginLedger::deserialize("{\"v\":2,\"records\":[]} \n").has_value()); +} + +// The legacy shape's own error cases: a type error is a REJECTION (blocking, blob +// preserved), never a silent degrade to empty — while a genuinely empty legacy list +// is valid and yields an empty, non-blocking ledger. +static void testLegacyShapeTypeErrorsAreRejectedButEmptyIsValid() { + CHECK(loadLedger("{\"owned\":[1,2]}").status == LedgerStatus::Unreadable); + CHECK(loadLedger("{\"owned\":\"x\"}").status == LedgerStatus::Unreadable); + + const LedgerLoad emptyLegacy = loadLedger("{\"owned\":[]}"); + CHECK(emptyLegacy.status == LedgerStatus::Loaded); + CHECK(emptyLegacy.ledger.empty()); +} + +// A blob written by a newer build is its own status: this build cannot trust records +// it parsed under v2 rules, so it must neither answer a destructive question from +// them nor overwrite the blob with its own truncation. +static void testFutureVersionIsNeitherLoadedNorMalformed() { + const std::string v3 = + "{\"v\":3,\"records\":[{\"path\":\"a.wav\",\"kind\":1}],\"newshape\":[1]}"; + const LedgerLoad load = loadLedger(v3); + CHECK(load.status == LedgerStatus::FutureVersion); + CHECK(load.ledger.empty()); // never a partial value + CHECK(ledgerDegraded(load.status)); // blocks answers AND suppresses the write + CHECK(!OriginLedger::deserialize(v3).has_value()); + + // Key order must not matter — "v" is validated after the object closes. + CHECK(loadLedger("{\"records\":[{\"path\":\"a.wav\"}],\"v\":9}").status == + LedgerStatus::FutureVersion); + + // The versions this build does read stay readable, and a nonsense version is + // corruption rather than a future shape. + CHECK(loadLedger("{\"v\":2,\"records\":[]}").status == LedgerStatus::Loaded); + CHECK(loadLedger("{\"v\":1,\"owned\":[\"a.wav\"]}").status == LedgerStatus::Loaded); + CHECK(loadLedger("{\"v\":-1,\"records\":[]}").status == LedgerStatus::Unreadable); +} + // A hand-edited or corrupt blob cannot smuggle an absolute or duplicate path past // the load — record() re-asserts the invariants on the way in. static void testLoadReassertsInvariants() { @@ -222,12 +299,41 @@ static void testLegacyPathOnlyManifestLiftsIn() { CHECK(r.parentSampleId.empty()); // no spurious lineage } - // Re-saving upgrades the shape without losing or inventing anything. - auto resaved = OriginLedger::deserialize(load.ledger.serialize()); + // Re-saving upgrades the shape without losing or inventing anything — pinned as a + // byte literal, not just a round-trip, so the lifted record's persisted integers + // (kind 0) and empty ids are fixed rather than merely self-consistent. + const std::string expected = + "{\"v\":2,\"records\":[" + "{\"path\":\"reasampler_bank/a.wav\",\"kind\":0,\"sample\":\"\",\"parent\":\"\"}," + "{\"path\":\"reasampler_bank/b.wav\",\"kind\":0,\"sample\":\"\",\"parent\":\"\"}" + "]}"; + CHECK(load.ledger.serialize() == expected); + auto resaved = OriginLedger::deserialize(expected); CHECK(resaved.has_value()); CHECK(*resaved == load.ledger); } +// A recapture regenerates the audio behind ONE bank id under a NEW file name +// (makeUniqueTag guarantees it), so the ledger legitimately ends up holding two +// records with the same sampleId and different paths. The original's lineage survives +// untouched — the Sample was rewritten in place, the birth record was not. +static void testRecaptureAddsSecondRecordUnderTheSameSampleId() { + OriginLedger l; + l.record(rec("bank/take-1.wav", OriginKind::Capture, "S-1", "S-parent")); + CHECK(l.record(rec("bank/take-2.wav", OriginKind::Recapture, "S-1", "S-other")) == + RecordResult::Recorded); + + CHECK(l.size() == 2); + CHECK(l.find("bank/take-1.wav")->sampleId == "S-1"); + CHECK(l.find("bank/take-2.wav")->sampleId == "S-1"); + // The ledger, not the (mutable) Sample, is authoritative for lineage: the + // recapture's differing parent did not rewrite the original's. + CHECK(l.find("bank/take-1.wav")->parentSampleId == "S-parent"); + CHECK(l.find("bank/take-1.wav")->kind == OriginKind::Capture); + // Both stay owned, so the superseded file is reclaimable rather than foreign. + CHECK(l.ownedPaths().size() == 2); +} + // --- never-recorded vs unreadable -------------------------------------------- // The two absences demand opposite treatment, so they must be distinguishable at // the load boundary — this is the only place that distinction is made. @@ -259,14 +365,19 @@ int main() { testEmptyLedger(); testRejectsEmptyAndAbsolutePaths(); testDedupPreservesInsertionOrder(); + testContainsIsExactStringNotPrefixOrSubstring(); testLineageIsNeverBackfilled(); testLineageQueryableImmediatelyAfterRecording(); testRoundTripWithLineage(); testRoundTripWithJsonMetacharacters(); - testSerializeGoldenLiteral(); + testSerializeGoldenLiteralPinsEveryPersistedKind(); testMalformedParsesToNullopt(); + testTrailingGarbageIsRejected(); + testLegacyShapeTypeErrorsAreRejectedButEmptyIsValid(); + testFutureVersionIsNeitherLoadedNorMalformed(); testLoadReassertsInvariants(); testLegacyPathOnlyManifestLiftsIn(); + testRecaptureAddsSecondRecordUnderTheSameSampleId(); testFreshLoadedUnreadableAreDistinct(); if (g_fail == 0) std::printf("origin_ledger: all tests passed\n"); diff --git a/tests/test_tracking_authority.cpp b/tests/test_tracking_authority.cpp index 61f71c5..b700fbb 100644 --- a/tests/test_tracking_authority.cpp +++ b/tests/test_tracking_authority.cpp @@ -148,8 +148,44 @@ static void testUnreadableUsageBlocksPruneAndNamesIt() { const ProtectionAnswer answer = pruneProtection(state); CHECK(answer.blocked); CHECK(!answer.ledgerUnreadable); + CHECK(!answer.ledgerFutureVersion); CHECK(answer.unreadableUsageKeys.size() == 1); CHECK(answer.unreadableUsageKeys[0] == "rsusage_BROKEN"); + + // The belt-and-braces guard is symmetric: a usage-only block ALSO withholds + // ownedPaths, even though the ledger itself read fine, so a caller that ignored + // `blocked` computes an empty orphan set rather than deleting with degraded + // protection. + CHECK(answer.ownedPaths.empty()); + CHECK(reclaim::pruneOrphans({"bank/orphan.wav"}, {}, answer.ownedPaths).empty()); + + // heldPaths is the one list that stays populated on a block — it only ever widens + // the protected set, so withholding it would be the unsafe direction. + CHECK(contains(answer.heldPaths, "bank/x.wav")); +} + +// A ledger written by a NEWER build blocks exactly like a corrupt one, but is +// reported separately: the operator advice differs (clearing a corrupt blob is +// repair; clearing a newer build's is destruction). +static void testFutureVersionLedgerBlocksAndIsReportedSeparately() { + const LedgerLoad load = loadLedger("{\"v\":99,\"records\":[]}"); + CHECK(load.status == LedgerStatus::FutureVersion); + + OriginLedger populated; + populated.record(originOf("bank/would-be-orphan.wav", OriginKind::Capture, "S-1")); + const UsageFoldResult fold = foldLive({}, {}); + + const TrackingState state{load.status, populated, fold}; + const ProtectionAnswer answer = pruneProtection(state); + CHECK(answer.blocked); + CHECK(answer.ledgerFutureVersion); + CHECK(!answer.ledgerUnreadable); + CHECK(answer.ownedPaths.empty()); + CHECK(reclaim::pruneOrphans({"bank/would-be-orphan.wav"}, {}, + answer.ownedPaths).empty()); + + // And the replace-vs-add question abstains rather than allowing a replace. + CHECK(tiedUsageExists(state, "bank/would-be-orphan.wav", "") == Answer::Indeterminate); } // An unreadable ledger blocks too, AND leaves ownedPaths empty — so a caller that @@ -178,12 +214,19 @@ static void testUnreadableLedgerBlocksAndYieldsNoOrphans() { // Both blockers at once must both be reported — the operator needs to fix both. static void testBothBlockersReported() { const OriginLedger empty; - const UsageFoldResult fold = foldLive({unreadable("rsusage_BROKEN")}, {"{T1}"}); + const UsageFoldResult fold = foldLive( + {usage("rsusage_A", "{T1}", {UsageHold{"S-held", "bank/held.wav"}}), + unreadable("rsusage_BROKEN")}, + {"{T1}"}); const TrackingState state{LedgerStatus::Unreadable, empty, fold}; const ProtectionAnswer answer = pruneProtection(state); CHECK(answer.blocked); CHECK(answer.ledgerUnreadable); CHECK(answer.unreadableUsageKeys.size() == 1); + // heldPaths survives a double block: the fold's protect-all set only ever widens + // what prune protects, so withholding it would be the unsafe direction. + CHECK(contains(answer.heldPaths, "bank/held.wav")); + CHECK(answer.ownedPaths.empty()); } // A record that exists but whose track hosts no identified instance still protects @@ -260,9 +303,6 @@ static void testSoleHolderExcludingItselfAnswersNo() { CHECK(tiedUsageExists(state, "bank/src.wav", "rsusage_ME") == Answer::No); } -// A `unioned` record carries more than one incarnation's holds, so it can never be -// attributed to a single owner — excluding it could hide a sibling's tie, which is -// the under-protecting direction. static void testUnionedRecordIsNeverExcludedAsOwn() { OriginLedger ledger; ledger.record(originOf("bank/src.wav", OriginKind::Capture, "S-src")); @@ -404,6 +444,7 @@ int main() { testLiveHoldProtectsDeReferencedCapture(); testUnreadableUsageBlocksPruneAndNamesIt(); testUnreadableLedgerBlocksAndYieldsNoOrphans(); + testFutureVersionLedgerBlocksAndIsReportedSeparately(); testBothBlockersReported(); testZeroIdentifiedInstancesStillProtects(); testUnreadableStateNeverAnswersNo();