tracking: read the ledger's version, not just write it; clear owned on any block; channel-correct prune recovery

This commit is contained in:
2026-07-30 20:11:05 -04:00
parent 7f70d94228
commit 45b87dc2ff
27 changed files with 432 additions and 168 deletions
+2 -2
View File
@@ -9,12 +9,12 @@ has — stay in the consumers; this module owns lexing/emitting only.
## Modules
- `json` (`core/json`) — the ONE hand-rolled JSON lexical layer (Q-W1): string/number/bool/null tokens, the scoped object `Writer`, and the bounds-checked `Reader` cursor, byte-compatible with the five pre-extraction per-module writers it replaced (`bank_model` / `bank_book` / `view_mode_model` / `owned_manifest` / `tail_control`). Domain grammars stay in the consumers; this owns lexing/emitting only.
- `json` (`core/json`) — the ONE hand-rolled JSON lexical layer (Q-W1): string/number/bool/null tokens, the scoped object `Writer`, and the bounds-checked `Reader` cursor, byte-compatible with the five pre-extraction per-module writers it replaced (`bank_model` / `bank_book` / `view_mode_model` / the retired `owned_manifest`, now `core/tracking/origin_ledger` / `tail_control`). Domain grammars stay in the consumers; this owns lexing/emitting only.
## Gotchas
- Byte-compatible with the five pre-extraction per-module writers it replaced
(`bank_model` / `bank_book` / `view_mode_model` / `owned_manifest` /
(`bank_model` / `bank_book` / `view_mode_model` / the retired `owned_manifest` /
`tail_control`) — a change here risks silently breaking round-trip compatibility
with ext-state blobs already persisted by projects written before the Q-W1
extraction.
+1 -1
View File
@@ -7,7 +7,7 @@
// unknown-value skipping, and the emit side (escaping, %.17g/%d/%lld rendering,
// the scoped object writer). Domain grammars — which keys exist, what shape each
// value takes — stay in the consumers (bank_model, bank_book, view_mode_model,
// owned_manifest, tail_control).
// origin_ledger, tail_control).
//
// Byte-compatibility contract (load-bearing): the emit helpers reproduce the prior
// per-module writers EXACTLY — writeEscaped's escape set, %.17g for doubles
+2 -18
View File
@@ -1,8 +1,7 @@
#include "core/model/bank_model.h"
#include <cctype>
#include "core/json/json.h"
#include "core/util/relative_path.h"
// bank_model implementation. JSON rides on the shared core/json lexical layer;
// only the Sample/index DOMAIN grammar lives here. Doubles are emitted with 17
@@ -44,22 +43,7 @@ bool Sample::operator==(const Sample& o) const {
provenance == o.provenance && createdTimestamp == o.createdTimestamp;
}
// -- path invariant -------------------------------------------------------
// Rejects absolute paths rather than normalizing them: the pure model has no
// knowledge of the project root, so any "normalization" would be a guess that
// could point at the wrong file. Covers POSIX ("/x"), Windows drive ("C:\x",
// "C:/x", "C:foo" drive-relative), and UNC ("\\host\share") forms. Any leading
// <alpha>: is rejected regardless of what follows — drive-relative paths
// ("C:foo.wav") resolve against the drive's current directory, not the project
// root, so they violate relative-paths-only just as much as "C:\foo.wav" does.
static bool isAbsolutePath(const std::string& p) {
if (p.empty()) return false;
if (p[0] == '/' || p[0] == '\\') return true; // POSIX root or UNC
if (p.size() >= 2 && std::isalpha(static_cast<unsigned char>(p[0])) && p[1] == ':')
return true; // Windows drive (C:\, C:/, C:foo, C:)
return false;
}
using util::isAbsolutePath;
// -- BankModel ------------------------------------------------------------
+1 -1
View File
@@ -32,7 +32,7 @@ enumeration and the actual deletion live in `shell/persist` (`prune_fs`), not he
## Modules
- `prune_reconcile` — pure prune core: `pruneOrphans(present, referenced, owned)` computes `(owned ∩ present) referenced`; the safety-critical "which files are orphans" decision, filesystem-free and hard-tested before any I/O exists. `mergeReferenced(bankRefs, heldPaths)` unions live instance holds into the referenced-set. Two of the three inputs (`owned`, and the held half of `referenced`) come from `core/tracking`'s authority, never assembled ad hoc by a shell. `PruneReport` carries the authority's verdict: `blockedByTracking` + `ledgerUnreadable` / `unreadableUsageKeys`.
- `prune_reconcile` — pure prune core: `pruneOrphans(present, referenced, owned)` computes `(owned ∩ present) referenced`; the safety-critical "which files are orphans" decision, filesystem-free and hard-tested before any I/O exists. `mergeReferenced(bankRefs, heldPaths)` unions live instance holds into the referenced-set. Two of the three inputs (`owned`, and the held half of `referenced`) come from `core/tracking`'s authority, never assembled ad hoc by a shell. `PruneReport` carries the authority's verdict: `blockedByTracking` + `ledgerUnreadable` / `ledgerFutureVersion` / `unreadableUsageKeys`.
## Gotchas
+6 -3
View File
@@ -53,9 +53,11 @@ namespace reasampler::reclaim {
// (count 0, empty list) and the prune must HALT — deleting with
// degraded protection is the data-loss direction. Set by the scan
// shell, never by buildPruneReport (which stays a pure tally).
// * ledgerUnreadable / unreadableUsageKeys — which side blocked, so the action can
// tell the operator what to recover. The key names are the exact
// "rsusage_<guid>" spellings.
// * ledgerUnreadable / ledgerFutureVersion / unreadableUsageKeys — which side
// blocked, so the action can tell the operator what to recover. The
// two ledger flags need opposite advice (a corrupt blob may be
// cleared; a newer build's blob must not be). The key names are the
// exact "rsusage_<guid>" spellings.
struct PruneReport {
std::size_t count = 0;
std::uint64_t totalBytes = 0;
@@ -63,6 +65,7 @@ struct PruneReport {
bool truncated = false;
bool blockedByTracking = false;
bool ledgerUnreadable = false;
bool ledgerFutureVersion = false;
std::vector<std::string> unreadableUsageKeys;
};
+28 -12
View File
@@ -21,21 +21,33 @@ non-destructive side of the question being asked — over-protection (prune skip
reclaimable file, or refuses to run; resample adds instead of replacing) is an
accepted residual; under-protection is a data-loss bug.
**No silent gaps.** A system-created file is tracked from the instant it exists.
`ReaSamplerSession::recordCreated` is the only writer, called at the same point the
`Sample` is added, and it reads lineage off that `Sample`'s own provenance — so the
recipe fingerprint and the lineage record come from one act and cannot disagree.
**No silent gaps — in memory at creation, on disk at the next save.**
`ReaSamplerSession::recordCreated` is the only writer, called in the same
straight-line block as the bank add with no I/O or early return between, so a
system-created file is tracked *in memory* the instant it exists. The record reaches
the `.rpp` only at the following `saveToActiveProject()`; a crash in that window, or a
whole session running with a degraded ledger (below), leaves the file untracked —
foreign, therefore never reclaimed, which is the safe direction. That residual is
accepted and stated here rather than papered over by a stronger claim.
**Lineage is never backfilled.** `OriginLedger::record` on a path already present is
`AlreadyPresent` and leaves the stored record untouched. A record says what was known
at birth or says nothing at all; nothing may later rewrite history from a guess.
Consequence, and the reason both blobs can carry a parent id: `recordCreated` *seeds*
the record from the `Sample`'s own `provenance.parentSampleId` (one source, one act),
but the `Sample` is mutable via `updateSampleInPlace` and the record is not — so where
the two ever diverge, **the ledger record is authoritative for lineage** and the
`Sample`'s copy is only the recipe facet's view of it.
**Never-recorded and unreadable are different absences.** `LedgerStatus` keeps them
**Never-recorded and degraded are different absences.** `LedgerStatus` keeps them
apart, and only `loadLedger` can tell them apart (an empty stored value is not valid
JSON, so the parser alone cannot). `Fresh` — a new project or a bank predating the
ledger — blocks nothing and yields definite answers. `Unreadable` blocks every
destructive answer AND suppresses the next write, so a corrupt blob survives for
recovery instead of being replaced by a ledger missing every earlier file.
ledger — blocks nothing and yields definite answers. `Unreadable` (corrupt) and
`FutureVersion` (a `"v"` newer than this build reads) are the degraded pair: each
blocks every destructive answer AND suppresses the next write, so the stored blob
survives for recovery instead of being replaced by a ledger missing every earlier
file. `ledgerDegraded()` is the one test for both treatments — they coincide today, so
the two axes are deliberately not split.
**Consumers cannot disagree.** Both answers come out of one `TrackingState`. The two
universes differ deliberately — prune protects bank-referenced paths, ledger-owned
@@ -54,9 +66,9 @@ birth record is written for every system-created file, ambiguous parentage or no
- `origin_ledger` — the record family: `OriginRecord` (project-relative path, `OriginKind`,
the sample id minted at birth, the parent sample id) and the insertion-ordered,
path-keyed, deduplicated `OriginLedger` that holds them, with its JSON codec and the
`loadLedger` three-way `Fresh` / `Loaded` / `Unreadable` classification. Persisted
under the FOREVER-STABLE `owned_files` ext-state key; the legacy path-only shape
(`{"owned":[...]}`) lifts in as `Unknown`-kind records with no lineage.
`loadLedger` `Fresh` / `Loaded` / `Unreadable` / `FutureVersion` classification.
Persisted under the FOREVER-STABLE `owned_files` ext-state key; the legacy path-only
shape (`{"owned":[...]}`) lifts in as `Unknown`-kind records with no lineage.
- `tracking_authority` — the one decision surface: `pruneProtection` (the `owned` and
held-path inputs prune's set algebra consumes, plus the blocked/blockers verdict)
and `tiedUsageExists` (`Yes` / `No` / `Indeterminate`, per capture, excluding the
@@ -66,7 +78,11 @@ birth record is written for every system-created file, ambiguous parentage or no
- `OriginKind` values are PERSISTED INTEGERS — never renumber, only append. An
unrecognized value degrades to `Unknown` rather than failing the parse: a vocabulary
gap must not halt the prune.
gap must not halt the prune. This degrade-don't-halt rule is about *field
vocabulary* only — an unrecognized **document version** (`"v"`) does the opposite
and blocks, because a changed record shape makes the whole ledger untrustworthy.
- `OriginRecord.sampleId` is NOT unique: a recapture writes a new file under the same
bank id, so two records can share a `sampleId`. `relativePath` is the only key.
- A `unioned` usage record can never be excluded as "my own" — it carries more than one
incarnation's holds, so attributing it to a single owner could hide a sibling's tie.
- The set algebra prune runs on `(owned ∩ present) referenced` is `core/reclaim`'s,
+42 -22
View File
@@ -1,8 +1,9 @@
#include "core/tracking/origin_ledger.h"
#include <cctype>
#include <utility>
#include "core/json/json.h"
#include "core/util/relative_path.h"
// Version ladder for the stored blob, under the FOREVER-STABLE "owned_files" key:
//
@@ -13,21 +14,20 @@
// v1 blobs lift to v2 records with kind Unknown and empty ids — a pre-existing bank
// keeps every protection it had (the paths are still owned) and gains no invented
// lineage. Both shapes parse; only v2 is written.
//
// "v" is READ and validated, not just written: a v3 record shape parsed by these v2
// rules would yield a plausible-but-partial ledger, and the next save would overwrite
// the v3 blob with that truncation. A version above kLedgerVersion is therefore its
// own degraded status, never a Loaded ledger.
namespace reasampler::tracking {
namespace {
// Any leading '/' or '\' (POSIX root / UNC), or a leading <alpha>: (Windows drive,
// incl. drive-relative "C:foo"). Must match bank_model's rejection exactly — the
// ledger holds the same kind of path as Sample.relativePath.
bool isAbsolutePath(const std::string& p) {
if (p.empty()) return false;
if (p[0] == '/' || p[0] == '\\') return true;
if (p.size() >= 2 && std::isalpha(static_cast<unsigned char>(p[0])) && p[1] == ':')
return true;
return false;
}
using util::isAbsolutePath;
// The version serialize() writes, and the highest this build can read.
constexpr int kLedgerVersion = 2;
// An unrecognized persisted integer degrades to Unknown rather than failing the
// parse: a newer channel's kind must not make the whole ledger unreadable, which
@@ -75,7 +75,7 @@ std::vector<std::string> OriginLedger::ownedPaths() const {
}
std::string OriginLedger::serialize() const {
std::string out = "{\"v\":2,\"records\":[";
std::string out = "{\"v\":" + json::numToStr(kLedgerVersion) + ",\"records\":[";
for (std::size_t i = 0; i < records_.size(); ++i) {
if (i) out += ',';
const OriginRecord& r = records_[i];
@@ -139,14 +139,18 @@ bool parseRecordArray(json::Reader& r, OriginLedger& out) {
}
}
bool parseLedger(json::Reader& r, OriginLedger& out) {
// `versionOut` stays 0 when no "v" key was present — the legacy path-only shape, or
// an empty object. Read regardless of key order, so it is validated after the close.
bool parseLedger(json::Reader& r, OriginLedger& out, int& versionOut) {
if (!r.consume('{')) return false;
r.skipWs();
if (r.consume('}')) return true;
for (;;) {
std::string key;
if (!r.parseKey(key)) return false;
if (key == "records") {
if (key == "v") {
if (!r.parseInt(versionOut)) return false;
} else if (key == "records") {
if (!parseRecordArray(r, out)) return false;
} else if (key == "owned") {
std::vector<std::string> paths;
@@ -162,25 +166,41 @@ bool parseLedger(json::Reader& r, OriginLedger& out) {
}
}
enum class ParseOutcome { Ok, Malformed, FutureVersion };
// The one parse; the two public entry points differ only in how much of the outcome
// they can express. `out` is written only on Ok — never a partial ledger.
ParseOutcome parseStored(const std::string& text, OriginLedger& out) {
OriginLedger parsed;
int version = 0;
::reasampler::json::Reader r(text);
if (!parseLedger(r, parsed, version)) return ParseOutcome::Malformed;
// Trailing garbage means the blob is not what it claims; accepting it would turn
// a detectably-corrupt value into a silently-partial ledger.
r.skipWs();
if (!r.eof()) return ParseOutcome::Malformed;
if (version < 0) return ParseOutcome::Malformed;
if (version > kLedgerVersion) return ParseOutcome::FutureVersion;
out = std::move(parsed);
return ParseOutcome::Ok;
}
} // namespace
std::optional<OriginLedger> OriginLedger::deserialize(const std::string& json) {
OriginLedger ledger;
::reasampler::json::Reader r(json);
if (!parseLedger(r, ledger)) return std::nullopt;
if (parseStored(json, ledger) != ParseOutcome::Ok) return std::nullopt;
return ledger;
}
LedgerLoad loadLedger(const std::string& stored) {
LedgerLoad load;
if (stored.empty()) return load; // absent key -> Fresh, not an error
std::optional<OriginLedger> parsed = OriginLedger::deserialize(stored);
if (!parsed) {
load.status = LedgerStatus::Unreadable;
return load;
switch (parseStored(stored, load.ledger)) {
case ParseOutcome::Ok: load.status = LedgerStatus::Loaded; break;
case ParseOutcome::Malformed: load.status = LedgerStatus::Unreadable; break;
case ParseOutcome::FutureVersion: load.status = LedgerStatus::FutureVersion; break;
}
load.status = LedgerStatus::Loaded;
load.ledger = std::move(*parsed);
return load;
}
+21 -11
View File
@@ -2,10 +2,7 @@
// origin_ledger — the one persisted record family behind file tracking: for every
// file the system itself created, what act created it and what it derives from.
// Supersedes the path-only owned manifest (same ext-state key, widened shape).
//
// Lineage is written at birth and never backfilled — record() on a path already
// present is a no-op, so a record says what was known when the file appeared or
// says nothing at all. Nothing here decides anything; tracking_authority does.
// Nothing here decides anything; tracking_authority does.
#include <cstddef>
#include <optional>
@@ -32,7 +29,11 @@ enum class OriginKind {
struct OriginRecord {
std::string relativePath;
OriginKind kind = OriginKind::Unknown;
std::string sampleId; // bank id minted at birth; "" when never recorded
// The bank id as of birth; "" when never recorded. NOT unique across records: a
// recapture writes a new file under the SAME bank id, so the ledger legitimately
// holds two records sharing a sampleId with different paths. `relativePath` is
// the only key; a by-sampleId lookup is ambiguous by construction.
std::string sampleId;
std::string parentSampleId; // the capture this derives from; "" = root / none
bool operator==(const OriginRecord& o) const;
@@ -77,7 +78,8 @@ public:
bool operator==(const OriginLedger& o) const { return records_ == o.records_; }
// Lossless round-trip: deserialize(serialize(x)) == x. std::nullopt on malformed
// input — the caller must treat that as unreadable, never as empty.
// input AND on a blob whose "v" is newer than this build can read — the caller
// must treat either as unreadable, never as empty. loadLedger tells them apart.
std::string serialize() const;
static std::optional<OriginLedger> deserialize(const std::string& json);
@@ -85,11 +87,19 @@ private:
std::vector<OriginRecord> records_;
};
// The three states of a stored ledger, kept apart because never-recorded and
// unreadable demand opposite treatment: `Fresh` is a legitimate empty (a new
// project, or a bank predating the ledger) and blocks nothing; `Unreadable` is a
// present-but-corrupt blob and must block every destructive answer.
enum class LedgerStatus { Fresh, Loaded, Unreadable };
// The states of a stored ledger. `Fresh` (absent key) and `Loaded` are the two
// usable ones. The other two are distinguished only so the operator gets the right
// recovery advice — clearing a corrupt blob is repair, clearing a newer build's blob
// is destruction.
enum class LedgerStatus { Fresh, Loaded, Unreadable, FutureVersion };
// The one degraded-state test, deliberately serving BOTH axes — "may I answer a
// destructive question?" and "may I overwrite the stored blob?" — because today
// every degraded status answers no to both. Split them only when a status needs one
// without the other.
inline bool ledgerDegraded(LedgerStatus s) {
return s == LedgerStatus::Unreadable || s == LedgerStatus::FutureVersion;
}
struct LedgerLoad {
LedgerStatus status = LedgerStatus::Fresh;
+7 -5
View File
@@ -14,20 +14,22 @@ ProtectionAnswer pruneProtection(const TrackingState& state) {
answer.unreadableUsageKeys = state.usage.offendingKeys;
}
if (state.ledgerStatus == LedgerStatus::Unreadable) {
if (ledgerDegraded(state.ledgerStatus)) {
answer.blocked = true;
answer.ledgerUnreadable = true;
return answer; // ownedPaths left empty -> (owned ∩ present) is empty
answer.ledgerUnreadable = state.ledgerStatus == LedgerStatus::Unreadable;
answer.ledgerFutureVersion = state.ledgerStatus == LedgerStatus::FutureVersion;
}
answer.ownedPaths = state.ledger.ownedPaths();
// Populated only on a clean answer: on ANY block an empty `owned` makes
// (owned ∩ present) empty, so even a caller that ignored `blocked` deletes nothing.
if (!answer.blocked) answer.ownedPaths = state.ledger.ownedPaths();
return answer;
}
Answer tiedUsageExists(const TrackingState& state, const std::string& capturePath,
const std::string& ownUsageKey) {
if (capturePath.empty()) return Answer::Indeterminate;
if (state.ledgerStatus == LedgerStatus::Unreadable) return Answer::Indeterminate;
if (ledgerDegraded(state.ledgerStatus)) return Answer::Indeterminate;
if (state.usage.abortPrune) return Answer::Indeterminate;
for (const wire::CountedUsage& counted : state.usage.counted) {
+7 -7
View File
@@ -26,12 +26,13 @@ struct TrackingState {
// blocker fields say what to tell the operator; the caller composes the message
// (the pure core does not know ext-state key spellings).
//
// Both path lists stay populated on a block as belt-and-braces: heldPaths carries
// the usage fold's protect-all set, and ownedPaths is left EMPTY on an unreadable
// ledger, so a caller that ignored `blocked` still computes an empty orphan set.
// Belt-and-braces on ANY block, whichever side blocked: heldPaths still carries the
// usage fold's protect-all set, and ownedPaths is left EMPTY — so a caller that
// ignored `blocked` still computes an empty orphan set rather than deleting.
struct ProtectionAnswer {
bool blocked = false;
bool ledgerUnreadable = false;
bool ledgerUnreadable = false; // corrupt blob — repairable by clearing the key
bool ledgerFutureVersion = false; // written by a newer build — must NOT be cleared
std::vector<std::string> unreadableUsageKeys;
std::vector<std::string> heldPaths; // union into prune's `referenced`
std::vector<std::string> ownedPaths; // prune's `owned`
@@ -46,13 +47,12 @@ enum class Answer { No, Yes, Indeterminate };
// Does a usage tied to `capturePath` exist, other than the asking instance's own?
//
// * unreadable ledger, or any unreadable usage record -> Indeterminate.
// * degraded ledger, or any unreadable usage record -> Indeterminate.
// * otherwise Yes iff some counting live record holds the path.
//
// `ownUsageKey` is the asking instance's own "rsusage_<guid>" key, excluded from the
// scan so a bake does not see itself; empty counts every holder. A `unioned` record
// is NEVER excluded — it carries more than one incarnation's holds, so attributing
// it to a single owner could hide a sibling's tie.
// is never excluded.
//
// Never-recorded (no ledger record for the path) is a definite answer, not an
// abstention: a pre-existing capture nothing holds answers No.
+2 -1
View File
@@ -3,12 +3,13 @@
## Scope
Tiny, dependency-free pure helpers linked by both artifacts: whole-file byte
loading and unit-interval clamping.
loading, unit-interval clamping, and the absolute-path rejection test.
## Modules
- `file_bytes` (`core/util`) — the ONE whole-file byte loader (Q-W1), linked by both artifacts; blocking I/O, off-audio-thread only.
- `clamp01` (`core/util`, header-only) — the ONE unit-interval clamp (Q-W1), replacing four per-module static copies; NaN passes through unchanged rather than collapsing to a bound.
- `relative_path` (`core/util`, header-only) — the ONE absolute-path rejection test behind the relative-paths-only invariant, shared by `bank_model` (`Sample.relativePath`) and `core/tracking/origin_ledger` (`OriginRecord.relativePath`). The two must reject identically or a path one accepts could be smuggled past the other; that is why it is one function and not two.
## Gotchas
+26
View File
@@ -0,0 +1,26 @@
#pragma once
// relative_path — the ONE absolute-path rejection test behind the
// relative-paths-only invariant, shared by every persisted path family
// (Sample.relativePath, OriginRecord.relativePath).
//
// Rejects rather than normalizes: the pure core has no knowledge of the project
// root, so any "normalization" would be a guess that could point at the wrong file.
#include <cctype>
#include <string>
namespace reasampler::util {
// True for POSIX root ("/x"), UNC ("\\host\share"), and any leading <alpha>: —
// including drive-RELATIVE forms ("C:foo.wav"), which resolve against the drive's
// current directory rather than the project root and so violate the invariant just
// as much as "C:\foo.wav" does.
inline bool isAbsolutePath(const std::string& p) {
if (p.empty()) return false;
if (p[0] == '/' || p[0] == '\\') return true;
if (p.size() >= 2 && std::isalpha(static_cast<unsigned char>(p[0])) && p[1] == ':')
return true;
return false;
}
} // namespace reasampler::util
+5 -10
View File
@@ -57,8 +57,7 @@ This directory owns two cross-artifact contracts specifically:
protected (identity-failure net — a matcher failure must never degrade toward
delete). The guarantee: a capture held by any live instance can never be deleted; if
the prune cannot determine with certainty which captures are held, it aborts
entirely (deletes nothing). Over-protection is the accepted residual; under-protection
is a data-loss bug. The fold reports `counted` — the live records still attributed to
entirely (deletes nothing). The fold reports `counted` — the live records still attributed to
their `rsusage_*` keys — because the flattened path list cannot answer "who holds
this"; `counted` is empty whenever `abortPrune` is set, since attribution is exactly
what an unreadable record destroys.
@@ -70,14 +69,10 @@ This directory owns two cross-artifact contracts specifically:
Resolution always leans over-protect: same-nonce + not-unioned → clean replace;
same-track foreign nonce or unioned → union; cross-track foreign nonce → remint under
a fresh key. None of the three directions can under-protect.
- **Deferred follow-up (TODO.md, deliberately NOT absorbed by the tracking
consolidation):** `ownerNonce` is not persisted, so after save→reopen an instance
cannot recognize its own prior-session usage record — it unions and marks the record
`unioned` forever, so prune stops reclaiming captures the instance once held but no
longer uses (safe, but the bank folder grows unbounded). This is a *completeness*
wart, not a safety one; every candidate fix examined so far trades it for a new
under-protection window, which the consolidation's own safety mandate forbids. See
`docs/TODO.md` for the constraint and the rejected session-epoch candidate.
- **Deferred follow-up:** `ownerNonce` is not persisted, so after save→reopen an
instance cannot recognize its own prior-session usage record and unions forever.
Reasoning, constraints, and the rejected candidates live in `docs/TODO.md`
("Persist ReaSampler 9000 instance identity…").
## Modules
+3 -5
View File
@@ -17,11 +17,9 @@
// per-instance key, never banks/view/tail/assign; the bridge's write entry
// point structurally accepts only "rsusage_"-prefixed keys.
//
// THE SAFETY PROPERTY (overrides every other consideration): every failure,
// ambiguity, or uncertainty here must fail-safe toward PROTECT. Over-protection
// (prune skips a reclaimable file, or refuses to run) is an accepted residual;
// under-protection (deleting a file an instance may still be playing) is a
// data-loss bug. Three folds enforce this:
// Every failure, ambiguity, or uncertainty here fails safe toward PROTECT (the
// territory-wide asymmetry, stated in core/tracking/CLAUDE.md). Three folds enforce
// it here:
// * sibling-collision -> UNION, never clean-replace over a foreign writer;
// * zero-identified -> records exist but no instance was identified live ->
// protect ALL records' paths (a matcher failure must
-2
View File
@@ -273,8 +273,6 @@ ImportResult importFileIntoActiveBank(const std::string& absoluteSourcePath) {
s.createdTimestamp = nowSec;
const AddResult r = book.activeIndex().add(s);
// Record the birth regardless of outcome — the tool WROTE the file, so prune must
// attribute it even in the narrow Collapsed race below.
g_session->recordCreated(s, tracking::OriginKind::Ingest);
switch (r) {
+18 -3
View File
@@ -6,6 +6,7 @@
#include <string>
#include <vector>
#include "core/version/app_version.h" // extStateNamespace — the printed recovery line must be channel-correct
#include "shell/persist/session.h" // ReaSamplerSession — pruneDryRun / pruneOrphanSet / pruneReclaim
#define REAPERAPI_MINIMAL
@@ -27,22 +28,36 @@ void doBankPruneFolder(ReaSamplerSession& session) {
// set unknowable, so the prune HALTS outright rather than proceed with degraded
// protection. Both blockers can fire at once; report each one that did.
if (report.blockedByTracking) {
// Every printed recovery line names THIS build's ext-state namespace: a beta
// user handed the stable spelling would clear the stable channel's key in
// their project and still be blocked.
const std::string& ns = version::extStateNamespace();
std::string msg =
"ReaSampler prune: ABORTED -- the file-tracking state could not be read. "
"Nothing was deleted.\n";
if (report.ledgerUnreadable) {
msg += "The stored file-tracking ledger is malformed. It has been left "
"intact rather than overwritten, so it can be repaired or cleared:\n"
" reaper.SetProjExtState(0, \"reasampler\", \"owned_files\", \"\")\n"
" reaper.SetProjExtState(0, \"" + ns + "\", \"owned_files\", \"\")\n"
"Clearing it makes every existing bank file un-reclaimable (they stop "
"being attributable to ReaSampler); no file is lost.\n";
"being attributable to ReaSampler); no file is lost. Reopen the "
"project afterwards -- the block is held for the rest of this "
"session, and until then no new capture is persisted to the ledger "
"either.\n";
}
if (report.ledgerFutureVersion) {
msg += "The stored file-tracking ledger was written by a NEWER version of "
"ReaSampler than this one, so its records cannot be read safely. It "
"has been left intact and will NOT be overwritten. Reopen the project "
"with that newer version -- do NOT clear this key from here, that "
"would discard tracking records this build cannot see.\n";
}
if (!report.unreadableUsageKeys.empty()) {
msg += "One or more instance usage records could not be read or decoded. "
"If the owning instance is still loaded it will republish its record "
"on the next poll tick, clearing the abort. If the instance no longer "
"exists (the key is an orphaned corrupt record), clear it manually:\n"
" reaper.SetProjExtState(0, \"reasampler\", \"<key>\", \"\")\n"
" reaper.SetProjExtState(0, \"" + ns + "\", \"<key>\", \"\")\n"
"Offending key(s):\n";
for (const std::string& key : report.unreadableUsageKeys) {
msg += " " + key + "\n";
+4 -4
View File
@@ -455,13 +455,13 @@ void RunRecaptureFromSource(ReaSamplerSession& session)
// One batched undo point around the in-place mutation; index-only ext-state,
// nothing placed on the timeline.
Undo_BeginBlock2(nullptr);
// Before the index outcome is even known: the render above already wrote the file,
// and makeUniqueTag guarantees a NEW path, so this is a second record carrying the
// same sampleId as the original. The superseded file becomes an orphan for prune.
session.recordCreated(updated, tracking::OriginKind::Recapture);
const bool changed = session.book().updateSampleInPlace(sampleId, updated);
if (changed)
{
// Record the regenerated file's birth; the superseded file becomes an orphan
// for prune to reclaim. A re-capture onto the same path is a dedup no-op, so
// the original birth record — not this one — stays authoritative.
session.recordCreated(updated, tracking::OriginKind::Recapture);
// Regenerating the same id's audio is exactly why instances need the generation
// bump — they'd otherwise keep playing stale audio until reload. Bumped inside
// the undo block so undo rolls back the generation with the rest of the blob.
@@ -250,9 +250,6 @@ CaptureResult captureAndIndexOne(ReaSamplerSession& session,
// AddResult tells a fresh add from a hash-dedup collapse, so the assign path (S8) can
// target the sample actually in the bank (the existing entry on a collapse).
const model::AddResult addResult = session.bank().add(res.sample);
// Record the birth at the same point the Sample is added, regardless of the index
// AddResult — even a hash-collapse still WROTE a file the tool owns, and the ledger
// dedups a repeat path itself (prune reconciles ledger vs index later).
session.recordCreated(res.sample, tracking::OriginKind::Capture);
// Resolve the LANDED bank-index id into res.sample.id for the S8 assign path: the new
+19 -7
View File
@@ -31,8 +31,6 @@ void CommitRealtimeResult(ReaSamplerSession& session, const CaptureResult& res)
return;
}
session.bank().add(res.sample);
// Record the birth regardless of the index AddResult — even a hash-collapse still
// wrote a file the tool owns; the ledger dedups a repeat path itself.
session.recordCreated(res.sample, tracking::OriginKind::Capture);
// A capture add changes what a live instance could play, so bump the generation
// before persisting to refresh instances.
@@ -56,14 +54,28 @@ void DriveRealtimeCapture(ReaSamplerSession& session)
{
RealtimeTickResult r = g_rtBackend.abort(*g_rtCapture);
// Log without persisting — we restored into the original project but must
// not persist into the now-active foreign one. A Failed abort surfaces
// not persist into the now-active foreign one: the ledger and bank we would
// have to write belong to the project we just left. A Failed abort surfaces
// abort()'s own message, distinguishing a clean tab-switch abort from the
// closed-project case (nothing restored because the pointers were already
// freed).
if (r.status == RealtimeTickStatus::Done)
ShowConsoleMsg("ReaSampler realtime capture: project switched mid-record -- "
"captured audio restored into the original project; not "
"persisted to avoid crossing projects.\n");
//
// The abort already moved the recorded WAV into the ORIGINAL project's bank
// folder, so it survives here untracked — deleting a user's just-recorded
// audio is the destructive direction and is not this shell's call, so the
// path is NAMED instead and the operator decides (docs/TODO.md).
if (r.status == RealtimeTickStatus::Done && r.result.status == CaptureStatus::Ok)
ShowConsoleMsg(("ReaSampler realtime capture: project switched mid-record -- "
"captured audio restored into the original project; not "
"persisted to avoid crossing projects. The recorded file was "
"left in the original project's bank folder as '" +
r.result.sample.relativePath +
"', untracked -- reopen that project and re-import it, or "
"delete it by hand.\n").c_str());
else if (r.status == RealtimeTickStatus::Done)
ShowConsoleMsg(("ReaSampler realtime capture: project switched mid-record -- "
"the capture was aborted and produced no usable file: " +
r.result.message + "\n").c_str());
else
ShowConsoleMsg(("ReaSampler realtime capture: project changed mid-record -- " +
r.result.message + "\n").c_str());
+12 -9
View File
@@ -34,17 +34,20 @@ REAPER/filesystem-facing half only, and it gathers rather than decides.
record/input chains, containers recursively, take FX), and folds via the pure
`sample_usage::foldUsageRecords`. Read-only at prune-scan time: `usage_scan` writes
no ext-state.
- **A malformed tracking ledger is Unreadable, never "empty".** `ext_state_io` keeps
- **A ledger this build cannot read is degraded, never "empty".** `ext_state_io` keeps
the `LedgerStatus` alongside the ledger, and `saveToActiveProject` SKIPS the
`owned_files` write while it is `Unreadable` — replacing a corrupt blob would
destroy the only record of every file created before the corruption, silently
turning them into permanently unreclaimable foreign files. Captures made during
such a session are recorded in memory but not persisted; they degrade to foreign
(untouchable), which is the safe direction.
`owned_files` write while `tracking::ledgerDegraded` holds — replacing a blob we
could not read would destroy the only record of every file created before it,
silently turning them into permanently unreclaimable foreign files. The status is
written only by `loadFromProject`, so it is sticky until the project is reloaded:
captures made during such a session are recorded in memory but not persisted, and
degrade to foreign (untouchable), which is the safe direction.
- **`PruneReport` carries `blockedByTracking` + `ledgerUnreadable` /
`unreadableUsageKeys`**; dry-run, orphan-set, and reclaim each independently abort
(delete nothing) on a block. `BANK_PRUNE_FOLDER` (in `shell/actions`) halts on the
flag and prints whichever blockers fired.
`ledgerFutureVersion` / `unreadableUsageKeys`**; dry-run, orphan-set, and reclaim
each independently abort (delete nothing) on a block. `BANK_PRUNE_FOLDER` (in
`shell/actions`) halts on the flag and prints whichever blockers fired, with
channel-correct recovery lines — a corrupt blob may be cleared, a newer build's
must not be.
## Modules
+10 -9
View File
@@ -161,11 +161,9 @@ bool ReaSamplerSession::saveToActiveProject() {
SetProjExtState(static_cast<ReaProject*>(proj), projExtNamespace(),
kProjExtTailKey, tailJson.c_str());
// Written on every save so the ledger and the bank stay in lockstep on disk —
// EXCEPT over a blob we could not read: replacing it would destroy the only
// record of every file created before the corruption, silently turning them
// into permanently unreclaimable foreign files.
if (trackingStatus_ != tracking::LedgerStatus::Unreadable) {
// Never write over a blob this build could not read (see this directory's
// CLAUDE.md for why the suppression, not a rewrite, is the safe direction).
if (!tracking::ledgerDegraded(trackingStatus_)) {
const std::string ledgerJson = tracking_.serialize();
SetProjExtState(static_cast<ReaProject*>(proj), projExtNamespace(),
kProjExtOwnedKey, ledgerJson.c_str());
@@ -234,10 +232,8 @@ capture::TailSetting loadTailSetting(ReaProject* proj) {
return *loaded;
}
// Absent/empty key -> Fresh (a new project, or a bank predating the ledger).
// Malformed -> Unreadable, which halts the prune and suppresses the next write
// rather than degrading to an empty ledger that looks like "nothing was ever
// created".
// Absent/empty key -> Fresh (a new project, or a bank predating the ledger). Anything
// this build cannot read is a degraded status, never an empty ledger.
tracking::LedgerLoad loadOriginLedger(ReaProject* proj) {
if (!proj) return tracking::LedgerLoad{};
tracking::LedgerLoad load = tracking::loadLedger(
@@ -246,6 +242,11 @@ tracking::LedgerLoad loadOriginLedger(ReaProject* proj) {
ShowConsoleMsg("ReaSampler: the stored file-tracking ledger is malformed. Prune "
"is halted for this project and the stored value is left intact "
"for recovery.\n");
} else if (load.status == tracking::LedgerStatus::FutureVersion) {
ShowConsoleMsg("ReaSampler: the stored file-tracking ledger was written by a "
"NEWER version of ReaSampler. Prune is halted for this project "
"and the stored value will not be overwritten -- reopen the "
"project with that version.\n");
}
return load;
}
+3
View File
@@ -72,6 +72,7 @@ struct PruneScan {
std::unordered_map<std::string, std::uint64_t> sizeByRel;
bool blocked = false;
bool ledgerUnreadable = false;
bool ledgerFutureVersion = false;
std::vector<std::string> unreadableUsageKeys;
};
@@ -134,6 +135,7 @@ PruneScan scanPruneOrphans(const BankBook& book, const tracking::OriginLedger& l
// action tell the user what to recover.
scan.blocked = true;
scan.ledgerUnreadable = protection.ledgerUnreadable;
scan.ledgerFutureVersion = protection.ledgerFutureVersion;
scan.unreadableUsageKeys = protection.unreadableUsageKeys;
return scan;
}
@@ -211,6 +213,7 @@ reclaim::PruneReport ReaSamplerSession::pruneDryRun() const {
// the prune refused to run.
report.blockedByTracking = scan.blocked;
report.ledgerUnreadable = scan.ledgerUnreadable;
report.ledgerFutureVersion = scan.ledgerFutureVersion;
report.unreadableUsageKeys = scan.unreadableUsageKeys;
return report;
}
+13 -1
View File
@@ -35,6 +35,7 @@
#define REAPERAPI_MINIMAL
#define REAPERAPI_WANT_MarkProjectDirty
#define REAPERAPI_WANT_SetProjExtState
#define REAPERAPI_WANT_ShowConsoleMsg
#include "reaper_plugin_functions.h"
namespace reasampler {
@@ -56,7 +57,18 @@ void ReaSamplerSession::recordCreated(const model::Sample& sample,
// rather than re-deriving keeps one source for the lineage fact. Absent
// provenance is a root capture, not a gap.
if (sample.provenance) rec.parentSampleId = sample.provenance->parentSampleId;
tracking_.record(rec);
const tracking::RecordResult result = tracking_.record(rec);
// A rejection means a file exists that nothing attributes to us — invisible
// otherwise, and exactly the gap this ledger exists to close. AlreadyPresent is
// the normal dedup outcome, not a gap.
if (result == tracking::RecordResult::RejectedEmptyPath ||
result == tracking::RecordResult::RejectedAbsolutePath) {
ShowConsoleMsg(("ReaSampler: could not record the origin of '" +
sample.relativePath +
"' -- the path is empty or absolute. The file is NOT tracked and "
"prune will treat it as a foreign file.\n").c_str());
}
}
bool ReaSamplerSession::consumeLoadSignal() {
+16 -14
View File
@@ -71,17 +71,20 @@ public:
capture::TailSetting& tail() { return tail_; }
const capture::TailSetting& tail() const { return tail_; }
// Birth records for the files the system itself created. Read as a PAIR —
// the ledger alone cannot say whether an absent record means never-recorded
// or unreadable, and the two demand opposite treatment. This is the reach
// any consumer outside persist uses to build a tracking::TrackingState.
const tracking::OriginLedger& tracking() const { return tracking_; }
tracking::LedgerStatus trackingStatus() const { return trackingStatus_; }
// Record a system-created file at the moment it exists — the ONLY way a
// birth record is written, so lineage can never be backfilled from a later
// guess. Lineage is read off the Sample's own provenance, the same act that
// stamped it, so the two cannot disagree. A repeat path is a no-op.
// Record a system-created file at the moment it exists — the ONLY way a birth
// record is written. Lineage is read off the Sample's own provenance, the same
// act that stamped it, so the two cannot disagree; where they later diverge the
// ledger record is authoritative (core/tracking/CLAUDE.md).
//
// Call it at EVERY creation site regardless of the bank's AddResult: a
// hash-collapse still wrote a file the tool owns, and an unrecorded file is a
// permanently unreclaimable foreign file. A rejected record is reported to the
// console — a file with no record is the gap this track exists to eliminate.
//
// A consumer outside persist that needs the ledger must take it WITH its
// LedgerStatus (a tracking::TrackingState); no accessor exposes one without the
// other, because an absent record and an unreadable ledger demand opposite
// treatment.
void recordCreated(const model::Sample& sample, tracking::OriginKind kind);
// The version that last wrote the active project: PreVersioning (no
@@ -153,9 +156,8 @@ private:
ViewModeModel view_; // reset to default on a project with no stored view_state
capture::TailSetting tail_; // reset to default (None / 2s) with no stored tail key
tracking::OriginLedger tracking_; // reset to empty/stored on EVERY load path, never inherited
// Unreadable is sticky for the project's session: it halts the prune AND
// suppresses the ledger write, so a corrupt blob survives for recovery
// instead of being silently replaced by a ledger missing every earlier file.
// Written only by loadFromProject, so a degraded status is sticky until the
// project is reloaded (see this directory's CLAUDE.md).
tracking::LedgerStatus trackingStatus_ = tracking::LedgerStatus::Fresh;
version::WritingVersion writingVersion_; // recovered per load; PreVersioning default
std::int64_t bankGeneration_ = 0; // recovered per load (absent -> 0); monotonic