tracking: read the ledger's version, not just write it; clear owned on any block; channel-correct prune recovery
This commit is contained in:
@@ -165,3 +165,17 @@ Forward-looking follow-ups. Deferred by decision, not oversight — each entry r
|
||||
**Priority / risk.** Not stated as a priority level; the source characterizes this as a "panel-polish detail."
|
||||
|
||||
**Done looks like.** Not stated in the source beyond choosing one of the three placement options.
|
||||
|
||||
## A realtime capture interrupted by a project switch leaves an untracked file behind
|
||||
|
||||
**Context (found by the tracking-consolidation review, 2026-07-30).** `DriveRealtimeCapture` detects that the active project is no longer the one the in-flight capture belongs to, aborts the backend, and drops the handle. On a `Done` abort the backend has *already* moved the recorded WAV into the **original** project's bank folder (`capture_realtime_finalize`), so a file the tool created exists with no bank entry and no ledger record.
|
||||
|
||||
**The wart.** This is the one hole in "no silent gaps": a system-created file that is never recorded. It is in the safe direction — an untracked file is foreign, so prune will never reclaim it — but it is permanent, and the bank folder grows by one orphan per interrupted record.
|
||||
|
||||
**Intended fix.** Record the birth against the project the capture belongs to. Neither half is available at the switch point: `session`'s ledger and `saveToActiveProject` both target the *active* project, which is by definition the wrong one here.
|
||||
|
||||
**The constraint the fix MUST handle.** Writing the record into the now-active project would attribute another project's file to it — a worse error than the gap, since prune would then consider deleting a file it does not own the folder for. Deleting the stranded file instead was considered and rejected: it is the user's just-recorded audio, and prune is the system's only deletion authority over bank-folder bytes (`shell/persist/CLAUDE.md`) — a shell self-cleanup exemption covers transient scratch, not a finished recording. The fix therefore needs a deferred write against a *named* project (or a re-entry into the original project on the next poll), not a change at the abort site.
|
||||
|
||||
**Priority / risk.** Low / deferred. Mitigated in the meantime: the console message names the stranded file's project-relative path, so the operator can recover or remove it rather than discovering it later as an unexplained orphan.
|
||||
|
||||
**Done looks like.** Switching projects mid-record leaves the recorded file with a ledger record in the project it belongs to, so a later prune of that project can reclaim it normally.
|
||||
|
||||
@@ -9,12 +9,12 @@ has — stay in the consumers; this module owns lexing/emitting only.
|
||||
|
||||
## Modules
|
||||
|
||||
- `json` (`core/json`) — the ONE hand-rolled JSON lexical layer (Q-W1): string/number/bool/null tokens, the scoped object `Writer`, and the bounds-checked `Reader` cursor, byte-compatible with the five pre-extraction per-module writers it replaced (`bank_model` / `bank_book` / `view_mode_model` / `owned_manifest` / `tail_control`). Domain grammars stay in the consumers; this owns lexing/emitting only.
|
||||
- `json` (`core/json`) — the ONE hand-rolled JSON lexical layer (Q-W1): string/number/bool/null tokens, the scoped object `Writer`, and the bounds-checked `Reader` cursor, byte-compatible with the five pre-extraction per-module writers it replaced (`bank_model` / `bank_book` / `view_mode_model` / the retired `owned_manifest`, now `core/tracking/origin_ledger` / `tail_control`). Domain grammars stay in the consumers; this owns lexing/emitting only.
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Byte-compatible with the five pre-extraction per-module writers it replaced
|
||||
(`bank_model` / `bank_book` / `view_mode_model` / `owned_manifest` /
|
||||
(`bank_model` / `bank_book` / `view_mode_model` / the retired `owned_manifest` /
|
||||
`tail_control`) — a change here risks silently breaking round-trip compatibility
|
||||
with ext-state blobs already persisted by projects written before the Q-W1
|
||||
extraction.
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
// unknown-value skipping, and the emit side (escaping, %.17g/%d/%lld rendering,
|
||||
// the scoped object writer). Domain grammars — which keys exist, what shape each
|
||||
// value takes — stay in the consumers (bank_model, bank_book, view_mode_model,
|
||||
// owned_manifest, tail_control).
|
||||
// origin_ledger, tail_control).
|
||||
//
|
||||
// Byte-compatibility contract (load-bearing): the emit helpers reproduce the prior
|
||||
// per-module writers EXACTLY — writeEscaped's escape set, %.17g for doubles
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
#include "core/model/bank_model.h"
|
||||
|
||||
#include <cctype>
|
||||
|
||||
#include "core/json/json.h"
|
||||
#include "core/util/relative_path.h"
|
||||
|
||||
// bank_model implementation. JSON rides on the shared core/json lexical layer;
|
||||
// only the Sample/index DOMAIN grammar lives here. Doubles are emitted with 17
|
||||
@@ -44,22 +43,7 @@ bool Sample::operator==(const Sample& o) const {
|
||||
provenance == o.provenance && createdTimestamp == o.createdTimestamp;
|
||||
}
|
||||
|
||||
// -- path invariant -------------------------------------------------------
|
||||
|
||||
// Rejects absolute paths rather than normalizing them: the pure model has no
|
||||
// knowledge of the project root, so any "normalization" would be a guess that
|
||||
// could point at the wrong file. Covers POSIX ("/x"), Windows drive ("C:\x",
|
||||
// "C:/x", "C:foo" drive-relative), and UNC ("\\host\share") forms. Any leading
|
||||
// <alpha>: is rejected regardless of what follows — drive-relative paths
|
||||
// ("C:foo.wav") resolve against the drive's current directory, not the project
|
||||
// root, so they violate relative-paths-only just as much as "C:\foo.wav" does.
|
||||
static bool isAbsolutePath(const std::string& p) {
|
||||
if (p.empty()) return false;
|
||||
if (p[0] == '/' || p[0] == '\\') return true; // POSIX root or UNC
|
||||
if (p.size() >= 2 && std::isalpha(static_cast<unsigned char>(p[0])) && p[1] == ':')
|
||||
return true; // Windows drive (C:\, C:/, C:foo, C:)
|
||||
return false;
|
||||
}
|
||||
using util::isAbsolutePath;
|
||||
|
||||
// -- BankModel ------------------------------------------------------------
|
||||
|
||||
|
||||
@@ -32,7 +32,7 @@ enumeration and the actual deletion live in `shell/persist` (`prune_fs`), not he
|
||||
|
||||
## Modules
|
||||
|
||||
- `prune_reconcile` — pure prune core: `pruneOrphans(present, referenced, owned)` computes `(owned ∩ present) − referenced`; the safety-critical "which files are orphans" decision, filesystem-free and hard-tested before any I/O exists. `mergeReferenced(bankRefs, heldPaths)` unions live instance holds into the referenced-set. Two of the three inputs (`owned`, and the held half of `referenced`) come from `core/tracking`'s authority, never assembled ad hoc by a shell. `PruneReport` carries the authority's verdict: `blockedByTracking` + `ledgerUnreadable` / `unreadableUsageKeys`.
|
||||
- `prune_reconcile` — pure prune core: `pruneOrphans(present, referenced, owned)` computes `(owned ∩ present) − referenced`; the safety-critical "which files are orphans" decision, filesystem-free and hard-tested before any I/O exists. `mergeReferenced(bankRefs, heldPaths)` unions live instance holds into the referenced-set. Two of the three inputs (`owned`, and the held half of `referenced`) come from `core/tracking`'s authority, never assembled ad hoc by a shell. `PruneReport` carries the authority's verdict: `blockedByTracking` + `ledgerUnreadable` / `ledgerFutureVersion` / `unreadableUsageKeys`.
|
||||
|
||||
## Gotchas
|
||||
|
||||
|
||||
@@ -53,9 +53,11 @@ namespace reasampler::reclaim {
|
||||
// (count 0, empty list) and the prune must HALT — deleting with
|
||||
// degraded protection is the data-loss direction. Set by the scan
|
||||
// shell, never by buildPruneReport (which stays a pure tally).
|
||||
// * ledgerUnreadable / unreadableUsageKeys — which side blocked, so the action can
|
||||
// tell the operator what to recover. The key names are the exact
|
||||
// "rsusage_<guid>" spellings.
|
||||
// * ledgerUnreadable / ledgerFutureVersion / unreadableUsageKeys — which side
|
||||
// blocked, so the action can tell the operator what to recover. The
|
||||
// two ledger flags need opposite advice (a corrupt blob may be
|
||||
// cleared; a newer build's blob must not be). The key names are the
|
||||
// exact "rsusage_<guid>" spellings.
|
||||
struct PruneReport {
|
||||
std::size_t count = 0;
|
||||
std::uint64_t totalBytes = 0;
|
||||
@@ -63,6 +65,7 @@ struct PruneReport {
|
||||
bool truncated = false;
|
||||
bool blockedByTracking = false;
|
||||
bool ledgerUnreadable = false;
|
||||
bool ledgerFutureVersion = false;
|
||||
std::vector<std::string> unreadableUsageKeys;
|
||||
};
|
||||
|
||||
|
||||
+28
-12
@@ -21,21 +21,33 @@ non-destructive side of the question being asked — over-protection (prune skip
|
||||
reclaimable file, or refuses to run; resample adds instead of replacing) is an
|
||||
accepted residual; under-protection is a data-loss bug.
|
||||
|
||||
**No silent gaps.** A system-created file is tracked from the instant it exists.
|
||||
`ReaSamplerSession::recordCreated` is the only writer, called at the same point the
|
||||
`Sample` is added, and it reads lineage off that `Sample`'s own provenance — so the
|
||||
recipe fingerprint and the lineage record come from one act and cannot disagree.
|
||||
**No silent gaps — in memory at creation, on disk at the next save.**
|
||||
`ReaSamplerSession::recordCreated` is the only writer, called in the same
|
||||
straight-line block as the bank add with no I/O or early return between, so a
|
||||
system-created file is tracked *in memory* the instant it exists. The record reaches
|
||||
the `.rpp` only at the following `saveToActiveProject()`; a crash in that window, or a
|
||||
whole session running with a degraded ledger (below), leaves the file untracked —
|
||||
foreign, therefore never reclaimed, which is the safe direction. That residual is
|
||||
accepted and stated here rather than papered over by a stronger claim.
|
||||
|
||||
**Lineage is never backfilled.** `OriginLedger::record` on a path already present is
|
||||
`AlreadyPresent` and leaves the stored record untouched. A record says what was known
|
||||
at birth or says nothing at all; nothing may later rewrite history from a guess.
|
||||
Consequence, and the reason both blobs can carry a parent id: `recordCreated` *seeds*
|
||||
the record from the `Sample`'s own `provenance.parentSampleId` (one source, one act),
|
||||
but the `Sample` is mutable via `updateSampleInPlace` and the record is not — so where
|
||||
the two ever diverge, **the ledger record is authoritative for lineage** and the
|
||||
`Sample`'s copy is only the recipe facet's view of it.
|
||||
|
||||
**Never-recorded and unreadable are different absences.** `LedgerStatus` keeps them
|
||||
**Never-recorded and degraded are different absences.** `LedgerStatus` keeps them
|
||||
apart, and only `loadLedger` can tell them apart (an empty stored value is not valid
|
||||
JSON, so the parser alone cannot). `Fresh` — a new project or a bank predating the
|
||||
ledger — blocks nothing and yields definite answers. `Unreadable` blocks every
|
||||
destructive answer AND suppresses the next write, so a corrupt blob survives for
|
||||
recovery instead of being replaced by a ledger missing every earlier file.
|
||||
ledger — blocks nothing and yields definite answers. `Unreadable` (corrupt) and
|
||||
`FutureVersion` (a `"v"` newer than this build reads) are the degraded pair: each
|
||||
blocks every destructive answer AND suppresses the next write, so the stored blob
|
||||
survives for recovery instead of being replaced by a ledger missing every earlier
|
||||
file. `ledgerDegraded()` is the one test for both treatments — they coincide today, so
|
||||
the two axes are deliberately not split.
|
||||
|
||||
**Consumers cannot disagree.** Both answers come out of one `TrackingState`. The two
|
||||
universes differ deliberately — prune protects bank-referenced paths, ledger-owned
|
||||
@@ -54,9 +66,9 @@ birth record is written for every system-created file, ambiguous parentage or no
|
||||
- `origin_ledger` — the record family: `OriginRecord` (project-relative path, `OriginKind`,
|
||||
the sample id minted at birth, the parent sample id) and the insertion-ordered,
|
||||
path-keyed, deduplicated `OriginLedger` that holds them, with its JSON codec and the
|
||||
`loadLedger` three-way `Fresh` / `Loaded` / `Unreadable` classification. Persisted
|
||||
under the FOREVER-STABLE `owned_files` ext-state key; the legacy path-only shape
|
||||
(`{"owned":[...]}`) lifts in as `Unknown`-kind records with no lineage.
|
||||
`loadLedger` `Fresh` / `Loaded` / `Unreadable` / `FutureVersion` classification.
|
||||
Persisted under the FOREVER-STABLE `owned_files` ext-state key; the legacy path-only
|
||||
shape (`{"owned":[...]}`) lifts in as `Unknown`-kind records with no lineage.
|
||||
- `tracking_authority` — the one decision surface: `pruneProtection` (the `owned` and
|
||||
held-path inputs prune's set algebra consumes, plus the blocked/blockers verdict)
|
||||
and `tiedUsageExists` (`Yes` / `No` / `Indeterminate`, per capture, excluding the
|
||||
@@ -66,7 +78,11 @@ birth record is written for every system-created file, ambiguous parentage or no
|
||||
|
||||
- `OriginKind` values are PERSISTED INTEGERS — never renumber, only append. An
|
||||
unrecognized value degrades to `Unknown` rather than failing the parse: a vocabulary
|
||||
gap must not halt the prune.
|
||||
gap must not halt the prune. This degrade-don't-halt rule is about *field
|
||||
vocabulary* only — an unrecognized **document version** (`"v"`) does the opposite
|
||||
and blocks, because a changed record shape makes the whole ledger untrustworthy.
|
||||
- `OriginRecord.sampleId` is NOT unique: a recapture writes a new file under the same
|
||||
bank id, so two records can share a `sampleId`. `relativePath` is the only key.
|
||||
- A `unioned` usage record can never be excluded as "my own" — it carries more than one
|
||||
incarnation's holds, so attributing it to a single owner could hide a sibling's tie.
|
||||
- The set algebra prune runs on `(owned ∩ present) − referenced` is `core/reclaim`'s,
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
#include "core/tracking/origin_ledger.h"
|
||||
|
||||
#include <cctype>
|
||||
#include <utility>
|
||||
|
||||
#include "core/json/json.h"
|
||||
#include "core/util/relative_path.h"
|
||||
|
||||
// Version ladder for the stored blob, under the FOREVER-STABLE "owned_files" key:
|
||||
//
|
||||
@@ -13,21 +14,20 @@
|
||||
// v1 blobs lift to v2 records with kind Unknown and empty ids — a pre-existing bank
|
||||
// keeps every protection it had (the paths are still owned) and gains no invented
|
||||
// lineage. Both shapes parse; only v2 is written.
|
||||
//
|
||||
// "v" is READ and validated, not just written: a v3 record shape parsed by these v2
|
||||
// rules would yield a plausible-but-partial ledger, and the next save would overwrite
|
||||
// the v3 blob with that truncation. A version above kLedgerVersion is therefore its
|
||||
// own degraded status, never a Loaded ledger.
|
||||
|
||||
namespace reasampler::tracking {
|
||||
|
||||
namespace {
|
||||
|
||||
// Any leading '/' or '\' (POSIX root / UNC), or a leading <alpha>: (Windows drive,
|
||||
// incl. drive-relative "C:foo"). Must match bank_model's rejection exactly — the
|
||||
// ledger holds the same kind of path as Sample.relativePath.
|
||||
bool isAbsolutePath(const std::string& p) {
|
||||
if (p.empty()) return false;
|
||||
if (p[0] == '/' || p[0] == '\\') return true;
|
||||
if (p.size() >= 2 && std::isalpha(static_cast<unsigned char>(p[0])) && p[1] == ':')
|
||||
return true;
|
||||
return false;
|
||||
}
|
||||
using util::isAbsolutePath;
|
||||
|
||||
// The version serialize() writes, and the highest this build can read.
|
||||
constexpr int kLedgerVersion = 2;
|
||||
|
||||
// An unrecognized persisted integer degrades to Unknown rather than failing the
|
||||
// parse: a newer channel's kind must not make the whole ledger unreadable, which
|
||||
@@ -75,7 +75,7 @@ std::vector<std::string> OriginLedger::ownedPaths() const {
|
||||
}
|
||||
|
||||
std::string OriginLedger::serialize() const {
|
||||
std::string out = "{\"v\":2,\"records\":[";
|
||||
std::string out = "{\"v\":" + json::numToStr(kLedgerVersion) + ",\"records\":[";
|
||||
for (std::size_t i = 0; i < records_.size(); ++i) {
|
||||
if (i) out += ',';
|
||||
const OriginRecord& r = records_[i];
|
||||
@@ -139,14 +139,18 @@ bool parseRecordArray(json::Reader& r, OriginLedger& out) {
|
||||
}
|
||||
}
|
||||
|
||||
bool parseLedger(json::Reader& r, OriginLedger& out) {
|
||||
// `versionOut` stays 0 when no "v" key was present — the legacy path-only shape, or
|
||||
// an empty object. Read regardless of key order, so it is validated after the close.
|
||||
bool parseLedger(json::Reader& r, OriginLedger& out, int& versionOut) {
|
||||
if (!r.consume('{')) return false;
|
||||
r.skipWs();
|
||||
if (r.consume('}')) return true;
|
||||
for (;;) {
|
||||
std::string key;
|
||||
if (!r.parseKey(key)) return false;
|
||||
if (key == "records") {
|
||||
if (key == "v") {
|
||||
if (!r.parseInt(versionOut)) return false;
|
||||
} else if (key == "records") {
|
||||
if (!parseRecordArray(r, out)) return false;
|
||||
} else if (key == "owned") {
|
||||
std::vector<std::string> paths;
|
||||
@@ -162,25 +166,41 @@ bool parseLedger(json::Reader& r, OriginLedger& out) {
|
||||
}
|
||||
}
|
||||
|
||||
enum class ParseOutcome { Ok, Malformed, FutureVersion };
|
||||
|
||||
// The one parse; the two public entry points differ only in how much of the outcome
|
||||
// they can express. `out` is written only on Ok — never a partial ledger.
|
||||
ParseOutcome parseStored(const std::string& text, OriginLedger& out) {
|
||||
OriginLedger parsed;
|
||||
int version = 0;
|
||||
::reasampler::json::Reader r(text);
|
||||
if (!parseLedger(r, parsed, version)) return ParseOutcome::Malformed;
|
||||
// Trailing garbage means the blob is not what it claims; accepting it would turn
|
||||
// a detectably-corrupt value into a silently-partial ledger.
|
||||
r.skipWs();
|
||||
if (!r.eof()) return ParseOutcome::Malformed;
|
||||
if (version < 0) return ParseOutcome::Malformed;
|
||||
if (version > kLedgerVersion) return ParseOutcome::FutureVersion;
|
||||
out = std::move(parsed);
|
||||
return ParseOutcome::Ok;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
std::optional<OriginLedger> OriginLedger::deserialize(const std::string& json) {
|
||||
OriginLedger ledger;
|
||||
::reasampler::json::Reader r(json);
|
||||
if (!parseLedger(r, ledger)) return std::nullopt;
|
||||
if (parseStored(json, ledger) != ParseOutcome::Ok) return std::nullopt;
|
||||
return ledger;
|
||||
}
|
||||
|
||||
LedgerLoad loadLedger(const std::string& stored) {
|
||||
LedgerLoad load;
|
||||
if (stored.empty()) return load; // absent key -> Fresh, not an error
|
||||
std::optional<OriginLedger> parsed = OriginLedger::deserialize(stored);
|
||||
if (!parsed) {
|
||||
load.status = LedgerStatus::Unreadable;
|
||||
return load;
|
||||
switch (parseStored(stored, load.ledger)) {
|
||||
case ParseOutcome::Ok: load.status = LedgerStatus::Loaded; break;
|
||||
case ParseOutcome::Malformed: load.status = LedgerStatus::Unreadable; break;
|
||||
case ParseOutcome::FutureVersion: load.status = LedgerStatus::FutureVersion; break;
|
||||
}
|
||||
load.status = LedgerStatus::Loaded;
|
||||
load.ledger = std::move(*parsed);
|
||||
return load;
|
||||
}
|
||||
|
||||
|
||||
@@ -2,10 +2,7 @@
|
||||
// origin_ledger — the one persisted record family behind file tracking: for every
|
||||
// file the system itself created, what act created it and what it derives from.
|
||||
// Supersedes the path-only owned manifest (same ext-state key, widened shape).
|
||||
//
|
||||
// Lineage is written at birth and never backfilled — record() on a path already
|
||||
// present is a no-op, so a record says what was known when the file appeared or
|
||||
// says nothing at all. Nothing here decides anything; tracking_authority does.
|
||||
// Nothing here decides anything; tracking_authority does.
|
||||
|
||||
#include <cstddef>
|
||||
#include <optional>
|
||||
@@ -32,7 +29,11 @@ enum class OriginKind {
|
||||
struct OriginRecord {
|
||||
std::string relativePath;
|
||||
OriginKind kind = OriginKind::Unknown;
|
||||
std::string sampleId; // bank id minted at birth; "" when never recorded
|
||||
// The bank id as of birth; "" when never recorded. NOT unique across records: a
|
||||
// recapture writes a new file under the SAME bank id, so the ledger legitimately
|
||||
// holds two records sharing a sampleId with different paths. `relativePath` is
|
||||
// the only key; a by-sampleId lookup is ambiguous by construction.
|
||||
std::string sampleId;
|
||||
std::string parentSampleId; // the capture this derives from; "" = root / none
|
||||
|
||||
bool operator==(const OriginRecord& o) const;
|
||||
@@ -77,7 +78,8 @@ public:
|
||||
bool operator==(const OriginLedger& o) const { return records_ == o.records_; }
|
||||
|
||||
// Lossless round-trip: deserialize(serialize(x)) == x. std::nullopt on malformed
|
||||
// input — the caller must treat that as unreadable, never as empty.
|
||||
// input AND on a blob whose "v" is newer than this build can read — the caller
|
||||
// must treat either as unreadable, never as empty. loadLedger tells them apart.
|
||||
std::string serialize() const;
|
||||
static std::optional<OriginLedger> deserialize(const std::string& json);
|
||||
|
||||
@@ -85,11 +87,19 @@ private:
|
||||
std::vector<OriginRecord> records_;
|
||||
};
|
||||
|
||||
// The three states of a stored ledger, kept apart because never-recorded and
|
||||
// unreadable demand opposite treatment: `Fresh` is a legitimate empty (a new
|
||||
// project, or a bank predating the ledger) and blocks nothing; `Unreadable` is a
|
||||
// present-but-corrupt blob and must block every destructive answer.
|
||||
enum class LedgerStatus { Fresh, Loaded, Unreadable };
|
||||
// The states of a stored ledger. `Fresh` (absent key) and `Loaded` are the two
|
||||
// usable ones. The other two are distinguished only so the operator gets the right
|
||||
// recovery advice — clearing a corrupt blob is repair, clearing a newer build's blob
|
||||
// is destruction.
|
||||
enum class LedgerStatus { Fresh, Loaded, Unreadable, FutureVersion };
|
||||
|
||||
// The one degraded-state test, deliberately serving BOTH axes — "may I answer a
|
||||
// destructive question?" and "may I overwrite the stored blob?" — because today
|
||||
// every degraded status answers no to both. Split them only when a status needs one
|
||||
// without the other.
|
||||
inline bool ledgerDegraded(LedgerStatus s) {
|
||||
return s == LedgerStatus::Unreadable || s == LedgerStatus::FutureVersion;
|
||||
}
|
||||
|
||||
struct LedgerLoad {
|
||||
LedgerStatus status = LedgerStatus::Fresh;
|
||||
|
||||
@@ -14,20 +14,22 @@ ProtectionAnswer pruneProtection(const TrackingState& state) {
|
||||
answer.unreadableUsageKeys = state.usage.offendingKeys;
|
||||
}
|
||||
|
||||
if (state.ledgerStatus == LedgerStatus::Unreadable) {
|
||||
if (ledgerDegraded(state.ledgerStatus)) {
|
||||
answer.blocked = true;
|
||||
answer.ledgerUnreadable = true;
|
||||
return answer; // ownedPaths left empty -> (owned ∩ present) is empty
|
||||
answer.ledgerUnreadable = state.ledgerStatus == LedgerStatus::Unreadable;
|
||||
answer.ledgerFutureVersion = state.ledgerStatus == LedgerStatus::FutureVersion;
|
||||
}
|
||||
|
||||
answer.ownedPaths = state.ledger.ownedPaths();
|
||||
// Populated only on a clean answer: on ANY block an empty `owned` makes
|
||||
// (owned ∩ present) empty, so even a caller that ignored `blocked` deletes nothing.
|
||||
if (!answer.blocked) answer.ownedPaths = state.ledger.ownedPaths();
|
||||
return answer;
|
||||
}
|
||||
|
||||
Answer tiedUsageExists(const TrackingState& state, const std::string& capturePath,
|
||||
const std::string& ownUsageKey) {
|
||||
if (capturePath.empty()) return Answer::Indeterminate;
|
||||
if (state.ledgerStatus == LedgerStatus::Unreadable) return Answer::Indeterminate;
|
||||
if (ledgerDegraded(state.ledgerStatus)) return Answer::Indeterminate;
|
||||
if (state.usage.abortPrune) return Answer::Indeterminate;
|
||||
|
||||
for (const wire::CountedUsage& counted : state.usage.counted) {
|
||||
|
||||
@@ -26,12 +26,13 @@ struct TrackingState {
|
||||
// blocker fields say what to tell the operator; the caller composes the message
|
||||
// (the pure core does not know ext-state key spellings).
|
||||
//
|
||||
// Both path lists stay populated on a block as belt-and-braces: heldPaths carries
|
||||
// the usage fold's protect-all set, and ownedPaths is left EMPTY on an unreadable
|
||||
// ledger, so a caller that ignored `blocked` still computes an empty orphan set.
|
||||
// Belt-and-braces on ANY block, whichever side blocked: heldPaths still carries the
|
||||
// usage fold's protect-all set, and ownedPaths is left EMPTY — so a caller that
|
||||
// ignored `blocked` still computes an empty orphan set rather than deleting.
|
||||
struct ProtectionAnswer {
|
||||
bool blocked = false;
|
||||
bool ledgerUnreadable = false;
|
||||
bool ledgerUnreadable = false; // corrupt blob — repairable by clearing the key
|
||||
bool ledgerFutureVersion = false; // written by a newer build — must NOT be cleared
|
||||
std::vector<std::string> unreadableUsageKeys;
|
||||
std::vector<std::string> heldPaths; // union into prune's `referenced`
|
||||
std::vector<std::string> ownedPaths; // prune's `owned`
|
||||
@@ -46,13 +47,12 @@ enum class Answer { No, Yes, Indeterminate };
|
||||
|
||||
// Does a usage tied to `capturePath` exist, other than the asking instance's own?
|
||||
//
|
||||
// * unreadable ledger, or any unreadable usage record -> Indeterminate.
|
||||
// * degraded ledger, or any unreadable usage record -> Indeterminate.
|
||||
// * otherwise Yes iff some counting live record holds the path.
|
||||
//
|
||||
// `ownUsageKey` is the asking instance's own "rsusage_<guid>" key, excluded from the
|
||||
// scan so a bake does not see itself; empty counts every holder. A `unioned` record
|
||||
// is NEVER excluded — it carries more than one incarnation's holds, so attributing
|
||||
// it to a single owner could hide a sibling's tie.
|
||||
// is never excluded.
|
||||
//
|
||||
// Never-recorded (no ledger record for the path) is a definite answer, not an
|
||||
// abstention: a pre-existing capture nothing holds answers No.
|
||||
|
||||
@@ -3,12 +3,13 @@
|
||||
## Scope
|
||||
|
||||
Tiny, dependency-free pure helpers linked by both artifacts: whole-file byte
|
||||
loading and unit-interval clamping.
|
||||
loading, unit-interval clamping, and the absolute-path rejection test.
|
||||
|
||||
## Modules
|
||||
|
||||
- `file_bytes` (`core/util`) — the ONE whole-file byte loader (Q-W1), linked by both artifacts; blocking I/O, off-audio-thread only.
|
||||
- `clamp01` (`core/util`, header-only) — the ONE unit-interval clamp (Q-W1), replacing four per-module static copies; NaN passes through unchanged rather than collapsing to a bound.
|
||||
- `relative_path` (`core/util`, header-only) — the ONE absolute-path rejection test behind the relative-paths-only invariant, shared by `bank_model` (`Sample.relativePath`) and `core/tracking/origin_ledger` (`OriginRecord.relativePath`). The two must reject identically or a path one accepts could be smuggled past the other; that is why it is one function and not two.
|
||||
|
||||
## Gotchas
|
||||
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
#pragma once
|
||||
// relative_path — the ONE absolute-path rejection test behind the
|
||||
// relative-paths-only invariant, shared by every persisted path family
|
||||
// (Sample.relativePath, OriginRecord.relativePath).
|
||||
//
|
||||
// Rejects rather than normalizes: the pure core has no knowledge of the project
|
||||
// root, so any "normalization" would be a guess that could point at the wrong file.
|
||||
|
||||
#include <cctype>
|
||||
#include <string>
|
||||
|
||||
namespace reasampler::util {
|
||||
|
||||
// True for POSIX root ("/x"), UNC ("\\host\share"), and any leading <alpha>: —
|
||||
// including drive-RELATIVE forms ("C:foo.wav"), which resolve against the drive's
|
||||
// current directory rather than the project root and so violate the invariant just
|
||||
// as much as "C:\foo.wav" does.
|
||||
inline bool isAbsolutePath(const std::string& p) {
|
||||
if (p.empty()) return false;
|
||||
if (p[0] == '/' || p[0] == '\\') return true;
|
||||
if (p.size() >= 2 && std::isalpha(static_cast<unsigned char>(p[0])) && p[1] == ':')
|
||||
return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
} // namespace reasampler::util
|
||||
+5
-10
@@ -57,8 +57,7 @@ This directory owns two cross-artifact contracts specifically:
|
||||
protected (identity-failure net — a matcher failure must never degrade toward
|
||||
delete). The guarantee: a capture held by any live instance can never be deleted; if
|
||||
the prune cannot determine with certainty which captures are held, it aborts
|
||||
entirely (deletes nothing). Over-protection is the accepted residual; under-protection
|
||||
is a data-loss bug. The fold reports `counted` — the live records still attributed to
|
||||
entirely (deletes nothing). The fold reports `counted` — the live records still attributed to
|
||||
their `rsusage_*` keys — because the flattened path list cannot answer "who holds
|
||||
this"; `counted` is empty whenever `abortPrune` is set, since attribution is exactly
|
||||
what an unreadable record destroys.
|
||||
@@ -70,14 +69,10 @@ This directory owns two cross-artifact contracts specifically:
|
||||
Resolution always leans over-protect: same-nonce + not-unioned → clean replace;
|
||||
same-track foreign nonce or unioned → union; cross-track foreign nonce → remint under
|
||||
a fresh key. None of the three directions can under-protect.
|
||||
- **Deferred follow-up (TODO.md, deliberately NOT absorbed by the tracking
|
||||
consolidation):** `ownerNonce` is not persisted, so after save→reopen an instance
|
||||
cannot recognize its own prior-session usage record — it unions and marks the record
|
||||
`unioned` forever, so prune stops reclaiming captures the instance once held but no
|
||||
longer uses (safe, but the bank folder grows unbounded). This is a *completeness*
|
||||
wart, not a safety one; every candidate fix examined so far trades it for a new
|
||||
under-protection window, which the consolidation's own safety mandate forbids. See
|
||||
`docs/TODO.md` for the constraint and the rejected session-epoch candidate.
|
||||
- **Deferred follow-up:** `ownerNonce` is not persisted, so after save→reopen an
|
||||
instance cannot recognize its own prior-session usage record and unions forever.
|
||||
Reasoning, constraints, and the rejected candidates live in `docs/TODO.md`
|
||||
("Persist ReaSampler 9000 instance identity…").
|
||||
|
||||
## Modules
|
||||
|
||||
|
||||
@@ -17,11 +17,9 @@
|
||||
// per-instance key, never banks/view/tail/assign; the bridge's write entry
|
||||
// point structurally accepts only "rsusage_"-prefixed keys.
|
||||
//
|
||||
// THE SAFETY PROPERTY (overrides every other consideration): every failure,
|
||||
// ambiguity, or uncertainty here must fail-safe toward PROTECT. Over-protection
|
||||
// (prune skips a reclaimable file, or refuses to run) is an accepted residual;
|
||||
// under-protection (deleting a file an instance may still be playing) is a
|
||||
// data-loss bug. Three folds enforce this:
|
||||
// Every failure, ambiguity, or uncertainty here fails safe toward PROTECT (the
|
||||
// territory-wide asymmetry, stated in core/tracking/CLAUDE.md). Three folds enforce
|
||||
// it here:
|
||||
// * sibling-collision -> UNION, never clean-replace over a foreign writer;
|
||||
// * zero-identified -> records exist but no instance was identified live ->
|
||||
// protect ALL records' paths (a matcher failure must
|
||||
|
||||
@@ -273,8 +273,6 @@ ImportResult importFileIntoActiveBank(const std::string& absoluteSourcePath) {
|
||||
s.createdTimestamp = nowSec;
|
||||
|
||||
const AddResult r = book.activeIndex().add(s);
|
||||
// Record the birth regardless of outcome — the tool WROTE the file, so prune must
|
||||
// attribute it even in the narrow Collapsed race below.
|
||||
g_session->recordCreated(s, tracking::OriginKind::Ingest);
|
||||
|
||||
switch (r) {
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "core/version/app_version.h" // extStateNamespace — the printed recovery line must be channel-correct
|
||||
#include "shell/persist/session.h" // ReaSamplerSession — pruneDryRun / pruneOrphanSet / pruneReclaim
|
||||
|
||||
#define REAPERAPI_MINIMAL
|
||||
@@ -27,22 +28,36 @@ void doBankPruneFolder(ReaSamplerSession& session) {
|
||||
// set unknowable, so the prune HALTS outright rather than proceed with degraded
|
||||
// protection. Both blockers can fire at once; report each one that did.
|
||||
if (report.blockedByTracking) {
|
||||
// Every printed recovery line names THIS build's ext-state namespace: a beta
|
||||
// user handed the stable spelling would clear the stable channel's key in
|
||||
// their project and still be blocked.
|
||||
const std::string& ns = version::extStateNamespace();
|
||||
std::string msg =
|
||||
"ReaSampler prune: ABORTED -- the file-tracking state could not be read. "
|
||||
"Nothing was deleted.\n";
|
||||
if (report.ledgerUnreadable) {
|
||||
msg += "The stored file-tracking ledger is malformed. It has been left "
|
||||
"intact rather than overwritten, so it can be repaired or cleared:\n"
|
||||
" reaper.SetProjExtState(0, \"reasampler\", \"owned_files\", \"\")\n"
|
||||
" reaper.SetProjExtState(0, \"" + ns + "\", \"owned_files\", \"\")\n"
|
||||
"Clearing it makes every existing bank file un-reclaimable (they stop "
|
||||
"being attributable to ReaSampler); no file is lost.\n";
|
||||
"being attributable to ReaSampler); no file is lost. Reopen the "
|
||||
"project afterwards -- the block is held for the rest of this "
|
||||
"session, and until then no new capture is persisted to the ledger "
|
||||
"either.\n";
|
||||
}
|
||||
if (report.ledgerFutureVersion) {
|
||||
msg += "The stored file-tracking ledger was written by a NEWER version of "
|
||||
"ReaSampler than this one, so its records cannot be read safely. It "
|
||||
"has been left intact and will NOT be overwritten. Reopen the project "
|
||||
"with that newer version -- do NOT clear this key from here, that "
|
||||
"would discard tracking records this build cannot see.\n";
|
||||
}
|
||||
if (!report.unreadableUsageKeys.empty()) {
|
||||
msg += "One or more instance usage records could not be read or decoded. "
|
||||
"If the owning instance is still loaded it will republish its record "
|
||||
"on the next poll tick, clearing the abort. If the instance no longer "
|
||||
"exists (the key is an orphaned corrupt record), clear it manually:\n"
|
||||
" reaper.SetProjExtState(0, \"reasampler\", \"<key>\", \"\")\n"
|
||||
" reaper.SetProjExtState(0, \"" + ns + "\", \"<key>\", \"\")\n"
|
||||
"Offending key(s):\n";
|
||||
for (const std::string& key : report.unreadableUsageKeys) {
|
||||
msg += " " + key + "\n";
|
||||
|
||||
@@ -455,13 +455,13 @@ void RunRecaptureFromSource(ReaSamplerSession& session)
|
||||
// One batched undo point around the in-place mutation; index-only ext-state,
|
||||
// nothing placed on the timeline.
|
||||
Undo_BeginBlock2(nullptr);
|
||||
// Before the index outcome is even known: the render above already wrote the file,
|
||||
// and makeUniqueTag guarantees a NEW path, so this is a second record carrying the
|
||||
// same sampleId as the original. The superseded file becomes an orphan for prune.
|
||||
session.recordCreated(updated, tracking::OriginKind::Recapture);
|
||||
const bool changed = session.book().updateSampleInPlace(sampleId, updated);
|
||||
if (changed)
|
||||
{
|
||||
// Record the regenerated file's birth; the superseded file becomes an orphan
|
||||
// for prune to reclaim. A re-capture onto the same path is a dedup no-op, so
|
||||
// the original birth record — not this one — stays authoritative.
|
||||
session.recordCreated(updated, tracking::OriginKind::Recapture);
|
||||
// Regenerating the same id's audio is exactly why instances need the generation
|
||||
// bump — they'd otherwise keep playing stale audio until reload. Bumped inside
|
||||
// the undo block so undo rolls back the generation with the rest of the blob.
|
||||
|
||||
@@ -250,9 +250,6 @@ CaptureResult captureAndIndexOne(ReaSamplerSession& session,
|
||||
// AddResult tells a fresh add from a hash-dedup collapse, so the assign path (S8) can
|
||||
// target the sample actually in the bank (the existing entry on a collapse).
|
||||
const model::AddResult addResult = session.bank().add(res.sample);
|
||||
// Record the birth at the same point the Sample is added, regardless of the index
|
||||
// AddResult — even a hash-collapse still WROTE a file the tool owns, and the ledger
|
||||
// dedups a repeat path itself (prune reconciles ledger vs index later).
|
||||
session.recordCreated(res.sample, tracking::OriginKind::Capture);
|
||||
|
||||
// Resolve the LANDED bank-index id into res.sample.id for the S8 assign path: the new
|
||||
|
||||
@@ -31,8 +31,6 @@ void CommitRealtimeResult(ReaSamplerSession& session, const CaptureResult& res)
|
||||
return;
|
||||
}
|
||||
session.bank().add(res.sample);
|
||||
// Record the birth regardless of the index AddResult — even a hash-collapse still
|
||||
// wrote a file the tool owns; the ledger dedups a repeat path itself.
|
||||
session.recordCreated(res.sample, tracking::OriginKind::Capture);
|
||||
// A capture add changes what a live instance could play, so bump the generation
|
||||
// before persisting to refresh instances.
|
||||
@@ -56,14 +54,28 @@ void DriveRealtimeCapture(ReaSamplerSession& session)
|
||||
{
|
||||
RealtimeTickResult r = g_rtBackend.abort(*g_rtCapture);
|
||||
// Log without persisting — we restored into the original project but must
|
||||
// not persist into the now-active foreign one. A Failed abort surfaces
|
||||
// not persist into the now-active foreign one: the ledger and bank we would
|
||||
// have to write belong to the project we just left. A Failed abort surfaces
|
||||
// abort()'s own message, distinguishing a clean tab-switch abort from the
|
||||
// closed-project case (nothing restored because the pointers were already
|
||||
// freed).
|
||||
if (r.status == RealtimeTickStatus::Done)
|
||||
ShowConsoleMsg("ReaSampler realtime capture: project switched mid-record -- "
|
||||
"captured audio restored into the original project; not "
|
||||
"persisted to avoid crossing projects.\n");
|
||||
//
|
||||
// The abort already moved the recorded WAV into the ORIGINAL project's bank
|
||||
// folder, so it survives here untracked — deleting a user's just-recorded
|
||||
// audio is the destructive direction and is not this shell's call, so the
|
||||
// path is NAMED instead and the operator decides (docs/TODO.md).
|
||||
if (r.status == RealtimeTickStatus::Done && r.result.status == CaptureStatus::Ok)
|
||||
ShowConsoleMsg(("ReaSampler realtime capture: project switched mid-record -- "
|
||||
"captured audio restored into the original project; not "
|
||||
"persisted to avoid crossing projects. The recorded file was "
|
||||
"left in the original project's bank folder as '" +
|
||||
r.result.sample.relativePath +
|
||||
"', untracked -- reopen that project and re-import it, or "
|
||||
"delete it by hand.\n").c_str());
|
||||
else if (r.status == RealtimeTickStatus::Done)
|
||||
ShowConsoleMsg(("ReaSampler realtime capture: project switched mid-record -- "
|
||||
"the capture was aborted and produced no usable file: " +
|
||||
r.result.message + "\n").c_str());
|
||||
else
|
||||
ShowConsoleMsg(("ReaSampler realtime capture: project changed mid-record -- " +
|
||||
r.result.message + "\n").c_str());
|
||||
|
||||
@@ -34,17 +34,20 @@ REAPER/filesystem-facing half only, and it gathers rather than decides.
|
||||
record/input chains, containers recursively, take FX), and folds via the pure
|
||||
`sample_usage::foldUsageRecords`. Read-only at prune-scan time: `usage_scan` writes
|
||||
no ext-state.
|
||||
- **A malformed tracking ledger is Unreadable, never "empty".** `ext_state_io` keeps
|
||||
- **A ledger this build cannot read is degraded, never "empty".** `ext_state_io` keeps
|
||||
the `LedgerStatus` alongside the ledger, and `saveToActiveProject` SKIPS the
|
||||
`owned_files` write while it is `Unreadable` — replacing a corrupt blob would
|
||||
destroy the only record of every file created before the corruption, silently
|
||||
turning them into permanently unreclaimable foreign files. Captures made during
|
||||
such a session are recorded in memory but not persisted; they degrade to foreign
|
||||
(untouchable), which is the safe direction.
|
||||
`owned_files` write while `tracking::ledgerDegraded` holds — replacing a blob we
|
||||
could not read would destroy the only record of every file created before it,
|
||||
silently turning them into permanently unreclaimable foreign files. The status is
|
||||
written only by `loadFromProject`, so it is sticky until the project is reloaded:
|
||||
captures made during such a session are recorded in memory but not persisted, and
|
||||
degrade to foreign (untouchable), which is the safe direction.
|
||||
- **`PruneReport` carries `blockedByTracking` + `ledgerUnreadable` /
|
||||
`unreadableUsageKeys`**; dry-run, orphan-set, and reclaim each independently abort
|
||||
(delete nothing) on a block. `BANK_PRUNE_FOLDER` (in `shell/actions`) halts on the
|
||||
flag and prints whichever blockers fired.
|
||||
`ledgerFutureVersion` / `unreadableUsageKeys`**; dry-run, orphan-set, and reclaim
|
||||
each independently abort (delete nothing) on a block. `BANK_PRUNE_FOLDER` (in
|
||||
`shell/actions`) halts on the flag and prints whichever blockers fired, with
|
||||
channel-correct recovery lines — a corrupt blob may be cleared, a newer build's
|
||||
must not be.
|
||||
|
||||
## Modules
|
||||
|
||||
|
||||
@@ -161,11 +161,9 @@ bool ReaSamplerSession::saveToActiveProject() {
|
||||
SetProjExtState(static_cast<ReaProject*>(proj), projExtNamespace(),
|
||||
kProjExtTailKey, tailJson.c_str());
|
||||
|
||||
// Written on every save so the ledger and the bank stay in lockstep on disk —
|
||||
// EXCEPT over a blob we could not read: replacing it would destroy the only
|
||||
// record of every file created before the corruption, silently turning them
|
||||
// into permanently unreclaimable foreign files.
|
||||
if (trackingStatus_ != tracking::LedgerStatus::Unreadable) {
|
||||
// Never write over a blob this build could not read (see this directory's
|
||||
// CLAUDE.md for why the suppression, not a rewrite, is the safe direction).
|
||||
if (!tracking::ledgerDegraded(trackingStatus_)) {
|
||||
const std::string ledgerJson = tracking_.serialize();
|
||||
SetProjExtState(static_cast<ReaProject*>(proj), projExtNamespace(),
|
||||
kProjExtOwnedKey, ledgerJson.c_str());
|
||||
@@ -234,10 +232,8 @@ capture::TailSetting loadTailSetting(ReaProject* proj) {
|
||||
return *loaded;
|
||||
}
|
||||
|
||||
// Absent/empty key -> Fresh (a new project, or a bank predating the ledger).
|
||||
// Malformed -> Unreadable, which halts the prune and suppresses the next write
|
||||
// rather than degrading to an empty ledger that looks like "nothing was ever
|
||||
// created".
|
||||
// Absent/empty key -> Fresh (a new project, or a bank predating the ledger). Anything
|
||||
// this build cannot read is a degraded status, never an empty ledger.
|
||||
tracking::LedgerLoad loadOriginLedger(ReaProject* proj) {
|
||||
if (!proj) return tracking::LedgerLoad{};
|
||||
tracking::LedgerLoad load = tracking::loadLedger(
|
||||
@@ -246,6 +242,11 @@ tracking::LedgerLoad loadOriginLedger(ReaProject* proj) {
|
||||
ShowConsoleMsg("ReaSampler: the stored file-tracking ledger is malformed. Prune "
|
||||
"is halted for this project and the stored value is left intact "
|
||||
"for recovery.\n");
|
||||
} else if (load.status == tracking::LedgerStatus::FutureVersion) {
|
||||
ShowConsoleMsg("ReaSampler: the stored file-tracking ledger was written by a "
|
||||
"NEWER version of ReaSampler. Prune is halted for this project "
|
||||
"and the stored value will not be overwritten -- reopen the "
|
||||
"project with that version.\n");
|
||||
}
|
||||
return load;
|
||||
}
|
||||
|
||||
@@ -72,6 +72,7 @@ struct PruneScan {
|
||||
std::unordered_map<std::string, std::uint64_t> sizeByRel;
|
||||
bool blocked = false;
|
||||
bool ledgerUnreadable = false;
|
||||
bool ledgerFutureVersion = false;
|
||||
std::vector<std::string> unreadableUsageKeys;
|
||||
};
|
||||
|
||||
@@ -134,6 +135,7 @@ PruneScan scanPruneOrphans(const BankBook& book, const tracking::OriginLedger& l
|
||||
// action tell the user what to recover.
|
||||
scan.blocked = true;
|
||||
scan.ledgerUnreadable = protection.ledgerUnreadable;
|
||||
scan.ledgerFutureVersion = protection.ledgerFutureVersion;
|
||||
scan.unreadableUsageKeys = protection.unreadableUsageKeys;
|
||||
return scan;
|
||||
}
|
||||
@@ -211,6 +213,7 @@ reclaim::PruneReport ReaSamplerSession::pruneDryRun() const {
|
||||
// the prune refused to run.
|
||||
report.blockedByTracking = scan.blocked;
|
||||
report.ledgerUnreadable = scan.ledgerUnreadable;
|
||||
report.ledgerFutureVersion = scan.ledgerFutureVersion;
|
||||
report.unreadableUsageKeys = scan.unreadableUsageKeys;
|
||||
return report;
|
||||
}
|
||||
|
||||
@@ -35,6 +35,7 @@
|
||||
#define REAPERAPI_MINIMAL
|
||||
#define REAPERAPI_WANT_MarkProjectDirty
|
||||
#define REAPERAPI_WANT_SetProjExtState
|
||||
#define REAPERAPI_WANT_ShowConsoleMsg
|
||||
#include "reaper_plugin_functions.h"
|
||||
|
||||
namespace reasampler {
|
||||
@@ -56,7 +57,18 @@ void ReaSamplerSession::recordCreated(const model::Sample& sample,
|
||||
// rather than re-deriving keeps one source for the lineage fact. Absent
|
||||
// provenance is a root capture, not a gap.
|
||||
if (sample.provenance) rec.parentSampleId = sample.provenance->parentSampleId;
|
||||
tracking_.record(rec);
|
||||
|
||||
const tracking::RecordResult result = tracking_.record(rec);
|
||||
// A rejection means a file exists that nothing attributes to us — invisible
|
||||
// otherwise, and exactly the gap this ledger exists to close. AlreadyPresent is
|
||||
// the normal dedup outcome, not a gap.
|
||||
if (result == tracking::RecordResult::RejectedEmptyPath ||
|
||||
result == tracking::RecordResult::RejectedAbsolutePath) {
|
||||
ShowConsoleMsg(("ReaSampler: could not record the origin of '" +
|
||||
sample.relativePath +
|
||||
"' -- the path is empty or absolute. The file is NOT tracked and "
|
||||
"prune will treat it as a foreign file.\n").c_str());
|
||||
}
|
||||
}
|
||||
|
||||
bool ReaSamplerSession::consumeLoadSignal() {
|
||||
|
||||
+16
-14
@@ -71,17 +71,20 @@ public:
|
||||
capture::TailSetting& tail() { return tail_; }
|
||||
const capture::TailSetting& tail() const { return tail_; }
|
||||
|
||||
// Birth records for the files the system itself created. Read as a PAIR —
|
||||
// the ledger alone cannot say whether an absent record means never-recorded
|
||||
// or unreadable, and the two demand opposite treatment. This is the reach
|
||||
// any consumer outside persist uses to build a tracking::TrackingState.
|
||||
const tracking::OriginLedger& tracking() const { return tracking_; }
|
||||
tracking::LedgerStatus trackingStatus() const { return trackingStatus_; }
|
||||
|
||||
// Record a system-created file at the moment it exists — the ONLY way a
|
||||
// birth record is written, so lineage can never be backfilled from a later
|
||||
// guess. Lineage is read off the Sample's own provenance, the same act that
|
||||
// stamped it, so the two cannot disagree. A repeat path is a no-op.
|
||||
// Record a system-created file at the moment it exists — the ONLY way a birth
|
||||
// record is written. Lineage is read off the Sample's own provenance, the same
|
||||
// act that stamped it, so the two cannot disagree; where they later diverge the
|
||||
// ledger record is authoritative (core/tracking/CLAUDE.md).
|
||||
//
|
||||
// Call it at EVERY creation site regardless of the bank's AddResult: a
|
||||
// hash-collapse still wrote a file the tool owns, and an unrecorded file is a
|
||||
// permanently unreclaimable foreign file. A rejected record is reported to the
|
||||
// console — a file with no record is the gap this track exists to eliminate.
|
||||
//
|
||||
// A consumer outside persist that needs the ledger must take it WITH its
|
||||
// LedgerStatus (a tracking::TrackingState); no accessor exposes one without the
|
||||
// other, because an absent record and an unreadable ledger demand opposite
|
||||
// treatment.
|
||||
void recordCreated(const model::Sample& sample, tracking::OriginKind kind);
|
||||
|
||||
// The version that last wrote the active project: PreVersioning (no
|
||||
@@ -153,9 +156,8 @@ private:
|
||||
ViewModeModel view_; // reset to default on a project with no stored view_state
|
||||
capture::TailSetting tail_; // reset to default (None / 2s) with no stored tail key
|
||||
tracking::OriginLedger tracking_; // reset to empty/stored on EVERY load path, never inherited
|
||||
// Unreadable is sticky for the project's session: it halts the prune AND
|
||||
// suppresses the ledger write, so a corrupt blob survives for recovery
|
||||
// instead of being silently replaced by a ledger missing every earlier file.
|
||||
// Written only by loadFromProject, so a degraded status is sticky until the
|
||||
// project is reloaded (see this directory's CLAUDE.md).
|
||||
tracking::LedgerStatus trackingStatus_ = tracking::LedgerStatus::Fresh;
|
||||
version::WritingVersion writingVersion_; // recovered per load; PreVersioning default
|
||||
std::int64_t bankGeneration_ = 0; // recovered per load (absent -> 0); monotonic
|
||||
|
||||
+125
-14
@@ -1,10 +1,10 @@
|
||||
// Standalone tests for reasampler::tracking::OriginLedger — no REAPER, no framework.
|
||||
//
|
||||
// The record family behind file tracking. Covers: the relative-paths-only invariant,
|
||||
// dedup, insertion order, the JSON round-trip (incl. a golden byte literal), the
|
||||
// no-backfill rule, the legacy path-only lift, and the three-way Fresh /
|
||||
// Loaded / Unreadable classification that keeps never-recorded apart from
|
||||
// unreadable.
|
||||
// exact-string ownership, dedup, insertion order, the JSON round-trip (incl. golden
|
||||
// byte literals over every persisted enum value), the no-backfill rule, the legacy
|
||||
// path-only lift, and the Fresh / Loaded / Unreadable / FutureVersion classification
|
||||
// that keeps never-recorded apart from the two degraded states.
|
||||
|
||||
#include "../src/core/tracking/origin_ledger.h"
|
||||
|
||||
@@ -151,17 +151,49 @@ static void testRoundTripWithJsonMetacharacters() {
|
||||
}
|
||||
|
||||
// Golden byte literal: pins the EXACT serialized bytes, so a format drift both
|
||||
// writer and reader agree on still fails here.
|
||||
static void testSerializeGoldenLiteral() {
|
||||
// writer and reader agree on still fails here. EVERY OriginKind appears, because the
|
||||
// integers are persisted — a swap of two values in both the enum and kindFromInt
|
||||
// round-trips perfectly and would mislabel every existing project. This literal is
|
||||
// the only thing standing in the way of that.
|
||||
static void testSerializeGoldenLiteralPinsEveryPersistedKind() {
|
||||
OriginLedger l;
|
||||
l.record(rec("reasampler_bank/a.wav", OriginKind::Capture, "S-a"));
|
||||
l.record(rec("reasampler_bank/b.wav", OriginKind::Resample, "S-b", "S-a"));
|
||||
l.record(rec("bank/unknown.wav", OriginKind::Unknown));
|
||||
l.record(rec("bank/capture.wav", OriginKind::Capture, "S-a"));
|
||||
l.record(rec("bank/ingest.wav", OriginKind::Ingest, "S-b"));
|
||||
l.record(rec("bank/recapture.wav", OriginKind::Recapture, "S-c", "S-a"));
|
||||
l.record(rec("bank/resample.wav", OriginKind::Resample, "S-d", "S-a"));
|
||||
const std::string expected =
|
||||
"{\"v\":2,\"records\":["
|
||||
"{\"path\":\"reasampler_bank/a.wav\",\"kind\":1,\"sample\":\"S-a\",\"parent\":\"\"},"
|
||||
"{\"path\":\"reasampler_bank/b.wav\",\"kind\":4,\"sample\":\"S-b\",\"parent\":\"S-a\"}"
|
||||
"{\"path\":\"bank/unknown.wav\",\"kind\":0,\"sample\":\"\",\"parent\":\"\"},"
|
||||
"{\"path\":\"bank/capture.wav\",\"kind\":1,\"sample\":\"S-a\",\"parent\":\"\"},"
|
||||
"{\"path\":\"bank/ingest.wav\",\"kind\":2,\"sample\":\"S-b\",\"parent\":\"\"},"
|
||||
"{\"path\":\"bank/recapture.wav\",\"kind\":3,\"sample\":\"S-c\",\"parent\":\"S-a\"},"
|
||||
"{\"path\":\"bank/resample.wav\",\"kind\":4,\"sample\":\"S-d\",\"parent\":\"S-a\"}"
|
||||
"]}";
|
||||
CHECK(l.serialize() == expected);
|
||||
|
||||
// And the reader agrees with the writer on the same bytes, kind by kind.
|
||||
auto back = OriginLedger::deserialize(expected);
|
||||
CHECK(back.has_value());
|
||||
CHECK(back->find("bank/unknown.wav")->kind == OriginKind::Unknown);
|
||||
CHECK(back->find("bank/capture.wav")->kind == OriginKind::Capture);
|
||||
CHECK(back->find("bank/ingest.wav")->kind == OriginKind::Ingest);
|
||||
CHECK(back->find("bank/recapture.wav")->kind == OriginKind::Recapture);
|
||||
CHECK(back->find("bank/resample.wav")->kind == OriginKind::Resample);
|
||||
}
|
||||
|
||||
// contains() is an EXACT-string predicate, never a prefix or substring match — the
|
||||
// prune's whole ownership attribution rests on it, and a looser match would attribute
|
||||
// (and so expose to deletion) a file the system never created.
|
||||
static void testContainsIsExactStringNotPrefixOrSubstring() {
|
||||
OriginLedger l;
|
||||
l.record(rec("reasampler_bank/a.wav", OriginKind::Capture));
|
||||
|
||||
CHECK(l.contains("reasampler_bank/a.wav"));
|
||||
CHECK(!l.contains("reasampler_bank/a")); // prefix of the stored path
|
||||
CHECK(!l.contains("a.wav")); // suffix of the stored path
|
||||
CHECK(!l.contains("reasampler_bank/a.wave")); // stored path is a prefix of this
|
||||
CHECK(!l.contains("REASAMPLER_BANK/A.WAV")); // no case folding
|
||||
}
|
||||
|
||||
// --- malformed input ---------------------------------------------------------
|
||||
@@ -177,12 +209,57 @@ static void testMalformedParsesToNullopt() {
|
||||
// Unknown rather than failing the whole ledger — a vocabulary gap must not halt
|
||||
// the prune.
|
||||
auto tolerated = OriginLedger::deserialize(
|
||||
"{\"v\":3,\"future\":{\"x\":1},\"records\":[{\"path\":\"a.wav\",\"kind\":99}]}");
|
||||
"{\"v\":2,\"future\":{\"x\":1},\"records\":[{\"path\":\"a.wav\",\"kind\":99}]}");
|
||||
CHECK(tolerated.has_value());
|
||||
CHECK(tolerated->size() == 1);
|
||||
CHECK(tolerated->find("a.wav")->kind == OriginKind::Unknown);
|
||||
}
|
||||
|
||||
// Trailing garbage is rejected outright: accepting it would turn a detectably-corrupt
|
||||
// blob into a silently-partial ledger, and the records it dropped would then be lost
|
||||
// on the next save.
|
||||
static void testTrailingGarbageIsRejected() {
|
||||
CHECK(!OriginLedger::deserialize("{\"v\":2,\"records\":[]}JUNK").has_value());
|
||||
CHECK(!OriginLedger::deserialize("{\"owned\":[\"a.wav\"]} {\"owned\":[]}").has_value());
|
||||
// ... but trailing whitespace alone is not garbage.
|
||||
CHECK(OriginLedger::deserialize("{\"v\":2,\"records\":[]} \n").has_value());
|
||||
}
|
||||
|
||||
// The legacy shape's own error cases: a type error is a REJECTION (blocking, blob
|
||||
// preserved), never a silent degrade to empty — while a genuinely empty legacy list
|
||||
// is valid and yields an empty, non-blocking ledger.
|
||||
static void testLegacyShapeTypeErrorsAreRejectedButEmptyIsValid() {
|
||||
CHECK(loadLedger("{\"owned\":[1,2]}").status == LedgerStatus::Unreadable);
|
||||
CHECK(loadLedger("{\"owned\":\"x\"}").status == LedgerStatus::Unreadable);
|
||||
|
||||
const LedgerLoad emptyLegacy = loadLedger("{\"owned\":[]}");
|
||||
CHECK(emptyLegacy.status == LedgerStatus::Loaded);
|
||||
CHECK(emptyLegacy.ledger.empty());
|
||||
}
|
||||
|
||||
// A blob written by a newer build is its own status: this build cannot trust records
|
||||
// it parsed under v2 rules, so it must neither answer a destructive question from
|
||||
// them nor overwrite the blob with its own truncation.
|
||||
static void testFutureVersionIsNeitherLoadedNorMalformed() {
|
||||
const std::string v3 =
|
||||
"{\"v\":3,\"records\":[{\"path\":\"a.wav\",\"kind\":1}],\"newshape\":[1]}";
|
||||
const LedgerLoad load = loadLedger(v3);
|
||||
CHECK(load.status == LedgerStatus::FutureVersion);
|
||||
CHECK(load.ledger.empty()); // never a partial value
|
||||
CHECK(ledgerDegraded(load.status)); // blocks answers AND suppresses the write
|
||||
CHECK(!OriginLedger::deserialize(v3).has_value());
|
||||
|
||||
// Key order must not matter — "v" is validated after the object closes.
|
||||
CHECK(loadLedger("{\"records\":[{\"path\":\"a.wav\"}],\"v\":9}").status ==
|
||||
LedgerStatus::FutureVersion);
|
||||
|
||||
// The versions this build does read stay readable, and a nonsense version is
|
||||
// corruption rather than a future shape.
|
||||
CHECK(loadLedger("{\"v\":2,\"records\":[]}").status == LedgerStatus::Loaded);
|
||||
CHECK(loadLedger("{\"v\":1,\"owned\":[\"a.wav\"]}").status == LedgerStatus::Loaded);
|
||||
CHECK(loadLedger("{\"v\":-1,\"records\":[]}").status == LedgerStatus::Unreadable);
|
||||
}
|
||||
|
||||
// A hand-edited or corrupt blob cannot smuggle an absolute or duplicate path past
|
||||
// the load — record() re-asserts the invariants on the way in.
|
||||
static void testLoadReassertsInvariants() {
|
||||
@@ -222,12 +299,41 @@ static void testLegacyPathOnlyManifestLiftsIn() {
|
||||
CHECK(r.parentSampleId.empty()); // no spurious lineage
|
||||
}
|
||||
|
||||
// Re-saving upgrades the shape without losing or inventing anything.
|
||||
auto resaved = OriginLedger::deserialize(load.ledger.serialize());
|
||||
// Re-saving upgrades the shape without losing or inventing anything — pinned as a
|
||||
// byte literal, not just a round-trip, so the lifted record's persisted integers
|
||||
// (kind 0) and empty ids are fixed rather than merely self-consistent.
|
||||
const std::string expected =
|
||||
"{\"v\":2,\"records\":["
|
||||
"{\"path\":\"reasampler_bank/a.wav\",\"kind\":0,\"sample\":\"\",\"parent\":\"\"},"
|
||||
"{\"path\":\"reasampler_bank/b.wav\",\"kind\":0,\"sample\":\"\",\"parent\":\"\"}"
|
||||
"]}";
|
||||
CHECK(load.ledger.serialize() == expected);
|
||||
auto resaved = OriginLedger::deserialize(expected);
|
||||
CHECK(resaved.has_value());
|
||||
CHECK(*resaved == load.ledger);
|
||||
}
|
||||
|
||||
// A recapture regenerates the audio behind ONE bank id under a NEW file name
|
||||
// (makeUniqueTag guarantees it), so the ledger legitimately ends up holding two
|
||||
// records with the same sampleId and different paths. The original's lineage survives
|
||||
// untouched — the Sample was rewritten in place, the birth record was not.
|
||||
static void testRecaptureAddsSecondRecordUnderTheSameSampleId() {
|
||||
OriginLedger l;
|
||||
l.record(rec("bank/take-1.wav", OriginKind::Capture, "S-1", "S-parent"));
|
||||
CHECK(l.record(rec("bank/take-2.wav", OriginKind::Recapture, "S-1", "S-other")) ==
|
||||
RecordResult::Recorded);
|
||||
|
||||
CHECK(l.size() == 2);
|
||||
CHECK(l.find("bank/take-1.wav")->sampleId == "S-1");
|
||||
CHECK(l.find("bank/take-2.wav")->sampleId == "S-1");
|
||||
// The ledger, not the (mutable) Sample, is authoritative for lineage: the
|
||||
// recapture's differing parent did not rewrite the original's.
|
||||
CHECK(l.find("bank/take-1.wav")->parentSampleId == "S-parent");
|
||||
CHECK(l.find("bank/take-1.wav")->kind == OriginKind::Capture);
|
||||
// Both stay owned, so the superseded file is reclaimable rather than foreign.
|
||||
CHECK(l.ownedPaths().size() == 2);
|
||||
}
|
||||
|
||||
// --- never-recorded vs unreadable --------------------------------------------
|
||||
// The two absences demand opposite treatment, so they must be distinguishable at
|
||||
// the load boundary — this is the only place that distinction is made.
|
||||
@@ -259,14 +365,19 @@ int main() {
|
||||
testEmptyLedger();
|
||||
testRejectsEmptyAndAbsolutePaths();
|
||||
testDedupPreservesInsertionOrder();
|
||||
testContainsIsExactStringNotPrefixOrSubstring();
|
||||
testLineageIsNeverBackfilled();
|
||||
testLineageQueryableImmediatelyAfterRecording();
|
||||
testRoundTripWithLineage();
|
||||
testRoundTripWithJsonMetacharacters();
|
||||
testSerializeGoldenLiteral();
|
||||
testSerializeGoldenLiteralPinsEveryPersistedKind();
|
||||
testMalformedParsesToNullopt();
|
||||
testTrailingGarbageIsRejected();
|
||||
testLegacyShapeTypeErrorsAreRejectedButEmptyIsValid();
|
||||
testFutureVersionIsNeitherLoadedNorMalformed();
|
||||
testLoadReassertsInvariants();
|
||||
testLegacyPathOnlyManifestLiftsIn();
|
||||
testRecaptureAddsSecondRecordUnderTheSameSampleId();
|
||||
testFreshLoadedUnreadableAreDistinct();
|
||||
|
||||
if (g_fail == 0) std::printf("origin_ledger: all tests passed\n");
|
||||
|
||||
@@ -148,8 +148,44 @@ static void testUnreadableUsageBlocksPruneAndNamesIt() {
|
||||
const ProtectionAnswer answer = pruneProtection(state);
|
||||
CHECK(answer.blocked);
|
||||
CHECK(!answer.ledgerUnreadable);
|
||||
CHECK(!answer.ledgerFutureVersion);
|
||||
CHECK(answer.unreadableUsageKeys.size() == 1);
|
||||
CHECK(answer.unreadableUsageKeys[0] == "rsusage_BROKEN");
|
||||
|
||||
// The belt-and-braces guard is symmetric: a usage-only block ALSO withholds
|
||||
// ownedPaths, even though the ledger itself read fine, so a caller that ignored
|
||||
// `blocked` computes an empty orphan set rather than deleting with degraded
|
||||
// protection.
|
||||
CHECK(answer.ownedPaths.empty());
|
||||
CHECK(reclaim::pruneOrphans({"bank/orphan.wav"}, {}, answer.ownedPaths).empty());
|
||||
|
||||
// heldPaths is the one list that stays populated on a block — it only ever widens
|
||||
// the protected set, so withholding it would be the unsafe direction.
|
||||
CHECK(contains(answer.heldPaths, "bank/x.wav"));
|
||||
}
|
||||
|
||||
// A ledger written by a NEWER build blocks exactly like a corrupt one, but is
|
||||
// reported separately: the operator advice differs (clearing a corrupt blob is
|
||||
// repair; clearing a newer build's is destruction).
|
||||
static void testFutureVersionLedgerBlocksAndIsReportedSeparately() {
|
||||
const LedgerLoad load = loadLedger("{\"v\":99,\"records\":[]}");
|
||||
CHECK(load.status == LedgerStatus::FutureVersion);
|
||||
|
||||
OriginLedger populated;
|
||||
populated.record(originOf("bank/would-be-orphan.wav", OriginKind::Capture, "S-1"));
|
||||
const UsageFoldResult fold = foldLive({}, {});
|
||||
|
||||
const TrackingState state{load.status, populated, fold};
|
||||
const ProtectionAnswer answer = pruneProtection(state);
|
||||
CHECK(answer.blocked);
|
||||
CHECK(answer.ledgerFutureVersion);
|
||||
CHECK(!answer.ledgerUnreadable);
|
||||
CHECK(answer.ownedPaths.empty());
|
||||
CHECK(reclaim::pruneOrphans({"bank/would-be-orphan.wav"}, {},
|
||||
answer.ownedPaths).empty());
|
||||
|
||||
// And the replace-vs-add question abstains rather than allowing a replace.
|
||||
CHECK(tiedUsageExists(state, "bank/would-be-orphan.wav", "") == Answer::Indeterminate);
|
||||
}
|
||||
|
||||
// An unreadable ledger blocks too, AND leaves ownedPaths empty — so a caller that
|
||||
@@ -178,12 +214,19 @@ static void testUnreadableLedgerBlocksAndYieldsNoOrphans() {
|
||||
// Both blockers at once must both be reported — the operator needs to fix both.
|
||||
static void testBothBlockersReported() {
|
||||
const OriginLedger empty;
|
||||
const UsageFoldResult fold = foldLive({unreadable("rsusage_BROKEN")}, {"{T1}"});
|
||||
const UsageFoldResult fold = foldLive(
|
||||
{usage("rsusage_A", "{T1}", {UsageHold{"S-held", "bank/held.wav"}}),
|
||||
unreadable("rsusage_BROKEN")},
|
||||
{"{T1}"});
|
||||
const TrackingState state{LedgerStatus::Unreadable, empty, fold};
|
||||
const ProtectionAnswer answer = pruneProtection(state);
|
||||
CHECK(answer.blocked);
|
||||
CHECK(answer.ledgerUnreadable);
|
||||
CHECK(answer.unreadableUsageKeys.size() == 1);
|
||||
// heldPaths survives a double block: the fold's protect-all set only ever widens
|
||||
// what prune protects, so withholding it would be the unsafe direction.
|
||||
CHECK(contains(answer.heldPaths, "bank/held.wav"));
|
||||
CHECK(answer.ownedPaths.empty());
|
||||
}
|
||||
|
||||
// A record that exists but whose track hosts no identified instance still protects
|
||||
@@ -260,9 +303,6 @@ static void testSoleHolderExcludingItselfAnswersNo() {
|
||||
CHECK(tiedUsageExists(state, "bank/src.wav", "rsusage_ME") == Answer::No);
|
||||
}
|
||||
|
||||
// A `unioned` record carries more than one incarnation's holds, so it can never be
|
||||
// attributed to a single owner — excluding it could hide a sibling's tie, which is
|
||||
// the under-protecting direction.
|
||||
static void testUnionedRecordIsNeverExcludedAsOwn() {
|
||||
OriginLedger ledger;
|
||||
ledger.record(originOf("bank/src.wav", OriginKind::Capture, "S-src"));
|
||||
@@ -404,6 +444,7 @@ int main() {
|
||||
testLiveHoldProtectsDeReferencedCapture();
|
||||
testUnreadableUsageBlocksPruneAndNamesIt();
|
||||
testUnreadableLedgerBlocksAndYieldsNoOrphans();
|
||||
testFutureVersionLedgerBlocksAndIsReportedSeparately();
|
||||
testBothBlockersReported();
|
||||
testZeroIdentifiedInstancesStillProtects();
|
||||
testUnreadableStateNeverAnswersNo();
|
||||
|
||||
Reference in New Issue
Block a user