daniel/reasampler
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

tracking: read the ledger's version, not just write it; clear owned on any block; channel-correct prune recovery

Browse Source
This commit is contained in:
Daniel Harvey
2026-07-30 20:11:05 -04:00
parent 7f70d94228
commit 45b87dc2ff
27 changed files with 432 additions and 168 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/shell/persist/CLAUDE.md
+12 -9
View File
@@ -34,17 +34,20 @@ REAPER/filesystem-facing half only, and it gathers rather than decides.
record/input chains, containers recursively, take FX), and folds via the pure
`sample_usage::foldUsageRecords`. Read-only at prune-scan time: `usage_scan` writes
no ext-state.
- **A malformed tracking ledger is Unreadable, never "empty".** `ext_state_io` keeps
- **A ledger this build cannot read is degraded, never "empty".** `ext_state_io` keeps
the `LedgerStatus` alongside the ledger, and `saveToActiveProject` SKIPS the
`owned_files` write while it is `Unreadable` — replacing a corrupt blob would
destroy the only record of every file created before the corruption, silently
turning them into permanently unreclaimable foreign files. Captures made during
such a session are recorded in memory but not persisted; they degrade to foreign
(untouchable), which is the safe direction.
`owned_files` write while `tracking::ledgerDegraded` holds — replacing a blob we
could not read would destroy the only record of every file created before it,
silently turning them into permanently unreclaimable foreign files. The status is
written only by `loadFromProject`, so it is sticky until the project is reloaded:
captures made during such a session are recorded in memory but not persisted, and
degrade to foreign (untouchable), which is the safe direction.
- **`PruneReport` carries `blockedByTracking` + `ledgerUnreadable` /
`unreadableUsageKeys`**; dry-run, orphan-set, and reclaim each independently abort
(delete nothing) on a block. `BANK_PRUNE_FOLDER` (in `shell/actions`) halts on the
flag and prints whichever blockers fired.
`ledgerFutureVersion` / `unreadableUsageKeys`**; dry-run, orphan-set, and reclaim
each independently abort (delete nothing) on a block. `BANK_PRUNE_FOLDER` (in
`shell/actions`) halts on the flag and prints whichever blockers fired, with
channel-correct recovery lines — a corrupt blob may be cleared, a newer build's
must not be.
## Modules
src/shell/persist/ext_state_io.cpp
+10 -9
View File
@@ -161,11 +161,9 @@ bool ReaSamplerSession::saveToActiveProject() {
SetProjExtState(static_cast<ReaProject*>(proj), projExtNamespace(),
kProjExtTailKey, tailJson.c_str());
// Written on every save so the ledger and the bank stay in lockstep on disk —
// EXCEPT over a blob we could not read: replacing it would destroy the only
// record of every file created before the corruption, silently turning them
// into permanently unreclaimable foreign files.
if (trackingStatus_ != tracking::LedgerStatus::Unreadable) {
// Never write over a blob this build could not read (see this directory's
// CLAUDE.md for why the suppression, not a rewrite, is the safe direction).
if (!tracking::ledgerDegraded(trackingStatus_)) {
const std::string ledgerJson = tracking_.serialize();
SetProjExtState(static_cast<ReaProject*>(proj), projExtNamespace(),
kProjExtOwnedKey, ledgerJson.c_str());
@@ -234,10 +232,8 @@ capture::TailSetting loadTailSetting(ReaProject* proj) {
return *loaded;
}
// Absent/empty key -> Fresh (a new project, or a bank predating the ledger).
// Malformed -> Unreadable, which halts the prune and suppresses the next write
// rather than degrading to an empty ledger that looks like "nothing was ever
// created".
// Absent/empty key -> Fresh (a new project, or a bank predating the ledger). Anything
// this build cannot read is a degraded status, never an empty ledger.
tracking::LedgerLoad loadOriginLedger(ReaProject* proj) {
if (!proj) return tracking::LedgerLoad{};
tracking::LedgerLoad load = tracking::loadLedger(
@@ -246,6 +242,11 @@ tracking::LedgerLoad loadOriginLedger(ReaProject* proj) {
ShowConsoleMsg("ReaSampler: the stored file-tracking ledger is malformed. Prune "
"is halted for this project and the stored value is left intact "
"for recovery.\n");
} else if (load.status == tracking::LedgerStatus::FutureVersion) {
ShowConsoleMsg("ReaSampler: the stored file-tracking ledger was written by a "
"NEWER version of ReaSampler. Prune is halted for this project "
"and the stored value will not be overwritten -- reopen the "
"project with that version.\n");
}
return load;
}
src/shell/persist/prune_fs.cpp
+3
View File
@@ -72,6 +72,7 @@ struct PruneScan {
std::unordered_map<std::string, std::uint64_t> sizeByRel;
bool blocked = false;
bool ledgerUnreadable = false;
bool ledgerFutureVersion = false;
std::vector<std::string> unreadableUsageKeys;
};
@@ -134,6 +135,7 @@ PruneScan scanPruneOrphans(const BankBook& book, const tracking::OriginLedger& l
// action tell the user what to recover.
scan.blocked = true;
scan.ledgerUnreadable = protection.ledgerUnreadable;
scan.ledgerFutureVersion = protection.ledgerFutureVersion;
scan.unreadableUsageKeys = protection.unreadableUsageKeys;
return scan;
}
@@ -211,6 +213,7 @@ reclaim::PruneReport ReaSamplerSession::pruneDryRun() const {
// the prune refused to run.
report.blockedByTracking = scan.blocked;
report.ledgerUnreadable = scan.ledgerUnreadable;
report.ledgerFutureVersion = scan.ledgerFutureVersion;
report.unreadableUsageKeys = scan.unreadableUsageKeys;
return report;
}
src/shell/persist/session.cpp
+13 -1
View File
@@ -35,6 +35,7 @@
#define REAPERAPI_MINIMAL
#define REAPERAPI_WANT_MarkProjectDirty
#define REAPERAPI_WANT_SetProjExtState
#define REAPERAPI_WANT_ShowConsoleMsg
#include "reaper_plugin_functions.h"
namespace reasampler {
@@ -56,7 +57,18 @@ void ReaSamplerSession::recordCreated(const model::Sample& sample,
// rather than re-deriving keeps one source for the lineage fact. Absent
// provenance is a root capture, not a gap.
if (sample.provenance) rec.parentSampleId = sample.provenance->parentSampleId;
tracking_.record(rec);
const tracking::RecordResult result = tracking_.record(rec);
// A rejection means a file exists that nothing attributes to us — invisible
// otherwise, and exactly the gap this ledger exists to close. AlreadyPresent is
// the normal dedup outcome, not a gap.
if (result == tracking::RecordResult::RejectedEmptyPath ||
result == tracking::RecordResult::RejectedAbsolutePath) {
ShowConsoleMsg(("ReaSampler: could not record the origin of '" +
sample.relativePath +
"' -- the path is empty or absolute. The file is NOT tracked and "
"prune will treat it as a foreign file.\n").c_str());
}
}
bool ReaSamplerSession::consumeLoadSignal() {
src/shell/persist/session.h
+16 -14
View File
@@ -71,17 +71,20 @@ public:
capture::TailSetting& tail() { return tail_; }
const capture::TailSetting& tail() const { return tail_; }
// Birth records for the files the system itself created. Read as a PAIR —
// the ledger alone cannot say whether an absent record means never-recorded
// or unreadable, and the two demand opposite treatment. This is the reach
// any consumer outside persist uses to build a tracking::TrackingState.
const tracking::OriginLedger& tracking() const { return tracking_; }
tracking::LedgerStatus trackingStatus() const { return trackingStatus_; }
// Record a system-created file at the moment it exists — the ONLY way a
// birth record is written, so lineage can never be backfilled from a later
// guess. Lineage is read off the Sample's own provenance, the same act that
// stamped it, so the two cannot disagree. A repeat path is a no-op.
// Record a system-created file at the moment it exists — the ONLY way a birth
// record is written. Lineage is read off the Sample's own provenance, the same
// act that stamped it, so the two cannot disagree; where they later diverge the
// ledger record is authoritative (core/tracking/CLAUDE.md).
//
// Call it at EVERY creation site regardless of the bank's AddResult: a
// hash-collapse still wrote a file the tool owns, and an unrecorded file is a
// permanently unreclaimable foreign file. A rejected record is reported to the
// console — a file with no record is the gap this track exists to eliminate.
//
// A consumer outside persist that needs the ledger must take it WITH its
// LedgerStatus (a tracking::TrackingState); no accessor exposes one without the
// other, because an absent record and an unreadable ledger demand opposite
// treatment.
void recordCreated(const model::Sample& sample, tracking::OriginKind kind);
// The version that last wrote the active project: PreVersioning (no
@@ -153,9 +156,8 @@ private:
ViewModeModel view_; // reset to default on a project with no stored view_state
capture::TailSetting tail_; // reset to default (None / 2s) with no stored tail key
tracking::OriginLedger tracking_; // reset to empty/stored on EVERY load path, never inherited
// Unreadable is sticky for the project's session: it halts the prune AND
// suppresses the ledger write, so a corrupt blob survives for recovery
// instead of being silently replaced by a ledger missing every earlier file.
// Written only by loadFromProject, so a degraded status is sticky until the
// project is reloaded (see this directory's CLAUDE.md).
tracking::LedgerStatus trackingStatus_ = tracking::LedgerStatus::Fresh;
version::WritingVersion writingVersion_; // recovered per load; PreVersioning default
std::int64_t bankGeneration_ = 0; // recovered per load (absent -> 0); monotonic