From 043558a54dab4d740e33036471d4df183d13d158 Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 07:27:43 -0400 Subject: [PATCH 01/24] =?UTF-8?q?Land=20src/core/package:=20the=20pure=20R?= =?UTF-8?q?SBK=20container=20=E2=80=94=20format=20ladder,=20JSON=20manifes?= =?UTF-8?q?t,=20framing/layout=20codec?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two-integer ladder (formatVersion/minReaderVersion), bare-name-only entries validated on encode and decode, prefix decode that proves exact file size without ever reading a payload. --- CMakeLists.txt | 1 + src/core/package/CLAUDE.md | 78 ++++++ src/core/package/CMakeLists.txt | 14 ++ src/core/package/bank_package.cpp | 139 +++++++++++ src/core/package/bank_package.h | 71 ++++++ src/core/package/package_format.cpp | 23 ++ src/core/package/package_format.h | 80 ++++++ src/core/package/package_manifest.cpp | 201 +++++++++++++++ src/core/package/package_manifest.h | 56 +++++ tests/test_bank_package.cpp | 344 ++++++++++++++++++++++++++ tests/test_package_format.cpp | 96 +++++++ tests/test_package_manifest.cpp | 248 +++++++++++++++++++ 12 files changed, 1351 insertions(+) create mode 100644 src/core/package/CLAUDE.md create mode 100644 src/core/package/CMakeLists.txt create mode 100644 src/core/package/bank_package.cpp create mode 100644 src/core/package/bank_package.h create mode 100644 src/core/package/package_format.cpp create mode 100644 src/core/package/package_format.h create mode 100644 src/core/package/package_manifest.cpp create mode 100644 src/core/package/package_manifest.h create mode 100644 tests/test_bank_package.cpp create mode 100644 tests/test_package_format.cpp create mode 100644 tests/test_package_manifest.cpp diff --git a/CMakeLists.txt b/CMakeLists.txt index f783871..ed9f193 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -92,3 +92,4 @@ enable_testing() add_subdirectory(src/core) add_subdirectory(src/app) add_subdirectory(src/shell/instrument) +add_subdirectory(src/core/package) diff --git a/src/core/package/CLAUDE.md b/src/core/package/CLAUDE.md new file mode 100644 index 0000000..cb56e08 --- /dev/null +++ b/src/core/package/CLAUDE.md @@ -0,0 +1,78 @@ +# src/core/package — the pure RSBK bank-package codec + +## Scope + +The hand-rolled `RSBK` bank-package container, entirely pure (REAPER-free, +unit-tested outside the DAW): the format contract and version ladder, the JSON +manifest, and the framing/layout codec. No filesystem — the shell +(`src/shell/package`) streams bytes against the layouts produced here. The +export/import *decisions* (`export_plan` / `import_plan`) are separate modules +landing after the format. + +## Invariants + +- **The container is the proprietary `RSBK` — ruled, not revisitable here.** No + ZIP, no compressor, no link edge to `vendor/WDL/WDL/zlib/`. The version + ladder, not a format swap, is how the format moves. +- **Two version integers, two jobs.** `formatVersion` = what the writer + emitted; `minReaderVersion` = the oldest reader that can read it safely. The + reader's whole rule is `minReaderVersion <= kPackageFormatVersion`. Additive + changes (a new optional manifest key, a new enum value with a defined + degrade) bump `formatVersion` only; structural changes bump both. The full + ladder lives as a comment in `package_format.h` and is READ and validated, + never merely written. +- **TooNew refuses whole.** A `minReaderVersion` above this build yields the + header (so the refusal can name the writer's semver and version) and nothing + else — no manifest, no layout, no half-success. The fields through the writer + semver are FROZEN for all future versions to keep that refusal producible. +- **Path expression is structurally impossible.** Entry names are bare file + names (`isValidEntryName`: no separators, no `..`, no drive/UNC/rooted form), + enforced on encode AND decode because a package can arrive from anywhere. + There is no field in the format capable of expressing a path. +- **Framing only, never a payload.** `bank_package` produces header bytes and + an ordered `{name, offset, length}` layout; it never holds, copies, or hashes + an entry's audio. `decodePackage` proves prefix + payload lengths equal the + observed file size exactly, so truncation and trailing garbage are Malformed + without any payload being read. +- **Per-sample shape has one owner.** Each entry nests a one-sample `BankModel` + blob emitted/parsed by `bank_model`'s own codec (the `bank_book_json` + precedent), so a future `Sample` field reaches packages with no change here. +- **Hostile input: error signaled, never UB** — the `bank_model.h` deserialize + standard, plus allocation caps on every length field so a forged header + cannot demand gigabytes. + +## Modules + +- `package_format` — the contract: magic, `kPackageFormatVersion` / + `kPackageMinReaderVersion`, the ladder comment, the three-way + `classifyPackageVersion` (`Readable` / `TooNew` / `Malformed`), the + entry-name rule, and `PackageHeader`. +- `package_manifest` — the manifest model (`PackageEntry` / `PackageManifest`) + and its JSON codec. Per entry: bare name, byte length, and a whole-file + `capture::hashBytes` digest (deliberately NOT `hashWavContent`, which skips + chunks and cannot answer "did these bytes survive") — the digest is carried + here, computed where payloads are streamed (shell). The bank's `slot_map` + rides along. Unknown keys skip at every level; duplicate entry names are + rejected both ways. +- `bank_package` — framing and arithmetic composing the two above: + `encodePackage` (prefix bytes + layout + total size, stamping this build's + ladder pair and `version::stampVersion()`), `decodePackage` (prefix + observed + file size in; header/manifest/layout out), and `requiredPrefixSize` (the + incremental-read seam for the shell). Framing rides `core/wire/bytes.h`. + +## Gotchas + +- Enums nested inside the `BankModel` blob follow `bank_model`'s own rule — an + out-of-range `sourceMode`/`tier` REJECTS the parse — so growing one of those + vocabularies is a `minReaderVersion` bump for packages, not an additive + change. Any enum integer the manifest itself ever adds must instead follow + the degrade-to-`Unknown` rule (`core/wire`'s `BakeStatus` precedent) to stay + additive. The manifest carries no enum of its own today. +- Sample-id rules (remap, collision, dedup across the destination) are + deliberately NOT enforced by the codec — they are `import_plan` decisions. The + codec rejects only what makes the container itself incoherent (duplicate + entry names, invalid names, a non-single-sample nested index). +- `requiredPrefixSize` trusts fields beyond the frozen region only when the + version pair classifies `Readable`; for `TooNew` it stops at the semver — + don't "fix" it to read the manifest length there, a future structural format + may have moved it. diff --git a/src/core/package/CMakeLists.txt b/src/core/package/CMakeLists.txt new file mode 100644 index 0000000..fc7fd2d --- /dev/null +++ b/src/core/package/CMakeLists.txt @@ -0,0 +1,14 @@ +reasampler_pure_library(package_format SOURCES package_format.cpp) +reasampler_test(package_format LINK package_format) + +reasampler_pure_library(package_manifest + SOURCES package_manifest.cpp + LINK PUBLIC bank_model slot_map PRIVATE package_format json) +reasampler_test(package_manifest LINK package_manifest) + +# bytes.h is header-only (see src/core/wire/CLAUDE.md) — no wire link edge needed. +reasampler_pure_library(bank_package + SOURCES bank_package.cpp + LINK PUBLIC package_format package_manifest PRIVATE app_version) +# app_version: the tests pin the stamped writer semver against stampVersion(). +reasampler_test(bank_package LINK bank_package app_version) diff --git a/src/core/package/bank_package.cpp b/src/core/package/bank_package.cpp new file mode 100644 index 0000000..b5a9389 --- /dev/null +++ b/src/core/package/bank_package.cpp @@ -0,0 +1,139 @@ +#include "core/package/bank_package.h" + +#include + +#include "core/version/app_version.h" +#include "core/wire/bytes.h" + +// Byte offsets (format 1, see package_format.h's ladder): magic at 0, u32 +// formatVersion at 4, u32 minReaderVersion at 8, u32 semver length W at 12, +// semver at 16, u32 manifest length M at 16+W, manifest at 20+W, payloads at +// 20+W+M. The region through the semver is the FROZEN refusal surface. + +namespace reasampler::package { + +namespace { + +constexpr std::size_t kMagicBytes = 4; +constexpr std::size_t kSemverLenAt = 12; +constexpr std::size_t kSemverAt = 16; + +bool magicMatches(const std::vector& bytes) { + return bytes.size() >= kMagicBytes && + std::memcmp(bytes.data(), kPackageMagic, kMagicBytes) == 0; +} + +std::uint32_t u32At(const std::vector& bytes, std::size_t at) { + std::uint32_t v = 0; + for (std::size_t b = 0; b < 4; ++b) + v |= static_cast(bytes[at + b]) << (b * 8); + return v; +} + +// Appends the payload spans for `entries` starting at `firstOffset`. False on +// u64 overflow (a forged length field summing past 2^64 must not wrap into a +// plausible layout). +bool appendSpans(const std::vector& entries, std::uint64_t firstOffset, + std::vector& out, std::uint64_t& end) { + std::uint64_t offset = firstOffset; + for (const auto& e : entries) { + out.push_back({e.fileName, offset, e.byteLength}); + if (offset + e.byteLength < offset) return false; + offset += e.byteLength; + } + end = offset; + return true; +} + +} // namespace + +std::optional encodePackage(const PackageManifest& m) { + auto manifestJson = serializeManifest(m); + if (!manifestJson) return std::nullopt; + if (manifestJson->size() > kMaxManifestBytes) return std::nullopt; + + const std::string& writer = version::stampVersion(); + if (writer.size() > kMaxWriterVersionBytes) return std::nullopt; + + EncodedPackage enc; + auto& out = enc.prefix; + out.insert(out.end(), kPackageMagic, kPackageMagic + kMagicBytes); + wire::putLE(out, kPackageFormatVersion); + wire::putLE(out, kPackageMinReaderVersion); + wire::putLE(out, static_cast(writer.size())); + out.insert(out.end(), writer.begin(), writer.end()); + wire::putLE(out, static_cast(manifestJson->size())); + out.insert(out.end(), manifestJson->begin(), manifestJson->end()); + + if (!appendSpans(m.entries, out.size(), enc.layout, enc.totalSize)) + return std::nullopt; + return enc; +} + +DecodedPackage decodePackage(const std::vector& prefix, + std::uint64_t totalFileSize) { + DecodedPackage dec; // status starts Malformed; every early return means it + + wire::ByteReader r(prefix); + if (r.str(kMagicBytes) != std::string(kPackageMagic, kMagicBytes)) return dec; + + const std::uint32_t formatVersion = r.u32(); + const std::uint32_t minReader = r.u32(); + if (!r.ok) return dec; + const PackageReadability verdict = classifyPackageVersion(formatVersion, minReader); + if (verdict == PackageReadability::Malformed) return dec; + + const std::uint32_t semverLen = r.u32(); + if (!r.ok || semverLen > kMaxWriterVersionBytes) return dec; + std::string writer = r.str(semverLen); + if (!r.ok) return dec; + + dec.header = PackageHeader{formatVersion, minReader, std::move(writer)}; + if (verdict == PackageReadability::TooNew) { + // Refuse whole: the header names the writer for the message; nothing + // past the frozen region is read, and no manifest is produced. + dec.status = PackageReadability::TooNew; + return dec; + } + + const std::uint32_t manifestLen = r.u32(); + if (!r.ok || manifestLen > kMaxManifestBytes) return dec; + const std::string manifestJson = r.str(manifestLen); + if (!r.ok) return dec; + + auto manifest = deserializeManifest(manifestJson); + if (!manifest) return dec; + + std::uint64_t end = 0; + std::vector layout; + if (!appendSpans(manifest->entries, r.pos, layout, end)) return dec; + // Exact-size proof: a byte missing (truncation) or a byte extra (trailing + // garbage) both fail, even though no payload is read here. + if (end != totalFileSize) return dec; + + dec.status = PackageReadability::Readable; + dec.manifest = std::move(*manifest); + dec.layout = std::move(layout); + dec.prefixSize = r.pos; + return dec; +} + +std::optional requiredPrefixSize(const std::vector& bytes) { + if (bytes.size() < kSemverAt) return kSemverAt; + if (!magicMatches(bytes)) return std::nullopt; + + const auto verdict = classifyPackageVersion(u32At(bytes, 4), u32At(bytes, 8)); + if (verdict == PackageReadability::Malformed) return std::nullopt; + + const std::uint32_t semverLen = u32At(bytes, kSemverLenAt); + if (semverLen > kMaxWriterVersionBytes) return std::nullopt; + const std::uint64_t throughSemver = kSemverAt + semverLen; + if (verdict == PackageReadability::TooNew) return throughSemver; + + if (bytes.size() < throughSemver + 4) return throughSemver + 4; + const std::uint32_t manifestLen = u32At(bytes, static_cast(throughSemver)); + if (manifestLen > kMaxManifestBytes) return std::nullopt; + return throughSemver + 4 + manifestLen; +} + +} // namespace reasampler::package diff --git a/src/core/package/bank_package.h b/src/core/package/bank_package.h new file mode 100644 index 0000000..5fb0e94 --- /dev/null +++ b/src/core/package/bank_package.h @@ -0,0 +1,71 @@ +#pragma once +// bank_package — RSBK framing and layout arithmetic: header encode, prefix +// decode, and the ordered {name, offset, length} entry layout. Framing only, +// never a payload: this module never holds, copies, or hashes an entry's audio +// — the shell streams payloads one at a time against the layout produced here. + +#include +#include +#include +#include + +#include "core/package/package_format.h" +#include "core/package/package_manifest.h" + +namespace reasampler::package { + +// Where one payload sits in the finished package file (absolute byte offset). +struct PackageEntrySpan { + std::string name; + std::uint64_t offset = 0; + std::uint64_t length = 0; + + bool operator==(const PackageEntrySpan& o) const { + return name == o.name && offset == o.offset && length == o.length; + } +}; + +// The write side's product: the prefix bytes (magic through manifest, written +// verbatim as the file's head), the layout to stream each payload at, and the +// finished file's exact size — what the shell verifies after the last append. +struct EncodedPackage { + std::vector prefix; + std::vector layout; + std::uint64_t totalSize = 0; +}; + +// Encodes the package prefix for `m`, stamping this build's version pair and +// version::stampVersion() as the writer semver. nullopt when the manifest +// cannot be represented (serializeManifest's rejections) — refused on encode so +// an undecodable package is never written. +std::optional encodePackage(const PackageManifest& m); + +// The read side's product. header is meaningful for Readable and TooNew (a +// refusal must still name the writer); manifest, layout, and prefixSize only +// for Readable — TooNew produces NO manifest, so a refused decode cannot +// half-succeed. +struct DecodedPackage { + PackageReadability status = PackageReadability::Malformed; + PackageHeader header; + PackageManifest manifest; + std::vector layout; + std::uint64_t prefixSize = 0; +}; + +// Decodes a package's leading bytes. `totalFileSize` is the on-disk size the +// caller observed: decode proves prefix + payload lengths equal it exactly, so +// a truncated or garbage-extended file is Malformed even though the payloads +// themselves are never read here. `prefix` may be the whole file or any head of +// it that requiredPrefixSize accepted. Error signaled, never UB. +DecodedPackage decodePackage(const std::vector& prefix, + std::uint64_t totalFileSize); + +// How many leading bytes decodePackage needs. May grow as bytes arrive: with +// fewer than the returned count on hand, read to that count and ask again. +// For a TooNew package it stops at the frozen region (through the writer +// semver) — field positions beyond it belong to the newer format and are not +// trusted. nullopt: these bytes can never frame a package (bad magic, +// incoherent versions, an over-cap length field) — stop reading. +std::optional requiredPrefixSize(const std::vector& bytes); + +} // namespace reasampler::package diff --git a/src/core/package/package_format.cpp b/src/core/package/package_format.cpp new file mode 100644 index 0000000..139163b --- /dev/null +++ b/src/core/package/package_format.cpp @@ -0,0 +1,23 @@ +#include "core/package/package_format.h" + +namespace reasampler::package { + +PackageReadability classifyPackageVersion(std::uint32_t formatVersion, + std::uint32_t minReaderVersion) { + if (formatVersion == 0 || minReaderVersion == 0) return PackageReadability::Malformed; + if (minReaderVersion > formatVersion) return PackageReadability::Malformed; + if (minReaderVersion > kPackageFormatVersion) return PackageReadability::TooNew; + return PackageReadability::Readable; +} + +bool isValidEntryName(const std::string& name) { + if (name.empty() || name.size() > kMaxEntryNameBytes) return false; + if (name == ".") return false; + if (name.find("..") != std::string::npos) return false; + for (char c : name) { + if (c == '/' || c == '\\' || c == ':') return false; + } + return true; +} + +} // namespace reasampler::package diff --git a/src/core/package/package_format.h b/src/core/package/package_format.h new file mode 100644 index 0000000..acb70f3 --- /dev/null +++ b/src/core/package/package_format.h @@ -0,0 +1,80 @@ +#pragma once +// package_format — the RSBK bank-package contract: magic, the version ladder, +// the readability classification, and the entry-name rule. Pure: standard +// library only. The framing codec that acts on this contract is bank_package; +// the manifest grammar is package_manifest. + +#include +#include + +namespace reasampler::package { + +// Version ladder for the RSBK container (read-and-validate, like the origin +// ledger's "v"): +// +// format 1 (current) magic "RSBK" | u32 formatVersion | u32 minReaderVersion +// | u32 len + writer semver | u32 len + JSON manifest +// | payloads concatenated in manifest entry order. +// All integers little-endian. +// +// Two integers, two jobs: formatVersion is what the writer emitted (monotonic, +// bumped on ANY change); minReaderVersion is the oldest reader that can read the +// package safely (bumped only on a STRUCTURAL change — a field's meaning shifts, +// a section is removed, framing moves; an additive change — a new optional +// manifest key, a new enum value with a defined degrade — leaves it alone). The +// reader's whole rule: read iff minReaderVersion <= kPackageFormatVersion. +// formatVersion beyond that is message text and log material only. +// +// FROZEN FOR ALL FUTURE VERSIONS: the fields through the writer semver. A +// too-new package must still yield the writer's version so the refusal can name +// what to install — a structural change may rearrange anything after the semver, +// never before it. +inline constexpr char kPackageMagic[4] = {'R', 'S', 'B', 'K'}; +inline constexpr std::uint32_t kPackageFormatVersion = 1; +inline constexpr std::uint32_t kPackageMinReaderVersion = 1; + +// Hostile-input allocation caps (error signaled, never a multi-gigabyte +// allocation off a forged length field). Generous against real content: a +// semver is ~10 bytes; a manifest for hundreds of samples is well under 1 MB. +inline constexpr std::uint32_t kMaxWriterVersionBytes = 64; +inline constexpr std::uint32_t kMaxManifestBytes = 64u * 1024u * 1024u; +inline constexpr std::size_t kMaxEntryNameBytes = 255; + +// The three-way read verdict (the FutureVersion precedent): TooNew refuses the +// whole package before anything is produced; Malformed is a header no honest +// writer emits. Also the status of a full prefix decode in bank_package. +enum class PackageReadability { + Readable, + TooNew, + Malformed, +}; + +// Classify a stored header pair against THIS build's ladder. minReaderVersion +// above kPackageFormatVersion is TooNew; a zero version or minReader > +// formatVersion is Malformed (a writer cannot require a reader newer than what +// it wrote). +PackageReadability classifyPackageVersion(std::uint32_t formatVersion, + std::uint32_t minReaderVersion); + +// The entry-name rule that makes path expression structurally impossible: a +// bare file name only. Rejects empty, ".", any ".." occurrence, any '/', '\\' +// or ':' (which also bans every absolute form — drive, UNC, rooted), and names +// over kMaxEntryNameBytes. Enforced on encode AND decode by package_manifest. +bool isValidEntryName(const std::string& name); + +// The fixed header, informational semver included. writerVersion is +// version::stampVersion() on the write side — it exists so a TooNew refusal can +// tell the user which build to install; it never gates. +struct PackageHeader { + std::uint32_t formatVersion = kPackageFormatVersion; + std::uint32_t minReaderVersion = kPackageMinReaderVersion; + std::string writerVersion; + + bool operator==(const PackageHeader& o) const { + return formatVersion == o.formatVersion && + minReaderVersion == o.minReaderVersion && + writerVersion == o.writerVersion; + } +}; + +} // namespace reasampler::package diff --git a/src/core/package/package_manifest.cpp b/src/core/package/package_manifest.cpp new file mode 100644 index 0000000..10cc267 --- /dev/null +++ b/src/core/package/package_manifest.cpp @@ -0,0 +1,201 @@ +#include "core/package/package_manifest.h" + +#include + +#include "core/json/json.h" +#include "core/package/package_format.h" + +// Duplicate entry names are rejected in both directions: two payloads landing +// on one destination name is incoherent, and on import it would be a silent +// overwrite. Sample-id rules (remap, collision) are deliberately NOT enforced +// here — they are the import plan's decisions, not the codec's. + +namespace reasampler::package { + +namespace { + +using json::numToStr; +using ObjWriter = json::Writer; + +bool duplicateName(const std::vector& entries) { + for (std::size_t i = 0; i < entries.size(); ++i) + for (std::size_t j = i + 1; j < entries.size(); ++j) + if (entries[i].fileName == entries[j].fileName) return true; + return false; +} + +// The one-sample BankModel image of `s` — bank_model's own writer, verbatim, so +// the per-sample shape has exactly one owner. nullopt when add() would reject +// the record (its guards are the format's guards too). +std::optional nestSample(const model::Sample& s) { + model::BankModel one; + if (one.add(s) != model::AddResult::Added) return std::nullopt; + return one.serialize(); +} + +} // namespace + +bool PackageEntry::operator==(const PackageEntry& o) const { + return fileName == o.fileName && byteLength == o.byteLength && + byteHash == o.byteHash && sample == o.sample; +} + +bool PackageManifest::operator==(const PackageManifest& o) const { + return bankDisplayName == o.bankDisplayName && exportTimestamp == o.exportTimestamp && + entries == o.entries && slots == o.slots; +} + +std::optional serializeManifest(const PackageManifest& m) { + for (const auto& e : m.entries) + if (!isValidEntryName(e.fileName)) return std::nullopt; + if (duplicateName(m.entries)) return std::nullopt; + + std::string out; + { + ObjWriter root(out); + root.keyStr("bankName", m.bankDisplayName); + root.keyRaw("exported", numToStr(m.exportTimestamp)); + + root.keyBegin("entries"); + out += '['; + for (std::size_t i = 0; i < m.entries.size(); ++i) { + const auto& e = m.entries[i]; + auto nested = nestSample(e.sample); + if (!nested) return std::nullopt; + if (i) out += ','; + ObjWriter w(out); + w.keyStr("name", e.fileName); + // byteLength rides as a signed decimal; 2^63 bytes is beyond any file. + w.keyRaw("length", numToStr(static_cast(e.byteLength))); + w.keyStr("hash", e.byteHash); + w.keyRaw("index", *nested); + } + out += ']'; + + root.keyBegin("slots"); + out += m.slots.serialize(); + } // root closes here (NRVO note in json::Writer) + return out; +} + +namespace { + +// Mirrors bank_book_json's private slots parser: [{id, slot}, ...] pairs handed +// to SlotMap::fromEntries, which owns the defensive repair rules. +bool parseSlots(json::Reader& r, model::SlotMap& out) { + std::vector> pairs; + if (!r.consume('[')) return false; + r.skipWs(); + if (r.consume(']')) { + out = model::SlotMap::fromEntries(pairs); + return true; + } + do { + if (!r.consume('{')) return false; + std::string id; + int slot = 0; + bool haveId = false, haveSlot = false; + do { + std::string k; + if (!r.parseKey(k)) return false; + if (k == "id") { if (!r.parseString(id)) return false; haveId = true; } + else if (k == "slot") { if (!r.parseInt(slot)) return false; haveSlot = true; } + else { if (!r.skipValue()) return false; } + } while (r.consume(',')); + if (!r.consume('}')) return false; + if (!haveId || !haveSlot) return false; + pairs.emplace_back(std::move(id), slot); + } while (r.consume(',')); + if (!r.consume(']')) return false; + out = model::SlotMap::fromEntries(pairs); + return true; +} + +bool parseEntry(json::Reader& r, PackageEntry& e) { + if (!r.consume('{')) return false; + r.skipWs(); + if (r.consume('}')) return false; // an entry needs all four fields + + bool haveName = false, haveLength = false, haveHash = false, haveSample = false; + do { + std::string key; + if (!r.parseKey(key)) return false; + + if (key == "name") { + if (!r.parseString(e.fileName)) return false; + haveName = true; + } else if (key == "length") { + std::int64_t v = 0; + if (!r.parseInt64(v)) return false; + if (v < 0) return false; + e.byteLength = static_cast(v); + haveLength = true; + } else if (key == "hash") { + if (!r.parseString(e.byteHash)) return false; + haveHash = true; + } else if (key == "index") { + std::string raw; + if (!r.captureValue(raw)) return false; + auto idx = model::BankModel::deserialize(raw); + // Exactly one sample: add()'s silent drop (rejected record) or a + // multi-sample blob both fail the entry rather than half-parse. + if (!idx || idx->size() != 1) return false; + e.sample = idx->all().front(); + haveSample = true; + } else { + if (!r.skipValue()) return false; // forward-compat unknown keys + } + } while (r.consume(',')); + + if (!r.consume('}')) return false; + if (!haveName || !haveLength || !haveHash || !haveSample) return false; + return isValidEntryName(e.fileName); +} + +bool parseManifest(json::Reader& r, PackageManifest& m) { + if (!r.consume('{')) return false; + r.skipWs(); + if (r.consume('}')) return true; // empty object: a valid empty manifest + + do { + std::string key; + if (!r.parseKey(key)) return false; + + if (key == "bankName") { + if (!r.parseString(m.bankDisplayName)) return false; + } else if (key == "exported") { + if (!r.parseInt64(m.exportTimestamp)) return false; + } else if (key == "entries") { + if (!r.consume('[')) return false; + r.skipWs(); + if (!r.consume(']')) { + do { + PackageEntry e; + if (!parseEntry(r, e)) return false; + m.entries.push_back(std::move(e)); + } while (r.consume(',')); + if (!r.consume(']')) return false; + } + } else if (key == "slots") { + if (!parseSlots(r, m.slots)) return false; + } else { + if (!r.skipValue()) return false; // forward-compat unknown keys + } + } while (r.consume(',')); + + if (!r.consume('}')) return false; + r.skipWs(); + if (!r.eof()) return false; // trailing garbage + return !duplicateName(m.entries); +} + +} // namespace + +std::optional deserializeManifest(const std::string& json) { + PackageManifest m; + json::Reader r(json); + if (!parseManifest(r, m)) return std::nullopt; + return m; +} + +} // namespace reasampler::package diff --git a/src/core/package/package_manifest.h b/src/core/package/package_manifest.h new file mode 100644 index 0000000..3afe70d --- /dev/null +++ b/src/core/package/package_manifest.h @@ -0,0 +1,56 @@ +#pragma once +// package_manifest — the RSBK manifest model and its JSON codec. Per-sample +// shape is NOT owned here: each entry nests a one-sample BankModel blob emitted +// by bank_model's own writer (the bank_book_json precedent), so a future Sample +// field reaches packages for free. Pure: no filesystem, no host types. + +#include +#include +#include +#include + +#include "core/model/bank_model.h" +#include "core/model/slot_map.h" + +namespace reasampler::package { + +// One payload's transport record. `byteHash` is capture::hashBytes over the +// payload's raw bytes — the whole-file digest, deliberately NOT hashWavContent +// (which skips chunks and so cannot answer "did these bytes survive the trip"). +// FNV-1a: a corruption detector, not a cryptographic checksum. The codec only +// carries the digest; hashing happens where the payload is streamed (shell). +struct PackageEntry { + std::string fileName; // bare name inside the package (isValidEntryName) + std::uint64_t byteLength = 0; + std::string byteHash; + model::Sample sample; + + bool operator==(const PackageEntry& o) const; +}; + +// Everything the manifest carries besides the payloads: informational envelope +// (source bank name, export moment), the entries, and the bank's display +// positions (a bank's arrangement is part of what the user built). +struct PackageManifest { + std::string bankDisplayName; + std::int64_t exportTimestamp = 0; // unix epoch seconds + std::vector entries; + model::SlotMap slots; + + bool operator==(const PackageManifest& o) const; +}; + +// Emits the manifest JSON. nullopt when the manifest cannot be represented: +// an invalid or duplicate entry name, or a sample record BankModel itself would +// reject (empty id, absolute path) — refusing on encode so an undecodable +// package is never written. +std::optional serializeManifest(const PackageManifest& m); + +// Parses manifest JSON (nullopt on malformed input, never UB). Unknown keys are +// skipped at every level, so an additive newer manifest still parses. Rejects +// what encode rejects — entry names are validated on BOTH directions because a +// package can arrive from anywhere — plus a missing per-entry field or a +// negative length. +std::optional deserializeManifest(const std::string& json); + +} // namespace reasampler::package diff --git a/tests/test_bank_package.cpp b/tests/test_bank_package.cpp new file mode 100644 index 0000000..5279bd7 --- /dev/null +++ b/tests/test_bank_package.cpp @@ -0,0 +1,344 @@ +// Standalone tests for reasampler::package::bank_package — no REAPER, no test +// framework. Byte-level suites hand-roll RSBK images with wire::putLE rather +// than calling encodePackage, so a layout regression in encode cannot hide from +// decode (the two sides are pinned against each other AND against raw bytes). + +#include "../src/core/package/bank_package.h" + +#include +#include +#include +#include + +#include "../src/core/version/app_version.h" +#include "../src/core/wire/bytes.h" + +using namespace reasampler::package; +using namespace reasampler::model; +namespace wire = reasampler::wire; +namespace version = reasampler::version; + +static int g_fail = 0; +#define CHECK(cond) do { if(!(cond)) { \ + std::printf("FAIL line %d: %s\n", __LINE__, #cond); ++g_fail; } } while(0) + +// --- fixtures ---------------------------------------------------------------- + +// Every Sample field populated, every optional PRESENT. +static Sample fullSample() { + Sample s; + s.id = "smp-full"; + s.displayName = "Kick (wet)"; + s.relativePath = "reasampler_bank/kick.wav"; + s.sourceMode = SourceMode::RazorArea; + s.sourceRange = {1.25, 3.5, 480.0, 1920.0}; + s.trackGuids = {"{AAA}", "{BBB}"}; + s.wetDry = 0.75; + s.channelCount = 2; + s.sampleRate = 48000; + s.lengthSeconds = 2.25; + s.lengthBeats = 4.5; + s.captureTempo = 120.5; + s.captureTimeSigNum = 7; + s.captureTimeSigDenom = 8; + s.key = "F#m"; + s.rootNote = 60; + s.loop = LoopPoints{100, 4800}; + s.levels = {-0.3, -12.7, -14.0}; + s.clipped = true; + s.tier = Tier::Archive; + s.contentHash = "W0123456789abcdef"; + s.provenance = Provenance{"smp-parent", "fx-snapshot"}; + s.createdTimestamp = 1754000000; + return s; +} + +// Every Sample optional ABSENT (key, rootNote, loop, provenance). +static Sample bareSample() { + Sample s; + s.id = "smp-bare"; + s.displayName = "Snare"; + s.relativePath = "reasampler_bank/snare.wav"; + s.sourceMode = SourceMode::Realtime; + s.contentHash = "Wfedcba9876543210"; + s.createdTimestamp = 1754000001; + return s; +} + +static PackageManifest fixture(std::uint64_t len0, std::uint64_t len1) { + PackageManifest m; + m.bankDisplayName = "Drums \"live\""; + m.exportTimestamp = 1754100000; + m.entries.push_back({"kick.wav", len0, "1111222233334444", fullSample()}); + m.entries.push_back({"snare.wav", len1, "5555666677778888", bareSample()}); + m.slots.append("smp-full"); + m.slots.append("smp-bare"); + return m; +} + +// A hand-rolled RSBK image: frozen region + a raw tail (manifest framing or +// deliberate garbage), independent of encodePackage. +static std::vector rawHeader(std::uint32_t fv, std::uint32_t mv, + const std::string& semver) { + std::vector out; + out.insert(out.end(), kPackageMagic, kPackageMagic + 4); + wire::putLE(out, fv); + wire::putLE(out, mv); + wire::putLE(out, static_cast(semver.size())); + out.insert(out.end(), semver.begin(), semver.end()); + return out; +} + +static void appendManifest(std::vector& out, const std::string& json) { + wire::putLE(out, static_cast(json.size())); + out.insert(out.end(), json.begin(), json.end()); +} + +// One-entry manifest JSON with `extra` spliced in as additional root content +// ("" for none) — for images a current writer would never emit. +static std::string handManifest(const std::string& name, int length, + const std::string& extra) { + return std::string("{") + extra + + "\"entries\":[{\"name\":\"" + name + + "\",\"length\":" + std::to_string(length) + ",\"hash\":\"h\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s1\"," + "\"relativePath\":\"bank/a.wav\"}]}}]}"; +} + +// --- encode / decode round trip ---------------------------------------------- + +static void testEncodeDecodeRoundTrip() { + const PackageManifest m = fixture(96000, 0); // a zero-length payload is legal + auto enc = encodePackage(m); + CHECK(enc.has_value()); + + // Layout arithmetic: payloads start at the prefix end, in manifest order. + CHECK(enc->layout.size() == 2); + CHECK(enc->layout[0].name == "kick.wav"); + CHECK(enc->layout[0].offset == enc->prefix.size()); + CHECK(enc->layout[0].length == 96000); + CHECK(enc->layout[1].offset == enc->prefix.size() + 96000); + CHECK(enc->layout[1].length == 0); + CHECK(enc->totalSize == enc->prefix.size() + 96000); + + // decodePackage(encodePackage(x)) == x — payloads are never read by the + // codec, so the prefix plus the true total size is the whole input. + const DecodedPackage dec = decodePackage(enc->prefix, enc->totalSize); + CHECK(dec.status == PackageReadability::Readable); + CHECK(dec.manifest == m); + CHECK(dec.layout == enc->layout); + CHECK(dec.prefixSize == enc->prefix.size()); + + // The header stamps this build's ladder pair and its informational semver. + CHECK(dec.header.formatVersion == kPackageFormatVersion); + CHECK(dec.header.minReaderVersion == kPackageMinReaderVersion); + CHECK(dec.header.writerVersion == version::stampVersion()); +} + +static void testEncodeRefusesWhatManifestRefuses() { + PackageManifest m = fixture(1, 1); + m.entries[0].fileName = "../evil.wav"; + CHECK(!encodePackage(m).has_value()); +} + +// --- truncation: every byte offset ------------------------------------------- + +static void testTruncationAtEveryByteOffsetIsMalformed() { + const PackageManifest m = fixture(3, 5); + auto enc = encodePackage(m); + CHECK(enc.has_value()); + + // The complete on-disk image: prefix + both payloads. + std::vector file = enc->prefix; + for (std::uint8_t b : {1, 2, 3, 10, 20, 30, 40, 50}) file.push_back(b); + CHECK(file.size() == enc->totalSize); + CHECK(decodePackage(file, file.size()).status == PackageReadability::Readable); + + for (std::size_t cut = 0; cut < file.size(); ++cut) { + const std::vector truncated(file.begin(), file.begin() + cut); + const DecodedPackage dec = decodePackage(truncated, truncated.size()); + if (dec.status != PackageReadability::Malformed) { + std::printf("FAIL: truncation at %zu not Malformed\n", cut); + ++g_fail; + break; + } + // A refused decode must not half-succeed at any cut either. + CHECK(dec.manifest.entries.empty()); + CHECK(dec.layout.empty()); + } + + // One byte extra (trailing garbage) is as Malformed as one byte missing. + std::vector extended = file; + extended.push_back(0); + CHECK(decodePackage(extended, extended.size()).status == PackageReadability::Malformed); + // And a file size that disagrees with the same bytes. + CHECK(decodePackage(file, file.size() + 1).status == PackageReadability::Malformed); + CHECK(decodePackage(file, file.size() - 1).status == PackageReadability::Malformed); +} + +// --- version ladder: TooNew refuses whole ------------------------------------ + +static void testTooNewProducesNoManifest() { + // A future structural format: only the frozen region is trustworthy, so the + // tail is deliberate garbage that would crash a parser that kept reading. + std::vector bytes = rawHeader(9, 9, "9.9.9"); + for (int i = 0; i < 32; ++i) bytes.push_back(0xFF); + + const DecodedPackage dec = decodePackage(bytes, bytes.size()); + CHECK(dec.status == PackageReadability::TooNew); + // The refusal message's three facts survive... + CHECK(dec.header.formatVersion == 9); + CHECK(dec.header.minReaderVersion == 9); + CHECK(dec.header.writerVersion == "9.9.9"); + // ...and nothing else is produced: no manifest, no layout, no half-success. + CHECK(dec.manifest.entries.empty()); + CHECK(dec.manifest == PackageManifest{}); + CHECK(dec.layout.empty()); + CHECK(dec.prefixSize == 0); + + // Boundary: minReader exactly one past this build. + auto boundary = rawHeader(kPackageFormatVersion + 1, kPackageFormatVersion + 1, "2.0.0"); + CHECK(decodePackage(boundary, boundary.size()).status == PackageReadability::TooNew); + + // A TooNew header truncated inside the frozen region cannot name the + // writer, so it is Malformed, not an unactionable refusal. + std::vector cut = rawHeader(9, 9, "9.9.9"); + cut.resize(18); // mid-semver + CHECK(decodePackage(cut, cut.size()).status == PackageReadability::Malformed); +} + +// --- version ladder: additive forward compatibility -------------------------- + +// The reason two integers exist: a NEWER formatVersion whose minReaderVersion +// still reaches back to this build must read, with its unknown keys skipped. +static void testNewerAdditiveFormatReads() { + const std::string manifest = handManifest( + "a.wav", 4, + "\"instrumentState\":{\"future\":[1,2,3]},\"anotherNewKey\":\"x\","); + std::vector bytes = + rawHeader(kPackageFormatVersion + 1, kPackageMinReaderVersion, "1.9.0"); + appendManifest(bytes, manifest); + const std::uint64_t total = bytes.size() + 4; // the one entry's payload + + const DecodedPackage dec = decodePackage(bytes, total); + CHECK(dec.status == PackageReadability::Readable); + CHECK(dec.header.formatVersion == kPackageFormatVersion + 1); + CHECK(dec.header.writerVersion == "1.9.0"); + CHECK(dec.manifest.entries.size() == 1); + CHECK(dec.manifest.entries[0].fileName == "a.wav"); + CHECK(dec.manifest.entries[0].sample.id == "s1"); + CHECK(dec.layout.size() == 1); + CHECK(dec.layout[0].offset == bytes.size()); + CHECK(dec.layout[0].length == 4); +} + +// --- hostile headers --------------------------------------------------------- + +static void testHostileHeadersAreMalformed() { + // Wrong magic. + std::vector bad = rawHeader(1, 1, "1.0.0"); + appendManifest(bad, "{}"); + bad[0] = 'Z'; + CHECK(decodePackage(bad, bad.size()).status == PackageReadability::Malformed); + + // Incoherent version pairs (the classify rules, proven through the framing). + for (auto [fv, mv] : {std::pair{0, 0}, {0, 1}, + {1, 0}, {1, 2}}) { + std::vector b = rawHeader(fv, mv, "1.0.0"); + appendManifest(b, "{}"); + CHECK(decodePackage(b, b.size()).status == PackageReadability::Malformed); + } + + // A forged semver length over the cap must not read past the buffer. + std::vector overSemver; + overSemver.insert(overSemver.end(), kPackageMagic, kPackageMagic + 4); + wire::putLE(overSemver, std::uint32_t{1}); + wire::putLE(overSemver, std::uint32_t{1}); + wire::putLE(overSemver, kMaxWriterVersionBytes + 1); + CHECK(decodePackage(overSemver, overSemver.size()).status == + PackageReadability::Malformed); + + // A forged manifest length over the cap: refused before any allocation. + std::vector overManifest = rawHeader(1, 1, "1.0.0"); + wire::putLE(overManifest, kMaxManifestBytes + 1); + CHECK(decodePackage(overManifest, overManifest.size()).status == + PackageReadability::Malformed); + + // A manifest whose entry name expresses a path: rejected on decode even + // though no current encoder would write it. + std::vector traversal = rawHeader(1, 1, "1.0.0"); + appendManifest(traversal, handManifest("../evil.wav", 4, "")); + CHECK(decodePackage(traversal, traversal.size() + 4).status == + PackageReadability::Malformed); +} + +// --- requiredPrefixSize ------------------------------------------------------ + +static void testRequiredPrefixSizeGrowsToTheFullPrefix() { + auto enc = encodePackage(fixture(3, 5)); + CHECK(enc.has_value()); + const auto& prefix = enc->prefix; + + // Empty: the fixed region first. + CHECK(requiredPrefixSize({}) == std::uint64_t{16}); + + // With the fixed region: asks through the semver + manifest-length field. + const std::string& semver = version::stampVersion(); + std::vector first16(prefix.begin(), prefix.begin() + 16); + CHECK(requiredPrefixSize(first16) == std::uint64_t{16 + semver.size() + 4}); + + // With that much: the full prefix size. And the answer is a fixpoint. + std::vector upToManifestLen( + prefix.begin(), prefix.begin() + 20 + static_cast(semver.size())); + CHECK(requiredPrefixSize(upToManifestLen) == std::uint64_t{prefix.size()}); + CHECK(requiredPrefixSize(prefix) == std::uint64_t{prefix.size()}); + + // The returned count is exactly enough for decodePackage. + CHECK(decodePackage(prefix, enc->totalSize).status == PackageReadability::Readable); +} + +static void testRequiredPrefixSizeRefusals() { + // Bad magic: stop reading. + std::vector bad(16, 0); + CHECK(!requiredPrefixSize(bad).has_value()); + + // Incoherent versions: stop reading. + std::vector zeroed = rawHeader(0, 0, "1.0.0"); + CHECK(!requiredPrefixSize(zeroed).has_value()); + + // TooNew: asks only through the frozen region — the manifest-length field + // belongs to the newer format and is never trusted. + std::vector tooNew = rawHeader(9, 9, "9.9.9"); + for (int i = 0; i < 8; ++i) tooNew.push_back(0xFF); // garbage where M would be + CHECK(requiredPrefixSize(tooNew) == std::uint64_t{16 + 5}); + + // Oversize length fields: stop reading. + std::vector overSemver; + overSemver.insert(overSemver.end(), kPackageMagic, kPackageMagic + 4); + wire::putLE(overSemver, std::uint32_t{1}); + wire::putLE(overSemver, std::uint32_t{1}); + wire::putLE(overSemver, kMaxWriterVersionBytes + 1); + CHECK(!requiredPrefixSize(overSemver).has_value()); + + std::vector overManifest = rawHeader(1, 1, "1.0.0"); + wire::putLE(overManifest, kMaxManifestBytes + 1); + CHECK(!requiredPrefixSize(overManifest).has_value()); +} + +int main() { + testEncodeDecodeRoundTrip(); + testEncodeRefusesWhatManifestRefuses(); + testTruncationAtEveryByteOffsetIsMalformed(); + testTooNewProducesNoManifest(); + testNewerAdditiveFormatReads(); + testHostileHeadersAreMalformed(); + testRequiredPrefixSizeRefusals(); + testRequiredPrefixSizeGrowsToTheFullPrefix(); + + if (g_fail == 0) { + std::printf("bank_package_tests: all passed\n"); + return 0; + } + std::printf("bank_package_tests: %d failure(s)\n", g_fail); + return 1; +} diff --git a/tests/test_package_format.cpp b/tests/test_package_format.cpp new file mode 100644 index 0000000..442d2ef --- /dev/null +++ b/tests/test_package_format.cpp @@ -0,0 +1,96 @@ +// Standalone tests for reasampler::package's format contract — no REAPER, no +// test framework. Pins the version-ladder classification (both integers, every +// branch) and the entry-name rule that makes path expression structurally +// impossible in a package. + +#include "../src/core/package/package_format.h" + +#include +#include + +using namespace reasampler::package; + +static int g_fail = 0; +#define CHECK(cond) do { if(!(cond)) { \ + std::printf("FAIL line %d: %s\n", __LINE__, #cond); ++g_fail; } } while(0) + +// --- classifyPackageVersion -------------------------------------------------- + +static void testClassifyReadable() { + CHECK(classifyPackageVersion(kPackageFormatVersion, kPackageMinReaderVersion) == + PackageReadability::Readable); + // The additive-forward-compat direction: a newer writer whose minReader + // still reaches back to this build reads fine. + CHECK(classifyPackageVersion(kPackageFormatVersion + 5, kPackageMinReaderVersion) == + PackageReadability::Readable); + // Boundary: minReader exactly this build's format version. + CHECK(classifyPackageVersion(kPackageFormatVersion + 1, kPackageFormatVersion) == + PackageReadability::Readable); +} + +static void testClassifyTooNew() { + // Boundary: one past this build's format version refuses. + CHECK(classifyPackageVersion(kPackageFormatVersion + 1, kPackageFormatVersion + 1) == + PackageReadability::TooNew); + CHECK(classifyPackageVersion(99, 42) == PackageReadability::TooNew); +} + +static void testClassifyMalformed() { + // Zero versions: no honest writer emits them (the ladder starts at 1). + CHECK(classifyPackageVersion(0, 0) == PackageReadability::Malformed); + CHECK(classifyPackageVersion(1, 0) == PackageReadability::Malformed); + CHECK(classifyPackageVersion(0, 1) == PackageReadability::Malformed); + // A writer cannot require a reader newer than what it wrote. + CHECK(classifyPackageVersion(1, 2) == PackageReadability::Malformed); + // Incoherence outranks TooNew: even with both above this build, minReader > + // formatVersion is Malformed, not a refusal message. + CHECK(classifyPackageVersion(5, 9) == PackageReadability::Malformed); +} + +// --- isValidEntryName -------------------------------------------------------- + +static void testEntryNameAccepts() { + CHECK(isValidEntryName("kick.wav")); + CHECK(isValidEntryName("Snare 03 (wet).wav")); + CHECK(isValidEntryName("no-extension")); + CHECK(isValidEntryName(".hidden")); // a leading dot is a bare name + CHECK(isValidEntryName("a.b.c.wav")); // single dots are fine + CHECK(isValidEntryName(std::string(kMaxEntryNameBytes, 'x'))); // at the cap +} + +static void testEntryNameRejectsSeparatorsAndDots() { + CHECK(!isValidEntryName("")); + CHECK(!isValidEntryName(".")); + CHECK(!isValidEntryName("..")); + CHECK(!isValidEntryName("..\\evil.wav")); + CHECK(!isValidEntryName("../evil.wav")); + CHECK(!isValidEntryName("a..b.wav")); // any ".." occurrence rejects + CHECK(!isValidEntryName("dir/inner.wav")); + CHECK(!isValidEntryName("dir\\inner.wav")); + CHECK(!isValidEntryName("/rooted.wav")); + CHECK(!isValidEntryName("\\rooted.wav")); +} + +static void testEntryNameRejectsAbsolutePrefixes() { + CHECK(!isValidEntryName("C:\\abs.wav")); + CHECK(!isValidEntryName("C:/abs.wav")); + CHECK(!isValidEntryName("c:relative-to-drive.wav")); // ':' bans drive forms + CHECK(!isValidEntryName("\\\\server\\share.wav")); // UNC + CHECK(!isValidEntryName(std::string(kMaxEntryNameBytes + 1, 'x'))); // over cap +} + +int main() { + testClassifyReadable(); + testClassifyTooNew(); + testClassifyMalformed(); + testEntryNameAccepts(); + testEntryNameRejectsSeparatorsAndDots(); + testEntryNameRejectsAbsolutePrefixes(); + + if (g_fail == 0) { + std::printf("package_format_tests: all passed\n"); + return 0; + } + std::printf("package_format_tests: %d failure(s)\n", g_fail); + return 1; +} diff --git a/tests/test_package_manifest.cpp b/tests/test_package_manifest.cpp new file mode 100644 index 0000000..c9ecbde --- /dev/null +++ b/tests/test_package_manifest.cpp @@ -0,0 +1,248 @@ +// Standalone tests for reasampler::package's manifest codec — no REAPER, no +// test framework. The round-trip fixture exercises every manifest field and +// every Sample optional in both present and absent states; the rejection suite +// pins the entry-name rule on encode AND decode. + +#include "../src/core/package/package_manifest.h" + +#include +#include +#include + +using namespace reasampler::package; +using namespace reasampler::model; + +static int g_fail = 0; +#define CHECK(cond) do { if(!(cond)) { \ + std::printf("FAIL line %d: %s\n", __LINE__, #cond); ++g_fail; } } while(0) + +// --- fixtures ---------------------------------------------------------------- + +// Every Sample field populated, every optional PRESENT. +static Sample fullSample() { + Sample s; + s.id = "smp-full"; + s.displayName = "Kick (wet)"; + s.relativePath = "reasampler_bank/kick.wav"; + s.sourceMode = SourceMode::RazorArea; + s.sourceRange = {1.25, 3.5, 480.0, 1920.0}; + s.trackGuids = {"{AAA}", "{BBB}"}; + s.wetDry = 0.75; + s.channelCount = 2; + s.sampleRate = 48000; + s.lengthSeconds = 2.25; + s.lengthBeats = 4.5; + s.captureTempo = 120.5; + s.captureTimeSigNum = 7; + s.captureTimeSigDenom = 8; + s.key = "F#m"; + s.rootNote = 60; + s.loop = LoopPoints{100, 4800}; + s.levels = {-0.3, -12.7, -14.0}; + s.clipped = true; + s.tier = Tier::Archive; + s.contentHash = "W0123456789abcdef"; + s.provenance = Provenance{"smp-parent", "fx-snapshot"}; + s.createdTimestamp = 1754000000; + return s; +} + +// Every Sample optional ABSENT (key, rootNote, loop, provenance). +static Sample bareSample() { + Sample s; + s.id = "smp-bare"; + s.displayName = "Snare"; + s.relativePath = "reasampler_bank/snare.wav"; + s.sourceMode = SourceMode::Realtime; + s.contentHash = "Wfedcba9876543210"; + s.createdTimestamp = 1754000001; + return s; +} + +static PackageManifest fixture() { + PackageManifest m; + m.bankDisplayName = "Drums \"live\""; // escaping exercised + m.exportTimestamp = 1754100000; + m.entries.push_back({"kick.wav", 96000, "1111222233334444", fullSample()}); + m.entries.push_back({"snare.wav", 0, "5555666677778888", bareSample()}); // 0-length legal + m.slots.append("smp-full"); + m.slots.append("smp-bare"); + m.slots.remove("smp-full"); // leaves a gap: slots round-trip must keep it + return m; +} + +// --- round trip -------------------------------------------------------------- + +static void testRoundTripEveryField() { + const PackageManifest m = fixture(); + auto json = serializeManifest(m); + CHECK(json.has_value()); + auto back = deserializeManifest(*json); + CHECK(back.has_value()); + CHECK(*back == m); + // Spot-check both optional states survived (== above proves it; these name + // the claim so a failure reads directly). + CHECK(back->entries[0].sample.loop.has_value()); + CHECK(back->entries[0].sample.provenance.has_value()); + CHECK(!back->entries[1].sample.key.has_value()); + CHECK(!back->entries[1].sample.rootNote.has_value()); + CHECK(back->slots.idAt(0).empty()); // the slot gap survived + CHECK(back->slots.slotOf("smp-bare") == 1); +} + +static void testEmptyManifestRoundTrips() { + PackageManifest m; + auto json = serializeManifest(m); + CHECK(json.has_value()); + auto back = deserializeManifest(*json); + CHECK(back.has_value()); + CHECK(*back == m); +} + +// --- forward compatibility --------------------------------------------------- + +static void testUnknownKeysSkippedAtEveryLevel() { + // A future additive manifest: unknown keys at the root and inside an entry. + const std::string json = + "{\"bankName\":\"B\",\"exported\":7," + "\"instrumentState\":{\"nested\":[1,2,{\"x\":\"y\"}]}," + "\"entries\":[{\"name\":\"a.wav\",\"length\":10,\"hash\":\"h\"," + "\"futureField\":\"ignored\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s1\"," + "\"relativePath\":\"bank/a.wav\"}]}}]," + "\"slots\":[],\"trailingUnknown\":null}"; + auto m = deserializeManifest(json); + CHECK(m.has_value()); + CHECK(m->bankDisplayName == "B"); + CHECK(m->exportTimestamp == 7); + CHECK(m->entries.size() == 1); + CHECK(m->entries[0].fileName == "a.wav"); + CHECK(m->entries[0].byteLength == 10); + CHECK(m->entries[0].sample.id == "s1"); +} + +// --- rejection: entry names, both directions --------------------------------- + +static void testEncodeRejectsBadEntryName() { + for (const char* bad : {"..\\evil.wav", "../evil.wav", "dir/a.wav", "C:\\a.wav", "", ".."}) { + PackageManifest m = fixture(); + m.entries[0].fileName = bad; + CHECK(!serializeManifest(m).has_value()); + } +} + +static void testDecodeRejectsBadEntryName() { + for (const char* bad : {"..\\\\evil.wav", "../evil.wav", "dir/a.wav", "C:\\\\a.wav", ".."}) { + // Hand-rolled JSON: a hostile package is not limited to what encode emits. + std::string json = + std::string("{\"entries\":[{\"name\":\"") + bad + + "\",\"length\":1,\"hash\":\"h\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s1\"," + "\"relativePath\":\"bank/a.wav\"}]}}]}"; + CHECK(!deserializeManifest(json).has_value()); + } +} + +static void testDuplicateEntryNamesRejectedBothWays() { + PackageManifest m = fixture(); + m.entries[1].fileName = m.entries[0].fileName; + CHECK(!serializeManifest(m).has_value()); + + // Decode side, from a hand-built duplicate (a hostile package is not + // limited to what encode emits). + const std::string dup = + "{\"entries\":[" + "{\"name\":\"a.wav\",\"length\":1,\"hash\":\"h\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s1\",\"relativePath\":\"p\"}]}}," + "{\"name\":\"a.wav\",\"length\":2,\"hash\":\"i\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s2\",\"relativePath\":\"q\"}]}}]}"; + CHECK(!deserializeManifest(dup).has_value()); +} + +// --- rejection: structural --------------------------------------------------- + +static void testEncodeRejectsUnrepresentableSample() { + PackageManifest m = fixture(); + m.entries[0].sample.id.clear(); // BankModel::add rejects an empty id + CHECK(!serializeManifest(m).has_value()); + + PackageManifest m2 = fixture(); + m2.entries[0].sample.relativePath = "C:/abs/kick.wav"; // and an absolute path + CHECK(!serializeManifest(m2).has_value()); +} + +static void testDecodeRejectsMalformedShapes() { + CHECK(!deserializeManifest("").has_value()); + CHECK(!deserializeManifest("not json").has_value()); + CHECK(!deserializeManifest("[]").has_value()); + CHECK(!deserializeManifest("{\"entries\":[{}]}").has_value()); // entry missing fields + // Missing one required entry field apiece. + CHECK(!deserializeManifest( + "{\"entries\":[{\"length\":1,\"hash\":\"h\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s\",\"relativePath\":\"p\"}]}}]}") + .has_value()); + CHECK(!deserializeManifest( + "{\"entries\":[{\"name\":\"a.wav\",\"hash\":\"h\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s\",\"relativePath\":\"p\"}]}}]}") + .has_value()); + CHECK(!deserializeManifest( + "{\"entries\":[{\"name\":\"a.wav\",\"length\":1," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s\",\"relativePath\":\"p\"}]}}]}") + .has_value()); + CHECK(!deserializeManifest( + "{\"entries\":[{\"name\":\"a.wav\",\"length\":1,\"hash\":\"h\"}]}").has_value()); + // Negative length. + CHECK(!deserializeManifest( + "{\"entries\":[{\"name\":\"a.wav\",\"length\":-1,\"hash\":\"h\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s\",\"relativePath\":\"p\"}]}}]}") + .has_value()); + // A nested index that is not exactly one sample (zero and two). + CHECK(!deserializeManifest( + "{\"entries\":[{\"name\":\"a.wav\",\"length\":1,\"hash\":\"h\"," + "\"index\":{\"version\":1,\"samples\":[]}}]}").has_value()); + CHECK(!deserializeManifest( + "{\"entries\":[{\"name\":\"a.wav\",\"length\":1,\"hash\":\"h\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s1\",\"relativePath\":\"p\"}," + "{\"id\":\"s2\",\"relativePath\":\"q\"}]}}]}").has_value()); + // A nested sample BankModel::add drops (absolute path) fails the entry — + // the silent drop must not half-parse into an empty index. + CHECK(!deserializeManifest( + "{\"entries\":[{\"name\":\"a.wav\",\"length\":1,\"hash\":\"h\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s\"," + "\"relativePath\":\"C:/abs.wav\"}]}}]}").has_value()); + // An out-of-range enum inside the nested Sample blob fails the entry: the + // manifest defines no enum of its own, and bank_model's codec REJECTS an + // unknown sourceMode/tier rather than degrading — so growing one of those + // vocabularies is a minReaderVersion bump, not an additive change (see this + // directory's CLAUDE.md). + CHECK(!deserializeManifest( + "{\"entries\":[{\"name\":\"a.wav\",\"length\":1,\"hash\":\"h\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s\",\"relativePath\":\"p\"," + "\"sourceMode\":99}]}}]}").has_value()); + // Trailing garbage after the root object. + auto json = serializeManifest(fixture()); + CHECK(json.has_value()); + CHECK(!deserializeManifest(*json + "x").has_value()); + // Truncation at a few JSON-level offsets (byte-level truncation of the whole + // package is bank_package's suite). + CHECK(!deserializeManifest(json->substr(0, json->size() / 2)).has_value()); + CHECK(!deserializeManifest(json->substr(0, 1)).has_value()); +} + +int main() { + testRoundTripEveryField(); + testEmptyManifestRoundTrips(); + testUnknownKeysSkippedAtEveryLevel(); + testEncodeRejectsBadEntryName(); + testDecodeRejectsBadEntryName(); + testDuplicateEntryNamesRejectedBothWays(); + testEncodeRejectsUnrepresentableSample(); + testDecodeRejectsMalformedShapes(); + + if (g_fail == 0) { + std::printf("package_manifest_tests: all passed\n"); + return 0; + } + std::printf("package_manifest_tests: %d failure(s)\n", g_fail); + return 1; +} From e0b4ec2e21256b55b25439e31b24bae801f804cd Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 07:56:45 -0400 Subject: [PATCH 02/24] Tighten RSBK package-format validation for review remediation Reject NUL/control bytes and Windows-hostile names in entry names, relax the over-broad ".." substring ban to component-only, close the trailing-garbage gap on empty manifests, and relocate the package CMake subdirectory to its ladder home. --- CMakeLists.txt | 1 - src/core/CMakeLists.txt | 1 + src/core/package/CLAUDE.md | 14 ++++-- src/core/package/bank_package.h | 5 +-- src/core/package/package_format.cpp | 35 +++++++++++++-- src/core/package/package_format.h | 22 ++++++--- src/core/package/package_manifest.cpp | 64 +++++++++++++-------------- tests/test_bank_package.cpp | 15 ++++--- tests/test_package_format.cpp | 39 +++++++++++++++- tests/test_package_manifest.cpp | 21 ++++++++- 10 files changed, 161 insertions(+), 56 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index ed9f193..f783871 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -92,4 +92,3 @@ enable_testing() add_subdirectory(src/core) add_subdirectory(src/app) add_subdirectory(src/shell/instrument) -add_subdirectory(src/core/package) diff --git a/src/core/CMakeLists.txt b/src/core/CMakeLists.txt index de07360..0eeee11 100644 --- a/src/core/CMakeLists.txt +++ b/src/core/CMakeLists.txt @@ -13,6 +13,7 @@ add_subdirectory(capture) add_subdirectory(tracking) add_subdirectory(reclaim) add_subdirectory(version) +add_subdirectory(package) add_subdirectory(view) add_subdirectory(ui) add_subdirectory(instrument) diff --git a/src/core/package/CLAUDE.md b/src/core/package/CLAUDE.md index cb56e08..a8bfe62 100644 --- a/src/core/package/CLAUDE.md +++ b/src/core/package/CLAUDE.md @@ -26,9 +26,12 @@ landing after the format. else — no manifest, no layout, no half-success. The fields through the writer semver are FROZEN for all future versions to keep that refusal producible. - **Path expression is structurally impossible.** Entry names are bare file - names (`isValidEntryName`: no separators, no `..`, no drive/UNC/rooted form), - enforced on encode AND decode because a package can arrive from anywhere. - There is no field in the format capable of expressing a path. + names (`isValidEntryName`: no separators, no `..` component, no + drive/UNC/rooted form, no control bytes, no Windows-reserved character, no + trailing dot/space, no DOS device name), enforced on encode AND decode + because a package can arrive from anywhere. There is no field in the format + capable of expressing a path. The rule set is the full authority; see + `package_format.h`'s doc comment for the itemized list. - **Framing only, never a payload.** `bank_package` produces header bytes and an ordered `{name, offset, length}` layout; it never holds, copies, or hashes an entry's audio. `decodePackage` proves prefix + payload lengths equal the @@ -76,3 +79,8 @@ landing after the format. version pair classifies `Readable`; for `TooNew` it stops at the semver — don't "fix" it to read the manifest length there, a future structural format may have moved it. +- The format carries no algorithm tag for `byteHash` — it is FNV-1a + (`capture::hashBytes`) implicitly. Changing the digest algorithm is a + `minReaderVersion` bump, not additive: an old reader would otherwise compare + a stored digest against bytes hashed the new way and silently misjudge + corruption. diff --git a/src/core/package/bank_package.h b/src/core/package/bank_package.h index 5fb0e94..677b567 100644 --- a/src/core/package/bank_package.h +++ b/src/core/package/bank_package.h @@ -1,8 +1,7 @@ #pragma once // bank_package — RSBK framing and layout arithmetic: header encode, prefix -// decode, and the ordered {name, offset, length} entry layout. Framing only, -// never a payload: this module never holds, copies, or hashes an entry's audio -// — the shell streams payloads one at a time against the layout produced here. +// decode, and the ordered {name, offset, length} entry layout. See this +// directory's CLAUDE.md for the framing-only invariant. Pure: no filesystem. #include #include diff --git a/src/core/package/package_format.cpp b/src/core/package/package_format.cpp index 139163b..d07aaec 100644 --- a/src/core/package/package_format.cpp +++ b/src/core/package/package_format.cpp @@ -1,5 +1,7 @@ #include "core/package/package_format.h" +#include + namespace reasampler::package { PackageReadability classifyPackageVersion(std::uint32_t formatVersion, @@ -10,13 +12,40 @@ PackageReadability classifyPackageVersion(std::uint32_t formatVersion, return PackageReadability::Readable; } +namespace { + +// Windows device names claim the whole entry regardless of extension +// (CON, CON.wav, con.WAV are all the same reserved device) — checked against +// the portion before the first dot only. +bool isDosDeviceName(const std::string& name) { + std::string base = name.substr(0, name.find('.')); + for (char& c : base) c = static_cast(std::toupper(static_cast(c))); + static const std::string kReserved[] = { + "CON", "PRN", "AUX", "NUL", + "COM1", "COM2", "COM3", "COM4", "COM5", "COM6", "COM7", "COM8", "COM9", + "LPT1", "LPT2", "LPT3", "LPT4", "LPT5", "LPT6", "LPT7", "LPT8", "LPT9", + }; + for (const auto& r : kReserved) if (base == r) return true; + return false; +} + +} // namespace + bool isValidEntryName(const std::string& name) { if (name.empty() || name.size() > kMaxEntryNameBytes) return false; - if (name == ".") return false; - if (name.find("..") != std::string::npos) return false; - for (char c : name) { + if (name == "." || name == "..") return false; + // fopen/CreateFileW both silently strip a trailing dot or space at + // creation, so "a.wav " and "a.wav" would collide on one file. + if (name.back() == '.' || name.back() == ' ') return false; + for (unsigned char c : name) { + // NUL and other control bytes truncate at the first filesystem call + // (std::ofstream, fopen, CreateFileW off .c_str()) — two names that + // differ only after the NUL land on the same file. + if (c < 0x20 || c == 0x7F) return false; if (c == '/' || c == '\\' || c == ':') return false; + if (c == '*' || c == '?' || c == '|' || c == '<' || c == '>' || c == '"') return false; } + if (isDosDeviceName(name)) return false; return true; } diff --git a/src/core/package/package_format.h b/src/core/package/package_format.h index acb70f3..53b9538 100644 --- a/src/core/package/package_format.h +++ b/src/core/package/package_format.h @@ -57,17 +57,27 @@ PackageReadability classifyPackageVersion(std::uint32_t formatVersion, std::uint32_t minReaderVersion); // The entry-name rule that makes path expression structurally impossible: a -// bare file name only. Rejects empty, ".", any ".." occurrence, any '/', '\\' -// or ':' (which also bans every absolute form — drive, UNC, rooted), and names -// over kMaxEntryNameBytes. Enforced on encode AND decode by package_manifest. +// bare file name only. Rejects empty, ".", the exact ".." component (a name +// can only ever be one component, since separators are banned below — a +// substring scan would over-reject legal names like "take..final.wav"), any +// control byte (NUL included — truncates at the first filesystem call and +// collides two distinct manifest entries onto one file) or 0x7F, any '/', +// '\\' or ':' (which also bans every absolute form — drive, UNC, rooted), any +// Windows-reserved character (`*?|<>"`), a trailing dot or space (silently +// stripped at file creation, so "a.wav " and "a.wav" would collide), a DOS +// device name (CON/PRN/AUX/NUL/COM1-9/LPT1-9, case-insensitive, with or +// without an extension), and names over kMaxEntryNameBytes. Enforced on +// encode AND decode by package_manifest. bool isValidEntryName(const std::string& name); // The fixed header, informational semver included. writerVersion is // version::stampVersion() on the write side — it exists so a TooNew refusal can -// tell the user which build to install; it never gates. +// tell the user which build to install; it never gates. Defaults are 0/0, NOT +// the current ladder pair — a caller reading `header` after a Malformed +// verdict must see an obviously-unset value, not a plausible-looking 1/1. struct PackageHeader { - std::uint32_t formatVersion = kPackageFormatVersion; - std::uint32_t minReaderVersion = kPackageMinReaderVersion; + std::uint32_t formatVersion = 0; + std::uint32_t minReaderVersion = 0; std::string writerVersion; bool operator==(const PackageHeader& o) const { diff --git a/src/core/package/package_manifest.cpp b/src/core/package/package_manifest.cpp index 10cc267..a4d4ba9 100644 --- a/src/core/package/package_manifest.cpp +++ b/src/core/package/package_manifest.cpp @@ -5,11 +5,6 @@ #include "core/json/json.h" #include "core/package/package_format.h" -// Duplicate entry names are rejected in both directions: two payloads landing -// on one destination name is incoherent, and on import it would be a silent -// overwrite. Sample-id rules (remap, collision) are deliberately NOT enforced -// here — they are the import plan's decisions, not the codec's. - namespace reasampler::package { namespace { @@ -17,6 +12,8 @@ namespace { using json::numToStr; using ObjWriter = json::Writer; +// Shared by serializeManifest and deserializeManifest — see this directory's +// CLAUDE.md for why duplicate names are rejected both ways. bool duplicateName(const std::vector& entries) { for (std::size_t i = 0; i < entries.size(); ++i) for (std::size_t j = i + 1; j < entries.size(); ++j) @@ -155,37 +152,40 @@ bool parseEntry(json::Reader& r, PackageEntry& e) { bool parseManifest(json::Reader& r, PackageManifest& m) { if (!r.consume('{')) return false; r.skipWs(); - if (r.consume('}')) return true; // empty object: a valid empty manifest + bool haveEntries = false; + if (!r.consume('}')) { // not the empty-object shortcut: parse the members + do { + std::string key; + if (!r.parseKey(key)) return false; - do { - std::string key; - if (!r.parseKey(key)) return false; - - if (key == "bankName") { - if (!r.parseString(m.bankDisplayName)) return false; - } else if (key == "exported") { - if (!r.parseInt64(m.exportTimestamp)) return false; - } else if (key == "entries") { - if (!r.consume('[')) return false; - r.skipWs(); - if (!r.consume(']')) { - do { - PackageEntry e; - if (!parseEntry(r, e)) return false; - m.entries.push_back(std::move(e)); - } while (r.consume(',')); - if (!r.consume(']')) return false; + if (key == "bankName") { + if (!r.parseString(m.bankDisplayName)) return false; + } else if (key == "exported") { + if (!r.parseInt64(m.exportTimestamp)) return false; + } else if (key == "entries") { + if (haveEntries) return false; // a repeated key must not accumulate + haveEntries = true; + if (!r.consume('[')) return false; + r.skipWs(); + if (!r.consume(']')) { + do { + PackageEntry e; + if (!parseEntry(r, e)) return false; + m.entries.push_back(std::move(e)); + } while (r.consume(',')); + if (!r.consume(']')) return false; + } + } else if (key == "slots") { + if (!parseSlots(r, m.slots)) return false; + } else { + if (!r.skipValue()) return false; // forward-compat unknown keys } - } else if (key == "slots") { - if (!parseSlots(r, m.slots)) return false; - } else { - if (!r.skipValue()) return false; // forward-compat unknown keys - } - } while (r.consume(',')); + } while (r.consume(',')); - if (!r.consume('}')) return false; + if (!r.consume('}')) return false; + } r.skipWs(); - if (!r.eof()) return false; // trailing garbage + if (!r.eof()) return false; // trailing garbage — even after an empty object return !duplicateName(m.entries); } diff --git a/tests/test_bank_package.cpp b/tests/test_bank_package.cpp index 5279bd7..565ab7c 100644 --- a/tests/test_bank_package.cpp +++ b/tests/test_bank_package.cpp @@ -95,13 +95,15 @@ static void appendManifest(std::vector& out, const std::string& js } // One-entry manifest JSON with `extra` spliced in as additional root content -// ("" for none) — for images a current writer would never emit. +// ("" for none) and `sampleExtra` spliced into the nested Sample object — for +// images a current writer would never emit. static std::string handManifest(const std::string& name, int length, - const std::string& extra) { + const std::string& extra, + const std::string& sampleExtra = "") { return std::string("{") + extra + "\"entries\":[{\"name\":\"" + name + "\",\"length\":" + std::to_string(length) + ",\"hash\":\"h\"," - "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s1\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s1\"," + sampleExtra + "\"relativePath\":\"bank/a.wav\"}]}}]}"; } @@ -210,11 +212,14 @@ static void testTooNewProducesNoManifest() { // --- version ladder: additive forward compatibility -------------------------- // The reason two integers exist: a NEWER formatVersion whose minReaderVersion -// still reaches back to this build must read, with its unknown keys skipped. +// still reaches back to this build must read, with its unknown keys skipped — +// at the manifest level AND inside the nested Sample blob, the actual +// motivating case for the two-integer ladder (see this directory's CLAUDE.md). static void testNewerAdditiveFormatReads() { const std::string manifest = handManifest( "a.wav", 4, - "\"instrumentState\":{\"future\":[1,2,3]},\"anotherNewKey\":\"x\","); + "\"instrumentState\":{\"future\":[1,2,3]},\"anotherNewKey\":\"x\",", + "\"someFutureSampleField\":42,"); std::vector bytes = rawHeader(kPackageFormatVersion + 1, kPackageMinReaderVersion, "1.9.0"); appendManifest(bytes, manifest); diff --git a/tests/test_package_format.cpp b/tests/test_package_format.cpp index 442d2ef..56d3cb8 100644 --- a/tests/test_package_format.cpp +++ b/tests/test_package_format.cpp @@ -56,6 +56,12 @@ static void testEntryNameAccepts() { CHECK(isValidEntryName(".hidden")); // a leading dot is a bare name CHECK(isValidEntryName("a.b.c.wav")); // single dots are fine CHECK(isValidEntryName(std::string(kMaxEntryNameBytes, 'x'))); // at the cap + // Legal names containing a ".." substring that is not the whole name: a + // name can only ever be one path component (separators are banned), so + // ".." as a component is the only expressible traversal. + CHECK(isValidEntryName("take..final.wav")); + CHECK(isValidEntryName("loop...wav")); + CHECK(isValidEntryName("a..b")); } static void testEntryNameRejectsSeparatorsAndDots() { @@ -64,11 +70,16 @@ static void testEntryNameRejectsSeparatorsAndDots() { CHECK(!isValidEntryName("..")); CHECK(!isValidEntryName("..\\evil.wav")); CHECK(!isValidEntryName("../evil.wav")); - CHECK(!isValidEntryName("a..b.wav")); // any ".." occurrence rejects CHECK(!isValidEntryName("dir/inner.wav")); CHECK(!isValidEntryName("dir\\inner.wav")); CHECK(!isValidEntryName("/rooted.wav")); CHECK(!isValidEntryName("\\rooted.wav")); + // Embedded NUL: every plausible filesystem call (ofstream, fopen, + // CreateFileW off .c_str()) truncates at it, so two names differing only + // after the NUL would collide on one file. + CHECK(!isValidEntryName(std::string("a\0b.wav", 7))); + // Other control bytes (newline here) are equally hostile to logs/UI. + CHECK(!isValidEntryName("a\nb.wav")); } static void testEntryNameRejectsAbsolutePrefixes() { @@ -79,6 +90,31 @@ static void testEntryNameRejectsAbsolutePrefixes() { CHECK(!isValidEntryName(std::string(kMaxEntryNameBytes + 1, 'x'))); // over cap } +static void testEntryNameRejectsWindowsHostileNames() { + // Reserved characters. + CHECK(!isValidEntryName("a*b.wav")); + CHECK(!isValidEntryName("a?b.wav")); + CHECK(!isValidEntryName("a|b.wav")); + CHECK(!isValidEntryName("a.wav")); + CHECK(!isValidEntryName("\"q\".wav")); + // Trailing dot or space (silently stripped at creation on Windows). + CHECK(!isValidEntryName("trailing ")); + CHECK(!isValidEntryName("trailing.")); + CHECK(!isValidEntryName(" ")); + CHECK(!isValidEntryName(" ")); + // DOS device names, case-insensitive, with and without an extension. + CHECK(!isValidEntryName("NUL")); + CHECK(!isValidEntryName("CON")); + CHECK(!isValidEntryName("con.wav")); + CHECK(!isValidEntryName("PRN")); + CHECK(!isValidEntryName("AUX")); + CHECK(!isValidEntryName("COM1")); + CHECK(!isValidEntryName("com1.txt")); + CHECK(!isValidEntryName("LPT1")); + // Not a device name: a real filename that merely starts with one. + CHECK(isValidEntryName("console.wav")); +} + int main() { testClassifyReadable(); testClassifyTooNew(); @@ -86,6 +122,7 @@ int main() { testEntryNameAccepts(); testEntryNameRejectsSeparatorsAndDots(); testEntryNameRejectsAbsolutePrefixes(); + testEntryNameRejectsWindowsHostileNames(); if (g_fail == 0) { std::printf("package_format_tests: all passed\n"); diff --git a/tests/test_package_manifest.cpp b/tests/test_package_manifest.cpp index c9ecbde..8d35475 100644 --- a/tests/test_package_manifest.cpp +++ b/tests/test_package_manifest.cpp @@ -102,13 +102,14 @@ static void testEmptyManifestRoundTrips() { // --- forward compatibility --------------------------------------------------- static void testUnknownKeysSkippedAtEveryLevel() { - // A future additive manifest: unknown keys at the root and inside an entry. + // A future additive manifest: unknown keys at the root, inside an entry, + // and inside the nested index blob itself. const std::string json = "{\"bankName\":\"B\",\"exported\":7," "\"instrumentState\":{\"nested\":[1,2,{\"x\":\"y\"}]}," "\"entries\":[{\"name\":\"a.wav\",\"length\":10,\"hash\":\"h\"," "\"futureField\":\"ignored\"," - "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s1\"," + "\"index\":{\"version\":1,\"futureIndexField\":42,\"samples\":[{\"id\":\"s1\"," "\"relativePath\":\"bank/a.wav\"}]}}]," "\"slots\":[],\"trailingUnknown\":null}"; auto m = deserializeManifest(json); @@ -159,6 +160,17 @@ static void testDuplicateEntryNamesRejectedBothWays() { CHECK(!deserializeManifest(dup).has_value()); } +static void testDuplicateEntriesKeyRejected() { + // A repeated "entries" key must not accumulate into two arrays' worth of + // entries — reject rather than silently union them. + const std::string json = + "{\"entries\":[{\"name\":\"a.wav\",\"length\":1,\"hash\":\"h\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s1\",\"relativePath\":\"p\"}]}}]," + "\"entries\":[{\"name\":\"b.wav\",\"length\":1,\"hash\":\"h\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s2\",\"relativePath\":\"q\"}]}}]}"; + CHECK(!deserializeManifest(json).has_value()); +} + // --- rejection: structural --------------------------------------------------- static void testEncodeRejectsUnrepresentableSample() { @@ -223,6 +235,10 @@ static void testDecodeRejectsMalformedShapes() { auto json = serializeManifest(fixture()); CHECK(json.has_value()); CHECK(!deserializeManifest(*json + "x").has_value()); + // Trailing garbage after the EMPTY-object shortcut specifically: this path + // returned early before reaching the eof check, so "{}JUNK" parsed valid. + CHECK(!deserializeManifest("{}JUNK").has_value()); + CHECK(deserializeManifest("{}").has_value()); // Truncation at a few JSON-level offsets (byte-level truncation of the whole // package is bank_package's suite). CHECK(!deserializeManifest(json->substr(0, json->size() / 2)).has_value()); @@ -236,6 +252,7 @@ int main() { testEncodeRejectsBadEntryName(); testDecodeRejectsBadEntryName(); testDuplicateEntryNamesRejectedBothWays(); + testDuplicateEntriesKeyRejected(); testEncodeRejectsUnrepresentableSample(); testDecodeRejectsMalformedShapes(); From 41a3016e634bd5fb1a954226dd3e743ae9c48bc5 Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 07:36:25 -0400 Subject: [PATCH 03/24] Land the package filesystem shell: streaming atomic package_io, journaled rollback carve-out, asymmetric platform pickers --- CMakeLists.txt | 1 + src/shell/package/CLAUDE.md | 61 +++++++ src/shell/package/CMakeLists.txt | 20 +++ src/shell/package/package_io.cpp | 170 ++++++++++++++++++ src/shell/package/package_io.h | 112 ++++++++++++ src/shell/package/package_pickers.cpp | 93 ++++++++++ src/shell/package/package_pickers.h | 21 +++ src/shell/package/package_rollback.cpp | 40 +++++ src/shell/package/package_rollback.h | 45 +++++ tests/test_package_io.cpp | 228 +++++++++++++++++++++++++ tests/test_package_rollback.cpp | 123 +++++++++++++ 11 files changed, 914 insertions(+) create mode 100644 src/shell/package/CLAUDE.md create mode 100644 src/shell/package/CMakeLists.txt create mode 100644 src/shell/package/package_io.cpp create mode 100644 src/shell/package/package_io.h create mode 100644 src/shell/package/package_pickers.cpp create mode 100644 src/shell/package/package_pickers.h create mode 100644 src/shell/package/package_rollback.cpp create mode 100644 src/shell/package/package_rollback.h create mode 100644 tests/test_package_io.cpp create mode 100644 tests/test_package_rollback.cpp diff --git a/CMakeLists.txt b/CMakeLists.txt index f783871..431b569 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -92,3 +92,4 @@ enable_testing() add_subdirectory(src/core) add_subdirectory(src/app) add_subdirectory(src/shell/instrument) +add_subdirectory(src/shell/package) diff --git a/src/shell/package/CLAUDE.md b/src/shell/package/CLAUDE.md new file mode 100644 index 0000000..6456fe9 --- /dev/null +++ b/src/shell/package/CLAUDE.md @@ -0,0 +1,61 @@ +# src/shell/package — package filesystem + dialog seam + +## Scope + +The filesystem and dialog acts behind bank-package export/import: streaming package +file I/O (`package_io`), the landed-file journal and its rollback delete +(`package_rollback`), and the platform file pickers (`package_pickers`). This seam +is bytes-only — the package format (magic, manifest, entry layout) is +`core/package`'s business, and the export/import verbs that orchestrate both do not +live here yet. No REAPER project state is touched in this directory: no ext-state +read or write, no undo block, no generation bump — those belong to the verbs. + +## Invariants + +- **Atomic write.** A package accumulates in a `.rsbanktmp` sibling in the + destination directory and reaches the destination only through `commit()`'s + rename (the mono-collapse temp+rename precedent). A failed, aborted, or abandoned + write leaves the destination absent or holding its prior contents — never a + partial `.rsbank`. +- **Streaming, both ways — at most ONE entry's payload in memory.** Writes append + one payload at a time; reads seek and materialize one range at a time. The claim + is structural, not aspirational: every payload crosses this seam as a move-only + `PayloadBuffer`, and `PayloadBuffer::alive()` is the seam counter the tests + assert against. There is no read-whole-package or write-whole-package entry + point; do not add one. +- **The rollback delete is prune's ONE carve-out, cited not restated.** The + citation and the discriminator live at `package_rollback.cpp`'s header. The + journal makes the discriminator structural: only paths its own `writeLandedFile` + successfully created are recorded, and `rollback()` consumes only the record — a + path this import did not write cannot be handed to it. +- **No overwrite of a bank-folder file, ever.** `writeLandedFile` refuses an + existing destination outright; collision handling (auto-rename) is the import + plan's job upstream. The package writer itself DOES replace an existing + destination — the export save dialog's own overwrite confirm is the consent — + and that asymmetry is deliberate. +- **The two pickers are asymmetric, and the asymmetry is real.** Import rides + REAPER's own `GetUserFileNameForRead` (both platforms); export goes native — + Win32 `GetSaveFileNameW` / SWELL `BrowseForSaveFile` — because the always-present + REAPER surface offers no save picker. Do not symmetrize; the newer + `GetUserFileName(mode=0)` alternative and why it is not used are recorded in + `package_pickers.cpp`'s header. + +## Modules + +- `package_io` — the streaming filesystem seam: `PayloadBuffer` (move-only payload + the `alive()` seam counter), `PackageFileWriter` (append-only temp+atomic-rename writer), `PackageFileReader` (seek-and-read one range per call, range-checked against the real file size), `readFilePayload` (one source file as one entry's payload), and `listFolderFileNames` (bare names, sorted, non-recursive, non-throwing). REAPER-free; tested without a DAW. +- `package_rollback` — `LandedFileJournal`: `writeLandedFile` (temp+rename land, recorded on success only, refuses an existing destination and an empty payload) and `rollback` (deletes exactly the recorded set, hard unlink — nothing ever referenced these bytes — tolerating a vanished file). REAPER-free; tested without a DAW. +- `package_pickers` — the two pickers in one platform TU (`#ifdef _WIN32` / `#else swell/swell.h`, the `draw_kit`/`prune_fs` split): `pickPackageForImport` (REAPER read picker) and `pickPackageSavePath` (native save dialog, UTF-8 in/out on Windows). Compile-only until the verbs land; nothing here can be exercised in a unit test. + +## Gotchas + +- A crash mid-write strands the `.rsbanktmp` sibling. It is not a `.rsbank` (no + picker filter matches it), and a later export to the same destination truncates + it — but one stranded in the BANK folder by a mid-import crash is a foreign file + to prune (not owned, so never an orphan) until removed by hand. `[verify — DAW]` + whether the import verb should pre-clean stale `.rsbanktmp` names when it lands. +- The picker `defext`/filter strings are spelled to the Win32 `lpstrDefExt` + convention (no dot) but are `[verify — DAW]` on all three platforms — neither + picker is exercised outside a live REAPER session. +- `readRange(_, 0)` returns an empty buffer — indistinguishable from failure, by + design (the one "nothing to work with" branch). A genuinely zero-length entry + cannot round-trip through this seam; the format layer must not emit one. diff --git a/src/shell/package/CMakeLists.txt b/src/shell/package/CMakeLists.txt new file mode 100644 index 0000000..ad716e4 --- /dev/null +++ b/src/shell/package/CMakeLists.txt @@ -0,0 +1,20 @@ +# The filesystem + dialog seam for bank packages. package_io / package_rollback are +# REAPER-free (standard filesystem only), so the pure-library/test helpers fit and +# their tests run without a DAW. The export/import verbs that drive all three targets +# are not in this directory yet. + +reasampler_pure_library(package_io SOURCES package_io.cpp LINK PUBLIC file_bytes) +reasampler_test(package_io LINK package_io) + +reasampler_pure_library(package_rollback SOURCES package_rollback.cpp LINK PUBLIC package_io) +reasampler_test(package_rollback LINK package_rollback) + +# The platform pickers touch the REAPER API and the native save dialog, so no test +# target can exercise them; declared as a library so both picker paths stay compiled. +add_library(package_pickers STATIC package_pickers.cpp) +target_include_directories(package_pickers PUBLIC ${REASAMPLER_SRC_DIR}) +target_include_directories(package_pickers PRIVATE ${SDK_INC} ${WDL_INC}) +if(NOT WIN32) + # Match the loadable modules: SWELL is provided by the host REAPER at runtime. + target_compile_definitions(package_pickers PRIVATE SWELL_PROVIDED_BY_APP) +endif() diff --git a/src/shell/package/package_io.cpp b/src/shell/package/package_io.cpp new file mode 100644 index 0000000..92f645b --- /dev/null +++ b/src/shell/package/package_io.cpp @@ -0,0 +1,170 @@ +// package_io.cpp — see package_io.h for the seam's contract. Non-throwing at the +// boundary: every filesystem call uses the error_code form so no filesystem_error +// crosses into a REAPER action body. + +#include "shell/package/package_io.h" + +#include +#include +#include +#include + +#include "core/util/file_bytes.h" + +namespace reasampler { + +namespace fs = std::filesystem; + +namespace { +std::atomic g_alivePayloads{0}; +} + +// --------------------------------------------------------------------------- +// PayloadBuffer + +PayloadBuffer::PayloadBuffer(std::vector bytes) + : bytes_(std::move(bytes)), counted_(!bytes_.empty()) { + if (counted_) g_alivePayloads.fetch_add(1, std::memory_order_relaxed); +} + +PayloadBuffer::~PayloadBuffer() { release(); } + +PayloadBuffer::PayloadBuffer(PayloadBuffer&& other) noexcept + : bytes_(std::move(other.bytes_)), counted_(other.counted_) { + // The count transfers with the bytes — a move must never double-count. + other.bytes_.clear(); + other.counted_ = false; +} + +PayloadBuffer& PayloadBuffer::operator=(PayloadBuffer&& other) noexcept { + if (this != &other) { + release(); + bytes_ = std::move(other.bytes_); + counted_ = other.counted_; + other.bytes_.clear(); + other.counted_ = false; + } + return *this; +} + +int PayloadBuffer::alive() { return g_alivePayloads.load(std::memory_order_relaxed); } + +void PayloadBuffer::release() { + if (counted_) g_alivePayloads.fetch_sub(1, std::memory_order_relaxed); + counted_ = false; + bytes_.clear(); +} + +// --------------------------------------------------------------------------- +// PackageFileWriter + +PackageFileWriter::PackageFileWriter(std::string destAbsPath) + : destPath_(std::move(destAbsPath)), tempPath_(destPath_ + ".rsbanktmp") { + out_.open(tempPath_, std::ios::binary | std::ios::trunc); + ok_ = static_cast(out_); +} + +PackageFileWriter::~PackageFileWriter() { + if (!done_) abort(); +} + +bool PackageFileWriter::appendRaw(const std::uint8_t* data, std::size_t len) { + if (!ok_ || done_) return false; + if (len == 0) return true; + out_.write(reinterpret_cast(data), + static_cast(len)); + ok_ = static_cast(out_); + return ok_; +} + +bool PackageFileWriter::appendPayload(const PayloadBuffer& payload) { + return appendRaw(payload.data(), payload.size()); +} + +bool PackageFileWriter::commit() { + if (done_) return false; + if (ok_) { + out_.flush(); + ok_ = static_cast(out_); + } + out_.close(); + if (!ok_) { + abort(); + return false; + } + // rename() replaces the destination in one step (the mono-collapse precedent): + // prior contents survive until the replacement is known-complete, and a failed + // rename self-cleans the temp rather than littering it. + std::error_code ec; + fs::rename(tempPath_, destPath_, ec); + if (ec) { + fs::remove(tempPath_, ec); + done_ = true; + ok_ = false; + return false; + } + done_ = true; + return true; +} + +void PackageFileWriter::abort() { + if (done_) return; + out_.close(); + std::error_code ec; + fs::remove(tempPath_, ec); + done_ = true; + ok_ = false; +} + +// --------------------------------------------------------------------------- +// PackageFileReader + +PackageFileReader::PackageFileReader(const std::string& srcAbsPath) { + std::error_code ec; + const std::uintmax_t sz = fs::file_size(srcAbsPath, ec); + if (ec) return; + in_.open(srcAbsPath, std::ios::binary); + if (!in_) return; + size_ = static_cast(sz); + ok_ = true; +} + +PayloadBuffer PackageFileReader::readRange(std::uint64_t offset, std::uint64_t length) { + // Overflow-safe range check: length is capped by the real file size before any + // allocation happens, so a hostile offset/length pair cannot demand the moon. + if (!ok_ || length == 0 || length > size_ || offset > size_ - length) { + return PayloadBuffer{}; + } + in_.clear(); // a prior failed read must not poison this one + in_.seekg(static_cast(offset)); + if (!in_) return PayloadBuffer{}; + std::vector bytes(static_cast(length)); + in_.read(reinterpret_cast(bytes.data()), + static_cast(length)); + if (static_cast(in_.gcount()) != length) return PayloadBuffer{}; + return PayloadBuffer(std::move(bytes)); +} + +// --------------------------------------------------------------------------- + +PayloadBuffer readFilePayload(const std::string& absPath) { + return PayloadBuffer(util::readFileBytes(absPath)); +} + +std::vector listFolderFileNames(const std::string& dirAbsPath) { + std::vector names; + std::error_code ec; + // Manual iterator form (it.increment(ec)) keeps the loop non-throwing on a + // mid-iteration failure, matching prune_fs's enumerate. + fs::directory_iterator it(dirAbsPath, ec); + for (; !ec && it != fs::directory_iterator{}; it.increment(ec)) { + const auto& entry = *it; + std::error_code reg_ec; + if (!entry.is_regular_file(reg_ec)) continue; // skip subdirs / specials + names.push_back(entry.path().filename().string()); + } + std::sort(names.begin(), names.end()); + return names; +} + +} // namespace reasampler diff --git a/src/shell/package/package_io.h b/src/shell/package/package_io.h new file mode 100644 index 0000000..6f36d2d --- /dev/null +++ b/src/shell/package/package_io.h @@ -0,0 +1,112 @@ +// shell/package/package_io — streaming filesystem seam for bank packages: append one +// payload at a time through a temp-file + atomic-rename writer, seek and read one +// payload at a time back out. Bytes only: what a package contains is core/package's +// business, never this seam's. Blocking I/O — UI-thread actions only, never the +// audio thread. + +#pragma once + +#include +#include +#include +#include + +namespace reasampler { + +// One entry's payload, and the seam counter that makes "never more than one entry in +// memory" assertable: alive() counts every buffer currently holding bytes, so the +// streaming claim is a test CHECK against this counter rather than a memory +// measurement. Move-only — copying a payload would silently double the held bytes. +class PayloadBuffer { +public: + PayloadBuffer() = default; + explicit PayloadBuffer(std::vector bytes); + ~PayloadBuffer(); + PayloadBuffer(PayloadBuffer&& other) noexcept; + PayloadBuffer& operator=(PayloadBuffer&& other) noexcept; + PayloadBuffer(const PayloadBuffer&) = delete; + PayloadBuffer& operator=(const PayloadBuffer&) = delete; + + const std::uint8_t* data() const { return bytes_.data(); } + std::size_t size() const { return bytes_.size(); } + bool empty() const { return bytes_.empty(); } + const std::vector& bytes() const { return bytes_; } + + // Buffers currently holding at least one byte, process-wide. + static int alive(); + +private: + void release(); + + std::vector bytes_; + bool counted_ = false; +}; + +// Streaming atomic writer. Bytes accumulate in ".rsbanktmp" beside the +// destination (same directory, so the final rename never crosses a volume); the +// destination itself is touched only by commit()'s rename, so a failed, aborted, or +// abandoned write leaves it absent or holding its prior contents — never a partial +// file. Destruction without commit() aborts and removes the temp. commit() REPLACES +// an existing destination: the export save dialog's own overwrite confirm is the +// consent (the never-overwrite rule for bank-folder files lives in +// LandedFileJournal, upstream of this writer). Neither copyable nor movable, and +// append-only — there is deliberately no way to hand it a whole package at once. +class PackageFileWriter { +public: + explicit PackageFileWriter(std::string destAbsPath); + ~PackageFileWriter(); + PackageFileWriter(const PackageFileWriter&) = delete; + PackageFileWriter& operator=(const PackageFileWriter&) = delete; + + bool ok() const { return ok_; } + // Framing/header bytes. False on a failed or already-finished writer. + bool appendRaw(const std::uint8_t* data, std::size_t len); + // One entry's bytes. Same contract as appendRaw. + bool appendPayload(const PayloadBuffer& payload); + // Flush, close, rename over the destination. False (and self-cleaning: the temp + // is removed, the destination untouched) on any failure or on a second call. + bool commit(); + // Close and remove the temp; the destination is never touched. Idempotent. + void abort(); + + const std::string& destPath() const { return destPath_; } + const std::string& tempPath() const { return tempPath_; } + +private: + std::string destPath_; + std::string tempPath_; + std::ofstream out_; + bool ok_ = false; + bool done_ = false; +}; + +// Seek-and-read reader: exactly one payload is materialized per readRange call, and +// there is deliberately no read-whole-file entry point. Empty buffer on ANY failure +// — unopenable file, zero length, out of range, short read — so the caller has one +// "nothing to work with" branch (file_bytes' contract). +class PackageFileReader { +public: + explicit PackageFileReader(const std::string& srcAbsPath); + + bool ok() const { return ok_; } + std::uint64_t fileSize() const { return size_; } + // Bytes [offset, offset+length). Range-checked against the real file size, so a + // hostile layout can never demand an allocation past the file's end. + PayloadBuffer readRange(std::uint64_t offset, std::uint64_t length); + +private: + std::ifstream in_; + std::uint64_t size_ = 0; + bool ok_ = false; +}; + +// One source file read whole as one entry's payload — a bank file IS the streaming +// unit, so whole-file here is one entry, released before the next is read. Empty on +// any failure, per readFileBytes. +PayloadBuffer readFilePayload(const std::string& absPath); + +// Bare file names (regular files only, never a path) in dirAbsPath, sorted so +// callers see a deterministic order; empty on a missing or unreadable folder. +std::vector listFolderFileNames(const std::string& dirAbsPath); + +} // namespace reasampler diff --git a/src/shell/package/package_pickers.cpp b/src/shell/package/package_pickers.cpp new file mode 100644 index 0000000..b122bd9 --- /dev/null +++ b/src/shell/package/package_pickers.cpp @@ -0,0 +1,93 @@ +// package_pickers.cpp — see package_pickers.h. Export is native rather than REAPER +// API: the SDK's newer GetUserFileName(mode=0) could save, but it resolves to null +// on older REAPER builds this extension still loads in, while GetSaveFileNameW / +// BrowseForSaveFile are always present. main.cpp owns the REAPER API pointers; this +// TU gets them extern. + +#include "shell/package/package_pickers.h" + +#ifdef _WIN32 +#include +#include +#else +#include "swell/swell.h" +#endif + +#define REAPERAPI_MINIMAL +#define REAPERAPI_WANT_GetUserFileNameForRead +#define REAPERAPI_WANT_GetMainHwnd +#include "reaper_plugin_functions.h" + +namespace reasampler { + +bool pickPackageForImport(std::string& outAbsPath) { + outAbsPath.clear(); + if (!GetUserFileNameForRead) return false; + char buf[4096]; + buf[0] = '\0'; + // defext spelled without the dot, matching Win32 lpstrDefExt. [verify — DAW] + // whether REAPER's picker applies it to the shown filter. + if (!GetUserFileNameForRead(buf, "Import bank package", "rsbank")) return false; + outAbsPath = buf; + return !outAbsPath.empty(); +} + +#ifdef _WIN32 + +bool pickPackageSavePath(const std::string& suggestedFileName, std::string& outAbsPath) { + outAbsPath.clear(); + + wchar_t file[4096]; + file[0] = L'\0'; + if (!suggestedFileName.empty()) { + const int wrote = MultiByteToWideChar(CP_UTF8, 0, suggestedFileName.c_str(), + -1, file, 4096); + if (wrote <= 0) file[0] = L'\0'; // unconvertible suggestion -> empty box + } + + OPENFILENAMEW ofn{}; + ofn.lStructSize = sizeof(ofn); + ofn.hwndOwner = GetMainHwnd ? GetMainHwnd() : nullptr; + ofn.lpstrFilter = + L"ReaSampler bank package (*.rsbank)\0*.rsbank\0All files (*.*)\0*.*\0"; + ofn.lpstrFile = file; + ofn.nMaxFile = 4096; + ofn.lpstrTitle = L"Export bank package"; + ofn.lpstrDefExt = L"rsbank"; + // NOCHANGEDIR: REAPER's process-wide working directory is not ours to move. + ofn.Flags = OFN_OVERWRITEPROMPT | OFN_PATHMUSTEXIST | OFN_NOCHANGEDIR | + OFN_HIDEREADONLY; + if (!GetSaveFileNameW(&ofn)) return false; + + const int need = + WideCharToMultiByte(CP_UTF8, 0, file, -1, nullptr, 0, nullptr, nullptr); + if (need <= 0) return false; + std::string utf8(static_cast(need), '\0'); + WideCharToMultiByte(CP_UTF8, 0, file, -1, &utf8[0], need, nullptr, nullptr); + if (!utf8.empty() && utf8.back() == '\0') utf8.pop_back(); + outAbsPath = std::move(utf8); + return !outAbsPath.empty(); +} + +#else // SWELL (macOS / Linux) + +bool pickPackageSavePath(const std::string& suggestedFileName, std::string& outAbsPath) { + outAbsPath.clear(); + // GetOpenFileName-style pair list; the literal's implicit terminator supplies the + // closing double-NUL. [verify — DAW] the exact filter strings SWELL accepts. + static const char kExtList[] = "ReaSampler bank package (*.rsbank)\0*.rsbank\0"; + char fn[4096]; + fn[0] = '\0'; + if (!BrowseForSaveFile("Export bank package", nullptr, + suggestedFileName.empty() ? nullptr + : suggestedFileName.c_str(), + kExtList, fn, static_cast(sizeof(fn)))) { + return false; + } + outAbsPath = fn; + return !outAbsPath.empty(); +} + +#endif + +} // namespace reasampler diff --git a/src/shell/package/package_pickers.h b/src/shell/package/package_pickers.h new file mode 100644 index 0000000..cdfbc1b --- /dev/null +++ b/src/shell/package/package_pickers.h @@ -0,0 +1,21 @@ +// shell/package/package_pickers — the two package file pickers, and they are +// deliberately asymmetric: import rides REAPER's own read picker +// (GetUserFileNameForRead, both platforms); export goes native — Win32 +// GetSaveFileNameW / SWELL BrowseForSaveFile — because the always-present REAPER +// surface offers no save picker. Do not symmetrize; the rationale is in the TU. + +#pragma once + +#include + +namespace reasampler { + +// REAPER's read picker. True with outAbsPath set iff the user chose a file. +bool pickPackageForImport(std::string& outAbsPath); + +// Native save picker, pre-filled with suggestedFileName (a bare name, e.g. +// "MyBank.rsbank"). True with outAbsPath set iff the user chose a destination; the +// dialog's own overwrite confirm has already run by then. +bool pickPackageSavePath(const std::string& suggestedFileName, std::string& outAbsPath); + +} // namespace reasampler diff --git a/src/shell/package/package_rollback.cpp b/src/shell/package/package_rollback.cpp new file mode 100644 index 0000000..ddface7 --- /dev/null +++ b/src/shell/package/package_rollback.cpp @@ -0,0 +1,40 @@ +// package_rollback.cpp — see package_rollback.h. The rollback delete below runs +// under the ONE carve-out from prune's exclusive file-deletion authority, stated at +// src/shell/persist/prune_fs.cpp:5-11; it satisfies that discriminator by +// construction — every recorded path was created by this journal's own +// writeLandedFile, and the abandoned index mutation means nothing ever referenced it. + +#include "shell/package/package_rollback.h" + +#include + +namespace reasampler { + +namespace fs = std::filesystem; + +bool LandedFileJournal::writeLandedFile(const std::string& absPath, + const PayloadBuffer& payload) { + if (payload.empty()) return false; + std::error_code ec; + if (fs::exists(absPath, ec) || ec) return false; + PackageFileWriter writer(absPath); + if (!writer.appendPayload(payload)) return false; // dtor aborts; temp removed + if (!writer.commit()) return false; + paths_.push_back(absPath); + return true; +} + +RollbackResult LandedFileJournal::rollback() { + RollbackResult result; + for (const std::string& path : paths_) { + std::error_code ec; + const bool removed = fs::remove(path, ec); + if (removed) ++result.deletedCount; + else if (ec) ++result.failedCount; + else ++result.alreadyAbsentCount; // no error, nothing there + } + paths_.clear(); + return result; +} + +} // namespace reasampler diff --git a/src/shell/package/package_rollback.h b/src/shell/package/package_rollback.h new file mode 100644 index 0000000..a72f89c --- /dev/null +++ b/src/shell/package/package_rollback.h @@ -0,0 +1,45 @@ +// shell/package/package_rollback — the files ONE import call has landed, as a +// journal: writes record themselves on success, and rollback() deletes exactly what +// is recorded — a path this import did not write is structurally impossible to hand +// it. The deletion carve-out this satisfies is cited at package_rollback.cpp's +// header. + +#pragma once + +#include +#include + +#include "shell/package/package_io.h" + +namespace reasampler { + +struct RollbackResult { + int deletedCount = 0; + int alreadyAbsentCount = 0; // vanished between land and rollback — not a failure + int failedCount = 0; // locked / permission — recorded, never thrown +}; + +// The evidence for the rollback discriminator: only paths this journal's own +// writeLandedFile successfully created are recorded, so rollback() can never touch a +// byte this import did not write. +class LandedFileJournal { +public: + // Lands one payload at absPath through the atomic temp+rename writer and records + // the path on success. REFUSES an existing destination — a bank-folder file is + // never overwritten; collision handling is the import plan's job, upstream. An + // empty payload is refused too: it signals an upstream read failure, never a + // real entry. + bool writeLandedFile(const std::string& absPath, const PayloadBuffer& payload); + + // Deletes exactly the recorded files and clears the journal, so a second call is + // a no-op. Hard unlink, not trash: nothing ever referenced these bytes. + RollbackResult rollback(); + + const std::vector& landedPaths() const { return paths_; } + bool empty() const { return paths_.empty(); } + +private: + std::vector paths_; +}; + +} // namespace reasampler diff --git a/tests/test_package_io.cpp b/tests/test_package_io.cpp new file mode 100644 index 0000000..7f10b73 --- /dev/null +++ b/tests/test_package_io.cpp @@ -0,0 +1,228 @@ +// Standalone tests for shell/package/package_io — no REAPER, no framework. Pins the +// two properties the seam exists for: an interrupted or failed write leaves the +// destination absent or holding its prior contents (failure injected at the writer +// seam — abandonment, open failure, rename failure), and a multi-entry round trip +// holds at most one entry's payload, asserted against PayloadBuffer::alive() — the +// seam counter — rather than a memory measurement. + +#include "../src/shell/package/package_io.h" + +#include +#include +#include +#include +#include +#include + +using namespace reasampler; +namespace fs = std::filesystem; + +static int g_fail = 0; +#define CHECK(cond) do { if(!(cond)) { \ + std::printf("FAIL line %d: %s\n", __LINE__, #cond); ++g_fail; } } while(0) + +static std::vector patternBytes(std::size_t n, std::uint8_t seed) { + std::vector v(n); + for (std::size_t i = 0; i < n; ++i) + v[i] = static_cast(seed + i * 7u); + return v; +} + +static void writeScratchFile(const std::string& path, + const std::vector& bytes) { + std::ofstream f(path, std::ios::binary | std::ios::trunc); + f.write(reinterpret_cast(bytes.data()), + static_cast(bytes.size())); +} + +static std::vector readAll(const std::string& path) { + std::ifstream f(path, std::ios::binary); + return std::vector(std::istreambuf_iterator(f), + std::istreambuf_iterator()); +} + +static void testPayloadCounterTracksMovesNotCopies() { + CHECK(PayloadBuffer::alive() == 0); + { + PayloadBuffer a(patternBytes(4, 1)); + CHECK(PayloadBuffer::alive() == 1); + PayloadBuffer b = std::move(a); + CHECK(PayloadBuffer::alive() == 1); // the count moved with the bytes + PayloadBuffer c; + c = std::move(b); + CHECK(PayloadBuffer::alive() == 1); + CHECK(c.size() == 4); + } + CHECK(PayloadBuffer::alive() == 0); + { + PayloadBuffer empty; + CHECK(PayloadBuffer::alive() == 0); // holding nothing counts as nothing + } +} + +static void testStreamingRoundTripHoldsOnePayload() { + const std::string dest = "pkg_io_scratch.rsbank"; + const std::vector header = patternBytes(16, 0xA0); + const std::vector> entries = { + patternBytes(1000, 1), patternBytes(500, 2), patternBytes(1, 3)}; + + std::vector srcs; + for (std::size_t i = 0; i < entries.size(); ++i) { + srcs.push_back("pkg_io_src" + std::to_string(i) + ".bin"); + writeScratchFile(srcs[i], entries[i]); + } + + CHECK(PayloadBuffer::alive() == 0); + std::vector> layout; // offset, length + { + PackageFileWriter writer(dest); + CHECK(writer.ok()); + CHECK(writer.appendRaw(header.data(), header.size())); + std::uint64_t offset = header.size(); + for (std::size_t i = 0; i < entries.size(); ++i) { + PayloadBuffer p = readFilePayload(srcs[i]); + CHECK(p.bytes() == entries[i]); + CHECK(PayloadBuffer::alive() == 1); // exactly one entry in memory + CHECK(writer.appendPayload(p)); + layout.emplace_back(offset, p.size()); + offset += p.size(); + } + CHECK(PayloadBuffer::alive() == 0); // each released before the next + CHECK(writer.commit()); + CHECK(!writer.commit()); // a second commit is refused + } + CHECK(!fs::exists(dest + ".rsbanktmp")); + CHECK(fs::exists(dest)); + + { + // Scoped: the reader holds the file open, and Windows refuses to delete an + // open file — cleanup below needs it closed first. + PackageFileReader reader(dest); + CHECK(reader.ok()); + CHECK(reader.fileSize() == header.size() + 1000 + 500 + 1); + for (std::size_t i = 0; i < entries.size(); ++i) { + PayloadBuffer p = reader.readRange(layout[i].first, layout[i].second); + CHECK(PayloadBuffer::alive() == 1); // one entry per readRange, no more + CHECK(p.bytes() == entries[i]); + } + CHECK(PayloadBuffer::alive() == 0); + } + + std::error_code ec; + for (const std::string& s : srcs) fs::remove(s, ec); + fs::remove(dest, ec); +} + +static void testAbandonedWriteLeavesNoDestination() { + const std::string dest = "pkg_io_abandon.rsbank"; + { + PackageFileWriter writer(dest); + const std::vector some = patternBytes(64, 9); + CHECK(writer.appendRaw(some.data(), some.size())); + // no commit — destruction is the injected interruption + } + CHECK(!fs::exists(dest)); + CHECK(!fs::exists(dest + ".rsbanktmp")); +} + +static void testAbortPreservesPriorContents() { + const std::string dest = "pkg_io_prior.rsbank"; + const std::vector prior = patternBytes(32, 0x40); + writeScratchFile(dest, prior); + { + PackageFileWriter writer(dest); + const std::vector some = patternBytes(64, 9); + CHECK(writer.appendRaw(some.data(), some.size())); + writer.abort(); + CHECK(!writer.appendRaw(some.data(), some.size())); // dead after abort + CHECK(!writer.commit()); + } + CHECK(readAll(dest) == prior); + CHECK(!fs::exists(dest + ".rsbanktmp")); + std::error_code ec; + fs::remove(dest, ec); +} + +static void testOpenFailureIsInert() { + const std::string dest = "pkg_io_no_such_dir/x.rsbank"; + PackageFileWriter writer(dest); + CHECK(!writer.ok()); + const std::vector some = patternBytes(8, 1); + CHECK(!writer.appendRaw(some.data(), some.size())); + CHECK(!writer.commit()); + CHECK(!fs::exists("pkg_io_no_such_dir")); +} + +static void testCommitRenameFailureSelfCleans() { + // A directory squatting on the destination makes the final rename fail — a real + // injected commit failure, not a simulated one. + const std::string dest = "pkg_io_dir.rsbank"; + std::error_code ec; + fs::create_directory(dest, ec); + CHECK(!ec); + { + PackageFileWriter writer(dest); + CHECK(writer.ok()); + const std::vector some = patternBytes(8, 1); + CHECK(writer.appendRaw(some.data(), some.size())); + CHECK(!writer.commit()); + } + CHECK(fs::is_directory(dest)); // prior state intact + CHECK(!fs::exists(dest + ".rsbanktmp")); + fs::remove(dest, ec); +} + +static void testReaderEdges() { + PackageFileReader missing("pkg_io_no_such_file.rsbank"); + CHECK(!missing.ok()); + CHECK(missing.readRange(0, 1).empty()); + + const std::string path = "pkg_io_edges.bin"; + const std::vector bytes = patternBytes(10, 5); + writeScratchFile(path, bytes); + { + // Scoped: the reader must be closed before the cleanup remove below. + PackageFileReader reader(path); + CHECK(reader.ok()); + CHECK(reader.fileSize() == 10); + CHECK(reader.readRange(5, 10).empty()); // past the end + CHECK(reader.readRange(10, 1).empty()); // starts at the end + CHECK(reader.readRange(0, 0).empty()); // zero length is failure, one branch + const PayloadBuffer slice = reader.readRange(2, 3); + CHECK(slice.bytes() == + std::vector(bytes.begin() + 2, bytes.begin() + 5)); + } + std::error_code ec; + fs::remove(path, ec); +} + +static void testListFolderFileNames() { + const std::string dir = "pkg_io_listdir"; + std::error_code ec; + fs::create_directory(dir, ec); + writeScratchFile(dir + "/b.bin", patternBytes(2, 1)); + writeScratchFile(dir + "/a.bin", patternBytes(2, 2)); + fs::create_directory(dir + "/sub", ec); + writeScratchFile(dir + "/sub/c.bin", patternBytes(2, 3)); + + const std::vector names = listFolderFileNames(dir); + CHECK(names == (std::vector{"a.bin", "b.bin"})); // sorted, bare, non-recursive + CHECK(listFolderFileNames("pkg_io_no_such_dir").empty()); + + fs::remove_all(dir, ec); +} + +int main() { + testPayloadCounterTracksMovesNotCopies(); + testStreamingRoundTripHoldsOnePayload(); + testAbandonedWriteLeavesNoDestination(); + testAbortPreservesPriorContents(); + testOpenFailureIsInert(); + testCommitRenameFailureSelfCleans(); + testReaderEdges(); + testListFolderFileNames(); + + if (g_fail == 0) std::printf("package_io: all tests passed\n"); + else std::printf("package_io: %d CHECK(s) FAILED\n", g_fail); + return g_fail == 0 ? 0 : 1; +} diff --git a/tests/test_package_rollback.cpp b/tests/test_package_rollback.cpp new file mode 100644 index 0000000..6c28bfa --- /dev/null +++ b/tests/test_package_rollback.cpp @@ -0,0 +1,123 @@ +// Standalone tests for shell/package/package_rollback — no REAPER, no framework. +// Pins the discriminator's mechanics: a file is recorded only when this journal's +// own write landed it, a pre-existing destination is refused untouched, and +// rollback deletes exactly the recorded set — a bystander file beside them stays, +// and a vanished file is tolerated rather than failed. + +#include "../src/shell/package/package_rollback.h" + +#include +#include +#include +#include +#include + +using namespace reasampler; +namespace fs = std::filesystem; + +static int g_fail = 0; +#define CHECK(cond) do { if(!(cond)) { \ + std::printf("FAIL line %d: %s\n", __LINE__, #cond); ++g_fail; } } while(0) + +static std::vector patternBytes(std::size_t n, std::uint8_t seed) { + std::vector v(n); + for (std::size_t i = 0; i < n; ++i) + v[i] = static_cast(seed + i * 7u); + return v; +} + +static void writeScratchFile(const std::string& path, + const std::vector& bytes) { + std::ofstream f(path, std::ios::binary | std::ios::trunc); + f.write(reinterpret_cast(bytes.data()), + static_cast(bytes.size())); +} + +static std::vector readAll(const std::string& path) { + std::ifstream f(path, std::ios::binary); + return std::vector(std::istreambuf_iterator(f), + std::istreambuf_iterator()); +} + +static void testLandRecordsOnSuccessOnly() { + LandedFileJournal journal; + const std::string path = "rb_land.bin"; + const std::vector bytes = patternBytes(32, 1); + CHECK(journal.writeLandedFile(path, PayloadBuffer(bytes))); + CHECK(readAll(path) == bytes); + CHECK(journal.landedPaths() == (std::vector{path})); + CHECK(!fs::exists(path + ".rsbanktmp")); + journal.rollback(); + CHECK(!fs::exists(path)); +} + +static void testExistingDestinationRefusedUntouched() { + LandedFileJournal journal; + const std::string path = "rb_existing.bin"; + const std::vector original = patternBytes(16, 0x60); + writeScratchFile(path, original); + + CHECK(!journal.writeLandedFile(path, PayloadBuffer(patternBytes(8, 1)))); + CHECK(readAll(path) == original); // never overwritten + CHECK(journal.empty()); // a refused write is not recorded + + const RollbackResult result = journal.rollback(); + CHECK(result.deletedCount == 0); + CHECK(fs::exists(path)); // rollback cannot touch a file it did not write + std::error_code ec; + fs::remove(path, ec); +} + +static void testEmptyPayloadRefused() { + LandedFileJournal journal; + CHECK(!journal.writeLandedFile("rb_empty.bin", PayloadBuffer{})); + CHECK(!fs::exists("rb_empty.bin")); + CHECK(journal.empty()); +} + +static void testRollbackDeletesExactlyTheRecordedSet() { + LandedFileJournal journal; + CHECK(journal.writeLandedFile("rb_a.bin", PayloadBuffer(patternBytes(8, 1)))); + CHECK(journal.writeLandedFile("rb_c.bin", PayloadBuffer(patternBytes(8, 2)))); + writeScratchFile("rb_bystander.bin", patternBytes(8, 3)); // not journal-written + + const RollbackResult result = journal.rollback(); + CHECK(result.deletedCount == 2); + CHECK(result.alreadyAbsentCount == 0); + CHECK(result.failedCount == 0); + CHECK(!fs::exists("rb_a.bin")); + CHECK(!fs::exists("rb_c.bin")); + CHECK(fs::exists("rb_bystander.bin")); // exactly the given files, nothing else + CHECK(journal.empty()); + + const RollbackResult second = journal.rollback(); // cleared: a no-op + CHECK(second.deletedCount == 0); + CHECK(fs::exists("rb_bystander.bin")); + std::error_code ec; + fs::remove("rb_bystander.bin", ec); +} + +static void testVanishedFileIsToleratedNotFailed() { + LandedFileJournal journal; + CHECK(journal.writeLandedFile("rb_gone.bin", PayloadBuffer(patternBytes(8, 1)))); + std::error_code ec; + fs::remove("rb_gone.bin", ec); // vanished between land and rollback + CHECK(!ec); + + const RollbackResult result = journal.rollback(); + CHECK(result.deletedCount == 0); + CHECK(result.alreadyAbsentCount == 1); + CHECK(result.failedCount == 0); +} + +int main() { + testLandRecordsOnSuccessOnly(); + testExistingDestinationRefusedUntouched(); + testEmptyPayloadRefused(); + testRollbackDeletesExactlyTheRecordedSet(); + testVanishedFileIsToleratedNotFailed(); + + if (g_fail == 0) std::printf("package_rollback: all tests passed\n"); + else std::printf("package_rollback: %d CHECK(s) FAILED\n", g_fail); + return g_fail == 0 ? 0 : 1; +} From 1aebf519386710abcf58e653a475106e6d12ffde Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 08:14:55 -0400 Subject: [PATCH 04/24] Relabel post-manifest-parse failure as TooNew; refuse zero-length package entries at encode An additively-tagged newer package that fails to parse now reports TooNew (with writer semver) instead of unactionable Malformed. Format layer also refuses encoding a zero-length entry, honoring the shell's appendPayload contract; both test-covered. --- src/core/package/CLAUDE.md | 14 ++++++++++ src/core/package/bank_package.cpp | 8 +++++- src/core/package/package_manifest.cpp | 6 ++++- src/core/package/package_manifest.h | 7 ++--- tests/test_bank_package.cpp | 38 ++++++++++++++++++++++++--- tests/test_package_manifest.cpp | 13 ++++++++- 6 files changed, 77 insertions(+), 9 deletions(-) diff --git a/src/core/package/CLAUDE.md b/src/core/package/CLAUDE.md index a8bfe62..373b512 100644 --- a/src/core/package/CLAUDE.md +++ b/src/core/package/CLAUDE.md @@ -79,8 +79,22 @@ landing after the format. version pair classifies `Readable`; for `TooNew` it stops at the semver — don't "fix" it to read the manifest length there, a future structural format may have moved it. +- A package whose header classifies `Readable` (fv > ours, minReader still + within reach — the additive case) but whose manifest fails to parse is + reported `TooNew`, not `Malformed`: the header is valid and already carries + the writer's semver, so the refusal can still name what to install. This + widens `TooNew` to cover "read and failed" as well as "stopped at the frozen + region" — both refuse whole and write nothing, so the safety property is + unchanged, only the message. `classifyPackageVersion` and the frozen-region + `TooNew` path are unaffected; this is the post-manifest-parse branch only. - The format carries no algorithm tag for `byteHash` — it is FNV-1a (`capture::hashBytes`) implicitly. Changing the digest algorithm is a `minReaderVersion` bump, not additive: an old reader would otherwise compare a stored digest against bytes hashed the new way and silently misjudge corruption. +- **Cross-module contract with `src/shell/package`:** a genuinely zero-length + entry cannot round-trip through the filesystem seam there (`appendPayload` + refuses an empty payload — an empty buffer signals an upstream read failure, + not a real entry). `serializeManifest` refuses a zero-length `PackageEntry` + at encode so this layer never produces one; decode does not enforce it (a + hostile/older package declaring one is not this track's concern). diff --git a/src/core/package/bank_package.cpp b/src/core/package/bank_package.cpp index b5a9389..9b0edbe 100644 --- a/src/core/package/bank_package.cpp +++ b/src/core/package/bank_package.cpp @@ -102,7 +102,13 @@ DecodedPackage decodePackage(const std::vector& prefix, if (!r.ok) return dec; auto manifest = deserializeManifest(manifestJson); - if (!manifest) return dec; + if (!manifest) { + // A newer additive format's parse failure reports TooNew, not the + // unactionable Malformed — the header (with the writer semver) is + // already valid here. See this directory's CLAUDE.md for the tradeoff. + if (formatVersion > kPackageFormatVersion) dec.status = PackageReadability::TooNew; + return dec; + } std::uint64_t end = 0; std::vector layout; diff --git a/src/core/package/package_manifest.cpp b/src/core/package/package_manifest.cpp index a4d4ba9..4740b3e 100644 --- a/src/core/package/package_manifest.cpp +++ b/src/core/package/package_manifest.cpp @@ -43,8 +43,12 @@ bool PackageManifest::operator==(const PackageManifest& o) const { } std::optional serializeManifest(const PackageManifest& m) { - for (const auto& e : m.entries) + for (const auto& e : m.entries) { if (!isValidEntryName(e.fileName)) return std::nullopt; + // Cross-module contract with src/shell/package — see this directory's + // CLAUDE.md. + if (e.byteLength == 0) return std::nullopt; + } if (duplicateName(m.entries)) return std::nullopt; std::string out; diff --git a/src/core/package/package_manifest.h b/src/core/package/package_manifest.h index 3afe70d..0b31ce0 100644 --- a/src/core/package/package_manifest.h +++ b/src/core/package/package_manifest.h @@ -41,9 +41,10 @@ struct PackageManifest { }; // Emits the manifest JSON. nullopt when the manifest cannot be represented: -// an invalid or duplicate entry name, or a sample record BankModel itself would -// reject (empty id, absolute path) — refusing on encode so an undecodable -// package is never written. +// an invalid or duplicate entry name, a zero-length entry (see this +// directory's CLAUDE.md — the shell's payload-append seam cannot round-trip +// one), or a sample record BankModel itself would reject (empty id, absolute +// path) — refusing on encode so an undecodable package is never written. std::optional serializeManifest(const PackageManifest& m); // Parses manifest JSON (nullopt on malformed input, never UB). Unknown keys are diff --git a/tests/test_bank_package.cpp b/tests/test_bank_package.cpp index 565ab7c..27f16af 100644 --- a/tests/test_bank_package.cpp +++ b/tests/test_bank_package.cpp @@ -110,7 +110,7 @@ static std::string handManifest(const std::string& name, int length, // --- encode / decode round trip ---------------------------------------------- static void testEncodeDecodeRoundTrip() { - const PackageManifest m = fixture(96000, 0); // a zero-length payload is legal + const PackageManifest m = fixture(96000, 48000); auto enc = encodePackage(m); CHECK(enc.has_value()); @@ -120,8 +120,8 @@ static void testEncodeDecodeRoundTrip() { CHECK(enc->layout[0].offset == enc->prefix.size()); CHECK(enc->layout[0].length == 96000); CHECK(enc->layout[1].offset == enc->prefix.size() + 96000); - CHECK(enc->layout[1].length == 0); - CHECK(enc->totalSize == enc->prefix.size() + 96000); + CHECK(enc->layout[1].length == 48000); + CHECK(enc->totalSize == enc->prefix.size() + 96000 + 48000); // decodePackage(encodePackage(x)) == x — payloads are never read by the // codec, so the prefix plus the true total size is the whole input. @@ -141,6 +141,13 @@ static void testEncodeRefusesWhatManifestRefuses() { PackageManifest m = fixture(1, 1); m.entries[0].fileName = "../evil.wav"; CHECK(!encodePackage(m).has_value()); + + // A zero-length entry cannot round-trip through the shell's filesystem + // seam (src/shell/package's appendPayload refuses an empty payload) — the + // format layer must never produce one. + PackageManifest zeroLen = fixture(1, 1); + zeroLen.entries[1].byteLength = 0; + CHECK(!encodePackage(zeroLen).has_value()); } // --- truncation: every byte offset ------------------------------------------- @@ -209,6 +216,30 @@ static void testTooNewProducesNoManifest() { CHECK(decodePackage(cut, cut.size()).status == PackageReadability::Malformed); } +// An additively-tagged package (fv > ours, minReader still within reach) whose +// manifest fails to parse: the header classifies Readable, so decode reads +// into the manifest and fails there. That failure must still report TooNew — +// the header is valid and already carries the writer's semver — not the +// unactionable Malformed a genuinely corrupt header produces. +static void testAdditiveUnparseableManifestIsTooNew() { + std::vector bytes = rawHeader(kPackageFormatVersion + 1, kPackageMinReaderVersion, "1.9.0"); + appendManifest(bytes, "not json"); + const DecodedPackage dec = decodePackage(bytes, bytes.size()); + CHECK(dec.status == PackageReadability::TooNew); + CHECK(dec.header.formatVersion == kPackageFormatVersion + 1); + CHECK(dec.header.minReaderVersion == kPackageMinReaderVersion); + CHECK(dec.header.writerVersion == "1.9.0"); + CHECK(dec.manifest.entries.empty()); + CHECK(dec.layout.empty()); + + // Same-version unparseable manifest stays Malformed: nothing "newer" + // excuses it, so this is not a blanket "unparseable == TooNew" rule. + std::vector sameVersion = + rawHeader(kPackageFormatVersion, kPackageMinReaderVersion, "1.0.0"); + appendManifest(sameVersion, "not json"); + CHECK(decodePackage(sameVersion, sameVersion.size()).status == PackageReadability::Malformed); +} + // --- version ladder: additive forward compatibility -------------------------- // The reason two integers exist: a NEWER formatVersion whose minReaderVersion @@ -335,6 +366,7 @@ int main() { testEncodeRefusesWhatManifestRefuses(); testTruncationAtEveryByteOffsetIsMalformed(); testTooNewProducesNoManifest(); + testAdditiveUnparseableManifestIsTooNew(); testNewerAdditiveFormatReads(); testHostileHeadersAreMalformed(); testRequiredPrefixSizeRefusals(); diff --git a/tests/test_package_manifest.cpp b/tests/test_package_manifest.cpp index 8d35475..e418ed7 100644 --- a/tests/test_package_manifest.cpp +++ b/tests/test_package_manifest.cpp @@ -64,7 +64,7 @@ static PackageManifest fixture() { m.bankDisplayName = "Drums \"live\""; // escaping exercised m.exportTimestamp = 1754100000; m.entries.push_back({"kick.wav", 96000, "1111222233334444", fullSample()}); - m.entries.push_back({"snare.wav", 0, "5555666677778888", bareSample()}); // 0-length legal + m.entries.push_back({"snare.wav", 48000, "5555666677778888", bareSample()}); m.slots.append("smp-full"); m.slots.append("smp-bare"); m.slots.remove("smp-full"); // leaves a gap: slots round-trip must keep it @@ -173,6 +173,16 @@ static void testDuplicateEntriesKeyRejected() { // --- rejection: structural --------------------------------------------------- +// A zero-length entry cannot round-trip through the shell's filesystem seam +// (src/shell/package's appendPayload refuses an empty payload) — the format +// layer must never produce one, so encode refuses it. Decode does not enforce +// this (a hostile/older package declaring one is not this codec's concern). +static void testEncodeRejectsZeroLengthEntry() { + PackageManifest m = fixture(); + m.entries[0].byteLength = 0; + CHECK(!serializeManifest(m).has_value()); +} + static void testEncodeRejectsUnrepresentableSample() { PackageManifest m = fixture(); m.entries[0].sample.id.clear(); // BankModel::add rejects an empty id @@ -253,6 +263,7 @@ int main() { testDecodeRejectsBadEntryName(); testDuplicateEntryNamesRejectedBothWays(); testDuplicateEntriesKeyRejected(); + testEncodeRejectsZeroLengthEntry(); testEncodeRejectsUnrepresentableSample(); testDecodeRejectsMalformedShapes(); From 35b2a3a15195fddd5dd914a6bc8746cddbce1b37 Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 07:17:01 -0400 Subject: [PATCH 05/24] tracking: append OriginKind::PackageImport as value 5 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An appended field-vocabulary value, so kLedgerVersion stays 2 — pinned by a test. Unknown kinds still degrade to Unknown with the ledger Loaded. --- src/core/tracking/origin_ledger.cpp | 1 + src/core/tracking/origin_ledger.h | 3 ++ tests/test_origin_ledger.cpp | 84 +++++++++++++++++++++++++++-- 3 files changed, 84 insertions(+), 4 deletions(-) diff --git a/src/core/tracking/origin_ledger.cpp b/src/core/tracking/origin_ledger.cpp index dbcafd4..82568b7 100644 --- a/src/core/tracking/origin_ledger.cpp +++ b/src/core/tracking/origin_ledger.cpp @@ -38,6 +38,7 @@ OriginKind kindFromInt(int v) { case 2: return OriginKind::Ingest; case 3: return OriginKind::Recapture; case 4: return OriginKind::Resample; + case 5: return OriginKind::PackageImport; default: return OriginKind::Unknown; } } diff --git a/src/core/tracking/origin_ledger.h b/src/core/tracking/origin_ledger.h index f04264e..01b99d1 100644 --- a/src/core/tracking/origin_ledger.h +++ b/src/core/tracking/origin_ledger.h @@ -20,6 +20,9 @@ enum class OriginKind { Ingest = 2, Recapture = 3, // regenerated in place from its recorded source recipe Resample = 4, // baked from an instrument's own processing chain + // Kept distinct from Ingest — both bring in a foreign file, but only this one + // can answer "which package did this bank come from" later. + PackageImport = 5, }; // One system-created file's birth record. `relativePath` is the key and is ALWAYS diff --git a/tests/test_origin_ledger.cpp b/tests/test_origin_ledger.cpp index ff2fb8f..f5363fc 100644 --- a/tests/test_origin_ledger.cpp +++ b/tests/test_origin_ledger.cpp @@ -2,9 +2,10 @@ // // The record family behind file tracking. Covers: the relative-paths-only invariant, // exact-string ownership, dedup, insertion order, the JSON round-trip (incl. golden -// byte literals over every persisted enum value), the no-backfill rule, the legacy -// path-only lift, and the Fresh / Loaded / Unreadable / FutureVersion classification -// that keeps never-recorded apart from the two degraded states. +// byte literals over every persisted enum value), the append-a-kind-without-moving-"v" +// rule and its degrade-don't-block twin, the no-backfill rule, the legacy path-only +// lift, and the Fresh / Loaded / Unreadable / FutureVersion classification that keeps +// never-recorded apart from the two degraded states. #include "../src/core/tracking/origin_ledger.h" @@ -162,13 +163,15 @@ static void testSerializeGoldenLiteralPinsEveryPersistedKind() { l.record(rec("bank/ingest.wav", OriginKind::Ingest, "S-b")); l.record(rec("bank/recapture.wav", OriginKind::Recapture, "S-c", "S-a")); l.record(rec("bank/resample.wav", OriginKind::Resample, "S-d", "S-a")); + l.record(rec("bank/import.wav", OriginKind::PackageImport, "S-e")); const std::string expected = "{\"v\":2,\"records\":[" "{\"path\":\"bank/unknown.wav\",\"kind\":0,\"sample\":\"\",\"parent\":\"\"}," "{\"path\":\"bank/capture.wav\",\"kind\":1,\"sample\":\"S-a\",\"parent\":\"\"}," "{\"path\":\"bank/ingest.wav\",\"kind\":2,\"sample\":\"S-b\",\"parent\":\"\"}," "{\"path\":\"bank/recapture.wav\",\"kind\":3,\"sample\":\"S-c\",\"parent\":\"S-a\"}," - "{\"path\":\"bank/resample.wav\",\"kind\":4,\"sample\":\"S-d\",\"parent\":\"S-a\"}" + "{\"path\":\"bank/resample.wav\",\"kind\":4,\"sample\":\"S-d\",\"parent\":\"S-a\"}," + "{\"path\":\"bank/import.wav\",\"kind\":5,\"sample\":\"S-e\",\"parent\":\"\"}" "]}"; CHECK(l.serialize() == expected); @@ -180,6 +183,76 @@ static void testSerializeGoldenLiteralPinsEveryPersistedKind() { CHECK(back->find("bank/ingest.wav")->kind == OriginKind::Ingest); CHECK(back->find("bank/recapture.wav")->kind == OriginKind::Recapture); CHECK(back->find("bank/resample.wav")->kind == OriginKind::Resample); + CHECK(back->find("bank/import.wav")->kind == OriginKind::PackageImport); + CHECK(*back == l); // every field, not just the kind, survives the trip +} + +// Appending a value to the kind vocabulary must NOT move the document version: the +// two rules sit side by side and pull in opposite directions — an unknown kind +// degrades, an unknown "v" blocks. Pinned on the emitted bytes rather than on the +// internal constant, because the byte is what an older build actually reads. +static void testAppendingAKindDoesNotMoveTheDocumentVersion() { + OriginLedger l; + l.record(rec("bank/import.wav", OriginKind::PackageImport, "S-e")); + const std::string json = l.serialize(); + CHECK(json.rfind("{\"v\":2,", 0) == 0); + CHECK(loadLedger(json).status == LedgerStatus::Loaded); + + // The ceiling did not move with it: v3 is still a future document shape. + CHECK(loadLedger("{\"v\":3,\"records\":[]}").status == LedgerStatus::FutureVersion); +} + +// The other half of the append rule: a kind this build does NOT know degrades to +// Unknown while the ledger still loads and the path stays owned. Losing the kind +// detail costs nothing today — no consumer reads it — but an Unreadable here would +// block prune entirely on nothing worse than a vocabulary gap. +static void testUnknownKindDegradesWithoutBlockingTheLedger() { + const std::string blob = + "{\"v\":2,\"records\":[" + "{\"path\":\"bank/next.wav\",\"kind\":6,\"sample\":\"S-1\",\"parent\":\"\"}," + "{\"path\":\"bank/far.wav\",\"kind\":99,\"sample\":\"S-2\",\"parent\":\"\"}," + "{\"path\":\"bank/bogus.wav\",\"kind\":-1,\"sample\":\"S-3\",\"parent\":\"\"}]}"; + + const LedgerLoad load = loadLedger(blob); + CHECK(load.status == LedgerStatus::Loaded); + CHECK(!ledgerDegraded(load.status)); + CHECK(load.ledger.size() == 3); + CHECK(load.ledger.find("bank/next.wav")->kind == OriginKind::Unknown); + CHECK(load.ledger.find("bank/far.wav")->kind == OriginKind::Unknown); + CHECK(load.ledger.find("bank/bogus.wav")->kind == OriginKind::Unknown); + + // Every path still owned, in order — the protection prune reads is untouched. + const std::vector paths = load.ledger.ownedPaths(); + CHECK(paths.size() == 3); + CHECK(paths[0] == "bank/next.wav"); + CHECK(paths[1] == "bank/far.wav"); + CHECK(paths[2] == "bank/bogus.wav"); + + // The rest of the record survives the degrade; only the kind is lost. + CHECK(load.ledger.find("bank/next.wav")->sampleId == "S-1"); +} + +// ownedPaths() is the ONLY thing pruneProtection reads out of a ledger, and it is +// kind-blind: two ledgers agreeing on paths and differing on every kind, new value +// included, yield identical protection input. Adding a kind therefore cannot change +// a prune decision for any existing kind. +static void testOwnedPathsAreKindIndependent() { + const OriginKind kinds[] = {OriginKind::Unknown, OriginKind::Capture, + OriginKind::Ingest, OriginKind::Recapture, + OriginKind::Resample, OriginKind::PackageImport}; + const std::size_t n = sizeof(kinds) / sizeof(kinds[0]); + + OriginLedger baseline; + OriginLedger varied; + for (std::size_t i = 0; i < n; ++i) { + const std::string path = "bank/f" + std::to_string(i) + ".wav"; + baseline.record(rec(path, OriginKind::Unknown, "S")); + varied.record(rec(path, kinds[i], "S")); + } + + CHECK(baseline.ownedPaths() == varied.ownedPaths()); + CHECK(baseline.ownedPaths().size() == n); + CHECK(!(baseline == varied)); // the ledgers really do differ, kind by kind } // contains() is an EXACT-string predicate, never a prefix or substring match — the @@ -383,6 +456,9 @@ int main() { testRoundTripWithLineage(); testRoundTripWithJsonMetacharacters(); testSerializeGoldenLiteralPinsEveryPersistedKind(); + testAppendingAKindDoesNotMoveTheDocumentVersion(); + testUnknownKindDegradesWithoutBlockingTheLedger(); + testOwnedPathsAreKindIndependent(); testMalformedParsesToNullopt(); testTrailingGarbageIsRejected(); testLegacyShapeTypeErrorsAreRejectedButEmptyIsValid(); From edfd7ead4d4c90cde982e5ff29a696ecf7ab69ed Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 08:15:12 -0400 Subject: [PATCH 06/24] Fix the package fs seam: UTF-8 paths, GetUserFileName pickers, exclusive-create landing, rollback arm/disarm Both pickers now ride GetUserFileName (mode 0/1); the "no save picker" premise was false. Landing uses O_EXCL so the create is the existence check, not a TOCTOU pair. --- src/shell/package/CLAUDE.md | 110 ++++++++++------ src/shell/package/CMakeLists.txt | 6 +- src/shell/package/package_io.cpp | 100 ++++++++++++-- src/shell/package/package_io.h | 75 ++++++----- src/shell/package/package_path.h | 19 +++ src/shell/package/package_pickers.cpp | 96 ++++---------- src/shell/package/package_pickers.h | 20 +-- src/shell/package/package_rollback.cpp | 35 +++-- src/shell/package/package_rollback.h | 34 +++-- tests/test_package_io.cpp | 173 ++++++++++++++++++++----- tests/test_package_rollback.cpp | 117 ++++++++++++----- 11 files changed, 532 insertions(+), 253 deletions(-) create mode 100644 src/shell/package/package_path.h diff --git a/src/shell/package/CLAUDE.md b/src/shell/package/CLAUDE.md index 6456fe9..0caf183 100644 --- a/src/shell/package/CLAUDE.md +++ b/src/shell/package/CLAUDE.md @@ -3,59 +3,89 @@ ## Scope The filesystem and dialog acts behind bank-package export/import: streaming package -file I/O (`package_io`), the landed-file journal and its rollback delete -(`package_rollback`), and the platform file pickers (`package_pickers`). This seam -is bytes-only — the package format (magic, manifest, entry layout) is -`core/package`'s business, and the export/import verbs that orchestrate both do not -live here yet. No REAPER project state is touched in this directory: no ext-state -read or write, no undo block, no generation bump — those belong to the verbs. +file I/O plus the file-status and exclusive-create acts (`package_io`), the +UTF-8 path conversion every one of them goes through (`package_path`), the landed-file +journal and its rollback delete (`package_rollback`), and the two file pickers +(`package_pickers`). This seam is bytes-only — the package format (magic, manifest, +entry layout) is `core/package`'s business, and the export/import verbs that +orchestrate both do not live here yet. No REAPER project state is touched in this +directory: no ext-state read or write, no undo block, no generation bump — those +belong to the verbs. ## Invariants -- **Atomic write.** A package accumulates in a `.rsbanktmp` sibling in the - destination directory and reaches the destination only through `commit()`'s - rename (the mono-collapse temp+rename precedent). A failed, aborted, or abandoned - write leaves the destination absent or holding its prior contents — never a - partial `.rsbank`. +- **Paths cross this seam as UTF-8 narrow strings and are converted through + `utf8Path()` before ANY filesystem call.** This is not decoration: on Windows + `std::filesystem` decodes a narrow path through the runtime ANSI code page (measured + `GetACP() == 1252`), so a bare `fs::path(std::string)` turns `café.rsbank` into + `café.rsbank` or fails to open it. Every path a verb hands in or gets back — + including `listFolderFileNames`' results, which use `u8string()` and never + `string()` — is UTF-8. `core/util/file_bytes` has the un-converted shape, which is + why `readFilePayload` reads through this module's own `PackageFileReader` instead. +- **Atomic package write, to the limit of a rename.** A package accumulates in a + `.rsbanktmp` sibling in the destination directory and reaches the destination only + through `commit()`'s rename (the mono-collapse temp+rename precedent). A failed, + aborted, or abandoned write leaves the destination absent or holding its prior + contents. This is process-crash atomic, NOT power-loss atomic: `commit()` flushes + and closes but does not `fsync`/`FlushFileBuffers`, so a power cut can still leave a + renamed-but-unflushed file. Deliberate — an fsync over a whole sample bank is a real + stall, and the failure this design targets is a refused or interrupted export. - **Streaming, both ways — at most ONE entry's payload in memory.** Writes append one payload at a time; reads seek and materialize one range at a time. The claim is structural, not aspirational: every payload crosses this seam as a move-only `PayloadBuffer`, and `PayloadBuffer::alive()` is the seam counter the tests assert against. There is no read-whole-package or write-whole-package entry point; do not add one. -- **The rollback delete is prune's ONE carve-out, cited not restated.** The - citation and the discriminator live at `package_rollback.cpp`'s header. The - journal makes the discriminator structural: only paths its own `writeLandedFile` - successfully created are recorded, and `rollback()` consumes only the record — a - path this import did not write cannot be handed to it. -- **No overwrite of a bank-folder file, ever.** `writeLandedFile` refuses an - existing destination outright; collision handling (auto-rename) is the import - plan's job upstream. The package writer itself DOES replace an existing - destination — the export save dialog's own overwrite confirm is the consent — - and that asymmetry is deliberate. -- **The two pickers are asymmetric, and the asymmetry is real.** Import rides - REAPER's own `GetUserFileNameForRead` (both platforms); export goes native — - Win32 `GetSaveFileNameW` / SWELL `BrowseForSaveFile` — because the always-present - REAPER surface offers no save picker. Do not symmetrize; the newer - `GetUserFileName(mode=0)` alternative and why it is not used are recorded in - `package_pickers.cpp`'s header. +- **An empty `PayloadBuffer` is a failure signal, never an entry.** It is the seam's + one "nothing to work with" branch, so both `PackageFileWriter::appendPayload` and + `writeFileExclusive` refuse it — appending it would let a verb commit framing that + claims bytes nobody wrote. `appendRaw(ptr, 0)` stays tolerated: framing has + legitimate zero-length edges. +- **No overwrite of a bank-folder file, ever — and the create is the check.** + `writeLandedFile` lands through `writeFileExclusive` (`O_EXCL` / `_O_EXCL`), so the + refusal of an occupied path is one atomic act rather than an `exists()` a concurrent + writer could win the race against. Collision handling (auto-rename) remains the + import plan's job upstream. The package writer itself DOES replace an existing + destination — the export save dialog's own overwrite confirm is the consent — and + that asymmetry is deliberate. +- **The rollback delete is prune's ONE carve-out, and only HALF of it is structural.** + The citation and the full discriminator live at `package_rollback.cpp`'s header. + "Did this call create it" is structural: only exclusively-created paths are + recorded, resolved absolute at record time so a later CWD change cannot re-aim the + delete. "Did anything ever reference it" is a **contract the import verb must + honour**: it MUST call `markIndexCommitted()` at the moment it commits the index, + after which `rollback()` refuses and `writeLandedFile` refuses. +- **Both pickers ride `GetUserFileName`** — mode 1 for import, mode 0 for export. + There is no platform split and no fallback: `main.cpp` defines `REAPERAPI_IMPLEMENT` + without `REAPERAPI_MINIMAL` and aborts the extension load if any single name fails + to resolve, so a build that can load us cannot lack it. ## Modules -- `package_io` — the streaming filesystem seam: `PayloadBuffer` (move-only payload + the `alive()` seam counter), `PackageFileWriter` (append-only temp+atomic-rename writer), `PackageFileReader` (seek-and-read one range per call, range-checked against the real file size), `readFilePayload` (one source file as one entry's payload), and `listFolderFileNames` (bare names, sorted, non-recursive, non-throwing). REAPER-free; tested without a DAW. -- `package_rollback` — `LandedFileJournal`: `writeLandedFile` (temp+rename land, recorded on success only, refuses an existing destination and an empty payload) and `rollback` (deletes exactly the recorded set, hard unlink — nothing ever referenced these bytes — tolerating a vanished file). REAPER-free; tested without a DAW. -- `package_pickers` — the two pickers in one platform TU (`#ifdef _WIN32` / `#else swell/swell.h`, the `draw_kit`/`prune_fs` split): `pickPackageForImport` (REAPER read picker) and `pickPackageSavePath` (native save dialog, UTF-8 in/out on Windows). Compile-only until the verbs land; nothing here can be exercised in a unit test. +- `package_path` — header-only; the ONE UTF-8-narrow → `fs::path` conversion, so the encoding contract has a single enforcement point. +- `package_io` — every filesystem act the verbs need: `PayloadBuffer` (move-only payload + the `alive()` seam counter), `PackageFileWriter` (append-only temp+atomic-rename writer), `PackageFileReader` (seek-and-read one range per call, range-checked against the real file size), `readFilePayload` (one source file as one entry's payload), `fileStatus` (Present/Absent/Unreadable — export's refusal message must distinguish the last two, and an empty payload cannot), `writeFileExclusive` (exclusive create + write, self-cleaning on a partial write), and `listFolderFileNames` (bare UTF-8 names, sorted, non-recursive, non-throwing). REAPER-free; tested without a DAW. +- `package_rollback` — `LandedFileJournal`: `writeLandedFile` (exclusive-create land, path resolved absolute, recorded on success only), `markIndexCommitted` (disarms the journal), and `rollback` (deletes exactly the recorded set, hard unlink, tolerating a vanished file; refuses once disarmed). REAPER-free; tested without a DAW. +- `package_pickers` — `pickPackageForImport` and `pickPackageSavePath`, both `GetUserFileName`. Compile-only until the verbs land; nothing here can be exercised in a unit test. ## Gotchas -- A crash mid-write strands the `.rsbanktmp` sibling. It is not a `.rsbank` (no - picker filter matches it), and a later export to the same destination truncates - it — but one stranded in the BANK folder by a mid-import crash is a foreign file - to prune (not owned, so never an orphan) until removed by hand. `[verify — DAW]` - whether the import verb should pre-clean stale `.rsbanktmp` names when it lands. -- The picker `defext`/filter strings are spelled to the Win32 `lpstrDefExt` - convention (no dot) but are `[verify — DAW]` on all three platforms — neither - picker is exercised outside a live REAPER session. +- A crash mid-export strands the `.rsbanktmp` sibling. It is not a `.rsbank` (no + picker filter matches it), and a later export to the same destination truncates it. + A crash mid-import strands a partial bank file under its real name instead — the + land is a direct exclusive create, not temp+rename. Either way the debris was never + recorded in the tracking ledger, so prune sees a foreign file (not owned, never an + orphan) and will not touch it; removal is by hand. `[verify — DAW]` whether the + import verb should pre-clean stale debris when it lands. +- The picker filter and mode arguments are spelled to `GetUserFileName`'s documented + pair format but are `[verify — DAW]` on all three platforms — neither picker is + exercised outside a live REAPER session. `GetUserFileName` also takes no owner + window, so dialog parenting is REAPER's to do; the superseded Win32 path passed + `GetMainHwnd()` explicitly. +- `pickPackageSavePath`'s `suggestedPath` doubles as the dialog's starting directory + when it is a full path. The verbs should seed it from the project directory — + passing a bare name leaves the dialog on REAPER's process working directory, which + is its install or resource path. - `readRange(_, 0)` returns an empty buffer — indistinguishable from failure, by - design (the one "nothing to work with" branch). A genuinely zero-length entry - cannot round-trip through this seam; the format layer must not emit one. + design (the one "nothing to work with" branch). **Cross-track contract, not a local + rule:** a genuinely zero-length entry cannot round-trip through this seam, so + `core/package`'s format layer must not emit one. diff --git a/src/shell/package/CMakeLists.txt b/src/shell/package/CMakeLists.txt index ad716e4..aff4414 100644 --- a/src/shell/package/CMakeLists.txt +++ b/src/shell/package/CMakeLists.txt @@ -3,14 +3,14 @@ # their tests run without a DAW. The export/import verbs that drive all three targets # are not in this directory yet. -reasampler_pure_library(package_io SOURCES package_io.cpp LINK PUBLIC file_bytes) +reasampler_pure_library(package_io SOURCES package_io.cpp) reasampler_test(package_io LINK package_io) reasampler_pure_library(package_rollback SOURCES package_rollback.cpp LINK PUBLIC package_io) reasampler_test(package_rollback LINK package_rollback) -# The platform pickers touch the REAPER API and the native save dialog, so no test -# target can exercise them; declared as a library so both picker paths stay compiled. +# The pickers call the REAPER API, so no test target can exercise them; declared as a +# library so the TU stays compiled. reaper_plugin.h pulls SWELL in on non-Windows. add_library(package_pickers STATIC package_pickers.cpp) target_include_directories(package_pickers PUBLIC ${REASAMPLER_SRC_DIR}) target_include_directories(package_pickers PRIVATE ${SDK_INC} ${WDL_INC}) diff --git a/src/shell/package/package_io.cpp b/src/shell/package/package_io.cpp index 92f645b..901dc3f 100644 --- a/src/shell/package/package_io.cpp +++ b/src/shell/package/package_io.cpp @@ -1,15 +1,24 @@ // package_io.cpp — see package_io.h for the seam's contract. Non-throwing at the -// boundary: every filesystem call uses the error_code form so no filesystem_error +// boundary: every filesystem call uses the error_code form, so no filesystem_error // crosses into a REAPER action body. #include "shell/package/package_io.h" #include #include -#include #include -#include "core/util/file_bytes.h" +#include "shell/package/package_path.h" + +#ifdef _WIN32 +#include +#include +#include +#include +#else +#include +#include +#endif namespace reasampler { @@ -58,8 +67,9 @@ void PayloadBuffer::release() { // --------------------------------------------------------------------------- // PackageFileWriter -PackageFileWriter::PackageFileWriter(std::string destAbsPath) - : destPath_(std::move(destAbsPath)), tempPath_(destPath_ + ".rsbanktmp") { +PackageFileWriter::PackageFileWriter(const std::string& destAbsPath) + : destPath_(utf8Path(destAbsPath)), tempPath_(destPath_) { + tempPath_ += ".rsbanktmp"; // += concatenates; / would make it a child out_.open(tempPath_, std::ios::binary | std::ios::trunc); ok_ = static_cast(out_); } @@ -78,6 +88,10 @@ bool PackageFileWriter::appendRaw(const std::uint8_t* data, std::size_t len) { } bool PackageFileWriter::appendPayload(const PayloadBuffer& payload) { + if (payload.empty()) { + ok_ = false; // the stream is now short of what the framing will claim + return false; + } return appendRaw(payload.data(), payload.size()); } @@ -120,10 +134,11 @@ void PackageFileWriter::abort() { // PackageFileReader PackageFileReader::PackageFileReader(const std::string& srcAbsPath) { + const fs::path path = utf8Path(srcAbsPath); std::error_code ec; - const std::uintmax_t sz = fs::file_size(srcAbsPath, ec); + const std::uintmax_t sz = fs::file_size(path, ec); if (ec) return; - in_.open(srcAbsPath, std::ios::binary); + in_.open(path, std::ios::binary); if (!in_) return; size_ = static_cast(sz); ok_ = true; @@ -147,8 +162,73 @@ PayloadBuffer PackageFileReader::readRange(std::uint64_t offset, std::uint64_t l // --------------------------------------------------------------------------- +// Deliberately NOT core/util/file_bytes: that loader takes an unconverted narrow path +// (see package_path.h), and this seam's own reader already goes through utf8Path. PayloadBuffer readFilePayload(const std::string& absPath) { - return PayloadBuffer(util::readFileBytes(absPath)); + PackageFileReader reader(absPath); + return reader.readRange(0, reader.fileSize()); +} + +FileStatus fileStatus(const std::string& absPath) { + const fs::path path = utf8Path(absPath); + std::error_code ec; + const fs::file_status st = fs::status(path, ec); + // status() reports not_found through the type AND sets ec, so the type is the + // discriminator; an ec with any other type is a real access failure. + if (st.type() == fs::file_type::not_found) return FileStatus::Absent; + if (ec || !fs::is_regular_file(st)) return FileStatus::Unreadable; + std::ifstream probe(path, std::ios::binary); + return probe ? FileStatus::Present : FileStatus::Unreadable; +} + +bool writeFileExclusive(const std::string& absPath, const PayloadBuffer& payload) { + if (payload.empty()) return false; + const fs::path path = utf8Path(absPath); + + int fd = -1; +#ifdef _WIN32 + if (_wsopen_s(&fd, path.wstring().c_str(), + _O_WRONLY | _O_CREAT | _O_EXCL | _O_BINARY, _SH_DENYNO, + _S_IREAD | _S_IWRITE) != 0) { + return false; + } +#else + fd = ::open(path.c_str(), O_WRONLY | O_CREAT | O_EXCL, 0644); +#endif + if (fd < 0) return false; + + bool ok = true; + std::size_t written = 0; + while (written < payload.size()) { + // Chunked because the Windows _write count is an unsigned int, not size_t. + const std::size_t chunk = + std::min(payload.size() - written, 1u << 20); +#ifdef _WIN32 + const int n = _write(fd, payload.data() + written, + static_cast(chunk)); +#else + const ssize_t n = ::write(fd, payload.data() + written, chunk); +#endif + if (n <= 0) { + ok = false; + break; + } + written += static_cast(n); + } + +#ifdef _WIN32 + _close(fd); +#else + ::close(fd); +#endif + + if (!ok) { + // Self-cleanup, not deletion authority: this call created the file moments + // ago and nothing has ever referenced it (prune_fs.cpp's carve-out). + std::error_code ec; + fs::remove(path, ec); + } + return ok; } std::vector listFolderFileNames(const std::string& dirAbsPath) { @@ -156,12 +236,12 @@ std::vector listFolderFileNames(const std::string& dirAbsPath) { std::error_code ec; // Manual iterator form (it.increment(ec)) keeps the loop non-throwing on a // mid-iteration failure, matching prune_fs's enumerate. - fs::directory_iterator it(dirAbsPath, ec); + fs::directory_iterator it(utf8Path(dirAbsPath), ec); for (; !ec && it != fs::directory_iterator{}; it.increment(ec)) { const auto& entry = *it; std::error_code reg_ec; if (!entry.is_regular_file(reg_ec)) continue; // skip subdirs / specials - names.push_back(entry.path().filename().string()); + names.push_back(entry.path().filename().u8string()); // never .string(): ANSI } std::sort(names.begin(), names.end()); return names; diff --git a/src/shell/package/package_io.h b/src/shell/package/package_io.h index 6f36d2d..332a5f1 100644 --- a/src/shell/package/package_io.h +++ b/src/shell/package/package_io.h @@ -1,22 +1,22 @@ -// shell/package/package_io — streaming filesystem seam for bank packages: append one -// payload at a time through a temp-file + atomic-rename writer, seek and read one -// payload at a time back out. Bytes only: what a package contains is core/package's -// business, never this seam's. Blocking I/O — UI-thread actions only, never the -// audio thread. +// shell/package/package_io — every filesystem act the export/import verbs need: +// streaming package read/write, whole-file payload read, folder listing, file status, +// and the exclusive create that lands one bank file. Bytes only — what a package +// contains is core/package's business. Blocking I/O: UI-thread actions only, never +// the audio thread. #pragma once #include +#include #include #include #include namespace reasampler { -// One entry's payload, and the seam counter that makes "never more than one entry in -// memory" assertable: alive() counts every buffer currently holding bytes, so the -// streaming claim is a test CHECK against this counter rather than a memory -// measurement. Move-only — copying a payload would silently double the held bytes. +// One entry's payload. Move-only, because a copy would silently double the bytes the +// seam promises to hold at most one of; alive() is the counter that makes that +// promise assertable instead of aspirational. class PayloadBuffer { public: PayloadBuffer() = default; @@ -30,7 +30,6 @@ public: const std::uint8_t* data() const { return bytes_.data(); } std::size_t size() const { return bytes_.size(); } bool empty() const { return bytes_.empty(); } - const std::vector& bytes() const { return bytes_; } // Buffers currently holding at least one byte, process-wide. static int alive(); @@ -42,26 +41,25 @@ private: bool counted_ = false; }; -// Streaming atomic writer. Bytes accumulate in ".rsbanktmp" beside the -// destination (same directory, so the final rename never crosses a volume); the -// destination itself is touched only by commit()'s rename, so a failed, aborted, or -// abandoned write leaves it absent or holding its prior contents — never a partial -// file. Destruction without commit() aborts and removes the temp. commit() REPLACES -// an existing destination: the export save dialog's own overwrite confirm is the -// consent (the never-overwrite rule for bank-folder files lives in -// LandedFileJournal, upstream of this writer). Neither copyable nor movable, and -// append-only — there is deliberately no way to hand it a whole package at once. +// Streaming atomic writer; paths cross this seam as UTF-8 narrow strings and are held +// as fs::path internally. The temp sibling is created in the DESTINATION's own +// directory so commit()'s rename never crosses a volume — a cross-device rename +// degrades to a copy and stops being atomic. commit() REPLACES an existing +// destination (the deliberate asymmetry against LandedFileJournal; see CLAUDE.md). class PackageFileWriter { public: - explicit PackageFileWriter(std::string destAbsPath); + explicit PackageFileWriter(const std::string& destAbsPath); ~PackageFileWriter(); PackageFileWriter(const PackageFileWriter&) = delete; PackageFileWriter& operator=(const PackageFileWriter&) = delete; bool ok() const { return ok_; } - // Framing/header bytes. False on a failed or already-finished writer. + // Framing/header bytes. False on a failed or already-finished writer. A zero + // length is accepted — framing has legitimate zero-length edges. bool appendRaw(const std::uint8_t* data, std::size_t len); - // One entry's bytes. Same contract as appendRaw. + // One entry's bytes. Also false — and the writer poisoned — on an EMPTY payload: + // empty is this seam's one "nothing to work with" signal, so accepting it would + // let a verb commit a package whose framing claims bytes nobody wrote. bool appendPayload(const PayloadBuffer& payload); // Flush, close, rename over the destination. False (and self-cleaning: the temp // is removed, the destination untouched) on any failure or on a second call. @@ -69,21 +67,21 @@ public: // Close and remove the temp; the destination is never touched. Idempotent. void abort(); - const std::string& destPath() const { return destPath_; } - const std::string& tempPath() const { return tempPath_; } + const std::filesystem::path& destPath() const { return destPath_; } + const std::filesystem::path& tempPath() const { return tempPath_; } private: - std::string destPath_; - std::string tempPath_; + std::filesystem::path destPath_; + std::filesystem::path tempPath_; std::ofstream out_; bool ok_ = false; bool done_ = false; }; // Seek-and-read reader: exactly one payload is materialized per readRange call, and -// there is deliberately no read-whole-file entry point. Empty buffer on ANY failure -// — unopenable file, zero length, out of range, short read — so the caller has one -// "nothing to work with" branch (file_bytes' contract). +// there is deliberately no read-whole-file entry point. Empty buffer on ANY failure — +// unopenable file, zero length, out of range, short read — so the caller has one +// "nothing to work with" branch. Use fileStatus() when the two must be told apart. class PackageFileReader { public: explicit PackageFileReader(const std::string& srcAbsPath); @@ -101,11 +99,24 @@ private: }; // One source file read whole as one entry's payload — a bank file IS the streaming -// unit, so whole-file here is one entry, released before the next is read. Empty on -// any failure, per readFileBytes. +// unit. Empty on any failure, per PackageFileReader. PayloadBuffer readFilePayload(const std::string& absPath); -// Bare file names (regular files only, never a path) in dirAbsPath, sorted so +// Export must tell a missing indexed file from an unreadable one in its refusal +// message; readFilePayload deliberately cannot, since both fail to an empty buffer. +enum class FileStatus { Present, Absent, Unreadable }; +FileStatus fileStatus(const std::string& absPath); + +// Creates absPath and writes the payload, failing if ANYTHING already occupies the +// path. The create IS the existence check (O_EXCL / CREATE_NEW), so nothing can slip +// in between: an exists()-then-write pair would let a file created in that window be +// overwritten and then deleted by a rollback that believes it wrote it. Refuses an +// empty payload, and removes its own partial file on a mid-write failure. Not +// temp+rename — an exclusive rename has no portable spelling, and the debris a crash +// leaves here is unrecorded and unindexed either way. +bool writeFileExclusive(const std::string& absPath, const PayloadBuffer& payload); + +// Bare file names (regular files only, never a path) in dirAbsPath, UTF-8, sorted so // callers see a deterministic order; empty on a missing or unreadable folder. std::vector listFolderFileNames(const std::string& dirAbsPath); diff --git a/src/shell/package/package_path.h b/src/shell/package/package_path.h new file mode 100644 index 0000000..9982496 --- /dev/null +++ b/src/shell/package/package_path.h @@ -0,0 +1,19 @@ +// shell/package/package_path — the ONE narrow-string -> fs::path conversion for this +// seam. std::filesystem decodes a narrow path through the RUNTIME ANSI code page on +// Windows (measured: GetACP() == 1252 here), never UTF-8, so a bare +// fs::path(std::string) turns every non-ASCII path this repo's UTF-8 convention +// produces into mojibake. u8path is the C++17 spelling; it is deprecated in C++20, so +// a standard bump replaces the body here rather than at every call site. + +#pragma once + +#include +#include + +namespace reasampler { + +inline std::filesystem::path utf8Path(const std::string& utf8) { + return std::filesystem::u8path(utf8); +} + +} // namespace reasampler diff --git a/src/shell/package/package_pickers.cpp b/src/shell/package/package_pickers.cpp index b122bd9..75af2a0 100644 --- a/src/shell/package/package_pickers.cpp +++ b/src/shell/package/package_pickers.cpp @@ -1,93 +1,45 @@ -// package_pickers.cpp — see package_pickers.h. Export is native rather than REAPER -// API: the SDK's newer GetUserFileName(mode=0) could save, but it resolves to null -// on older REAPER builds this extension still loads in, while GetSaveFileNameW / -// BrowseForSaveFile are always present. main.cpp owns the REAPER API pointers; this -// TU gets them extern. +// package_pickers.cpp — see package_pickers.h. GetUserFileName cannot be null here: +// main.cpp defines REAPERAPI_IMPLEMENT without REAPERAPI_MINIMAL, so the generated +// resolver walks the FULL table, and it aborts the extension load if any single name +// fails to resolve. A fallback picker would be unreachable code. #include "shell/package/package_pickers.h" -#ifdef _WIN32 -#include -#include -#else -#include "swell/swell.h" -#endif - #define REAPERAPI_MINIMAL -#define REAPERAPI_WANT_GetUserFileNameForRead -#define REAPERAPI_WANT_GetMainHwnd +#define REAPERAPI_WANT_GetUserFileName #include "reaper_plugin_functions.h" namespace reasampler { -bool pickPackageForImport(std::string& outAbsPath) { +namespace { + +// GetUserFileName's documented pair format: label|pattern|label|pattern. +const char kExtList[] = + "ReaSampler bank package (*.rsbank)|*.rsbank|All files (*.*)|*.*"; + +bool runPicker(int mode, const char* caption, const char* initial, + std::string& outAbsPath) { outAbsPath.clear(); - if (!GetUserFileNameForRead) return false; char buf[4096]; buf[0] = '\0'; - // defext spelled without the dot, matching Win32 lpstrDefExt. [verify — DAW] - // whether REAPER's picker applies it to the shown filter. - if (!GetUserFileNameForRead(buf, "Import bank package", "rsbank")) return false; + if (!GetUserFileName(mode, caption, initial, kExtList, buf, + static_cast(sizeof(buf)))) { + return false; + } outAbsPath = buf; return !outAbsPath.empty(); } -#ifdef _WIN32 +} // namespace -bool pickPackageSavePath(const std::string& suggestedFileName, std::string& outAbsPath) { - outAbsPath.clear(); - - wchar_t file[4096]; - file[0] = L'\0'; - if (!suggestedFileName.empty()) { - const int wrote = MultiByteToWideChar(CP_UTF8, 0, suggestedFileName.c_str(), - -1, file, 4096); - if (wrote <= 0) file[0] = L'\0'; // unconvertible suggestion -> empty box - } - - OPENFILENAMEW ofn{}; - ofn.lStructSize = sizeof(ofn); - ofn.hwndOwner = GetMainHwnd ? GetMainHwnd() : nullptr; - ofn.lpstrFilter = - L"ReaSampler bank package (*.rsbank)\0*.rsbank\0All files (*.*)\0*.*\0"; - ofn.lpstrFile = file; - ofn.nMaxFile = 4096; - ofn.lpstrTitle = L"Export bank package"; - ofn.lpstrDefExt = L"rsbank"; - // NOCHANGEDIR: REAPER's process-wide working directory is not ours to move. - ofn.Flags = OFN_OVERWRITEPROMPT | OFN_PATHMUSTEXIST | OFN_NOCHANGEDIR | - OFN_HIDEREADONLY; - if (!GetSaveFileNameW(&ofn)) return false; - - const int need = - WideCharToMultiByte(CP_UTF8, 0, file, -1, nullptr, 0, nullptr, nullptr); - if (need <= 0) return false; - std::string utf8(static_cast(need), '\0'); - WideCharToMultiByte(CP_UTF8, 0, file, -1, &utf8[0], need, nullptr, nullptr); - if (!utf8.empty() && utf8.back() == '\0') utf8.pop_back(); - outAbsPath = std::move(utf8); - return !outAbsPath.empty(); +bool pickPackageForImport(std::string& outAbsPath) { + return runPicker(1, "Import bank package", "", outAbsPath); } -#else // SWELL (macOS / Linux) - -bool pickPackageSavePath(const std::string& suggestedFileName, std::string& outAbsPath) { - outAbsPath.clear(); - // GetOpenFileName-style pair list; the literal's implicit terminator supplies the - // closing double-NUL. [verify — DAW] the exact filter strings SWELL accepts. - static const char kExtList[] = "ReaSampler bank package (*.rsbank)\0*.rsbank\0"; - char fn[4096]; - fn[0] = '\0'; - if (!BrowseForSaveFile("Export bank package", nullptr, - suggestedFileName.empty() ? nullptr - : suggestedFileName.c_str(), - kExtList, fn, static_cast(sizeof(fn)))) { - return false; - } - outAbsPath = fn; - return !outAbsPath.empty(); +bool pickPackageSavePath(const std::string& suggestedPath, std::string& outAbsPath) { + // [verify — DAW] GetUserFileName takes no owner window, so the dialog's parenting + // is REAPER's to do; the previous Win32 path passed GetMainHwnd() explicitly. + return runPicker(0, "Export bank package", suggestedPath.c_str(), outAbsPath); } -#endif - } // namespace reasampler diff --git a/src/shell/package/package_pickers.h b/src/shell/package/package_pickers.h index cdfbc1b..2cd8cf7 100644 --- a/src/shell/package/package_pickers.h +++ b/src/shell/package/package_pickers.h @@ -1,8 +1,7 @@ -// shell/package/package_pickers — the two package file pickers, and they are -// deliberately asymmetric: import rides REAPER's own read picker -// (GetUserFileNameForRead, both platforms); export goes native — Win32 -// GetSaveFileNameW / SWELL BrowseForSaveFile — because the always-present REAPER -// surface offers no save picker. Do not symmetrize; the rationale is in the TU. +// shell/package/package_pickers — the two package file pickers, both on REAPER's own +// GetUserFileName (mode 1 = existing file, mode 0 = new file). No native/platform +// split: the SDK's save mode is not optional on any build that can load this +// extension. Paths in and out are UTF-8, per the REAPER API contract. #pragma once @@ -10,12 +9,13 @@ namespace reasampler { -// REAPER's read picker. True with outAbsPath set iff the user chose a file. +// True with outAbsPath set iff the user chose a file. bool pickPackageForImport(std::string& outAbsPath); -// Native save picker, pre-filled with suggestedFileName (a bare name, e.g. -// "MyBank.rsbank"). True with outAbsPath set iff the user chose a destination; the -// dialog's own overwrite confirm has already run by then. -bool pickPackageSavePath(const std::string& suggestedFileName, std::string& outAbsPath); +// suggestedPath is a bare file name ("MyBank.rsbank") or a full path — a full one +// also seeds the dialog's starting directory, which is how a caller keeps the picker +// off REAPER's process working directory. True with outAbsPath set iff the user chose +// a destination; the dialog's own overwrite confirm has already run by then. +bool pickPackageSavePath(const std::string& suggestedPath, std::string& outAbsPath); } // namespace reasampler diff --git a/src/shell/package/package_rollback.cpp b/src/shell/package/package_rollback.cpp index ddface7..1219292 100644 --- a/src/shell/package/package_rollback.cpp +++ b/src/shell/package/package_rollback.cpp @@ -1,34 +1,45 @@ -// package_rollback.cpp — see package_rollback.h. The rollback delete below runs -// under the ONE carve-out from prune's exclusive file-deletion authority, stated at -// src/shell/persist/prune_fs.cpp:5-11; it satisfies that discriminator by -// construction — every recorded path was created by this journal's own -// writeLandedFile, and the abandoned index mutation means nothing ever referenced it. +// package_rollback.cpp — see package_rollback.h. The rollback delete below runs under +// the ONE carve-out from prune's exclusive file-deletion authority, stated at +// src/shell/persist/prune_fs.cpp:5-11. That discriminator has two clauses and this +// journal makes only the FIRST structural: "did this call create it" is guaranteed by +// recording exclusively-created paths, but "did anything ever reference it" is a +// claim about the caller's ordering — hence markIndexCommitted(), which the import +// verb must fire at the index commit so a later rollback() refuses instead of +// deleting indexed files. #include "shell/package/package_rollback.h" #include +#include "shell/package/package_path.h" + namespace reasampler { namespace fs = std::filesystem; -bool LandedFileJournal::writeLandedFile(const std::string& absPath, +bool LandedFileJournal::writeLandedFile(const std::string& destPath, const PayloadBuffer& payload) { - if (payload.empty()) return false; + if (indexCommitted_) return false; + std::error_code ec; - if (fs::exists(absPath, ec) || ec) return false; - PackageFileWriter writer(absPath); - if (!writer.appendPayload(payload)) return false; // dtor aborts; temp removed - if (!writer.commit()) return false; + const fs::path resolved = fs::absolute(utf8Path(destPath), ec); + if (ec) return false; + const std::string absPath = resolved.u8string(); + + if (!writeFileExclusive(absPath, payload)) return false; paths_.push_back(absPath); return true; } RollbackResult LandedFileJournal::rollback() { RollbackResult result; + if (indexCommitted_) { + result.refused = true; + return result; + } for (const std::string& path : paths_) { std::error_code ec; - const bool removed = fs::remove(path, ec); + const bool removed = fs::remove(utf8Path(path), ec); if (removed) ++result.deletedCount; else if (ec) ++result.failedCount; else ++result.alreadyAbsentCount; // no error, nothing there diff --git a/src/shell/package/package_rollback.h b/src/shell/package/package_rollback.h index a72f89c..cfc663d 100644 --- a/src/shell/package/package_rollback.h +++ b/src/shell/package/package_rollback.h @@ -1,8 +1,7 @@ // shell/package/package_rollback — the files ONE import call has landed, as a // journal: writes record themselves on success, and rollback() deletes exactly what -// is recorded — a path this import did not write is structurally impossible to hand -// it. The deletion carve-out this satisfies is cited at package_rollback.cpp's -// header. +// is recorded. The deletion carve-out this satisfies, and the half of it the caller +// still owns, are at package_rollback.cpp's header. #pragma once @@ -17,22 +16,30 @@ struct RollbackResult { int deletedCount = 0; int alreadyAbsentCount = 0; // vanished between land and rollback — not a failure int failedCount = 0; // locked / permission — recorded, never thrown + bool refused = false; // markIndexCommitted() ran: nothing was deleted }; -// The evidence for the rollback discriminator: only paths this journal's own -// writeLandedFile successfully created are recorded, so rollback() can never touch a -// byte this import did not write. class LandedFileJournal { public: - // Lands one payload at absPath through the atomic temp+rename writer and records - // the path on success. REFUSES an existing destination — a bank-folder file is - // never overwritten; collision handling is the import plan's job, upstream. An - // empty payload is refused too: it signals an upstream read failure, never a - // real entry. - bool writeLandedFile(const std::string& absPath, const PayloadBuffer& payload); + // Lands one payload at destPath through the exclusive create (which refuses an + // occupied path outright — a bank-folder file is never overwritten, and collision + // handling is the import plan's job upstream) and records it on success. An empty + // payload is refused, per writeFileExclusive. Relative paths are resolved against + // the process CWD before the write, so the journal's record is always absolute + // and a later CWD change cannot re-aim the delete. Refused once + // markIndexCommitted() has run. + bool writeLandedFile(const std::string& destPath, const PayloadBuffer& payload); + + // Disarms the journal: the index mutation these files back is committed, so they + // are now referenced bytes and the carve-out no longer covers them. This is the + // half of prune's discriminator the journal cannot make structural on its own — + // the import verb MUST call it at the moment the index is committed. + void markIndexCommitted() { indexCommitted_ = true; } + bool indexCommitted() const { return indexCommitted_; } // Deletes exactly the recorded files and clears the journal, so a second call is - // a no-op. Hard unlink, not trash: nothing ever referenced these bytes. + // a no-op. Hard unlink, not trash: nothing ever referenced these bytes. Refuses + // (deleting nothing, keeping the record) once markIndexCommitted() has run. RollbackResult rollback(); const std::vector& landedPaths() const { return paths_; } @@ -40,6 +47,7 @@ public: private: std::vector paths_; + bool indexCommitted_ = false; }; } // namespace reasampler diff --git a/tests/test_package_io.cpp b/tests/test_package_io.cpp index 7f10b73..a533c72 100644 --- a/tests/test_package_io.cpp +++ b/tests/test_package_io.cpp @@ -1,12 +1,11 @@ -// Standalone tests for shell/package/package_io — no REAPER, no framework. Pins the -// two properties the seam exists for: an interrupted or failed write leaves the -// destination absent or holding its prior contents (failure injected at the writer -// seam — abandonment, open failure, rename failure), and a multi-entry round trip -// holds at most one entry's payload, asserted against PayloadBuffer::alive() — the -// seam counter — rather than a memory measurement. +// Standalone tests for shell/package/package_io — no REAPER, no framework. Failure is +// injected at the writer seam (abandonment, open failure, rename failure) rather than +// simulated, and the streaming claim is asserted against PayloadBuffer::alive(). #include "../src/shell/package/package_io.h" +#include "../src/shell/package/package_path.h" +#include #include #include #include @@ -28,19 +27,31 @@ static std::vector patternBytes(std::size_t n, std::uint8_t seed) return v; } +static bool sameBytes(const PayloadBuffer& p, const std::vector& want) { + return p.size() == want.size() && + (want.empty() || std::equal(want.begin(), want.end(), p.data())); +} + static void writeScratchFile(const std::string& path, const std::vector& bytes) { - std::ofstream f(path, std::ios::binary | std::ios::trunc); + std::ofstream f(utf8Path(path), std::ios::binary | std::ios::trunc); f.write(reinterpret_cast(bytes.data()), static_cast(bytes.size())); } static std::vector readAll(const std::string& path) { - std::ifstream f(path, std::ios::binary); + std::ifstream f(utf8Path(path), std::ios::binary); return std::vector(std::istreambuf_iterator(f), std::istreambuf_iterator()); } +static bool exists(const std::string& path) { return fs::exists(utf8Path(path)); } + +static void removeQuietly(const std::string& path) { + std::error_code ec; + fs::remove(utf8Path(path), ec); +} + static void testPayloadCounterTracksMovesNotCopies() { CHECK(PayloadBuffer::alive() == 0); { @@ -81,7 +92,7 @@ static void testStreamingRoundTripHoldsOnePayload() { std::uint64_t offset = header.size(); for (std::size_t i = 0; i < entries.size(); ++i) { PayloadBuffer p = readFilePayload(srcs[i]); - CHECK(p.bytes() == entries[i]); + CHECK(sameBytes(p, entries[i])); CHECK(PayloadBuffer::alive() == 1); // exactly one entry in memory CHECK(writer.appendPayload(p)); layout.emplace_back(offset, p.size()); @@ -91,8 +102,8 @@ static void testStreamingRoundTripHoldsOnePayload() { CHECK(writer.commit()); CHECK(!writer.commit()); // a second commit is refused } - CHECK(!fs::exists(dest + ".rsbanktmp")); - CHECK(fs::exists(dest)); + CHECK(!exists(dest + ".rsbanktmp")); + CHECK(exists(dest)); { // Scoped: the reader holds the file open, and Windows refuses to delete an @@ -103,14 +114,36 @@ static void testStreamingRoundTripHoldsOnePayload() { for (std::size_t i = 0; i < entries.size(); ++i) { PayloadBuffer p = reader.readRange(layout[i].first, layout[i].second); CHECK(PayloadBuffer::alive() == 1); // one entry per readRange, no more - CHECK(p.bytes() == entries[i]); + CHECK(sameBytes(p, entries[i])); } CHECK(PayloadBuffer::alive() == 0); } - std::error_code ec; - for (const std::string& s : srcs) fs::remove(s, ec); - fs::remove(dest, ec); + for (const std::string& s : srcs) removeQuietly(s); + removeQuietly(dest); +} + +static void testEmptyPayloadIsRefusedAndPoisonsTheWriter() { + // The failure this guards: an unreadable source yields an empty payload, and a + // verb that trusted a `true` here would commit framing claiming bytes nobody wrote. + const std::string dest = "pkg_io_emptypayload.rsbank"; + { + PackageFileWriter writer(dest); + const std::vector head = patternBytes(4, 1); + CHECK(writer.appendRaw(head.data(), head.size())); + CHECK(!writer.appendPayload(PayloadBuffer{})); + CHECK(!writer.ok()); + CHECK(!writer.commit()); // the short stream can never reach the destination + } + CHECK(!exists(dest)); + CHECK(!exists(dest + ".rsbanktmp")); + // A zero-length appendRaw stays tolerated — framing has legitimate empty edges. + { + PackageFileWriter writer(dest); + CHECK(writer.appendRaw(nullptr, 0)); + CHECK(writer.ok()); + writer.abort(); + } } static void testAbandonedWriteLeavesNoDestination() { @@ -121,8 +154,8 @@ static void testAbandonedWriteLeavesNoDestination() { CHECK(writer.appendRaw(some.data(), some.size())); // no commit — destruction is the injected interruption } - CHECK(!fs::exists(dest)); - CHECK(!fs::exists(dest + ".rsbanktmp")); + CHECK(!exists(dest)); + CHECK(!exists(dest + ".rsbanktmp")); } static void testAbortPreservesPriorContents() { @@ -138,9 +171,26 @@ static void testAbortPreservesPriorContents() { CHECK(!writer.commit()); } CHECK(readAll(dest) == prior); - CHECK(!fs::exists(dest + ".rsbanktmp")); - std::error_code ec; - fs::remove(dest, ec); + CHECK(!exists(dest + ".rsbanktmp")); + removeQuietly(dest); +} + +static void testCommitReplacesAnExistingFile() { + // The module's one deliberate asymmetry against LandedFileJournal's never-overwrite + // rule: the save dialog's overwrite confirm is the consent, so commit() replaces. + const std::string dest = "pkg_io_replace.rsbank"; + const std::vector prior = patternBytes(40, 0x11); + const std::vector fresh = patternBytes(7, 0x22); + writeScratchFile(dest, prior); + { + PackageFileWriter writer(dest); + CHECK(writer.ok()); + CHECK(writer.appendRaw(fresh.data(), fresh.size())); + CHECK(writer.commit()); + } + CHECK(readAll(dest) == fresh); + CHECK(!exists(dest + ".rsbanktmp")); + removeQuietly(dest); } static void testOpenFailureIsInert() { @@ -150,7 +200,7 @@ static void testOpenFailureIsInert() { const std::vector some = patternBytes(8, 1); CHECK(!writer.appendRaw(some.data(), some.size())); CHECK(!writer.commit()); - CHECK(!fs::exists("pkg_io_no_such_dir")); + CHECK(!exists("pkg_io_no_such_dir")); } static void testCommitRenameFailureSelfCleans() { @@ -158,7 +208,7 @@ static void testCommitRenameFailureSelfCleans() { // injected commit failure, not a simulated one. const std::string dest = "pkg_io_dir.rsbank"; std::error_code ec; - fs::create_directory(dest, ec); + fs::create_directory(utf8Path(dest), ec); CHECK(!ec); { PackageFileWriter writer(dest); @@ -167,9 +217,9 @@ static void testCommitRenameFailureSelfCleans() { CHECK(writer.appendRaw(some.data(), some.size())); CHECK(!writer.commit()); } - CHECK(fs::is_directory(dest)); // prior state intact - CHECK(!fs::exists(dest + ".rsbanktmp")); - fs::remove(dest, ec); + CHECK(fs::is_directory(utf8Path(dest))); // prior state intact + CHECK(!exists(dest + ".rsbanktmp")); + fs::remove(utf8Path(dest), ec); } static void testReaderEdges() { @@ -188,38 +238,97 @@ static void testReaderEdges() { CHECK(reader.readRange(5, 10).empty()); // past the end CHECK(reader.readRange(10, 1).empty()); // starts at the end CHECK(reader.readRange(0, 0).empty()); // zero length is failure, one branch - const PayloadBuffer slice = reader.readRange(2, 3); - CHECK(slice.bytes() == - std::vector(bytes.begin() + 2, bytes.begin() + 5)); + CHECK(sameBytes(reader.readRange(2, 3), + std::vector(bytes.begin() + 2, bytes.begin() + 5))); } + removeQuietly(path); +} + +static void testFileStatusSeparatesAbsentFromUnreadable() { + CHECK(fileStatus("pkg_io_no_such_file.bin") == FileStatus::Absent); + + const std::string path = "pkg_io_status.bin"; + writeScratchFile(path, patternBytes(4, 1)); + CHECK(fileStatus(path) == FileStatus::Present); + removeQuietly(path); + + // A directory in a file's place is not a readable file — the export message must + // not report it as simply missing. + const std::string dir = "pkg_io_status_dir"; std::error_code ec; - fs::remove(path, ec); + fs::create_directory(utf8Path(dir), ec); + CHECK(fileStatus(dir) == FileStatus::Unreadable); + fs::remove(utf8Path(dir), ec); +} + +static void testNonAsciiPathsRoundTripAsUtf8() { + // "café" in UTF-8. Windows decodes a narrow std::filesystem path through the ANSI + // code page, so an unconverted seam lands "café" or fails outright. + const std::string dir = "pkg_io_caf\xC3\xA9_dir"; + const std::string name = "caf\xC3\xA9.rsbank"; + std::error_code ec; + fs::create_directory(utf8Path(dir), ec); + CHECK(!ec); + + const std::string dest = dir + "/" + name; + const std::vector bytes = patternBytes(24, 0x5A); + { + PackageFileWriter writer(dest); + CHECK(writer.ok()); + CHECK(writer.appendRaw(bytes.data(), bytes.size())); + CHECK(writer.commit()); + } + CHECK(exists(dest)); + CHECK(fileStatus(dest) == FileStatus::Present); + CHECK(sameBytes(readFilePayload(dest), bytes)); + // The listing must hand the name back in the same encoding it was given. + CHECK(listFolderFileNames(dir) == (std::vector{name})); + + fs::remove_all(utf8Path(dir), ec); +} + +static void testWriteFileExclusiveRefusesAnOccupiedPath() { + const std::string path = "pkg_io_excl.bin"; + const std::vector mine = patternBytes(12, 3); + CHECK(writeFileExclusive(path, PayloadBuffer(mine))); + CHECK(readAll(path) == mine); + // The create IS the check: a second call cannot replace the first file's bytes. + CHECK(!writeFileExclusive(path, PayloadBuffer(patternBytes(9, 8)))); + CHECK(readAll(path) == mine); + CHECK(!writeFileExclusive("pkg_io_excl_empty.bin", PayloadBuffer{})); + CHECK(!exists("pkg_io_excl_empty.bin")); + removeQuietly(path); } static void testListFolderFileNames() { const std::string dir = "pkg_io_listdir"; std::error_code ec; - fs::create_directory(dir, ec); + fs::create_directory(utf8Path(dir), ec); writeScratchFile(dir + "/b.bin", patternBytes(2, 1)); writeScratchFile(dir + "/a.bin", patternBytes(2, 2)); - fs::create_directory(dir + "/sub", ec); + fs::create_directory(utf8Path(dir + "/sub"), ec); writeScratchFile(dir + "/sub/c.bin", patternBytes(2, 3)); const std::vector names = listFolderFileNames(dir); CHECK(names == (std::vector{"a.bin", "b.bin"})); // sorted, bare, non-recursive CHECK(listFolderFileNames("pkg_io_no_such_dir").empty()); - fs::remove_all(dir, ec); + fs::remove_all(utf8Path(dir), ec); } int main() { testPayloadCounterTracksMovesNotCopies(); testStreamingRoundTripHoldsOnePayload(); + testEmptyPayloadIsRefusedAndPoisonsTheWriter(); testAbandonedWriteLeavesNoDestination(); testAbortPreservesPriorContents(); + testCommitReplacesAnExistingFile(); testOpenFailureIsInert(); testCommitRenameFailureSelfCleans(); testReaderEdges(); + testFileStatusSeparatesAbsentFromUnreadable(); + testNonAsciiPathsRoundTripAsUtf8(); + testWriteFileExclusiveRefusesAnOccupiedPath(); testListFolderFileNames(); if (g_fail == 0) std::printf("package_io: all tests passed\n"); diff --git a/tests/test_package_rollback.cpp b/tests/test_package_rollback.cpp index 6c28bfa..8ddeb32 100644 --- a/tests/test_package_rollback.cpp +++ b/tests/test_package_rollback.cpp @@ -1,11 +1,12 @@ // Standalone tests for shell/package/package_rollback — no REAPER, no framework. -// Pins the discriminator's mechanics: a file is recorded only when this journal's -// own write landed it, a pre-existing destination is refused untouched, and -// rollback deletes exactly the recorded set — a bystander file beside them stays, -// and a vanished file is tolerated rather than failed. +// Pins both halves of the deletion discriminator: the structural half (only an +// exclusively-created path is recorded, and the record is absolute so a CWD change +// cannot re-aim it) and the contract half (markIndexCommitted disarms rollback). #include "../src/shell/package/package_rollback.h" +#include "../src/shell/package/package_path.h" +#include #include #include #include @@ -19,6 +20,11 @@ static int g_fail = 0; #define CHECK(cond) do { if(!(cond)) { \ std::printf("FAIL line %d: %s\n", __LINE__, #cond); ++g_fail; } } while(0) +// The journal records absolute paths, so every expectation is built the same way. +static std::string scratch(const std::string& name) { + return (fs::current_path() / utf8Path(name)).u8string(); +} + static std::vector patternBytes(std::size_t n, std::uint8_t seed) { std::vector v(n); for (std::size_t i = 0; i < n; ++i) @@ -28,32 +34,55 @@ static std::vector patternBytes(std::size_t n, std::uint8_t seed) static void writeScratchFile(const std::string& path, const std::vector& bytes) { - std::ofstream f(path, std::ios::binary | std::ios::trunc); + std::ofstream f(utf8Path(path), std::ios::binary | std::ios::trunc); f.write(reinterpret_cast(bytes.data()), static_cast(bytes.size())); } static std::vector readAll(const std::string& path) { - std::ifstream f(path, std::ios::binary); + std::ifstream f(utf8Path(path), std::ios::binary); return std::vector(std::istreambuf_iterator(f), std::istreambuf_iterator()); } +static bool exists(const std::string& path) { return fs::exists(utf8Path(path)); } + +static void removeQuietly(const std::string& path) { + std::error_code ec; + fs::remove(utf8Path(path), ec); +} + static void testLandRecordsOnSuccessOnly() { LandedFileJournal journal; - const std::string path = "rb_land.bin"; + const std::string path = scratch("rb_land.bin"); const std::vector bytes = patternBytes(32, 1); CHECK(journal.writeLandedFile(path, PayloadBuffer(bytes))); CHECK(readAll(path) == bytes); - CHECK(journal.landedPaths() == (std::vector{path})); - CHECK(!fs::exists(path + ".rsbanktmp")); + CHECK(journal.landedPaths().size() == 1); + CHECK(!exists(path + ".rsbanktmp")); // the land is a direct exclusive create journal.rollback(); - CHECK(!fs::exists(path)); + CHECK(!exists(path)); // the recorded path denoted the file we asked for +} + +static void testRelativeInputIsRecordedAbsolute() { + // The hazard: a bare name recorded verbatim, then a CWD change, and rollback + // unlinks whatever now sits at that name in the new directory. + LandedFileJournal journal; + CHECK(journal.writeLandedFile("rb_relative.bin", PayloadBuffer(patternBytes(8, 4)))); + CHECK(journal.landedPaths().size() == 1); + const std::string recorded = journal.landedPaths().front(); + CHECK(utf8Path(recorded).is_absolute()); + std::error_code ec; + // Absolute AND still the same file — a spelling check alone would not prove that. + CHECK(fs::equivalent(utf8Path(recorded), utf8Path(scratch("rb_relative.bin")), ec)); + CHECK(!ec); + journal.rollback(); + CHECK(!exists(scratch("rb_relative.bin"))); } static void testExistingDestinationRefusedUntouched() { LandedFileJournal journal; - const std::string path = "rb_existing.bin"; + const std::string path = scratch("rb_existing.bin"); const std::vector original = patternBytes(16, 0x60); writeScratchFile(path, original); @@ -63,46 +92,49 @@ static void testExistingDestinationRefusedUntouched() { const RollbackResult result = journal.rollback(); CHECK(result.deletedCount == 0); - CHECK(fs::exists(path)); // rollback cannot touch a file it did not write - std::error_code ec; - fs::remove(path, ec); + CHECK(exists(path)); // rollback cannot touch a file it did not write + removeQuietly(path); } static void testEmptyPayloadRefused() { LandedFileJournal journal; - CHECK(!journal.writeLandedFile("rb_empty.bin", PayloadBuffer{})); - CHECK(!fs::exists("rb_empty.bin")); + const std::string path = scratch("rb_empty.bin"); + CHECK(!journal.writeLandedFile(path, PayloadBuffer{})); + CHECK(!exists(path)); CHECK(journal.empty()); } static void testRollbackDeletesExactlyTheRecordedSet() { LandedFileJournal journal; - CHECK(journal.writeLandedFile("rb_a.bin", PayloadBuffer(patternBytes(8, 1)))); - CHECK(journal.writeLandedFile("rb_c.bin", PayloadBuffer(patternBytes(8, 2)))); - writeScratchFile("rb_bystander.bin", patternBytes(8, 3)); // not journal-written + const std::string a = scratch("rb_a.bin"); + const std::string c = scratch("rb_c.bin"); + const std::string bystander = scratch("rb_bystander.bin"); + CHECK(journal.writeLandedFile(a, PayloadBuffer(patternBytes(8, 1)))); + CHECK(journal.writeLandedFile(c, PayloadBuffer(patternBytes(8, 2)))); + writeScratchFile(bystander, patternBytes(8, 3)); // not journal-written const RollbackResult result = journal.rollback(); CHECK(result.deletedCount == 2); CHECK(result.alreadyAbsentCount == 0); CHECK(result.failedCount == 0); - CHECK(!fs::exists("rb_a.bin")); - CHECK(!fs::exists("rb_c.bin")); - CHECK(fs::exists("rb_bystander.bin")); // exactly the given files, nothing else + CHECK(!result.refused); + CHECK(!exists(a)); + CHECK(!exists(c)); + CHECK(exists(bystander)); // exactly the given files, nothing else CHECK(journal.empty()); const RollbackResult second = journal.rollback(); // cleared: a no-op CHECK(second.deletedCount == 0); - CHECK(fs::exists("rb_bystander.bin")); - std::error_code ec; - fs::remove("rb_bystander.bin", ec); + CHECK(exists(bystander)); + removeQuietly(bystander); } static void testVanishedFileIsToleratedNotFailed() { LandedFileJournal journal; - CHECK(journal.writeLandedFile("rb_gone.bin", PayloadBuffer(patternBytes(8, 1)))); - std::error_code ec; - fs::remove("rb_gone.bin", ec); // vanished between land and rollback - CHECK(!ec); + const std::string path = scratch("rb_gone.bin"); + CHECK(journal.writeLandedFile(path, PayloadBuffer(patternBytes(8, 1)))); + removeQuietly(path); // vanished between land and rollback + CHECK(!exists(path)); const RollbackResult result = journal.rollback(); CHECK(result.deletedCount == 0); @@ -110,12 +142,39 @@ static void testVanishedFileIsToleratedNotFailed() { CHECK(result.failedCount == 0); } +static void testIndexCommitDisarmsRollback() { + // Once the index references these files the carve-out no longer covers them, so a + // late failure in the verb must not be able to delete indexed bytes. + LandedFileJournal journal; + const std::string path = scratch("rb_committed.bin"); + const std::vector bytes = patternBytes(8, 1); + CHECK(journal.writeLandedFile(path, PayloadBuffer(bytes))); + + journal.markIndexCommitted(); + CHECK(journal.indexCommitted()); + + const RollbackResult result = journal.rollback(); + CHECK(result.refused); + CHECK(result.deletedCount == 0); + CHECK(readAll(path) == bytes); // untouched + CHECK(!journal.empty()); // the record survives the refusal + + // Landing more files after the commit would produce unrollbackable state. + const std::string late = scratch("rb_late.bin"); + CHECK(!journal.writeLandedFile(late, PayloadBuffer(patternBytes(8, 2)))); + CHECK(!exists(late)); + + removeQuietly(path); +} + int main() { testLandRecordsOnSuccessOnly(); + testRelativeInputIsRecordedAbsolute(); testExistingDestinationRefusedUntouched(); testEmptyPayloadRefused(); testRollbackDeletesExactlyTheRecordedSet(); testVanishedFileIsToleratedNotFailed(); + testIndexCommitDisarmsRollback(); if (g_fail == 0) std::printf("package_rollback: all tests passed\n"); else std::printf("package_rollback: %d CHECK(s) FAILED\n", g_fail); From 3909b1072cd563764a7464462b77bdfe8857ef4a Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 08:44:24 -0400 Subject: [PATCH 07/24] Close the RSBK name-collision class: ASCII case folding, UTF-8 well-formedness, nested-path traversal All three are format-locked and validated on encode and decode. Repeated known keys now reject at the root and inside an entry rather than last-wins. --- src/core/package/CLAUDE.md | 52 ++++++++-- src/core/package/package_format.cpp | 79 +++++++++++++-- src/core/package/package_format.h | 42 +++++--- src/core/package/package_manifest.cpp | 30 ++++-- src/core/package/package_manifest.h | 21 ++-- tests/test_bank_package.cpp | 40 +++++++- tests/test_package_format.cpp | 113 ++++++++++++++++++++- tests/test_package_manifest.cpp | 139 ++++++++++++++++++++++---- 8 files changed, 442 insertions(+), 74 deletions(-) diff --git a/src/core/package/CLAUDE.md b/src/core/package/CLAUDE.md index 373b512..3c37197 100644 --- a/src/core/package/CLAUDE.md +++ b/src/core/package/CLAUDE.md @@ -25,13 +25,22 @@ landing after the format. header (so the refusal can name the writer's semver and version) and nothing else — no manifest, no layout, no half-success. The fields through the writer semver are FROZEN for all future versions to keep that refusal producible. -- **Path expression is structurally impossible.** Entry names are bare file - names (`isValidEntryName`: no separators, no `..` component, no - drive/UNC/rooted form, no control bytes, no Windows-reserved character, no - trailing dot/space, no DOS device name), enforced on encode AND decode - because a package can arrive from anywhere. There is no field in the format - capable of expressing a path. The rule set is the full authority; see - `package_format.h`'s doc comment for the itemized list. +- **Every name and path in the format is validated on encode AND decode**, + because a package can arrive from anywhere. Three rules, all in + `package_format`, whose doc comments are the itemized authority: + - `isValidEntryName` — a payload's name is a bare file name (no separators, + no `..` component, no drive/UNC/rooted form, no control bytes, no + Windows-reserved character, no trailing dot/space, no DOS device name, + well-formed UTF-8 only). Path expression is impossible in this field. + - `sameEntryName` — two entry names differing only by ASCII case are ONE + name. Windows and macOS's default APFS would extract them onto a single + file, and a bank authored on a case-sensitive filesystem produces the pair + honestly. + - `isValidNestedSamplePath` — the nested `Sample::relativePath` IS a path by + design, and is the one field here that can express one. It refuses a `..` + component and every absolute form; `BankModel::add` checks only the latter, + so traversal would otherwise reach a future `import_plan` inside a record + the format vouched for. - **Framing only, never a payload.** `bank_package` produces header bytes and an ordered `{name, offset, length}` layout; it never holds, copies, or hashes an entry's audio. `decodePackage` proves prefix + payload lengths equal the @@ -48,8 +57,8 @@ landing after the format. - `package_format` — the contract: magic, `kPackageFormatVersion` / `kPackageMinReaderVersion`, the ladder comment, the three-way - `classifyPackageVersion` (`Readable` / `TooNew` / `Malformed`), the - entry-name rule, and `PackageHeader`. + `classifyPackageVersion` (`Readable` / `TooNew` / `Malformed`), the three + naming rules above, and `PackageHeader`. - `package_manifest` — the manifest model (`PackageEntry` / `PackageManifest`) and its JSON codec. Per entry: bare name, byte length, and a whole-file `capture::hashBytes` digest (deliberately NOT `hashWavContent`, which skips @@ -87,11 +96,36 @@ landing after the format. region" — both refuse whole and write nothing, so the safety property is unchanged, only the message. `classifyPackageVersion` and the frozen-region `TooNew` path are unaffected; this is the post-manifest-parse branch only. +- **The parse branch is the ONLY one that relabels**, deliberately: a newer + package that trips the manifest cap, a short manifest read, the layout + overflow, or the exact-size proof still reports `Malformed` even with + `formatVersion` above ours. The size proof clearly should — "install 1.9.0" + does not fix a truncated download — and the other three are indistinguishable + from ordinary corruption at the point they fail. Don't "complete" the relabel + across them for symmetry; the split is the answer, not an omission. - The format carries no algorithm tag for `byteHash` — it is FNV-1a (`capture::hashBytes`) implicitly. Changing the digest algorithm is a `minReaderVersion` bump, not additive: an old reader would otherwise compare a stored digest against bytes hashed the new way and silently misjudge corruption. +- **Obligation on the export track: sanitize, don't relay the refusal.** + `serializeManifest` returns one indistinguishable `nullopt` for every rejection + — an unrepresentable name, a case-folded collision, a traversing nested path, a + zero-length entry, a record `BankModel::add` refuses — and most of the naming + rules are Windows'. A bank ingested on macOS/Linux legitimately holds + `Hit?.wav`, `snare .wav`, or two names differing only by case, and a nested + `relativePath` is only checked for the absolute forms where it is written. + Relaying the `nullopt` makes ONE such file an unactionable total failure of the + whole export. `export_plan` must map bank entries to package + names that satisfy these rules (and disambiguate case-folded collisions) before + calling this layer; the codec's refusal is the backstop, not the user-facing + behaviour. +- **NFC/NFD normalization collisions are accepted, not solved.** macOS compares + file names normalization-insensitively, so the NFC and NFD spellings of one + accented name are two manifest entries that extract onto one file — the same + collision class as the ASCII case fold, which `sameEntryName` does catch. A + table-free fix does not exist, and restricting names to ASCII would be + genuinely over-strict for non-English users. Left open knowingly. - **Cross-module contract with `src/shell/package`:** a genuinely zero-length entry cannot round-trip through the filesystem seam there (`appendPayload` refuses an empty payload — an empty buffer signals an upstream read failure, diff --git a/src/core/package/package_format.cpp b/src/core/package/package_format.cpp index d07aaec..76808ac 100644 --- a/src/core/package/package_format.cpp +++ b/src/core/package/package_format.cpp @@ -1,6 +1,8 @@ #include "core/package/package_format.h" -#include +#include + +#include "core/util/relative_path.h" namespace reasampler::package { @@ -14,21 +16,64 @@ PackageReadability classifyPackageVersion(std::uint32_t formatVersion, namespace { +// Hand-rolled rather than std::tolower: that fold is locale-dependent, so two +// machines reading the same package could disagree on which names collide. +char lowerAscii(unsigned char c) { + return (c >= 'A' && c <= 'Z') ? static_cast(c - 'A' + 'a') : static_cast(c); +} + // Windows device names claim the whole entry regardless of extension // (CON, CON.wav, con.WAV are all the same reserved device) — checked against -// the portion before the first dot only. +// the portion before the first dot only. The trailing three pairs are the UTF-8 +// spellings of COM¹/COM²/COM³/LPT¹/LPT²/LPT³: Windows reads those ISO 8859-1 +// superscripts as digits in a device name. COM0/LPT0 are NOT reserved. bool isDosDeviceName(const std::string& name) { std::string base = name.substr(0, name.find('.')); - for (char& c : base) c = static_cast(std::toupper(static_cast(c))); + for (char& c : base) c = lowerAscii(static_cast(c)); static const std::string kReserved[] = { - "CON", "PRN", "AUX", "NUL", - "COM1", "COM2", "COM3", "COM4", "COM5", "COM6", "COM7", "COM8", "COM9", - "LPT1", "LPT2", "LPT3", "LPT4", "LPT5", "LPT6", "LPT7", "LPT8", "LPT9", + "con", "prn", "aux", "nul", + "com1", "com2", "com3", "com4", "com5", "com6", "com7", "com8", "com9", + "lpt1", "lpt2", "lpt3", "lpt4", "lpt5", "lpt6", "lpt7", "lpt8", "lpt9", + "com\xC2\xB9", "com\xC2\xB2", "com\xC2\xB3", + "lpt\xC2\xB9", "lpt\xC2\xB2", "lpt\xC2\xB3", }; for (const auto& r : kReserved) if (base == r) return true; return false; } +// Table-free UTF-8 well-formedness. Overlong encodings, surrogate halves and +// code points past U+10FFFF are rejected as hard as a structural length error: +// the UTF-8 -> UTF-16 conversion a host must perform maps an ill-formed +// sequence to U+FFFD unless it opts into failing, so two names differing only +// in invalid bytes would otherwise collapse onto one destination file. +bool isWellFormedUtf8(const std::string& s) { + const auto* p = reinterpret_cast(s.data()); + const std::size_t n = s.size(); + for (std::size_t i = 0; i < n;) { + const unsigned char c = p[i]; + std::size_t extra = 0; + std::uint32_t cp = 0; + if (c < 0x80) { ++i; continue; } + else if ((c & 0xE0) == 0xC0) { extra = 1; cp = c & 0x1Fu; } + else if ((c & 0xF0) == 0xE0) { extra = 2; cp = c & 0x0Fu; } + else if ((c & 0xF8) == 0xF0) { extra = 3; cp = c & 0x07u; } + else return false; // a stray continuation byte, or a 5/6-byte lead + if (i + extra >= n) return false; + for (std::size_t k = 1; k <= extra; ++k) { + const unsigned char cont = p[i + k]; + if ((cont & 0xC0) != 0x80) return false; + cp = (cp << 6) | (cont & 0x3Fu); + } + if (extra == 1 && cp < 0x80) return false; + if (extra == 2 && cp < 0x800) return false; + if (extra == 3 && cp < 0x10000) return false; + if (cp > 0x10FFFF) return false; + if (cp >= 0xD800 && cp <= 0xDFFF) return false; + i += extra + 1; + } + return true; +} + } // namespace bool isValidEntryName(const std::string& name) { @@ -46,7 +91,29 @@ bool isValidEntryName(const std::string& name) { if (c == '*' || c == '?' || c == '|' || c == '<' || c == '>' || c == '"') return false; } if (isDosDeviceName(name)) return false; + return isWellFormedUtf8(name); +} + +bool sameEntryName(const std::string& a, const std::string& b) { + if (a.size() != b.size()) return false; + for (std::size_t i = 0; i < a.size(); ++i) + if (lowerAscii(static_cast(a[i])) != + lowerAscii(static_cast(b[i]))) + return false; return true; } +bool isValidNestedSamplePath(const std::string& path) { + if (util::isAbsolutePath(path)) return false; + // Component-wise, not a substring scan: "take..final/a.wav" is a legal + // relative path, "bank/../evil.wav" is not. + for (std::size_t start = 0;; ) { + const std::size_t sep = path.find_first_of("/\\", start); + const std::size_t end = (sep == std::string::npos) ? path.size() : sep; + if (path.compare(start, end - start, "..") == 0) return false; + if (sep == std::string::npos) return true; + start = sep + 1; + } +} + } // namespace reasampler::package diff --git a/src/core/package/package_format.h b/src/core/package/package_format.h index 53b9538..185f890 100644 --- a/src/core/package/package_format.h +++ b/src/core/package/package_format.h @@ -56,25 +56,39 @@ enum class PackageReadability { PackageReadability classifyPackageVersion(std::uint32_t formatVersion, std::uint32_t minReaderVersion); -// The entry-name rule that makes path expression structurally impossible: a -// bare file name only. Rejects empty, ".", the exact ".." component (a name -// can only ever be one component, since separators are banned below — a -// substring scan would over-reject legal names like "take..final.wav"), any -// control byte (NUL included — truncates at the first filesystem call and -// collides two distinct manifest entries onto one file) or 0x7F, any '/', -// '\\' or ':' (which also bans every absolute form — drive, UNC, rooted), any -// Windows-reserved character (`*?|<>"`), a trailing dot or space (silently -// stripped at file creation, so "a.wav " and "a.wav" would collide), a DOS -// device name (CON/PRN/AUX/NUL/COM1-9/LPT1-9, case-insensitive, with or -// without an extension), and names over kMaxEntryNameBytes. Enforced on -// encode AND decode by package_manifest. +// The entry-name rule: a bare file name only. Rejects empty, ".", the exact +// ".." component (a name can only ever be one component, since separators are +// banned below — a substring scan would over-reject legal names like +// "take..final.wav"), any control byte (NUL included — truncates at the first +// filesystem call and collides two distinct manifest entries onto one file) or +// 0x7F, any '/', '\\' or ':' (which also bans every absolute form — drive, UNC, +// rooted), any Windows-reserved character (`*?|<>"`), a trailing dot or space +// (silently stripped at file creation, so "a.wav " and "a.wav" would collide), +// a DOS device name (CON/PRN/AUX/NUL/COM1-9/LPT1-9 plus the superscript +// COM/LPT 1-3 forms, case-insensitive, with or without an extension), names +// over kMaxEntryNameBytes, and any byte sequence that is not well-formed UTF-8. bool isValidEntryName(const std::string& name); +// The format's name-equivalence rule: two entry names that differ only by ASCII +// case are ONE name. Windows and macOS's default APFS are case-insensitive, so +// "Kick.wav" and "kick.wav" would extract onto a single file — and a bank +// authored on a case-sensitive filesystem produces that pair honestly. Non-ASCII +// bytes compare exactly (see this directory's CLAUDE.md on NFC/NFD). +bool sameEntryName(const std::string& a, const std::string& b); + +// The one field in the format that CAN express a path: a nested Sample's +// relativePath, which is bank-relative by design. Rejects every absolute form +// (the shared util::isAbsolutePath test) and any ".." component — BankModel::add +// checks only the former, so traversal reaches the format without this. +bool isValidNestedSamplePath(const std::string& path); + // The fixed header, informational semver included. writerVersion is // version::stampVersion() on the write side — it exists so a TooNew refusal can // tell the user which build to install; it never gates. Defaults are 0/0, NOT -// the current ladder pair — a caller reading `header` after a Malformed -// verdict must see an obviously-unset value, not a plausible-looking 1/1. +// the current ladder pair, so a header that never parsed reads as obviously +// unset rather than as a plausible 1/1. The fields are meaningful whenever they +// are non-zero, not only on success: a decode that got past the header and +// failed later (a corrupt manifest) reports the real pair alongside Malformed. struct PackageHeader { std::uint32_t formatVersion = 0; std::uint32_t minReaderVersion = 0; diff --git a/src/core/package/package_manifest.cpp b/src/core/package/package_manifest.cpp index 4740b3e..5a2763e 100644 --- a/src/core/package/package_manifest.cpp +++ b/src/core/package/package_manifest.cpp @@ -13,11 +13,12 @@ using json::numToStr; using ObjWriter = json::Writer; // Shared by serializeManifest and deserializeManifest — see this directory's -// CLAUDE.md for why duplicate names are rejected both ways. +// CLAUDE.md for why duplicate names are rejected both ways. Equivalence is the +// format's, not std::string's: sameEntryName folds ASCII case. bool duplicateName(const std::vector& entries) { for (std::size_t i = 0; i < entries.size(); ++i) for (std::size_t j = i + 1; j < entries.size(); ++j) - if (entries[i].fileName == entries[j].fileName) return true; + if (sameEntryName(entries[i].fileName, entries[j].fileName)) return true; return false; } @@ -45,6 +46,7 @@ bool PackageManifest::operator==(const PackageManifest& o) const { std::optional serializeManifest(const PackageManifest& m) { for (const auto& e : m.entries) { if (!isValidEntryName(e.fileName)) return std::nullopt; + if (!isValidNestedSamplePath(e.sample.relativePath)) return std::nullopt; // Cross-module contract with src/shell/package — see this directory's // CLAUDE.md. if (e.byteLength == 0) return std::nullopt; @@ -122,19 +124,22 @@ bool parseEntry(json::Reader& r, PackageEntry& e) { std::string key; if (!r.parseKey(key)) return false; + // A repeated key is rejected here exactly as at the root — same format + // question, one level down. if (key == "name") { - if (!r.parseString(e.fileName)) return false; + if (haveName || !r.parseString(e.fileName)) return false; haveName = true; } else if (key == "length") { std::int64_t v = 0; - if (!r.parseInt64(v)) return false; + if (haveLength || !r.parseInt64(v)) return false; if (v < 0) return false; e.byteLength = static_cast(v); haveLength = true; } else if (key == "hash") { - if (!r.parseString(e.byteHash)) return false; + if (haveHash || !r.parseString(e.byteHash)) return false; haveHash = true; } else if (key == "index") { + if (haveSample) return false; std::string raw; if (!r.captureValue(raw)) return false; auto idx = model::BankModel::deserialize(raw); @@ -150,25 +155,31 @@ bool parseEntry(json::Reader& r, PackageEntry& e) { if (!r.consume('}')) return false; if (!haveName || !haveLength || !haveHash || !haveSample) return false; - return isValidEntryName(e.fileName); + return isValidEntryName(e.fileName) && isValidNestedSamplePath(e.sample.relativePath); } bool parseManifest(json::Reader& r, PackageManifest& m) { if (!r.consume('{')) return false; r.skipWs(); - bool haveEntries = false; + // "Which duplicate keys are legal" is a format contract, so it is answered + // for every root key rather than only for the one that would accumulate: + // a repeated key is rejected, never last-wins. Unknown keys may repeat — + // they are skipped, and a future format must stay free to add them. + bool haveBankName = false, haveExported = false, haveEntries = false, haveSlots = false; + const auto firstTime = [](bool& seen) { const bool ok = !seen; seen = true; return ok; }; if (!r.consume('}')) { // not the empty-object shortcut: parse the members do { std::string key; if (!r.parseKey(key)) return false; if (key == "bankName") { + if (!firstTime(haveBankName)) return false; if (!r.parseString(m.bankDisplayName)) return false; } else if (key == "exported") { + if (!firstTime(haveExported)) return false; if (!r.parseInt64(m.exportTimestamp)) return false; } else if (key == "entries") { - if (haveEntries) return false; // a repeated key must not accumulate - haveEntries = true; + if (!firstTime(haveEntries)) return false; if (!r.consume('[')) return false; r.skipWs(); if (!r.consume(']')) { @@ -180,6 +191,7 @@ bool parseManifest(json::Reader& r, PackageManifest& m) { if (!r.consume(']')) return false; } } else if (key == "slots") { + if (!firstTime(haveSlots)) return false; if (!parseSlots(r, m.slots)) return false; } else { if (!r.skipValue()) return false; // forward-compat unknown keys diff --git a/src/core/package/package_manifest.h b/src/core/package/package_manifest.h index 0b31ce0..906f0b6 100644 --- a/src/core/package/package_manifest.h +++ b/src/core/package/package_manifest.h @@ -40,18 +40,21 @@ struct PackageManifest { bool operator==(const PackageManifest& o) const; }; -// Emits the manifest JSON. nullopt when the manifest cannot be represented: -// an invalid or duplicate entry name, a zero-length entry (see this -// directory's CLAUDE.md — the shell's payload-append seam cannot round-trip -// one), or a sample record BankModel itself would reject (empty id, absolute -// path) — refusing on encode so an undecodable package is never written. +// Emits the manifest JSON. nullopt when the manifest cannot be represented: an +// invalid or duplicate entry name (duplicate by sameEntryName, not string +// equality), a nested relativePath isValidNestedSamplePath refuses, a +// zero-length entry (see this directory's CLAUDE.md — the shell's payload-append +// seam cannot round-trip one), or a sample record BankModel itself would reject +// (empty id, absolute path) — refusing on encode so an undecodable package is +// never written. std::optional serializeManifest(const PackageManifest& m); // Parses manifest JSON (nullopt on malformed input, never UB). Unknown keys are -// skipped at every level, so an additive newer manifest still parses. Rejects -// what encode rejects — entry names are validated on BOTH directions because a -// package can arrive from anywhere — plus a missing per-entry field or a -// negative length. +// skipped at every level, so an additive newer manifest still parses; a repeated +// KNOWN root key is rejected rather than last-wins. Rejects what encode rejects +// except the zero-length entry — names and nested paths are validated on BOTH +// directions because a package can arrive from anywhere — plus a missing +// per-entry field or a negative length. std::optional deserializeManifest(const std::string& json); } // namespace reasampler::package diff --git a/tests/test_bank_package.cpp b/tests/test_bank_package.cpp index 27f16af..8783230 100644 --- a/tests/test_bank_package.cpp +++ b/tests/test_bank_package.cpp @@ -142,9 +142,7 @@ static void testEncodeRefusesWhatManifestRefuses() { m.entries[0].fileName = "../evil.wav"; CHECK(!encodePackage(m).has_value()); - // A zero-length entry cannot round-trip through the shell's filesystem - // seam (src/shell/package's appendPayload refuses an empty payload) — the - // format layer must never produce one. + // See src/core/package/CLAUDE.md for the shell seam that forces this. PackageManifest zeroLen = fixture(1, 1); zeroLen.entries[1].byteLength = 0; CHECK(!encodePackage(zeroLen).has_value()); @@ -185,6 +183,41 @@ static void testTruncationAtEveryByteOffsetIsMalformed() { CHECK(decodePackage(file, file.size() - 1).status == PackageReadability::Malformed); } +// --- header defaults --------------------------------------------------------- + +// The 0/0 defaults are not the current ladder pair, so a header that never +// parsed cannot be mistaken for a plausible 1/1. They mean "unset", NOT "the +// decode failed": a decode that got past the header reports the real pair +// alongside its Malformed verdict. +static void testHeaderDefaultsMeanUnparsed() { + CHECK(PackageHeader{}.formatVersion == 0); + CHECK(PackageHeader{}.minReaderVersion == 0); + CHECK(PackageHeader{}.writerVersion.empty()); + + // Failed before the header: bad magic, and a semver truncated mid-string. + std::vector badMagic = rawHeader(1, 1, "1.0.0"); + appendManifest(badMagic, "{}"); + badMagic[0] = 'Z'; + const DecodedPackage magic = decodePackage(badMagic, badMagic.size()); + CHECK(magic.status == PackageReadability::Malformed); + CHECK(magic.header == PackageHeader{}); + + std::vector cutSemver = rawHeader(1, 1, "1.0.0"); + cutSemver.resize(18); + CHECK(decodePackage(cutSemver, cutSemver.size()).header == PackageHeader{}); + + // Failed after it: a same-version package with a corrupt manifest is + // Malformed, and its header is fully populated. + std::vector corrupt = + rawHeader(kPackageFormatVersion, kPackageMinReaderVersion, "1.0.0"); + appendManifest(corrupt, "not json"); + const DecodedPackage late = decodePackage(corrupt, corrupt.size()); + CHECK(late.status == PackageReadability::Malformed); + CHECK(late.header.formatVersion == kPackageFormatVersion); + CHECK(late.header.minReaderVersion == kPackageMinReaderVersion); + CHECK(late.header.writerVersion == "1.0.0"); +} + // --- version ladder: TooNew refuses whole ------------------------------------ static void testTooNewProducesNoManifest() { @@ -365,6 +398,7 @@ int main() { testEncodeDecodeRoundTrip(); testEncodeRefusesWhatManifestRefuses(); testTruncationAtEveryByteOffsetIsMalformed(); + testHeaderDefaultsMeanUnparsed(); testTooNewProducesNoManifest(); testAdditiveUnparseableManifestIsTooNew(); testNewerAdditiveFormatReads(); diff --git a/tests/test_package_format.cpp b/tests/test_package_format.cpp index 56d3cb8..adf79d6 100644 --- a/tests/test_package_format.cpp +++ b/tests/test_package_format.cpp @@ -1,7 +1,7 @@ // Standalone tests for reasampler::package's format contract — no REAPER, no // test framework. Pins the version-ladder classification (both integers, every -// branch) and the entry-name rule that makes path expression structurally -// impossible in a package. +// branch) and the three naming rules: the entry-name rule, the ASCII-folding +// name equivalence, and the nested-path traversal guard. #include "../src/core/package/package_format.h" @@ -111,8 +111,110 @@ static void testEntryNameRejectsWindowsHostileNames() { CHECK(!isValidEntryName("COM1")); CHECK(!isValidEntryName("com1.txt")); CHECK(!isValidEntryName("LPT1")); - // Not a device name: a real filename that merely starts with one. + // The superscript device forms (COM¹ COM² COM³ LPT¹ LPT² LPT³ in UTF-8): + // Windows reads those as digits in a device name, so "COM².wav" is COM2. + CHECK(!isValidEntryName("COM\xC2\xB9.wav")); + CHECK(!isValidEntryName("com\xC2\xB2")); + CHECK(!isValidEntryName("COM\xC2\xB3.wav")); + CHECK(!isValidEntryName("LPT\xC2\xB9")); + CHECK(!isValidEntryName("lpt\xC2\xB2.txt")); + CHECK(!isValidEntryName("LPT\xC2\xB3.wav")); + // Not a device name: a real filename that merely starts with one, and the + // zero forms, which Windows does not reserve. CHECK(isValidEntryName("console.wav")); + CHECK(isValidEntryName("COM0.wav")); + CHECK(isValidEntryName("LPT0")); + // Nor does a superscript past 3 name a device. + CHECK(isValidEntryName("COM\xE2\x81\xB4.wav")); // U+2074 SUPERSCRIPT FOUR +} + +// --- isValidEntryName: UTF-8 well-formedness --------------------------------- + +static void testEntryNameAcceptsWellFormedUtf8() { + CHECK(isValidEntryName("caf\xC3\xA9.wav")); // 2-byte: é + CHECK(isValidEntryName("\xE2\x99\xAA.wav")); // 3-byte: ♪ + CHECK(isValidEntryName("\xF0\x9F\x8E\xB5.wav")); // 4-byte: 🎵 + CHECK(isValidEntryName("\xEF\xBB\xBF.wav")); // U+FEFF, ugly but well-formed + CHECK(isValidEntryName("\xF4\x8F\xBF\xBF.wav")); // U+10FFFF, the last code point +} + +static void testEntryNameRejectsIllFormedUtf8() { + // Two names differing ONLY in their invalid bytes: a host converting to + // UTF-16 substitutes U+FFFD for both by default, collapsing them onto one + // file — the duplicate-name collision the manifest cannot otherwise see. + CHECK(!isValidEntryName("a\x80.wav")); // stray continuation byte + CHECK(!isValidEntryName("a\x81.wav")); + // Structural: truncated sequences (a lead byte the name ends inside). + CHECK(!isValidEntryName("a\xC3")); + CHECK(!isValidEntryName("a\xE2\x99")); + CHECK(!isValidEntryName("a\xF0\x9F\x8E")); + // A lead byte followed by a non-continuation. + CHECK(!isValidEntryName("a\xC3\x41.wav")); + // Overlong encodings: an alternate spelling of an ASCII byte we ban. + CHECK(!isValidEntryName("a\xC0\xAF.wav")); // overlong '/' + CHECK(!isValidEntryName("a\xC0\x80.wav")); // overlong NUL + CHECK(!isValidEntryName("a\xE0\x80\xAF.wav")); // overlong '/', 3-byte + CHECK(!isValidEntryName("a\xF0\x80\x80\xAF.wav")); // overlong '/', 4-byte + // Surrogate halves: no code point, and unrepresentable in UTF-16. + CHECK(!isValidEntryName("a\xED\xA0\x80.wav")); // U+D800 + CHECK(!isValidEntryName("a\xED\xBF\xBF.wav")); // U+DFFF + // Past U+10FFFF, and the 5/6-byte leads that never encode anything. + CHECK(!isValidEntryName("a\xF4\x90\x80\x80.wav")); // U+110000 + CHECK(!isValidEntryName("a\xF5\x80\x80\x80.wav")); + CHECK(!isValidEntryName("a\xFC\x80\x80\x80\x80\x80.wav")); + CHECK(!isValidEntryName("a\xFF.wav")); +} + +// --- sameEntryName ----------------------------------------------------------- + +static void testSameEntryNameFoldsAsciiCase() { + // A bank authored on a case-sensitive filesystem produces this pair + // honestly; Windows and default APFS would extract both onto one file. + CHECK(sameEntryName("Kick.wav", "kick.wav")); + CHECK(sameEntryName("KICK.WAV", "kick.wav")); + CHECK(sameEntryName("kick.wav", "kick.wav")); + CHECK(!sameEntryName("kick.wav", "snare.wav")); + CHECK(!sameEntryName("kick.wav", "kick.wave")); // length alone decides + CHECK(!sameEntryName("", "a")); + CHECK(sameEntryName("", "")); + // ASCII only: "é" vs "É" are two names here (the NFC/NFD limitation this + // shares — see this directory's CLAUDE.md). + CHECK(!sameEntryName("caf\xC3\xA9.wav", "caf\xC3\x89.wav")); + // Only the letters fold — the bytes flanking the ASCII range must not. + CHECK(!sameEntryName("a[b", "a{b")); // 0x5B vs 0x7B, 'Z'+1 and 'z'+1 + CHECK(!sameEntryName("a@b", "a`b")); // 0x40 vs 0x60, 'A'-1 and 'a'-1 +} + +// --- isValidNestedSamplePath ------------------------------------------------- + +static void testNestedSamplePathAcceptsRelative() { + CHECK(isValidNestedSamplePath("a.wav")); + CHECK(isValidNestedSamplePath("reasampler_bank/kick.wav")); + CHECK(isValidNestedSamplePath("reasampler_bank\\kick.wav")); + CHECK(isValidNestedSamplePath("deep/dir/tree/a.wav")); + // A ".." that is not a whole component is an ordinary name. + CHECK(isValidNestedSamplePath("take..final/a.wav")); + CHECK(isValidNestedSamplePath("bank/..hidden")); + CHECK(isValidNestedSamplePath("a..b")); +} + +static void testNestedSamplePathRejectsTraversalAndAbsolute() { + // BankModel::add catches only the absolute forms, so traversal reaches the + // format unless this rule stops it. + CHECK(!isValidNestedSamplePath("..")); + CHECK(!isValidNestedSamplePath("../evil.wav")); + CHECK(!isValidNestedSamplePath("..\\evil.wav")); + CHECK(!isValidNestedSamplePath("bank/../../evil.wav")); + CHECK(!isValidNestedSamplePath("bank\\..\\evil.wav")); + CHECK(!isValidNestedSamplePath("bank/..")); + CHECK(!isValidNestedSamplePath("bank/../")); + // Everything util::isAbsolutePath already catches. + CHECK(!isValidNestedSamplePath("/rooted.wav")); + CHECK(!isValidNestedSamplePath("\\rooted.wav")); + CHECK(!isValidNestedSamplePath("C:/abs.wav")); + CHECK(!isValidNestedSamplePath("C:\\abs.wav")); + CHECK(!isValidNestedSamplePath("c:relative-to-drive.wav")); + CHECK(!isValidNestedSamplePath("\\\\server\\share.wav")); } int main() { @@ -123,6 +225,11 @@ int main() { testEntryNameRejectsSeparatorsAndDots(); testEntryNameRejectsAbsolutePrefixes(); testEntryNameRejectsWindowsHostileNames(); + testEntryNameAcceptsWellFormedUtf8(); + testEntryNameRejectsIllFormedUtf8(); + testSameEntryNameFoldsAsciiCase(); + testNestedSamplePathAcceptsRelative(); + testNestedSamplePathRejectsTraversalAndAbsolute(); if (g_fail == 0) { std::printf("package_format_tests: all passed\n"); diff --git a/tests/test_package_manifest.cpp b/tests/test_package_manifest.cpp index e418ed7..f644f22 100644 --- a/tests/test_package_manifest.cpp +++ b/tests/test_package_manifest.cpp @@ -1,7 +1,7 @@ // Standalone tests for reasampler::package's manifest codec — no REAPER, no // test framework. The round-trip fixture exercises every manifest field and // every Sample optional in both present and absent states; the rejection suite -// pins the entry-name rule on encode AND decode. +// pins the naming, case-folding and traversal rules on encode AND decode. #include "../src/core/package/package_manifest.h" @@ -132,16 +132,58 @@ static void testEncodeRejectsBadEntryName() { } } +// One entry, `name` spliced in as raw manifest text so a hostile spelling +// (escapes included) is expressible — a package is not limited to what encode +// emits. +static std::string oneEntryJson(const std::string& name, + const std::string& relativePath = "bank/a.wav") { + return "{\"entries\":[{\"name\":\"" + name + + "\",\"length\":1,\"hash\":\"h\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s1\"," + "\"relativePath\":\"" + relativePath + "\"}]}}]}"; +} + static void testDecodeRejectsBadEntryName() { - for (const char* bad : {"..\\\\evil.wav", "../evil.wav", "dir/a.wav", "C:\\\\a.wav", ".."}) { - // Hand-rolled JSON: a hostile package is not limited to what encode emits. - std::string json = - std::string("{\"entries\":[{\"name\":\"") + bad + - "\",\"length\":1,\"hash\":\"h\"," - "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s1\"," - "\"relativePath\":\"bank/a.wav\"}]}}]}"; - CHECK(!deserializeManifest(json).has_value()); - } + for (const char* bad : {"..\\\\evil.wav", "../evil.wav", "dir/a.wav", "C:\\\\a.wav", ".."}) + CHECK(!deserializeManifest(oneEntryJson(bad)).has_value()); + + // The rest of the rule set through decode — the direction that matters, + // since a package can arrive from anywhere. + CHECK(!deserializeManifest(oneEntryJson("CON.wav")).has_value()); // DOS device + CHECK(!deserializeManifest(oneEntryJson("COM\xC2\xB2.wav")).has_value()); // COM² + CHECK(!deserializeManifest(oneEntryJson("a.wav ")).has_value()); // trailing space + CHECK(!deserializeManifest(oneEntryJson("a.wav.")).has_value()); // trailing dot + CHECK(!deserializeManifest(oneEntryJson("a*b.wav")).has_value()); // reserved char + // A NUL smuggled in as a JSON escape: the manifest text is legal, the + // decoded name is not. + CHECK(!deserializeManifest(oneEntryJson("a\\u0000b.wav")).has_value()); + // Ill-formed UTF-8 as raw bytes. + CHECK(!deserializeManifest(oneEntryJson("a\xC3.wav")).has_value()); + // A lone surrogate never reaches the name rule — the JSON reader refuses + // the unpaired \uD800 first. Pinned so that refusal cannot silently become + // "decoded to U+FFFD and accepted". + CHECK(!deserializeManifest(oneEntryJson("a\\ud800b.wav")).has_value()); +} + +static void testDecodeRejectsTraversalInNestedPath() { + // The one field in the format that CAN express a path. BankModel::add + // catches the absolute forms only, so ".." arrives unless the package layer + // refuses it. + CHECK(!deserializeManifest(oneEntryJson("a.wav", "../../evil.wav")).has_value()); + CHECK(!deserializeManifest(oneEntryJson("a.wav", "bank/../evil.wav")).has_value()); + CHECK(!deserializeManifest(oneEntryJson("a.wav", "..")).has_value()); + // A ".." that is not a whole component still reads. + CHECK(deserializeManifest(oneEntryJson("a.wav", "take..final/a.wav")).has_value()); +} + +static void testEncodeRejectsTraversalInNestedPath() { + PackageManifest m = fixture(); + m.entries[0].sample.relativePath = "../../evil.wav"; + CHECK(!serializeManifest(m).has_value()); + + PackageManifest m2 = fixture(); + m2.entries[0].sample.relativePath = "bank/../evil.wav"; + CHECK(!serializeManifest(m2).has_value()); } static void testDuplicateEntryNamesRejectedBothWays() { @@ -149,40 +191,92 @@ static void testDuplicateEntryNamesRejectedBothWays() { m.entries[1].fileName = m.entries[0].fileName; CHECK(!serializeManifest(m).has_value()); + // Case-folded: two names one case-insensitive filesystem extracts onto a + // single file are one name here too, in both directions. + PackageManifest folded = fixture(); + folded.entries[1].fileName = "KICK.WAV"; // entries[0] is "kick.wav" + CHECK(!serializeManifest(folded).has_value()); + // Decode side, from a hand-built duplicate (a hostile package is not // limited to what encode emits). const std::string dup = "{\"entries\":[" "{\"name\":\"a.wav\",\"length\":1,\"hash\":\"h\"," "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s1\",\"relativePath\":\"p\"}]}}," - "{\"name\":\"a.wav\",\"length\":2,\"hash\":\"i\"," + "{\"name\":\"A.WAV\",\"length\":2,\"hash\":\"i\"," "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s2\",\"relativePath\":\"q\"}]}}]}"; CHECK(!deserializeManifest(dup).has_value()); + + // Two names that differ outside the ASCII letters are still two names. + const std::string distinct = + "{\"entries\":[" + "{\"name\":\"a1.wav\",\"length\":1,\"hash\":\"h\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s1\",\"relativePath\":\"p\"}]}}," + "{\"name\":\"a2.wav\",\"length\":2,\"hash\":\"i\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s2\",\"relativePath\":\"q\"}]}}]}"; + CHECK(deserializeManifest(distinct).has_value()); } -static void testDuplicateEntriesKeyRejected() { - // A repeated "entries" key must not accumulate into two arrays' worth of - // entries — reject rather than silently union them. - const std::string json = +static void testRepeatedRootKeyRejected() { + // A repeated "entries" must not accumulate into two arrays' worth of + // entries; the other three assign rather than append, but "which duplicate + // keys are legal" is one format answer, not four. + const std::string entriesTwice = "{\"entries\":[{\"name\":\"a.wav\",\"length\":1,\"hash\":\"h\"," "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s1\",\"relativePath\":\"p\"}]}}]," "\"entries\":[{\"name\":\"b.wav\",\"length\":1,\"hash\":\"h\"," "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s2\",\"relativePath\":\"q\"}]}}]}"; - CHECK(!deserializeManifest(json).has_value()); + CHECK(!deserializeManifest(entriesTwice).has_value()); + + CHECK(!deserializeManifest("{\"bankName\":\"A\",\"bankName\":\"B\"}").has_value()); + CHECK(!deserializeManifest("{\"exported\":1,\"exported\":2}").has_value()); + CHECK(!deserializeManifest("{\"slots\":[],\"slots\":[]}").has_value()); + // Unknown keys stay repeatable: they are skipped, and a future format must + // be free to add them. + CHECK(deserializeManifest("{\"future\":1,\"future\":2}").has_value()); + + // The same answer one level down, so a hostile manifest cannot make two + // readers disagree about which spelling of an entry field is the real one. + CHECK(!deserializeManifest( + "{\"entries\":[{\"name\":\"a.wav\",\"name\":\"b.wav\",\"length\":1,\"hash\":\"h\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s1\",\"relativePath\":\"p\"}]}}]}") + .has_value()); + CHECK(!deserializeManifest( + "{\"entries\":[{\"name\":\"a.wav\",\"length\":1,\"length\":2,\"hash\":\"h\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s1\",\"relativePath\":\"p\"}]}}]}") + .has_value()); + CHECK(!deserializeManifest( + "{\"entries\":[{\"name\":\"a.wav\",\"length\":1,\"hash\":\"h\",\"hash\":\"i\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s1\",\"relativePath\":\"p\"}]}}]}") + .has_value()); + CHECK(!deserializeManifest( + "{\"entries\":[{\"name\":\"a.wav\",\"length\":1,\"hash\":\"h\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s1\",\"relativePath\":\"p\"}]}," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s2\",\"relativePath\":\"q\"}]}}]}") + .has_value()); } // --- rejection: structural --------------------------------------------------- -// A zero-length entry cannot round-trip through the shell's filesystem seam -// (src/shell/package's appendPayload refuses an empty payload) — the format -// layer must never produce one, so encode refuses it. Decode does not enforce -// this (a hostile/older package declaring one is not this codec's concern). +// See src/core/package/CLAUDE.md for the shell seam that forces this. static void testEncodeRejectsZeroLengthEntry() { PackageManifest m = fixture(); m.entries[0].byteLength = 0; CHECK(!serializeManifest(m).has_value()); } +// The asymmetry is deliberate: refusing a zero-length entry is an obligation on +// what this layer WRITES, not a claim about what a package may declare. Pinned +// so it is not "fixed" into a decode-side rejection. +static void testDecodeAcceptsZeroLengthEntry() { + auto m = deserializeManifest( + "{\"entries\":[{\"name\":\"a.wav\",\"length\":0,\"hash\":\"h\"," + "\"index\":{\"version\":1,\"samples\":[{\"id\":\"s1\",\"relativePath\":\"p\"}]}}]}"); + CHECK(m.has_value()); + CHECK(m->entries.size() == 1); + CHECK(m->entries[0].byteLength == 0); +} + static void testEncodeRejectsUnrepresentableSample() { PackageManifest m = fixture(); m.entries[0].sample.id.clear(); // BankModel::add rejects an empty id @@ -261,9 +355,12 @@ int main() { testUnknownKeysSkippedAtEveryLevel(); testEncodeRejectsBadEntryName(); testDecodeRejectsBadEntryName(); + testDecodeRejectsTraversalInNestedPath(); + testEncodeRejectsTraversalInNestedPath(); testDuplicateEntryNamesRejectedBothWays(); - testDuplicateEntriesKeyRejected(); + testRepeatedRootKeyRejected(); testEncodeRejectsZeroLengthEntry(); + testDecodeAcceptsZeroLengthEntry(); testEncodeRejectsUnrepresentableSample(); testDecodeRejectsMalformedShapes(); From a197ff7d68231ebbba6475637a8929b6411bd791 Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 07:24:30 -0400 Subject: [PATCH 08/24] docs: fix overclaiming OriginKind comment and trim restated test comments Enum comment claimed package-id lookup that no persisted field supports; reworded to the real distinction. Trimmed CLAUDE.md-duplicated test comments and the header. --- src/core/tracking/origin_ledger.h | 14 ++++++-------- tests/test_origin_ledger.cpp | 21 +++++++-------------- 2 files changed, 13 insertions(+), 22 deletions(-) diff --git a/src/core/tracking/origin_ledger.h b/src/core/tracking/origin_ledger.h index 01b99d1..da8d0e3 100644 --- a/src/core/tracking/origin_ledger.h +++ b/src/core/tracking/origin_ledger.h @@ -15,14 +15,12 @@ namespace reasampler::tracking { // lifted from a legacy path-only manifest, or one whose creator did not know. // PERSISTED AS INTEGERS: never renumber an existing value, only append. enum class OriginKind { - Unknown = 0, - Capture = 1, - Ingest = 2, - Recapture = 3, // regenerated in place from its recorded source recipe - Resample = 4, // baked from an instrument's own processing chain - // Kept distinct from Ingest — both bring in a foreign file, but only this one - // can answer "which package did this bank come from" later. - PackageImport = 5, + Unknown = 0, + Capture = 1, + Ingest = 2, + Recapture = 3, // regenerated in place from its recorded source recipe + Resample = 4, // baked from an instrument's own processing chain + PackageImport = 5, // package-sourced vs Ingest's user-picked; unrecoverable once merged }; // One system-created file's birth record. `relativePath` is the key and is ALWAYS diff --git a/tests/test_origin_ledger.cpp b/tests/test_origin_ledger.cpp index f5363fc..aa1eecd 100644 --- a/tests/test_origin_ledger.cpp +++ b/tests/test_origin_ledger.cpp @@ -1,11 +1,9 @@ // Standalone tests for reasampler::tracking::OriginLedger — no REAPER, no framework. // -// The record family behind file tracking. Covers: the relative-paths-only invariant, -// exact-string ownership, dedup, insertion order, the JSON round-trip (incl. golden -// byte literals over every persisted enum value), the append-a-kind-without-moving-"v" -// rule and its degrade-don't-block twin, the no-backfill rule, the legacy path-only -// lift, and the Fresh / Loaded / Unreadable / FutureVersion classification that keeps -// never-recorded apart from the two degraded states. +// Covers: relative-paths-only, exact-string ownership, dedup, insertion order, the +// JSON round-trip (incl. golden bytes over every persisted enum value), the append-a- +// kind rules, the no-backfill rule, the legacy path-only lift, and the Fresh / Loaded +// / Unreadable / FutureVersion classification. #include "../src/core/tracking/origin_ledger.h" @@ -187,10 +185,7 @@ static void testSerializeGoldenLiteralPinsEveryPersistedKind() { CHECK(*back == l); // every field, not just the kind, survives the trip } -// Appending a value to the kind vocabulary must NOT move the document version: the -// two rules sit side by side and pull in opposite directions — an unknown kind -// degrades, an unknown "v" blocks. Pinned on the emitted bytes rather than on the -// internal constant, because the byte is what an older build actually reads. +// Pins the emitted "v" byte, not the internal constant — an older build reads bytes. static void testAppendingAKindDoesNotMoveTheDocumentVersion() { OriginLedger l; l.record(rec("bank/import.wav", OriginKind::PackageImport, "S-e")); @@ -202,10 +197,8 @@ static void testAppendingAKindDoesNotMoveTheDocumentVersion() { CHECK(loadLedger("{\"v\":3,\"records\":[]}").status == LedgerStatus::FutureVersion); } -// The other half of the append rule: a kind this build does NOT know degrades to -// Unknown while the ledger still loads and the path stays owned. Losing the kind -// detail costs nothing today — no consumer reads it — but an Unreadable here would -// block prune entirely on nothing worse than a vocabulary gap. +// Pins three unrecognized kind values (future, far-future, negative) all landing on +// Unknown with the ledger still Loaded and the path still owned. static void testUnknownKindDegradesWithoutBlockingTheLedger() { const std::string blob = "{\"v\":2,\"records\":[" From 655159ceac471e112bd2d329fb9e4dfebe2e3d61 Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 08:44:29 -0400 Subject: [PATCH 09/24] Close package fs review findings: readRange bounds, picker ext, non-ASCII tests Cap readRange's allocation and reject size_t overflow instead of truncating; re-append .rsbank when the export picker omits it; add cafe coverage for writeFileExclusive and writeLandedFile; loop write() on EINTR. --- src/shell/package/CLAUDE.md | 14 +++++++++++--- src/shell/package/package_io.cpp | 20 ++++++++++++++++---- src/shell/package/package_io.h | 4 +++- src/shell/package/package_path.h | 15 +++++++++++---- src/shell/package/package_pickers.cpp | 20 +++++++++++++++++++- src/shell/package/package_pickers.h | 5 ++++- src/shell/package/package_rollback.cpp | 2 +- src/shell/package/package_rollback.h | 8 +++++++- tests/test_package_io.cpp | 11 +++++++++++ tests/test_package_rollback.cpp | 15 ++++++++++++++- 10 files changed, 97 insertions(+), 17 deletions(-) diff --git a/src/shell/package/CLAUDE.md b/src/shell/package/CLAUDE.md index 0caf183..7835e65 100644 --- a/src/shell/package/CLAUDE.md +++ b/src/shell/package/CLAUDE.md @@ -53,8 +53,12 @@ belong to the verbs. "Did this call create it" is structural: only exclusively-created paths are recorded, resolved absolute at record time so a later CWD change cannot re-aim the delete. "Did anything ever reference it" is a **contract the import verb must - honour**: it MUST call `markIndexCommitted()` at the moment it commits the index, - after which `rollback()` refuses and `writeLandedFile` refuses. + honour**: it MUST call `markIndexCommitted()` only AFTER the index write has + returned success — calling it before, then having that write fail, strands the + landed files with no index entry and a journal that now refuses to roll them + back — after which `rollback()` refuses and `writeLandedFile` refuses. (Destroying + an armed journal without calling either does NOT roll it back — see + `LandedFileJournal`'s own doc comment.) - **Both pickers ride `GetUserFileName`** — mode 1 for import, mode 0 for export. There is no platform split and no fallback: `main.cpp` defines `REAPERAPI_IMPLEMENT` without `REAPERAPI_MINIMAL` and aborts the extension load if any single name fails @@ -80,7 +84,11 @@ belong to the verbs. pair format but are `[verify — DAW]` on all three platforms — neither picker is exercised outside a live REAPER session. `GetUserFileName` also takes no owner window, so dialog parenting is REAPER's to do; the superseded Win32 path passed - `GetMainHwnd()` explicitly. + `GetMainHwnd()` explicitly. Also `[verify — DAW]`: whether mode 0's picker appends + an extension from `extension_list` when the user omits one — `pickPackageSavePath` + re-appends `.rsbank` itself so the returned path is correct regardless of how that + lands (the superseded Win32 path had `ofn.lpstrDefExt` for this; `GetUserFileName` + has no equivalent parameter). - `pickPackageSavePath`'s `suggestedPath` doubles as the dialog's starting directory when it is a full path. The verbs should seed it from the project directory — passing a bare name leaves the dialog on REAPER's process working directory, which diff --git a/src/shell/package/package_io.cpp b/src/shell/package/package_io.cpp index 901dc3f..26cd7e6 100644 --- a/src/shell/package/package_io.cpp +++ b/src/shell/package/package_io.cpp @@ -1,11 +1,13 @@ // package_io.cpp — see package_io.h for the seam's contract. Non-throwing at the -// boundary: every filesystem call uses the error_code form, so no filesystem_error -// crosses into a REAPER action body. +// boundary about filesystem_error: every filesystem call uses the error_code form. +// Allocation can still throw bad_alloc — readRange's sanity ceiling exists to keep +// that surface small, not to remove it. #include "shell/package/package_io.h" #include #include +#include #include #include "shell/package/package_path.h" @@ -16,6 +18,7 @@ #include #include #else +#include #include #include #endif @@ -147,7 +150,15 @@ PackageFileReader::PackageFileReader(const std::string& srcAbsPath) { PayloadBuffer PackageFileReader::readRange(std::uint64_t offset, std::uint64_t length) { // Overflow-safe range check: length is capped by the real file size before any // allocation happens, so a hostile offset/length pair cannot demand the moon. - if (!ok_ || length == 0 || length > size_ || offset > size_ - length) { + // Also rejected here rather than truncated: a length that would not fit in + // size_t (possible on a 32-bit build, where streamsize below stays 64-bit and + // so would read past a truncated allocation) and a length past the sanity + // ceiling, which exists so a merely large-but-real file size can't still hand + // std::vector a multi-gigabyte demand. + constexpr std::uint64_t kMaxReadRangeBytes = std::uint64_t{4} << 30; // 4 GiB + if (!ok_ || length == 0 || length > size_ || offset > size_ - length || + length > kMaxReadRangeBytes || + length > static_cast(std::numeric_limits::max())) { return PayloadBuffer{}; } in_.clear(); // a prior failed read must not poison this one @@ -208,6 +219,7 @@ bool writeFileExclusive(const std::string& absPath, const PayloadBuffer& payload static_cast(chunk)); #else const ssize_t n = ::write(fd, payload.data() + written, chunk); + if (n < 0 && errno == EINTR) continue; // a signal on the UI thread isn't a failure #endif if (n <= 0) { ok = false; @@ -241,7 +253,7 @@ std::vector listFolderFileNames(const std::string& dirAbsPath) { const auto& entry = *it; std::error_code reg_ec; if (!entry.is_regular_file(reg_ec)) continue; // skip subdirs / specials - names.push_back(entry.path().filename().u8string()); // never .string(): ANSI + names.push_back(pathToUtf8(entry.path().filename())); // never .string(): ANSI } std::sort(names.begin(), names.end()); return names; diff --git a/src/shell/package/package_io.h b/src/shell/package/package_io.h index 332a5f1..a33f66e 100644 --- a/src/shell/package/package_io.h +++ b/src/shell/package/package_io.h @@ -89,7 +89,9 @@ public: bool ok() const { return ok_; } std::uint64_t fileSize() const { return size_; } // Bytes [offset, offset+length). Range-checked against the real file size, so a - // hostile layout can never demand an allocation past the file's end. + // hostile layout can never demand an allocation past the file's end, and capped + // against a 4 GiB sanity ceiling so a merely large-but-real file can't still + // force a multi-gigabyte allocation out of one call. PayloadBuffer readRange(std::uint64_t offset, std::uint64_t length); private: diff --git a/src/shell/package/package_path.h b/src/shell/package/package_path.h index 9982496..e70db66 100644 --- a/src/shell/package/package_path.h +++ b/src/shell/package/package_path.h @@ -1,9 +1,9 @@ -// shell/package/package_path — the ONE narrow-string -> fs::path conversion for this -// seam. std::filesystem decodes a narrow path through the RUNTIME ANSI code page on -// Windows (measured: GetACP() == 1252 here), never UTF-8, so a bare +// shell/package/package_path — the ONE narrow-string <-> fs::path conversion pair for +// this seam. std::filesystem decodes a narrow path through the RUNTIME ANSI code page +// on Windows (measured: GetACP() == 1252 here), never UTF-8, so a bare // fs::path(std::string) turns every non-ASCII path this repo's UTF-8 convention // produces into mojibake. u8path is the C++17 spelling; it is deprecated in C++20, so -// a standard bump replaces the body here rather than at every call site. +// a standard bump replaces both bodies here rather than at every call site. #pragma once @@ -16,4 +16,11 @@ inline std::filesystem::path utf8Path(const std::string& utf8) { return std::filesystem::u8path(utf8); } +// u8string() returns std::u8string in C++20 — this is the one place that narrows it +// back to std::string, so a standard bump only widens this one body. +inline std::string pathToUtf8(const std::filesystem::path& path) { + const auto u8 = path.u8string(); + return std::string(u8.begin(), u8.end()); +} + } // namespace reasampler diff --git a/src/shell/package/package_pickers.cpp b/src/shell/package/package_pickers.cpp index 75af2a0..16395a6 100644 --- a/src/shell/package/package_pickers.cpp +++ b/src/shell/package/package_pickers.cpp @@ -5,6 +5,9 @@ #include "shell/package/package_pickers.h" +#include +#include + #define REAPERAPI_MINIMAL #define REAPERAPI_WANT_GetUserFileName #include "reaper_plugin_functions.h" @@ -30,6 +33,14 @@ bool runPicker(int mode, const char* caption, const char* initial, return !outAbsPath.empty(); } +bool hasCaseInsensitiveSuffix(const std::string& path, const std::string& suffix) { + if (path.size() < suffix.size()) return false; + return std::equal(suffix.rbegin(), suffix.rend(), path.rbegin(), + [](unsigned char a, unsigned char b) { + return std::tolower(a) == std::tolower(b); + }); +} + } // namespace bool pickPackageForImport(std::string& outAbsPath) { @@ -39,7 +50,14 @@ bool pickPackageForImport(std::string& outAbsPath) { bool pickPackageSavePath(const std::string& suggestedPath, std::string& outAbsPath) { // [verify — DAW] GetUserFileName takes no owner window, so the dialog's parenting // is REAPER's to do; the previous Win32 path passed GetMainHwnd() explicitly. - return runPicker(0, "Export bank package", suggestedPath.c_str(), outAbsPath); + if (!runPicker(0, "Export bank package", suggestedPath.c_str(), outAbsPath)) { + return false; + } + // GetUserFileName has no lpstrDefExt equivalent (the old Win32 picker's + // ofn.lpstrDefExt = L"rsbank"); whether mode 0 appends one itself from + // kExtList is [verify — DAW], so append it ourselves whenever it's missing. + if (!hasCaseInsensitiveSuffix(outAbsPath, ".rsbank")) outAbsPath += ".rsbank"; + return true; } } // namespace reasampler diff --git a/src/shell/package/package_pickers.h b/src/shell/package/package_pickers.h index 2cd8cf7..9f60512 100644 --- a/src/shell/package/package_pickers.h +++ b/src/shell/package/package_pickers.h @@ -1,7 +1,10 @@ // shell/package/package_pickers — the two package file pickers, both on REAPER's own // GetUserFileName (mode 1 = existing file, mode 0 = new file). No native/platform // split: the SDK's save mode is not optional on any build that can load this -// extension. Paths in and out are UTF-8, per the REAPER API contract. +// extension. Paths in and out are UTF-8, matching this tree's established practice +// for narrow strings crossing the REAPER API (see instrument_drop_win.cpp's +// path.u8string() to TrackFX_SetPreset, or prune_fs.cpp's CP_UTF8 conversion) — the +// SDK header itself never says "UTF-8". #pragma once diff --git a/src/shell/package/package_rollback.cpp b/src/shell/package/package_rollback.cpp index 1219292..c3162d0 100644 --- a/src/shell/package/package_rollback.cpp +++ b/src/shell/package/package_rollback.cpp @@ -24,7 +24,7 @@ bool LandedFileJournal::writeLandedFile(const std::string& destPath, std::error_code ec; const fs::path resolved = fs::absolute(utf8Path(destPath), ec); if (ec) return false; - const std::string absPath = resolved.u8string(); + const std::string absPath = pathToUtf8(resolved); if (!writeFileExclusive(absPath, payload)) return false; paths_.push_back(absPath); diff --git a/src/shell/package/package_rollback.h b/src/shell/package/package_rollback.h index cfc663d..438f168 100644 --- a/src/shell/package/package_rollback.h +++ b/src/shell/package/package_rollback.h @@ -19,6 +19,10 @@ struct RollbackResult { bool refused = false; // markIndexCommitted() ran: nothing was deleted }; +// Destroying an armed (uncommitted, un-rolled-back) journal is NOT an implicit +// rollback — the caller must call rollback() itself on the failure path it wants +// to undo. That's the fail-safe direction: a journal dropped by an unrelated early +// return leaves the landed files in place rather than silently deleting them. class LandedFileJournal { public: // Lands one payload at destPath through the exclusive create (which refuses an @@ -33,7 +37,9 @@ public: // Disarms the journal: the index mutation these files back is committed, so they // are now referenced bytes and the carve-out no longer covers them. This is the // half of prune's discriminator the journal cannot make structural on its own — - // the import verb MUST call it at the moment the index is committed. + // the import verb MUST call this only AFTER the index write has returned success. + // Calling it before, then having that write fail, strands the landed files with + // no index entry and a journal that now refuses to roll them back. void markIndexCommitted() { indexCommitted_ = true; } bool indexCommitted() const { return indexCommitted_; } diff --git a/tests/test_package_io.cpp b/tests/test_package_io.cpp index a533c72..d6237f6 100644 --- a/tests/test_package_io.cpp +++ b/tests/test_package_io.cpp @@ -293,11 +293,22 @@ static void testWriteFileExclusiveRefusesAnOccupiedPath() { CHECK(writeFileExclusive(path, PayloadBuffer(mine))); CHECK(readAll(path) == mine); // The create IS the check: a second call cannot replace the first file's bytes. + // (this pins the refusal, not O_EXCL's atomicity — a genuine race isn't portable) CHECK(!writeFileExclusive(path, PayloadBuffer(patternBytes(9, 8)))); CHECK(readAll(path) == mine); CHECK(!writeFileExclusive("pkg_io_excl_empty.bin", PayloadBuffer{})); CHECK(!exists("pkg_io_excl_empty.bin")); removeQuietly(path); + + // writeFileExclusive is the one call site using the wstring()/c_str() Windows + // open form rather than the fstream(fs::path) overload every other test here + // exercises — the only conversion whose reversion this file would otherwise miss. + const std::string cafePath = "pkg_io_excl_caf\xC3\xA9.bin"; + const std::vector cafeBytes = patternBytes(6, 4); + CHECK(writeFileExclusive(cafePath, PayloadBuffer(cafeBytes))); + CHECK(readAll(cafePath) == cafeBytes); + CHECK(!writeFileExclusive(cafePath, PayloadBuffer(patternBytes(3, 9)))); + removeQuietly(cafePath); } static void testListFolderFileNames() { diff --git a/tests/test_package_rollback.cpp b/tests/test_package_rollback.cpp index 8ddeb32..6f68bfc 100644 --- a/tests/test_package_rollback.cpp +++ b/tests/test_package_rollback.cpp @@ -22,7 +22,7 @@ static int g_fail = 0; // The journal records absolute paths, so every expectation is built the same way. static std::string scratch(const std::string& name) { - return (fs::current_path() / utf8Path(name)).u8string(); + return pathToUtf8(fs::current_path() / utf8Path(name)); } static std::vector patternBytes(std::size_t n, std::uint8_t seed) { @@ -64,6 +64,18 @@ static void testLandRecordsOnSuccessOnly() { CHECK(!exists(path)); // the recorded path denoted the file we asked for } +static void testLandNonAsciiPathRoundTripsAsUtf8() { + // The fs::absolute -> u8string round trip at writeLandedFile is otherwise + // untested with a non-ASCII path. + LandedFileJournal journal; + const std::string path = scratch("rb_caf\xC3\xA9.bin"); + const std::vector bytes = patternBytes(16, 2); + CHECK(journal.writeLandedFile(path, PayloadBuffer(bytes))); + CHECK(readAll(path) == bytes); + journal.rollback(); + CHECK(!exists(path)); +} + static void testRelativeInputIsRecordedAbsolute() { // The hazard: a bare name recorded verbatim, then a CWD change, and rollback // unlinks whatever now sits at that name in the new directory. @@ -169,6 +181,7 @@ static void testIndexCommitDisarmsRollback() { int main() { testLandRecordsOnSuccessOnly(); + testLandNonAsciiPathRoundTripsAsUtf8(); testRelativeInputIsRecordedAbsolute(); testExistingDestinationRefusedUntouched(); testEmptyPayloadRefused(); From 181b4f2edb33c0ece38cd39e4de020bb4cc15307 Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 09:02:23 -0400 Subject: [PATCH 10/24] Record parseSlots exemption and fix stale package doc comments Documents why parseSlots skips repeat-key rejection, corrects two drifted doc lines (naming-rule count, Malformed-after-header header validity), and records two forward obligations for import_plan in CLAUDE.md. --- src/core/package/CLAUDE.md | 24 ++++++++++++++++++++---- src/core/package/bank_package.h | 6 ++++-- src/core/package/package_format.h | 6 +++--- src/core/package/package_manifest.cpp | 9 ++++++++- 4 files changed, 35 insertions(+), 10 deletions(-) diff --git a/src/core/package/CLAUDE.md b/src/core/package/CLAUDE.md index 3c37197..9114f83 100644 --- a/src/core/package/CLAUDE.md +++ b/src/core/package/CLAUDE.md @@ -25,9 +25,10 @@ landing after the format. header (so the refusal can name the writer's semver and version) and nothing else — no manifest, no layout, no half-success. The fields through the writer semver are FROZEN for all future versions to keep that refusal producible. -- **Every name and path in the format is validated on encode AND decode**, - because a package can arrive from anywhere. Three rules, all in - `package_format`, whose doc comments are the itemized authority: +- **Every name and path in the format is validated on encode AND decode, to + the extent stated below**, because a package can arrive from anywhere. + Three rules, all in `package_format`, whose doc comments are the itemized + authority: - `isValidEntryName` — a payload's name is a bare file name (no separators, no `..` component, no drive/UNC/rooted form, no control bytes, no Windows-reserved character, no trailing dot/space, no DOS device name, @@ -40,7 +41,13 @@ landing after the format. design, and is the one field here that can express one. It refuses a `..` component and every absolute form; `BankModel::add` checks only the latter, so traversal would otherwise reach a future `import_plan` inside a record - the format vouched for. + the format vouched for. **Scope is traversal and absolute-form only** — no + UTF-8 well-formedness check (unlike `isValidEntryName`), no device-name + check, no case-fold dedup on `relativePath` (unlike `sameEntryName` on the + entry name). Correct for what this field is — a *record* field, not a + filesystem destination; `BankModel::add` owns the rest. Forward contract + for `import_plan`: **the destination file is derived from the entry name, + never from `relativePath`.** - **Framing only, never a payload.** `bank_package` produces header bytes and an ordered `{name, offset, length}` layout; it never holds, copies, or hashes an entry's audio. `decodePackage` proves prefix + payload lengths equal the @@ -126,6 +133,15 @@ landing after the format. collision class as the ASCII case fold, which `sameEntryName` does catch. A table-free fix does not exist, and restricting names to ASCII would be genuinely over-strict for non-English users. Left open knowingly. +- **`duplicateName` is O(n²) over `entries` on the decode path** — pre-existing + shape (the double loop is unchanged since `af35fc5`; only the comparator + changed). Under the `kMaxManifestBytes` cap (64 MB) a minimal entry is + ~100 bytes, so a hostile package can declare ~670k entries — ~2×10¹¹ pair + comparisons, a multi-minute hang on import. It signals an error rather than + UB, so the hostile-input invariant above still holds, but it sits against + this module's "a forged header cannot demand gigabytes" posture. Forward + obligation for `import_plan`: fold this into a sorted vector or hash set + when that track lands; not changed here. - **Cross-module contract with `src/shell/package`:** a genuinely zero-length entry cannot round-trip through the filesystem seam there (`appendPayload` refuses an empty payload — an empty buffer signals an upstream read failure, diff --git a/src/core/package/bank_package.h b/src/core/package/bank_package.h index 677b567..84de7e9 100644 --- a/src/core/package/bank_package.h +++ b/src/core/package/bank_package.h @@ -40,8 +40,10 @@ struct EncodedPackage { std::optional encodePackage(const PackageManifest& m); // The read side's product. header is meaningful for Readable and TooNew (a -// refusal must still name the writer); manifest, layout, and prefixSize only -// for Readable — TooNew produces NO manifest, so a refused decode cannot +// refusal must still name the writer), and on any Malformed reached after the +// header parsed (a corrupt manifest at this build's own version carries the +// real pair, not the 0/0 unparsed default); manifest, layout, and prefixSize +// only for Readable — TooNew produces NO manifest, so a refused decode cannot // half-succeed. struct DecodedPackage { PackageReadability status = PackageReadability::Malformed; diff --git a/src/core/package/package_format.h b/src/core/package/package_format.h index 185f890..fd7c7a9 100644 --- a/src/core/package/package_format.h +++ b/src/core/package/package_format.h @@ -1,8 +1,8 @@ #pragma once // package_format — the RSBK bank-package contract: magic, the version ladder, -// the readability classification, and the entry-name rule. Pure: standard -// library only. The framing codec that acts on this contract is bank_package; -// the manifest grammar is package_manifest. +// the readability classification, and the three naming rules below. Pure: +// standard library only. The framing codec that acts on this contract is +// bank_package; the manifest grammar is package_manifest. #include #include diff --git a/src/core/package/package_manifest.cpp b/src/core/package/package_manifest.cpp index 5a2763e..df2f772 100644 --- a/src/core/package/package_manifest.cpp +++ b/src/core/package/package_manifest.cpp @@ -84,7 +84,14 @@ std::optional serializeManifest(const PackageManifest& m) { namespace { // Mirrors bank_book_json's private slots parser: [{id, slot}, ...] pairs handed -// to SlotMap::fromEntries, which owns the defensive repair rules. +// to SlotMap::fromEntries, which owns the defensive repair rules. Deliberately +// does NOT reject a repeated "id"/"slot" key the way the root and entry parsers +// below reject theirs — this grammar belongs to core/model's bank_book_json, and +// diverging here would give one wire shape two behaviours in two files. The +// stakes differ too: a repeated "name" decides which file an entry lands on, +// while a repeated "id" here still feeds SlotMap::fromEntries's deterministic +// first-wins/never-double-occupy repair, so no ambiguity survives. Do not +// "finish" the repeat-key rejection here to match the parsers below. bool parseSlots(json::Reader& r, model::SlotMap& out) { std::vector> pairs; if (!r.consume('[')) return false; From f18884637017245d5b486508369d5892fd306e4a Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 08:52:25 -0400 Subject: [PATCH 11/24] docs(package): record known gaps and correct stale claims Notes the export verb's overwrite-consent obligation post-append, the append's extension-divergence behavior, and readFilePayload's 4GiB blind spot; fixes a stale u8string() reference and marks the 4GiB guard as accepted-unexercised. --- src/shell/package/CLAUDE.md | 19 ++++++++++++++++--- src/shell/package/package_io.cpp | 6 ++++++ src/shell/package/package_io.h | 4 +++- 3 files changed, 25 insertions(+), 4 deletions(-) diff --git a/src/shell/package/CLAUDE.md b/src/shell/package/CLAUDE.md index 7835e65..b4db0c4 100644 --- a/src/shell/package/CLAUDE.md +++ b/src/shell/package/CLAUDE.md @@ -19,7 +19,7 @@ belong to the verbs. `std::filesystem` decodes a narrow path through the runtime ANSI code page (measured `GetACP() == 1252`), so a bare `fs::path(std::string)` turns `café.rsbank` into `café.rsbank` or fails to open it. Every path a verb hands in or gets back — - including `listFolderFileNames`' results, which use `u8string()` and never + including `listFolderFileNames`' results, which go through `pathToUtf8()` and never `string()` — is UTF-8. `core/util/file_bytes` has the un-converted shape, which is why `readFilePayload` reads through this module's own `PackageFileReader` instead. - **Atomic package write, to the limit of a rename.** A package accumulates in a @@ -47,7 +47,14 @@ belong to the verbs. writer could win the race against. Collision handling (auto-rename) remains the import plan's job upstream. The package writer itself DOES replace an existing destination — the export save dialog's own overwrite confirm is the consent — and - that asymmetry is deliberate. + that asymmetry is deliberate. **Known gap, obligation on the export verb:** + `pickPackageSavePath`'s own `.rsbank` re-append (see its Gotcha below) can turn a + confirmed path `X` into a write target `X.rsbank` that the dialog never asked about. + The export verb MUST re-check `fileStatus()` on the path actually handed to + `PackageFileWriter` — after any extension append — and get its own consent if that + re-checked path is `Present`; the dialog's confirm only ever covered the pre-append + path. Not fixed at this seam: prompting is verb-level UX, and `pickPackageSavePath` + has no caller yet, so the gap is latent, not live. - **The rollback delete is prune's ONE carve-out, and only HALF of it is structural.** The citation and the full discriminator live at `package_rollback.cpp`'s header. "Did this call create it" is structural: only exclusively-created paths are @@ -88,7 +95,13 @@ belong to the verbs. an extension from `extension_list` when the user omits one — `pickPackageSavePath` re-appends `.rsbank` itself so the returned path is correct regardless of how that lands (the superseded Win32 path had `ofn.lpstrDefExt` for this; `GetUserFileName` - has no equivalent parameter). + has no equivalent parameter). The re-append is suffix-blind: it only skips when the + path already ends in `.rsbank`, so a path carrying a DIFFERENT extension gets + `.rsbank` appended after it (`mybank.bak` → `mybank.bak.rsbank`), unlike the + superseded `ofn.lpstrDefExt`, which appended only when the path had no extension at + all. Defensible for a format-locked export, but a real divergence from the old + picker's behavior — whoever tests the picker under `[verify — DAW]` should expect + the double-extension result on a path that already has one. - `pickPackageSavePath`'s `suggestedPath` doubles as the dialog's starting directory when it is a full path. The verbs should seed it from the project directory — passing a bare name leaves the dialog on REAPER's process working directory, which diff --git a/src/shell/package/package_io.cpp b/src/shell/package/package_io.cpp index 26cd7e6..c268742 100644 --- a/src/shell/package/package_io.cpp +++ b/src/shell/package/package_io.cpp @@ -155,6 +155,12 @@ PayloadBuffer PackageFileReader::readRange(std::uint64_t offset, std::uint64_t l // so would read past a truncated allocation) and a length past the sanity // ceiling, which exists so a merely large-but-real file size can't still hand // std::vector a multi-gigabyte demand. + // + // `length > size_` short-circuits before the two branches below ever see a real + // file, so neither is reachable without a genuine >4 GiB fixture — this guard + // ships unexercised by test_package_io.cpp, which covers past-the-end, + // starts-at-the-end, and zero-length only. The ordering (cheap size check first) + // is deliberate and correct; it is not reordered to make the branch testable. constexpr std::uint64_t kMaxReadRangeBytes = std::uint64_t{4} << 30; // 4 GiB if (!ok_ || length == 0 || length > size_ || offset > size_ - length || length > kMaxReadRangeBytes || diff --git a/src/shell/package/package_io.h b/src/shell/package/package_io.h index a33f66e..adf60ec 100644 --- a/src/shell/package/package_io.h +++ b/src/shell/package/package_io.h @@ -101,7 +101,9 @@ private: }; // One source file read whole as one entry's payload — a bank file IS the streaming -// unit. Empty on any failure, per PackageFileReader. +// unit. Empty on any failure, per PackageFileReader — including a source file over +// readRange's 4 GiB ceiling, which reads as empty exactly like an unreadable file; +// fileStatus() cannot tell the two apart either, since it only checks openability. PayloadBuffer readFilePayload(const std::string& absPath); // Export must tell a missing indexed file from an unreadable one in its refusal From 081b6f10281c93cdb07e2ffd58a95c924e89add7 Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 13:21:43 -0400 Subject: [PATCH 12/24] package: one bank leaves the project as one .rsbank, or the export refuses and says why Pure planner classifies missing/unreadable/unrepresentable and repairs transport names; the verb digests, streams and commits atomically over a const session. --- src/app/CMakeLists.txt | 3 +- src/app/main.cpp | 6 + src/core/package/CLAUDE.md | 14 + src/core/package/CMakeLists.txt | 5 + src/core/package/export_plan.cpp | 165 +++++++++ src/core/package/export_plan.h | 89 +++++ src/shell/actions/CLAUDE.md | 1 + src/shell/actions/package_export_action.cpp | 190 ++++++++++ src/shell/actions/package_export_action.h | 18 + src/shell/package/CLAUDE.md | 6 +- src/shell/package/CMakeLists.txt | 9 + src/shell/package/export_bank.cpp | 184 ++++++++++ src/shell/package/export_bank.h | 91 +++++ src/shell/panel/panel_bank_ops.cpp | 4 + tests/test_export_bank.cpp | 375 ++++++++++++++++++++ tests/test_export_plan.cpp | 268 ++++++++++++++ 16 files changed, 1426 insertions(+), 2 deletions(-) create mode 100644 src/core/package/export_plan.cpp create mode 100644 src/core/package/export_plan.h create mode 100644 src/shell/actions/package_export_action.cpp create mode 100644 src/shell/actions/package_export_action.h create mode 100644 src/shell/package/export_bank.cpp create mode 100644 src/shell/package/export_bank.h create mode 100644 tests/test_export_bank.cpp create mode 100644 tests/test_export_plan.cpp diff --git a/src/app/CMakeLists.txt b/src/app/CMakeLists.txt index 044ab51..fa863d4 100644 --- a/src/app/CMakeLists.txt +++ b/src/app/CMakeLists.txt @@ -46,13 +46,14 @@ add_library(reaper_reasampler MODULE ${REASAMPLER_SRC_DIR}/shell/actions/design_view_actions.cpp ${REASAMPLER_SRC_DIR}/shell/actions/bank_actions.cpp ${REASAMPLER_SRC_DIR}/shell/actions/prune_action.cpp + ${REASAMPLER_SRC_DIR}/shell/actions/package_export_action.cpp ${REASAMPLER_SRC_DIR}/shell/actions/ingest.cpp ${REASAMPLER_SRC_DIR}/shell/actions/arrange_drop_win.cpp ${REASAMPLER_SRC_DIR}/shell/actions/drag_out_win.cpp ${REASAMPLER_SRC_DIR}/shell/actions/instrument_drop_win.cpp ${REASAMPLER_SRC_DIR}/shell/persist/usage_scan.cpp ) -target_link_libraries(reaper_reasampler PRIVATE json wire file_bytes bank_model capture_paths capture_name peaks bank_grid mode_switch tab_strip view_mode_model view_tree guid_diff lane_keys solo_cache insert_plan render_settings render_window track_topology batch_capture tail_control capture_realtime bank_book wav_codec origin_ledger tracking_authority prune_reconcile prune_button app_version provenance drag_out instrument_drop theme component_geometry action_bar footer_bar overflow_menu mode_enable tooltip card_meta card_drag assignment_request bank_sync sample_usage bake_wire resample_name) +target_link_libraries(reaper_reasampler PRIVATE json wire file_bytes bank_model capture_paths capture_name peaks bank_grid mode_switch tab_strip view_mode_model view_tree guid_diff lane_keys solo_cache insert_plan render_settings render_window track_topology batch_capture tail_control capture_realtime bank_book wav_codec origin_ledger tracking_authority prune_reconcile prune_button app_version provenance drag_out instrument_drop theme component_geometry action_bar footer_bar overflow_menu mode_enable tooltip card_meta card_drag assignment_request bank_sync sample_usage bake_wire resample_name export_bank package_pickers) # NOT linked here, deliberately: sampler_core / pitch_shift / the filter. The instrument # renders its own bake in its own process, which is what keeps the extension's link graph # free of the voice engine — a link edge to it here means the design drifted. diff --git a/src/app/main.cpp b/src/app/main.cpp index 57696ae..599003c 100644 --- a/src/app/main.cpp +++ b/src/app/main.cpp @@ -26,6 +26,7 @@ #include "shell/actions/action_registry.h" // the registration table #include "shell/actions/bank_actions.h" // multi-bank action family #include "shell/actions/design_view_actions.h" // Design View action family +#include "shell/actions/package_export_action.h" // bank-package export action body #include "core/wire/bake_wire.h" // kBakeActionSuffix (the shared action id) #include "shell/capture/bake_land.h" // resample-bake landing action body #include "shell/capture/capture_batch.h" // batch + recapture action bodies @@ -91,6 +92,9 @@ static void RunCancelRealtime(int) { capture::RunCancelRealtime(g_session); } static void RunRecaptureFromSource(int) { capture::RunRecaptureFromSource(g_session); } static void RunRenderTrackInPlace(int) { capture::RunRenderTrackInPlace(g_session); } static void RunResampleBake(int) { capture::RunResampleBake(g_session); } +static void RunExportBankPackage(int) { + reasampler::doBankPackageExport(g_session, g_session.book().activeBankId()); +} static void RunShowVersion(int) { // On-demand only — no unconditional startup print (routine console chatter pops // the console window). @@ -147,6 +151,8 @@ static std::vector buildMainActionTable() { rows.push_back({reasampler::wire::kBakeActionSuffix, "land pending ReaSampler 9000 resample bake", &RunResampleBake}); + rows.push_back({"EXPORT_BANK_PACKAGE", "export active bank as package", + &RunExportBankPackage}); rows.push_back({"SHOW_VERSION", "show version", &RunShowVersion}); return rows; diff --git a/src/core/package/CLAUDE.md b/src/core/package/CLAUDE.md index 9114f83..80f475f 100644 --- a/src/core/package/CLAUDE.md +++ b/src/core/package/CLAUDE.md @@ -73,6 +73,12 @@ landing after the format. here, computed where payloads are streamed (shell). The bank's `slot_map` rides along. Unknown keys skip at every level; duplicate entry names are rejected both ways. +- `export_plan` — the pure export decision over value inputs (the bank's members + plus the shell's per-file probe result): the verdict (`Ready` / `Incomplete` / + `Refused`), the transport name per shipping entry, and what is excluded and why + (missing / unreadable / an index record the format cannot represent). Owns the + name repair the codec's refusal backstops, and normalizes each shipping record's + `relativePath` to the bare package name — see the transport-name gotcha below. - `bank_package` — framing and arithmetic composing the two above: `encodePackage` (prefix bytes + layout + total size, stamping this build's ladder pair and `version::stampVersion()`), `decodePackage` (prefix + observed @@ -115,6 +121,14 @@ landing after the format. `minReaderVersion` bump, not additive: an old reader would otherwise compare a stored digest against bytes hashed the new way and silently misjudge corruption. +- **A written package carries no path in ANY field.** `isValidNestedSamplePath` + permits a relative `relativePath` because a *record* may hold one, but + `export_plan` writes each shipping entry's `relativePath` as its bare package + name, so an emitted manifest has no separator anywhere and the entry name is the + single naming authority on both sides. The directory component it drops carries + no information — the bank subfolder is a fixed `capture_paths` constant the + importer re-spells. The nested-path rule stays as the decode-side backstop for a + package this build did not write. - **Obligation on the export track: sanitize, don't relay the refusal.** `serializeManifest` returns one indistinguishable `nullopt` for every rejection — an unrepresentable name, a case-folded collision, a traversing nested path, a diff --git a/src/core/package/CMakeLists.txt b/src/core/package/CMakeLists.txt index fc7fd2d..32ae08d 100644 --- a/src/core/package/CMakeLists.txt +++ b/src/core/package/CMakeLists.txt @@ -6,6 +6,11 @@ reasampler_pure_library(package_manifest LINK PUBLIC bank_model slot_map PRIVATE package_format json) reasampler_test(package_manifest LINK package_manifest) +reasampler_pure_library(export_plan + SOURCES export_plan.cpp + LINK PUBLIC package_manifest PRIVATE package_format) +reasampler_test(export_plan LINK export_plan package_format) + # bytes.h is header-only (see src/core/wire/CLAUDE.md) — no wire link edge needed. reasampler_pure_library(bank_package SOURCES bank_package.cpp diff --git a/src/core/package/export_plan.cpp b/src/core/package/export_plan.cpp new file mode 100644 index 0000000..aa4e6a0 --- /dev/null +++ b/src/core/package/export_plan.cpp @@ -0,0 +1,165 @@ +// export_plan.cpp — see export_plan.h for the contract. + +#include "core/package/export_plan.h" + +#include + +#include "core/package/package_format.h" + +namespace reasampler::package { + +namespace { + +// The bare file name a bank-relative path ends in. Separators are matched in both +// spellings: a persisted index may hold either on Windows. +std::string baseNameOf(const std::string& path) { + const std::size_t sep = path.find_last_of("/\\"); + return sep == std::string::npos ? path : path.substr(sep + 1); +} + +// Trailing dots and spaces are stripped at file creation on Windows, so a name +// carrying them would collide with its stripped twin (isValidEntryName refuses them +// for that reason). +std::string stripTrailingDotsAndSpaces(std::string s) { + while (!s.empty() && (s.back() == '.' || s.back() == ' ')) s.pop_back(); + return s; +} + +// Truncates to at most `max` bytes without splitting a UTF-8 sequence — a split one +// would leave the name ill-formed, which isValidEntryName refuses outright. +std::string truncateUtf8(std::string s, std::size_t max) { + if (s.size() <= max) return s; + s.resize(max); + while (!s.empty() && (static_cast(s.back()) & 0xC0) == 0x80) s.pop_back(); + if (!s.empty() && static_cast(s.back()) >= 0xC0) s.pop_back(); + return s; +} + +// `name` with `suffix` inserted before its extension, trimmed so the result still +// fits the entry-name cap. +std::string insertSuffix(const std::string& name, const std::string& suffix) { + const std::size_t dot = name.find_last_of('.'); + const bool hasExt = dot != std::string::npos && dot > 0; + std::string stem = hasExt ? name.substr(0, dot) : name; + const std::string ext = hasExt ? name.substr(dot) : std::string(); + const std::size_t room = kMaxEntryNameBytes - suffix.size() - ext.size(); + stem = truncateUtf8(std::move(stem), room); + return stem + suffix + ext; +} + +bool nameTaken(const std::string& candidate, const std::vector& taken) { + for (const std::string& t : taken) + if (sameEntryName(candidate, t)) return true; + return false; +} + +// A transport name distinct from every name already claimed, under the format's own +// case-folding equivalence (two names differing only by ASCII case would extract onto +// one file on Windows and default APFS). +std::string uniqueEntryName(const std::string& base, const std::vector& taken) { + if (!nameTaken(base, taken)) return base; + // Bounded by construction: each iteration either returns or collides with a + // distinct member of `taken`, and the suffixed names are pairwise distinct. + std::string candidate = base; + for (std::size_t n = 2; n <= taken.size() + 2; ++n) { + candidate = insertSuffix(base, "_" + std::to_string(n)); + if (!nameTaken(candidate, taken) && isValidEntryName(candidate)) return candidate; + } + return candidate; +} + +// What BankModel::add and the manifest's nested-path rule together accept — the pair +// package_manifest::serializeManifest checks per entry. The codec's refusal is the +// backstop; classifying here is what lets the export name the offending entry. +bool recordRepresentable(const model::Sample& s) { + return !s.id.empty() && isValidNestedSamplePath(s.relativePath); +} + +ExcludedEntry excludedFrom(const model::Sample& s, ExclusionReason reason) { + ExcludedEntry e; + e.sampleId = s.id; + e.displayName = s.displayName; + e.relativePath = s.relativePath; + e.reason = reason; + return e; +} + +} // namespace + +std::string sanitizeEntryName(const std::string& rawFileName) { + std::string n = rawFileName; + for (char& c : n) { + const unsigned char u = static_cast(c); + if (u < 0x20 || u == 0x7F || u == '/' || u == '\\' || u == ':' || u == '*' || + u == '?' || u == '|' || u == '<' || u == '>' || u == '"') + c = '_'; + } + // One byte of headroom so the prefix repair below still fits the cap. + n = stripTrailingDotsAndSpaces(truncateUtf8(std::move(n), kMaxEntryNameBytes - 1)); + if (isValidEntryName(n)) return n; + + // One prefix answers every remaining reserved form at once: "." / "..", a DOS + // device name, and a name the strips emptied. + std::string prefixed = stripTrailingDotsAndSpaces("_" + n); + if (isValidEntryName(prefixed)) return prefixed; + + // Ill-formed UTF-8 is what is left, and isValidEntryName is the only authority on + // it here, so fold the whole non-ASCII range rather than re-deriving the scanner. + for (char& c : prefixed) + if (static_cast(c) >= 0x80) c = '_'; + prefixed = stripTrailingDotsAndSpaces(prefixed); + return isValidEntryName(prefixed) ? prefixed : std::string("entry"); +} + +ExportPlan planExport(const ExportInputs& in) { + ExportPlan plan; + plan.manifest.bankDisplayName = in.bankDisplayName; + + bool anyAbsent = false; + bool anyUnrepresentable = false; + std::vector takenNames; + std::vector shippedIds; + + for (const ExportCandidate& c : in.candidates) { + if (!recordRepresentable(c.sample)) { + plan.excluded.push_back( + excludedFrom(c.sample, ExclusionReason::RecordUnrepresentable)); + anyUnrepresentable = true; + continue; + } + if (c.fileState != SourceFileState::Present) { + plan.excluded.push_back(excludedFrom( + c.sample, c.fileState == SourceFileState::Unreadable + ? ExclusionReason::FileUnreadable + : ExclusionReason::FileMissing)); + anyAbsent = true; + continue; + } + + PackageEntry e; + e.sample = c.sample; + e.fileName = uniqueEntryName(sanitizeEntryName(baseNameOf(c.sample.relativePath)), + takenNames); + // The transport record names its payload by the package name and nothing + // else, so the manifest carries no path at all — the bank subfolder is a + // fixed constant the importer re-spells through capture_paths. + e.sample.relativePath = e.fileName; + + takenNames.push_back(e.fileName); + shippedIds.push_back(c.sample.id); + plan.sourceRelativePaths.push_back(c.sample.relativePath); + plan.manifest.entries.push_back(std::move(e)); + } + + // Display positions follow membership: an excluded entry's slot marker would name + // a sample the package does not carry. + plan.manifest.slots = in.slots; + plan.manifest.slots.reconcile(shippedIds); + + plan.verdict = anyUnrepresentable ? ExportVerdict::Refused + : anyAbsent ? ExportVerdict::Incomplete + : ExportVerdict::Ready; + return plan; +} + +} // namespace reasampler::package diff --git a/src/core/package/export_plan.h b/src/core/package/export_plan.h new file mode 100644 index 0000000..77df29d --- /dev/null +++ b/src/core/package/export_plan.h @@ -0,0 +1,89 @@ +#pragma once +// export_plan — the pure export decision: which bank entries ship, what each one is +// named inside the package, what is absent, and therefore whether the export may +// proceed at all. Values in, verdict out — the shell probes the filesystem and hands +// the results here. Pure: no filesystem, no host types. + +#include +#include + +#include "core/model/bank_model.h" +#include "core/model/slot_map.h" +#include "core/package/package_manifest.h" + +namespace reasampler::package { + +// What the shell's filesystem probe found for one indexed entry. Missing and +// Unreadable stay distinct all the way to the refusal message: the file is gone vs. +// the file is there and will not open, which have opposite recoveries. +enum class SourceFileState { + Present, + Missing, + Unreadable, +}; + +struct ExportCandidate { + model::Sample sample; + SourceFileState fileState = SourceFileState::Missing; +}; + +// One bank as the planner sees it: the display name that rides in the manifest +// envelope, the members in bank insertion order, and the bank's display positions. +struct ExportInputs { + std::string bankDisplayName; + std::vector candidates; + model::SlotMap slots; +}; + +// Why an indexed entry cannot ship. +enum class ExclusionReason { + FileMissing, + FileUnreadable, + // The index record itself cannot be written: an empty id, or a relativePath the + // format's nested-path rule refuses. Not something a confirm can proceed past. + RecordUnrepresentable, +}; + +struct ExcludedEntry { + std::string sampleId; + std::string displayName; + std::string relativePath; + ExclusionReason reason = ExclusionReason::FileMissing; +}; + +enum class ExportVerdict { + Ready, // every candidate ships + Incomplete, // a file is absent or unreadable; the rest may ship behind an explicit confirm + Refused, // an index record the format cannot represent — no confirm path +}; + +struct ExportPlan { + ExportVerdict verdict = ExportVerdict::Ready; + + // Entries in bank order, each carrying its transport name and its record. The + // shell measures `byteLength`/`byteHash` from the payload, so they are 0/"" here; + // `exportTimestamp` is the shell's clock read and is 0 here too. + PackageManifest manifest; + + // Where each shipping entry's bytes are read from, parallel to + // `manifest.entries` — the record's own relativePath is normalized to the bare + // package name (see the transport-name note in this directory's CLAUDE.md), so + // the source spelling has to survive separately. + std::vector sourceRelativePaths; + + std::vector excluded; +}; + +ExportPlan planExport(const ExportInputs& in); + +// The smallest repair of one bare file name that satisfies isValidEntryName — +// separators, reserved characters and control bytes to '_', an over-long name +// truncated on a UTF-8 boundary, and an underscore prefix for the reserved forms +// ("." / ".." / a DOS device name). Never returns a name isValidEntryName refuses. +// +// A bank ingested on macOS/Linux legitimately holds names Windows cannot spell, and +// relaying the codec's one indistinguishable refusal would make a single such file +// an unactionable total failure of the whole export. +std::string sanitizeEntryName(const std::string& rawFileName); + +} // namespace reasampler::package diff --git a/src/shell/actions/CLAUDE.md b/src/shell/actions/CLAUDE.md index befbaed..6b4b412 100644 --- a/src/shell/actions/CLAUDE.md +++ b/src/shell/actions/CLAUDE.md @@ -39,6 +39,7 @@ is owned by other directories and only skinned here. ## Modules - `shell/actions` (`action_registry` / `design_view_actions` / `bank_actions` / `prune_action`) — the bindable action families, all routed via the `command_id`/`gaccel`/`hookcommand` contract. `action_registry` owns the shared registration plumbing (interned channel-qualified id strings; register and mirror-unregister present the identical pointer) **and the Q-W6 registration TABLE**: `main.cpp`'s own family (capture scopes, panel toggle, insert, batch, realtime, recapture, version) is one `ActionTableRow` array — suffix, phrase, flat function-pointer handler — that registration, hookcommand dispatch, and the unload mirror-unregister all iterate, so adding an action touches the table only (OCP). Bank mutations flow through the promptless `shell/bank_ops` verbs (`bankOp*` + `persistBankOp`, taking `ReaSamplerSession&`), which the panel menus and `bank_actions` consume as thin UX skins. **Every bank index verb wraps its mutation in a batched REAPER undo point (`Undo_BeginBlock2`/`EndBlock2`, `UNDO_STATE_MISCCFG`) so one bank operation is one Ctrl-Z.** The prune action (`prune_action`, `BANK_PRUNE_FOLDER`) is **the ONLY file-deletion action in the system**; it opens no undo point (file deletion is not REAPER-undoable). `BANK_PRUNE_FOLDER` halts on `blockedByTracking` and prints each blocker that fired, with recovery instructions. +- `package_export_action` — the "export bank as package" skin: survey and report first, confirm what is absent (and, separately, a destination being replaced), pick a destination, write. Every prompt in the flow lives here so `shell/package/export_bank` stays promptless. Read-only against the project — it holds the session by `const&`, so no ext-state write, generation bump or undo point is reachable. Registration rides `main.cpp`'s action table (`EXPORT_BANK_PACKAGE`); the panel's tab menu is the second skin over the same body. - `drag_out_win` — OS drag-out shell: Windows OLE `DoDragDrop`/`CF_HDROP`, copy-only (`DROPEFFECT_MOVE` not offered); macOS/Linux via `SWELL_InitiateDragDropOfFileList`. - `instrument_drop_win` — instrument-drop shell: `probeDropTarget` resolves a screen point to a track + a `ReaperSurface` (via the pure `wire::classifyReaperSurface`, whose token rules `core/wire/CLAUDE.md` owns), and the drop half adds a ReaSampler 9000 instance and applies the dragged capture's state via a transient `.vstpreset` + `TrackFX_SetPreset` (the former `TrackFX_SetNamedConfigParm` "vst_chunk" write was silently unappliable for VST3). Exposes `loadInstrumentOntoTrack` (inner half, no own undo block) and `performInstrumentDrop` (wraps in its own undo block). **Never captures, never writes the bank, never inserts a timeline item.** - `arrange_drop_win` — the drag-out gesture's arrange outcome: `arrangeTimeAtScreenX` (pointer column → time via `GetSet_ArrangeView2`'s one-pixel-span reading — inferred, not SDK-documented) and `performArrangeDrop` (snap the drop time, then one `InsertMedia` per capture on the pointer's track — assumed, not confirmed, to land end-to-end via REAPER's own cursor advance — in ONE undo block, counting only InsertMedia's reported successes, with the caller's track selection and edit cursor restored). The one timeline-placing shell here, per the invariant above; it never captures and never writes the bank. diff --git a/src/shell/actions/package_export_action.cpp b/src/shell/actions/package_export_action.cpp new file mode 100644 index 0000000..9b83c6c --- /dev/null +++ b/src/shell/actions/package_export_action.cpp @@ -0,0 +1,190 @@ +// package_export_action.cpp — see package_export_action.h for the contract this TU +// preserves. main.cpp owns the API pointers; this TU gets them extern. + +#include "shell/actions/package_export_action.h" + +#include +#include +#include +#include +#include + +#include "core/capture/capture_paths.h" // projectDirOfRpp, sanitizeStem +#include "shell/package/export_bank.h" +#include "shell/package/package_pickers.h" +#include "shell/persist/session.h" + +#define REAPERAPI_MINIMAL +#define REAPERAPI_WANT_EnumProjects +#define REAPERAPI_WANT_ShowConsoleMsg +#define REAPERAPI_WANT_ShowMessageBox +#include "reaper_plugin_functions.h" + +namespace reasampler { + +namespace { + +constexpr const char* kUnsavedProjectMsg = + "ReaSampler export: save the project first -- an unsaved project has no bank folder " + "to read from.\n"; + +std::string currentProjectDir() { + std::vector buf(4096, '\0'); + EnumProjects(-1, buf.data(), static_cast(buf.size())); + return capture::projectDirOfRpp(std::string(buf.data())); +} + +// 6 == YES; anything else cancels (SDK ~6544). +bool confirmed(const std::string& msg, const char* title) { + return ShowMessageBox(msg.c_str(), title, 4) == 6; +} + +std::string entryLine(const package::ExcludedEntry& e) { + const char* why = e.reason == package::ExclusionReason::FileMissing ? "missing" + : e.reason == package::ExclusionReason::FileUnreadable ? "unreadable" + : "unusable index record"; + return " " + (e.displayName.empty() ? e.sampleId : e.displayName) + " [" + why + + "] " + e.relativePath + "\n"; +} + +// `maxLines` == 0 lists everything (the console record); a positive cap keeps a +// confirm dialog readable on a bank with hundreds of absent files, prune's own +// truncate-the-confirm-not-the-report discipline. +std::string excludedManifest(const std::vector& excluded, + std::size_t maxLines) { + std::string msg; + std::size_t shown = 0; + for (const package::ExcludedEntry& e : excluded) { + if (maxLines != 0 && shown == maxLines) { + msg += " ... (" + std::to_string(excluded.size() - shown) + + " more, listed in the console)\n"; + break; + } + msg += entryLine(e); + ++shown; + } + return msg; +} + +void reportOutcome(const ExportOutcome& out, const std::string& destPath) { + switch (out.status) { + case ExportStatus::Written: + ShowConsoleMsg(("ReaSampler export: wrote " + std::to_string(out.entriesWritten) + + " entry/entries (" + std::to_string(out.bytesWritten) + + " bytes) to " + destPath + "\n") + .c_str()); + return; + case ExportStatus::SourceReadFailed: + ShowConsoleMsg(("ReaSampler export: ABORTED -- \"" + out.offendingName + + "\" could not be read. Nothing was written.\n") + .c_str()); + return; + case ExportStatus::SourceChanged: + ShowConsoleMsg(("ReaSampler export: ABORTED -- \"" + out.offendingName + + "\" changed on disk while the package was being written. " + "Nothing was written; run the export again.\n") + .c_str()); + return; + case ExportStatus::EncodeFailed: + ShowConsoleMsg("ReaSampler export: ABORTED -- this bank could not be encoded " + "as a package. Nothing was written.\n"); + return; + // Both refusals are re-derived from a FRESH plan, so reaching them after the + // survey means the bank changed under the export, not that the user declined. + case ExportStatus::RefusedIncomplete: + case ExportStatus::RefusedUnrepresentable: + ShowConsoleMsg("ReaSampler export: ABORTED -- the bank changed between the " + "report and the write. Nothing was written; run the export " + "again.\n"); + return; + case ExportStatus::RefusedDestinationExists: + ShowConsoleMsg("ReaSampler export: cancelled -- nothing was written.\n"); + return; + case ExportStatus::NoSuchBank: + ShowConsoleMsg("ReaSampler export: that bank no longer exists.\n"); + return; + case ExportStatus::NoProjectDir: + ShowConsoleMsg(kUnsavedProjectMsg); + return; + case ExportStatus::WriteFailed: + ShowConsoleMsg(("ReaSampler export: FAILED writing " + destPath + + ". No package was left behind; any file already at that path is " + "untouched.\n") + .c_str()); + return; + } +} + +} // namespace + +void doBankPackageExport(const ReaSamplerSession& session, const std::string& bankId) { + const std::string projectDir = currentProjectDir(); + if (projectDir.empty()) { + ShowConsoleMsg(kUnsavedProjectMsg); + return; + } + + // Report before acting, and before the picker opens: a refusal the user cannot + // act on should not cost them a trip through a save dialog first. + const ExportSurvey survey = surveyBankExport(session, projectDir, bankId); + if (!survey.bankFound) { + ShowConsoleMsg("ReaSampler export: no such bank.\n"); + return; + } + const std::string bankName = survey.plan.manifest.bankDisplayName; + + bool allowIncomplete = false; + if (survey.plan.verdict == package::ExportVerdict::Refused) { + ShowConsoleMsg(("ReaSampler export: ABORTED -- \"" + bankName + + "\" holds index record(s) a package cannot carry. Nothing was " + "written.\n" + + excludedManifest(survey.plan.excluded, 0)) + .c_str()); + return; + } + if (survey.plan.verdict == package::ExportVerdict::Incomplete) { + const std::string headline = + "ReaSampler export: \"" + bankName + "\" has " + + std::to_string(survey.plan.excluded.size()) + + " entry/entries whose file is missing or unreadable:\n"; + ShowConsoleMsg((headline + excludedManifest(survey.plan.excluded, 0)).c_str()); + if (!confirmed(headline + excludedManifest(survey.plan.excluded, 10) + + "\nExport the " + + std::to_string(survey.plan.manifest.entries.size()) + + " present entry/entries anyway?", + "ReaSampler: incomplete bank")) { + ShowConsoleMsg("ReaSampler export: cancelled -- nothing was written.\n"); + return; + } + allowIncomplete = true; + } + + // The bank's own name, not the project's: the artifact is a bank, and a user + // exporting three banks from one project needs three distinguishable files. + const std::string suggested = + projectDir + "/" + capture::sanitizeStem(bankName) + ".rsbank"; + std::string dest; + if (!pickPackageSavePath(suggested, dest)) return; // user cancelled the picker + + ExportRequest req; + req.projectDir = projectDir; + req.bankId = bankId; + req.destAbsPath = dest; + req.exportTimestamp = static_cast(std::time(nullptr)); + req.allowIncomplete = allowIncomplete; + + ExportOutcome out = exportBank(session, req); + if (out.status == ExportStatus::RefusedDestinationExists) { + if (!confirmed("A file already exists at:\n\n " + dest + + "\n\nReplace it with this bank package?", + "ReaSampler: replace package")) { + ShowConsoleMsg("ReaSampler export: cancelled -- nothing was written.\n"); + return; + } + req.allowOverwrite = true; + out = exportBank(session, req); + } + reportOutcome(out, dest); +} + +} // namespace reasampler diff --git a/src/shell/actions/package_export_action.h b/src/shell/actions/package_export_action.h new file mode 100644 index 0000000..695d5a3 --- /dev/null +++ b/src/shell/actions/package_export_action.h @@ -0,0 +1,18 @@ +#pragma once +// package_export_action — the "export bank as package" action body: survey and +// report first, confirm what is absent, pick a destination, write. Every prompt in +// the flow lives here; shell/package/export_bank stays promptless. Registration and +// dispatch for its FOREVER-STABLE id ride main.cpp's action table. + +#include + +namespace reasampler { + +class ReaSamplerSession; + +// Exports one bank (the pool included — it is structurally a bank) to a .rsbank the +// user picks. Read-only against the project: the session is const, so no ext-state +// write, generation bump or undo point is reachable from here. +void doBankPackageExport(const ReaSamplerSession& session, const std::string& bankId); + +} // namespace reasampler diff --git a/src/shell/package/CLAUDE.md b/src/shell/package/CLAUDE.md index b4db0c4..0ab8adc 100644 --- a/src/shell/package/CLAUDE.md +++ b/src/shell/package/CLAUDE.md @@ -54,7 +54,10 @@ belong to the verbs. `PackageFileWriter` — after any extension append — and get its own consent if that re-checked path is `Present`; the dialog's confirm only ever covered the pre-append path. Not fixed at this seam: prompting is verb-level UX, and `pickPackageSavePath` - has no caller yet, so the gap is latent, not live. + has no caller yet, so the gap is latent, not live. **Closed on the export side:** + `exportBank` re-checks `fileStatus()` on the post-append path and refuses + `RefusedDestinationExists` until the caller sets `allowOverwrite`, which + `package_export_action` does only after its own confirm naming that exact path. - **The rollback delete is prune's ONE carve-out, and only HALF of it is structural.** The citation and the full discriminator live at `package_rollback.cpp`'s header. "Did this call create it" is structural: only exclusively-created paths are @@ -77,6 +80,7 @@ belong to the verbs. - `package_io` — every filesystem act the verbs need: `PayloadBuffer` (move-only payload + the `alive()` seam counter), `PackageFileWriter` (append-only temp+atomic-rename writer), `PackageFileReader` (seek-and-read one range per call, range-checked against the real file size), `readFilePayload` (one source file as one entry's payload), `fileStatus` (Present/Absent/Unreadable — export's refusal message must distinguish the last two, and an empty payload cannot), `writeFileExclusive` (exclusive create + write, self-cleaning on a partial write), and `listFolderFileNames` (bare UTF-8 names, sorted, non-recursive, non-throwing). REAPER-free; tested without a DAW. - `package_rollback` — `LandedFileJournal`: `writeLandedFile` (exclusive-create land, path resolved absolute, recorded on success only), `markIndexCommitted` (disarms the journal), and `rollback` (deletes exactly the recorded set, hard unlink, tolerating a vanished file; refuses once disarmed). REAPER-free; tested without a DAW. - `package_pickers` — `pickPackageForImport` and `pickPackageSavePath`, both `GetUserFileName`. Compile-only until the verbs land; nothing here can be exercised in a unit test. +- `export_bank` — the promptless export verb, in three composable public steps: `surveyBankExport` (the read-only plan, report-before-acting), `digestSources` (measures each entry's length + `hashBytes` digest, one payload at a time), and `writePackageFile` (prefix, then each payload re-read and re-verified against that digest before it is appended, then commit). `exportBank` composes the three and gates on the plan verdict, the incomplete confirm and the destination confirm. The session arrives **const** — every mutator on it is non-const, so "an export writes no ext state, opens no undo point and never bumps the generation" is enforced by the type rather than remembered. Reads the session through inline accessors only, which is why its tests link and run without a DAW. ## Gotchas diff --git a/src/shell/package/CMakeLists.txt b/src/shell/package/CMakeLists.txt index aff4414..caa87da 100644 --- a/src/shell/package/CMakeLists.txt +++ b/src/shell/package/CMakeLists.txt @@ -9,6 +9,15 @@ reasampler_test(package_io LINK package_io) reasampler_pure_library(package_rollback SOURCES package_rollback.cpp LINK PUBLIC package_io) reasampler_test(package_rollback LINK package_rollback) +# export_bank reads the live session through ReaSamplerSession's INLINE accessors only, +# so it pulls in no REAPER-facing TU and its tests link (and run) without a DAW. +reasampler_pure_library(export_bank + SOURCES export_bank.cpp + LINK PUBLIC export_plan bank_package package_io PRIVATE capture_paths wav_codec) +reasampler_test(export_bank + LINK export_bank bank_book slot_map view_mode_model tail_control origin_ledger + tracking_authority prune_reconcile app_version capture_paths) + # The pickers call the REAPER API, so no test target can exercise them; declared as a # library so the TU stays compiled. reaper_plugin.h pulls SWELL in on non-Windows. add_library(package_pickers STATIC package_pickers.cpp) diff --git a/src/shell/package/export_bank.cpp b/src/shell/package/export_bank.cpp new file mode 100644 index 0000000..d2aee61 --- /dev/null +++ b/src/shell/package/export_bank.cpp @@ -0,0 +1,184 @@ +// export_bank.cpp — see export_bank.h for the contract. +// +// wav_codec is called for hashBytes ONLY. Payload bytes are copied and hashed, never +// rebuilt, trimmed, normalized or collapsed — the capture path's mono collapse must +// not reach an export. + +#include "shell/package/export_bank.h" + +#include +#include +#include + +#include "core/capture/capture_paths.h" // resolveBankFile — the index's relative -> absolute +#include "core/capture/wav_codec.h" // hashBytes +#include "core/model/bank_book.h" +#include "shell/package/package_io.h" +#include "shell/persist/session.h" // ReaSamplerSession — read through its inline book() only + +namespace reasampler { + +namespace { + +package::SourceFileState stateOf(const std::string& absPath) { + switch (fileStatus(absPath)) { + case FileStatus::Present: return package::SourceFileState::Present; + case FileStatus::Unreadable: return package::SourceFileState::Unreadable; + case FileStatus::Absent: break; + } + return package::SourceFileState::Missing; +} + +std::vector absoluteSources(const std::string& projectDir, + const std::vector& relativePaths) { + std::vector out; + out.reserve(relativePaths.size()); + for (const std::string& rel : relativePaths) + out.push_back(capture::resolveBankFile(projectDir, rel)); + return out; +} + +} // namespace + +ExportSurvey surveyBankExport(const ReaSamplerSession& session, + const std::string& projectDir, + const std::string& bankId) { + ExportSurvey survey; + const Bank* bank = session.book().bank(bankId); + if (!bank) return survey; + survey.bankFound = true; + + package::ExportInputs inputs; + inputs.bankDisplayName = bank->displayName; + inputs.slots = bank->slots; + for (const model::Sample& s : bank->index.all()) { + package::ExportCandidate c; + c.sample = s; + c.fileState = stateOf(capture::resolveBankFile(projectDir, s.relativePath)); + inputs.candidates.push_back(std::move(c)); + } + survey.plan = package::planExport(inputs); + return survey; +} + +bool digestSources(package::PackageManifest& manifest, + const std::vector& sourceAbsPaths, + std::string& outFailedName) { + outFailedName.clear(); + if (sourceAbsPaths.size() != manifest.entries.size()) return false; + for (std::size_t i = 0; i < manifest.entries.size(); ++i) { + const PayloadBuffer payload = readFilePayload(sourceAbsPaths[i]); + if (payload.empty()) { + outFailedName = manifest.entries[i].fileName; + return false; + } + manifest.entries[i].byteLength = payload.size(); + manifest.entries[i].byteHash = capture::hashBytes(payload.data(), payload.size()); + } + return true; +} + +ExportOutcome writePackageFile(const package::EncodedPackage& encoded, + const package::PackageManifest& manifest, + const std::vector& sourceAbsPaths, + const std::string& destAbsPath) { + ExportOutcome out; + if (sourceAbsPaths.size() != manifest.entries.size() || + encoded.layout.size() != manifest.entries.size()) { + out.status = ExportStatus::EncodeFailed; + return out; + } + + // Every early return below abandons the writer through its destructor, which + // removes the temp and leaves the destination untouched. + PackageFileWriter writer(destAbsPath); + if (!writer.ok() || !writer.appendRaw(encoded.prefix.data(), encoded.prefix.size())) { + out.status = ExportStatus::WriteFailed; + return out; + } + std::uint64_t written = encoded.prefix.size(); + + for (std::size_t i = 0; i < manifest.entries.size(); ++i) { + const package::PackageEntry& entry = manifest.entries[i]; + const PayloadBuffer payload = readFilePayload(sourceAbsPaths[i]); + if (payload.empty()) { + out.status = ExportStatus::SourceReadFailed; + out.offendingName = entry.fileName; + return out; + } + if (payload.size() != entry.byteLength || + capture::hashBytes(payload.data(), payload.size()) != entry.byteHash) { + out.status = ExportStatus::SourceChanged; + out.offendingName = entry.fileName; + return out; + } + if (!writer.appendPayload(payload)) { + out.status = ExportStatus::WriteFailed; + return out; + } + written += payload.size(); + } + + if (written != encoded.totalSize || !writer.commit()) { + out.status = ExportStatus::WriteFailed; + return out; + } + out.status = ExportStatus::Written; + out.entriesWritten = manifest.entries.size(); + out.bytesWritten = written; + return out; +} + +ExportOutcome exportBank(const ReaSamplerSession& session, const ExportRequest& req) { + ExportOutcome out; + if (req.projectDir.empty()) { + out.status = ExportStatus::NoProjectDir; + return out; + } + + const ExportSurvey survey = surveyBankExport(session, req.projectDir, req.bankId); + if (!survey.bankFound) { + out.status = ExportStatus::NoSuchBank; + return out; + } + out.bankDisplayName = survey.plan.manifest.bankDisplayName; + out.excluded = survey.plan.excluded; + + if (survey.plan.verdict == package::ExportVerdict::Refused) { + out.status = ExportStatus::RefusedUnrepresentable; + return out; + } + if (survey.plan.verdict == package::ExportVerdict::Incomplete && !req.allowIncomplete) { + out.status = ExportStatus::RefusedIncomplete; + return out; + } + // The save dialog's own overwrite confirm covered the path the USER chose, which + // is not necessarily the path handed here (the picker re-appends `.rsbank`), so + // consent for the real target is re-taken by the skin. + if (!req.allowOverwrite && fileStatus(req.destAbsPath) == FileStatus::Present) { + out.status = ExportStatus::RefusedDestinationExists; + return out; + } + + package::PackageManifest manifest = survey.plan.manifest; + manifest.exportTimestamp = req.exportTimestamp; + const std::vector sources = + absoluteSources(req.projectDir, survey.plan.sourceRelativePaths); + + if (!digestSources(manifest, sources, out.offendingName)) { + out.status = ExportStatus::SourceReadFailed; + return out; + } + const std::optional encoded = package::encodePackage(manifest); + if (!encoded) { + out.status = ExportStatus::EncodeFailed; + return out; + } + + ExportOutcome written = writePackageFile(*encoded, manifest, sources, req.destAbsPath); + written.bankDisplayName = out.bankDisplayName; + written.excluded = std::move(out.excluded); + return written; +} + +} // namespace reasampler diff --git a/src/shell/package/export_bank.h b/src/shell/package/export_bank.h new file mode 100644 index 0000000..a6deae8 --- /dev/null +++ b/src/shell/package/export_bank.h @@ -0,0 +1,91 @@ +// shell/package/export_bank — the promptless bank-export verb: survey, digest, +// stream, commit. No prompts and no message boxes (shell/actions/ +// package_export_action is the skin). The session arrives CONST, which is how "an +// export writes no ext state, opens no undo point and never bumps the bank +// generation" is enforced rather than remembered — every mutator on the session is +// non-const. Blocking I/O: UI-thread actions only. + +#pragma once + +#include +#include +#include +#include + +#include "core/package/bank_package.h" +#include "core/package/export_plan.h" + +namespace reasampler { + +class ReaSamplerSession; + +struct ExportRequest { + std::string projectDir; // absolute; the root the index's relative paths hang off + std::string bankId; + std::string destAbsPath; // the .rsbank to write + std::int64_t exportTimestamp = 0; // manifest envelope; the caller's clock read + // Both default false and are set ONLY after the skin's explicit confirm: one + // lists what is absent, the other names the destination being replaced. + bool allowIncomplete = false; + bool allowOverwrite = false; +}; + +enum class ExportStatus { + Written, + NoSuchBank, + NoProjectDir, + RefusedIncomplete, + RefusedUnrepresentable, + RefusedDestinationExists, + SourceReadFailed, // a file the plan classified Present would not read, or is empty + SourceChanged, // a payload's bytes moved between the digest pass and the stream pass + EncodeFailed, + WriteFailed, +}; + +struct ExportOutcome { + ExportStatus status = ExportStatus::WriteFailed; + std::size_t entriesWritten = 0; + std::uint64_t bytesWritten = 0; + std::string bankDisplayName; + std::vector excluded; + std::string offendingName; // the entry a SourceReadFailed / SourceChanged names +}; + +struct ExportSurvey { + bool bankFound = false; + package::ExportPlan plan; +}; + +// Report-before-acting: the same plan exportBank recomputes, with nothing written. +// Read-only against both the project and the filesystem. +ExportSurvey surveyBankExport(const ReaSamplerSession& session, + const std::string& projectDir, + const std::string& bankId); + +// Fills each manifest entry's byteLength and byteHash from its source file — the +// digest pass, one payload in memory at a time. False with `outFailedName` set when a +// source will not read or is empty; a zero-length entry cannot round-trip the +// format's own seam, so it is a failure here rather than an entry. +bool digestSources(package::PackageManifest& manifest, + const std::vector& sourceAbsPaths, + std::string& outFailedName); + +// Streams one package to `destAbsPath`: the encoded prefix, then each payload re-read +// from `sourceAbsPaths` (parallel to `manifest.entries`) and re-checked against the +// length and digest recorded for it before it is appended — so the digest the +// manifest claims describes the bytes actually written, not the bytes a concurrent +// edit replaced. Any failure abandons the writer, leaving the destination absent or +// holding its prior contents. +// +// Public because that atomicity is this function's property: proving it needs a +// failure injected mid-stream, which is a call to this seam, not to exportBank. +ExportOutcome writePackageFile(const package::EncodedPackage& encoded, + const package::PackageManifest& manifest, + const std::vector& sourceAbsPaths, + const std::string& destAbsPath); + +// The verb: plan, gate on the verdict and the destination, digest, encode, stream. +ExportOutcome exportBank(const ReaSamplerSession& session, const ExportRequest& req); + +} // namespace reasampler diff --git a/src/shell/panel/panel_bank_ops.cpp b/src/shell/panel/panel_bank_ops.cpp index 0ce2fbb..f946c4f 100644 --- a/src/shell/panel/panel_bank_ops.cpp +++ b/src/shell/panel/panel_bank_ops.cpp @@ -16,6 +16,7 @@ #include "shell/panel/panel_state.h" #include "shell/panel/panel_bank_ops.h" +#include "shell/actions/package_export_action.h" // doBankPackageExport — the export skin #include "shell/bank_ops/bank_ops.h" // bankOp* promptless verbs #include "shell/persist/session.h" // ReaSamplerSession — the live session the ops mutate @@ -241,6 +242,7 @@ enum : unsigned int { kMenuDelete, kMenuEvacuate, kMenuCreate, + kMenuExport, // export this bank as a .rsbank package kMenuRemove, // remove selected sample(s) from the source bank kMenuMoveBase = 1000, // move-to-bank: kMenuMoveBase + destination index kMenuCopyBase = 2000, // copy-to-bank: kMenuCopyBase + destination index @@ -265,6 +267,7 @@ void showTabMenu(int screenX, int screenY, const std::string& bankId) { menuAppend(menu, kMenuRename, "Rename..."); menuAppend(menu, kMenuEvacuate, "Evacuate to pool", /*grayed=*/!nonEmpty); menuAppend(menu, kMenuDelete, "Delete..."); + menuAppend(menu, kMenuExport, "Export as package..."); menuSeparator(menu); menuAppend(menu, kMenuCreate, "New bank..."); @@ -277,6 +280,7 @@ void showTabMenu(int screenX, int screenY, const std::string& bankId) { case kMenuRename: doRenameBank(bankId); break; case kMenuEvacuate: doEvacuateBank(bankId); break; case kMenuDelete: doDeleteBank(bankId); break; + case kMenuExport: doBankPackageExport(*g_panel.session, bankId); break; case kMenuCreate: doCreateBank(); break; default: break; } diff --git a/tests/test_export_bank.cpp b/tests/test_export_bank.cpp new file mode 100644 index 0000000..25e1b23 --- /dev/null +++ b/tests/test_export_bank.cpp @@ -0,0 +1,375 @@ +// Standalone tests for shell/package/export_bank — no REAPER, no framework. The +// export reads the session through inline accessors only, so a real ReaSamplerSession +// and a real bank folder on disk are both constructible here. +// +// Mid-stream failure is INJECTED rather than simulated: the digest pass and the +// stream pass are separate public calls, so a source file removed or rewritten +// between them is exactly the concurrent-edit case the stream pass re-checks for. + +#include "../src/shell/package/export_bank.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../src/core/capture/wav_codec.h" +#include "../src/core/model/bank_book.h" +#include "../src/core/package/bank_package.h" +#include "../src/shell/package/package_io.h" +#include "../src/shell/package/package_path.h" +#include "../src/shell/persist/session.h" + +using namespace reasampler; +namespace fs = std::filesystem; + +static int g_fail = 0; +#define CHECK(cond) do { if(!(cond)) { \ + std::printf("FAIL line %d: %s\n", __LINE__, #cond); ++g_fail; } } while(0) + +// --- scratch filesystem ------------------------------------------------------- + +static std::string g_root; + +static std::string scratchRoot() { + if (g_root.empty()) { + std::error_code ec; + const fs::path p = fs::temp_directory_path(ec) / "reasampler_export_tests"; + fs::remove_all(p, ec); + fs::create_directories(p, ec); + g_root = p.generic_string(); + } + return g_root; +} + +static std::vector patternBytes(std::size_t n, std::uint8_t seed) { + std::vector v(n); + for (std::size_t i = 0; i < n; ++i) v[i] = static_cast(seed + i * 7u); + return v; +} + +static void writeFile(const std::string& path, const std::vector& bytes) { + std::ofstream f(utf8Path(path), std::ios::binary | std::ios::trunc); + f.write(reinterpret_cast(bytes.data()), + static_cast(bytes.size())); +} + +static std::vector readFile(const std::string& path) { + std::ifstream f(utf8Path(path), std::ios::binary); + return std::vector(std::istreambuf_iterator(f), + std::istreambuf_iterator()); +} + +static bool exists(const std::string& path) { return fs::exists(utf8Path(path)); } + +// --- fixture project ---------------------------------------------------------- + +// One scratch project directory with a bank folder, plus the session whose book +// names its contents. `fileNames` are written into the bank folder with distinct +// byte patterns; a name in `omit` gets an index entry but NO file on disk. +struct Fixture { + std::string projectDir; + ReaSamplerSession session; + std::string bankId = "bank-1"; + + explicit Fixture(const std::string& tag) { + projectDir = scratchRoot() + "/" + tag; + std::error_code ec; + fs::create_directories(utf8Path(projectDir + "/reasampler_bank"), ec); + session.book().createBank(bankId, "Drums " + tag); + } + + void addSample(const std::string& id, const std::string& fileName, + std::size_t bytes, std::uint8_t seed, bool writeToDisk = true) { + model::Sample s; + s.id = id; + s.displayName = id; + s.relativePath = std::string("reasampler_bank/") + fileName; + s.sampleRate = 48000; + s.channelCount = 2; + s.contentHash = "hash-" + id; + CHECK(session.book().index(bankId)->add(s) == model::AddResult::Added); + session.book().reconcileSlots(); + if (writeToDisk) writeFile(absPathOf(fileName), patternBytes(bytes, seed)); + } + + std::string absPathOf(const std::string& fileName) const { + return projectDir + "/reasampler_bank/" + fileName; + } + std::string destPath() const { return projectDir + "/out.rsbank"; } + + ExportRequest request() const { + ExportRequest req; + req.projectDir = projectDir; + req.bankId = bankId; + req.destAbsPath = destPath(); + req.exportTimestamp = 1234567890; + return req; + } +}; + +// --- package readback --------------------------------------------------------- + +// Decodes an emitted package straight off disk, growing the prefix read the way the +// format's own requiredPrefixSize seam asks callers to. +static package::DecodedPackage decodeFromDisk(const std::string& path) { + PackageFileReader reader(path); + const std::uint64_t size = reader.fileSize(); + std::vector prefix; + for (int guard = 0; guard < 8; ++guard) { + const std::optional need = package::requiredPrefixSize(prefix); + if (!need) break; + if (*need <= prefix.size()) break; + PayloadBuffer buf = reader.readRange(0, *need); + if (buf.empty()) break; + prefix.assign(buf.data(), buf.data() + buf.size()); + } + return package::decodePackage(prefix, size); +} + +// --- tests -------------------------------------------------------------------- + +static void testHealthyExportCarriesEveryPayloadByteExact() { + Fixture fx("healthy"); + fx.addSample("s1", "kick.wav", 800, 1); + fx.addSample("s2", "snare.wav", 1300, 60); + fx.addSample("s3", "hat.wav", 97, 200); + + const ExportOutcome out = exportBank(fx.session, fx.request()); + CHECK(out.status == ExportStatus::Written); + CHECK(out.entriesWritten == 3); + CHECK(out.excluded.empty()); + CHECK(PayloadBuffer::alive() == 0); + + const package::DecodedPackage decoded = decodeFromDisk(fx.destPath()); + CHECK(decoded.status == package::PackageReadability::Readable); + CHECK(decoded.manifest.entries.size() == 3); + CHECK(decoded.layout.size() == 3); + CHECK(decoded.manifest.bankDisplayName == "Drums healthy"); + CHECK(decoded.manifest.exportTimestamp == 1234567890); + + // PER ENTRY, not in aggregate: the digest the package records, the digest of the + // payload actually stored at that entry's span, and the digest of the source file + // on disk must all be the same string. + PackageFileReader reader(fx.destPath()); + const std::vector sourceNames = {"kick.wav", "snare.wav", "hat.wav"}; + CHECK(decoded.manifest.entries.size() == sourceNames.size()); + for (std::size_t i = 0; i < decoded.manifest.entries.size(); ++i) { + const package::PackageEntry& entry = decoded.manifest.entries[i]; + const PayloadBuffer stored = reader.readRange(decoded.layout[i].offset, + decoded.layout[i].length); + CHECK(!stored.empty()); + const std::vector source = readFile(fx.absPathOf(sourceNames[i])); + const std::string sourceDigest = capture::hashBytes(source.data(), source.size()); + CHECK(entry.byteLength == source.size()); + CHECK(entry.byteHash == sourceDigest); + CHECK(capture::hashBytes(stored.data(), stored.size()) == sourceDigest); + CHECK(stored.size() == source.size()); + CHECK(std::equal(source.begin(), source.end(), stored.data())); + } + CHECK(PayloadBuffer::alive() == 0); +} + +static void testEmittedManifestBytesCarryNoPath() { + Fixture fx("nopath"); + // Both source records carry a directory component; neither may reach the file. + fx.addSample("s1", "kick.wav", 200, 3); + fx.addSample("s2", "snare take 2.wav", 200, 9); + CHECK(exportBank(fx.session, fx.request()).status == ExportStatus::Written); + + // Scan the MANIFEST REGION of the emitted file, located by walking the frozen + // header the way a reader does: magic | fv | minReader | len+semver | len+JSON. + // The binary length fields are deliberately excluded — a length whose byte + // happens to be 0x2F is not a separator. + const std::vector file = readFile(fx.destPath()); + CHECK(file.size() > 20); + auto le32 = [&](std::size_t at) { + return static_cast(file[at]) | + (static_cast(file[at + 1]) << 8) | + (static_cast(file[at + 2]) << 16) | + (static_cast(file[at + 3]) << 24); + }; + const std::uint32_t semverLen = le32(12); + const std::size_t manifestLenAt = 16 + semverLen; + const std::uint32_t manifestLen = le32(manifestLenAt); + const std::size_t manifestAt = manifestLenAt + 4; + CHECK(manifestAt + manifestLen <= file.size()); + const std::string manifest(reinterpret_cast(file.data() + manifestAt), + manifestLen); + + CHECK(manifest.find("kick.wav") != std::string::npos); // the scan is looking at the manifest + CHECK(manifest.find('/') == std::string::npos); + CHECK(manifest.find('\\') == std::string::npos); + CHECK(manifest.find("..") == std::string::npos); + CHECK(manifest.find("reasampler_bank") == std::string::npos); + // ':' cannot be banned outright — it is JSON's own key separator — so the check + // is for the drive form specifically: a string value opening with ':'. + // With '/' and '\\' already absent, that covers the drive-relative spelling too. + bool driveForm = false; + for (std::size_t i = 0; i + 2 < manifest.size(); ++i) + if (manifest[i] == '"' && + std::isalpha(static_cast(manifest[i + 1])) && + manifest[i + 2] == ':') + driveForm = true; + CHECK(!driveForm); +} + +static void testFailureInjectedMidWriteLeavesNoPackageAndSparesThePriorFile() { + Fixture fx("midwrite"); + fx.addSample("s1", "kick.wav", 500, 1); + fx.addSample("s2", "snare.wav", 500, 2); + + const std::vector prior = patternBytes(64, 99); + writeFile(fx.destPath(), prior); + + ExportSurvey survey = surveyBankExport(fx.session, fx.projectDir, fx.bankId); + CHECK(survey.plan.verdict == package::ExportVerdict::Ready); + package::PackageManifest manifest = survey.plan.manifest; + const std::vector sources = {fx.absPathOf("kick.wav"), + fx.absPathOf("snare.wav")}; + std::string failed; + CHECK(digestSources(manifest, sources, failed)); + const std::optional encoded = package::encodePackage(manifest); + CHECK(encoded.has_value()); + + // Injection: the second payload vanishes after the framing that claims it was + // already encoded, so the failure lands with the prefix and one payload written. + std::error_code ec; + fs::remove(utf8Path(fx.absPathOf("snare.wav")), ec); + + const ExportOutcome out = + writePackageFile(*encoded, manifest, sources, fx.destPath()); + CHECK(out.status == ExportStatus::SourceReadFailed); + CHECK(out.offendingName == "snare.wav"); + CHECK(readFile(fx.destPath()) == prior); // the prior file is untouched + CHECK(!exists(fx.destPath() + ".rsbanktmp")); // and no debris is left behind + CHECK(PayloadBuffer::alive() == 0); +} + +static void testPayloadChangedBetweenDigestAndStreamAborts() { + Fixture fx("changed"); + fx.addSample("s1", "kick.wav", 500, 1); + CHECK(!exists(fx.destPath())); + + ExportSurvey survey = surveyBankExport(fx.session, fx.projectDir, fx.bankId); + package::PackageManifest manifest = survey.plan.manifest; + const std::vector sources = {fx.absPathOf("kick.wav")}; + std::string failed; + CHECK(digestSources(manifest, sources, failed)); + const std::optional encoded = package::encodePackage(manifest); + CHECK(encoded.has_value()); + + // Same length, different bytes — only the digest re-check can catch this. + writeFile(fx.absPathOf("kick.wav"), patternBytes(500, 77)); + + const ExportOutcome out = + writePackageFile(*encoded, manifest, sources, fx.destPath()); + CHECK(out.status == ExportStatus::SourceChanged); + CHECK(out.offendingName == "kick.wav"); + CHECK(!exists(fx.destPath())); +} + +static void testExportTouchesNoProjectState() { + Fixture fx("readonly"); + fx.addSample("s1", "kick.wav", 400, 5); + fx.addSample("s2", "snare.wav", 400, 6); + + // The ext-state blob IS the serialized book (shell/persist/ext_state_io), so + // byte-identity of that string is byte-identity of what a persist would write. + const std::string extStateBefore = fx.session.book().serialize(); + const std::int64_t generationBefore = fx.session.bankGeneration(); + + CHECK(exportBank(fx.session, fx.request()).status == ExportStatus::Written); + + CHECK(fx.session.book().serialize() == extStateBefore); + CHECK(fx.session.bankGeneration() == generationBefore); +} + +static void testEmptyBankExportsAsAValidZeroEntryPackage() { + Fixture fx("empty"); + const ExportOutcome out = exportBank(fx.session, fx.request()); + CHECK(out.status == ExportStatus::Written); + CHECK(out.entriesWritten == 0); + + const package::DecodedPackage decoded = decodeFromDisk(fx.destPath()); + CHECK(decoded.status == package::PackageReadability::Readable); + CHECK(decoded.manifest.entries.empty()); + CHECK(decoded.layout.empty()); + CHECK(decoded.manifest.bankDisplayName == "Drums empty"); + // The size proof is decodePackage's, and it ran against the real on-disk size. + CHECK(decoded.prefixSize == readFile(fx.destPath()).size()); +} + +static void testIncompleteBankRefusesUntilConfirmed() { + Fixture fx("incomplete"); + fx.addSample("s1", "kick.wav", 300, 1); + fx.addSample("s2", "gone.wav", 300, 2, /*writeToDisk=*/false); + + ExportRequest req = fx.request(); + const ExportOutcome refused = exportBank(fx.session, req); + CHECK(refused.status == ExportStatus::RefusedIncomplete); + CHECK(refused.excluded.size() == 1); + CHECK(refused.excluded[0].sampleId == "s2"); + CHECK(refused.excluded[0].reason == package::ExclusionReason::FileMissing); + CHECK(!exists(fx.destPath())); + + req.allowIncomplete = true; + const ExportOutcome allowed = exportBank(fx.session, req); + CHECK(allowed.status == ExportStatus::Written); + CHECK(allowed.entriesWritten == 1); + CHECK(allowed.excluded.size() == 1); // the report survives into the summary + CHECK(decodeFromDisk(fx.destPath()).manifest.entries.size() == 1); +} + +static void testExistingDestinationRefusesUntilConfirmed() { + Fixture fx("overwrite"); + fx.addSample("s1", "kick.wav", 300, 1); + const std::vector prior = patternBytes(32, 11); + writeFile(fx.destPath(), prior); + + ExportRequest req = fx.request(); + const ExportOutcome refused = exportBank(fx.session, req); + CHECK(refused.status == ExportStatus::RefusedDestinationExists); + CHECK(readFile(fx.destPath()) == prior); + + req.allowOverwrite = true; + CHECK(exportBank(fx.session, req).status == ExportStatus::Written); + CHECK(readFile(fx.destPath()) != prior); +} + +static void testUnknownBankAndUnsavedProjectAreNamedSeparately() { + Fixture fx("guards"); + ExportRequest req = fx.request(); + req.bankId = "no-such-bank"; + CHECK(exportBank(fx.session, req).status == ExportStatus::NoSuchBank); + + ExportRequest unsaved = fx.request(); + unsaved.projectDir.clear(); + CHECK(exportBank(fx.session, unsaved).status == ExportStatus::NoProjectDir); + CHECK(!exists(fx.destPath())); +} + +int main() { + testHealthyExportCarriesEveryPayloadByteExact(); + testEmittedManifestBytesCarryNoPath(); + testFailureInjectedMidWriteLeavesNoPackageAndSparesThePriorFile(); + testPayloadChangedBetweenDigestAndStreamAborts(); + testExportTouchesNoProjectState(); + testEmptyBankExportsAsAValidZeroEntryPackage(); + testIncompleteBankRefusesUntilConfirmed(); + testExistingDestinationRefusesUntilConfirmed(); + testUnknownBankAndUnsavedProjectAreNamedSeparately(); + + if (g_fail == 0) { + std::printf("export_bank_tests: all passed\n"); + return 0; + } + std::printf("export_bank_tests: %d failure(s)\n", g_fail); + return 1; +} diff --git a/tests/test_export_plan.cpp b/tests/test_export_plan.cpp new file mode 100644 index 0000000..f29c356 --- /dev/null +++ b/tests/test_export_plan.cpp @@ -0,0 +1,268 @@ +// Standalone tests for reasampler::package::export_plan — no REAPER, no filesystem, +// no test framework. The planner's totality claim is the point: every input class +// (missing / unreadable / unrepresentable / zero / one) classifies here, and the +// names it produces are asserted against the codec's OWN predicates rather than +// against a hand-copied rule. + +#include "../src/core/package/export_plan.h" + +#include +#include +#include +#include +#include + +#include "../src/core/package/package_format.h" +#include "../src/core/package/package_manifest.h" + +using namespace reasampler::package; +using namespace reasampler::model; + +static int g_fail = 0; +#define CHECK(cond) do { if(!(cond)) { \ + std::printf("FAIL line %d: %s\n", __LINE__, #cond); ++g_fail; } } while(0) + +// --- fixtures ---------------------------------------------------------------- + +static Sample sampleAt(const std::string& id, const std::string& relativePath) { + Sample s; + s.id = id; + s.displayName = id + " display"; + s.relativePath = relativePath; + s.sampleRate = 48000; + s.channelCount = 2; + s.contentHash = "0123456789abcdef"; + return s; +} + +static ExportCandidate present(const std::string& id, const std::string& rel) { + return ExportCandidate{sampleAt(id, rel), SourceFileState::Present}; +} + +static ExportCandidate withState(const std::string& id, const std::string& rel, + SourceFileState state) { + return ExportCandidate{sampleAt(id, rel), state}; +} + +static ExportInputs bankOf(std::vector candidates) { + ExportInputs in; + in.bankDisplayName = "Drums"; + in.candidates = std::move(candidates); + std::vector ids; + for (const ExportCandidate& c : in.candidates) ids.push_back(c.sample.id); + in.slots.resetDense(ids); + return in; +} + +static bool hasExclusion(const ExportPlan& p, const std::string& id, ExclusionReason why) { + for (const ExcludedEntry& e : p.excluded) + if (e.sampleId == id && e.reason == why) return true; + return false; +} + +// --- the four classification inputs the plan names --------------------------- + +static void testZeroSamplesIsAReadyEmptyPlan() { + const ExportPlan p = planExport(bankOf({})); + CHECK(p.verdict == ExportVerdict::Ready); + CHECK(p.manifest.entries.empty()); + CHECK(p.sourceRelativePaths.empty()); + CHECK(p.excluded.empty()); + CHECK(p.manifest.bankDisplayName == "Drums"); + CHECK(p.manifest.slots.empty()); +} + +static void testOneSamplePresentShips() { + const ExportPlan p = planExport(bankOf({present("s1", "reasampler_bank/kick.wav")})); + CHECK(p.verdict == ExportVerdict::Ready); + CHECK(p.manifest.entries.size() == 1); + CHECK(p.excluded.empty()); + CHECK(p.manifest.entries[0].fileName == "kick.wav"); + CHECK(p.manifest.entries[0].sample.id == "s1"); + // The source spelling survives only on the side channel; the transport record + // names the payload by its bare package name. + CHECK(p.sourceRelativePaths.size() == 1); + CHECK(p.sourceRelativePaths[0] == "reasampler_bank/kick.wav"); + CHECK(p.manifest.entries[0].sample.relativePath == "kick.wav"); +} + +static void testMissingFileIsIncompleteNotRefused() { + const ExportPlan p = planExport(bankOf({ + present("s1", "reasampler_bank/kick.wav"), + withState("s2", "reasampler_bank/gone.wav", SourceFileState::Missing), + })); + CHECK(p.verdict == ExportVerdict::Incomplete); + CHECK(p.manifest.entries.size() == 1); + CHECK(p.manifest.entries[0].sample.id == "s1"); + CHECK(p.excluded.size() == 1); + CHECK(hasExclusion(p, "s2", ExclusionReason::FileMissing)); + CHECK(p.excluded[0].relativePath == "reasampler_bank/gone.wav"); +} + +static void testUnreadableFileStaysDistinctFromMissing() { + const ExportPlan p = planExport(bankOf({ + withState("s1", "reasampler_bank/locked.wav", SourceFileState::Unreadable), + withState("s2", "reasampler_bank/gone.wav", SourceFileState::Missing), + })); + CHECK(p.verdict == ExportVerdict::Incomplete); + CHECK(p.manifest.entries.empty()); + CHECK(p.excluded.size() == 2); + CHECK(hasExclusion(p, "s1", ExclusionReason::FileUnreadable)); + CHECK(hasExclusion(p, "s2", ExclusionReason::FileMissing)); + CHECK(!hasExclusion(p, "s1", ExclusionReason::FileMissing)); +} + +static void testUnrepresentableRecordRefusesWholeExport() { + // A traversing nested path — the one thing an index record can carry that + // BankModel::add does not itself refuse. + const ExportPlan traversal = + planExport(bankOf({present("s1", "reasampler_bank/kick.wav"), + present("s2", "reasampler_bank/../evil.wav")})); + CHECK(traversal.verdict == ExportVerdict::Refused); + CHECK(hasExclusion(traversal, "s2", ExclusionReason::RecordUnrepresentable)); + CHECK(traversal.manifest.entries.size() == 1); // still reports what WOULD ship + + const ExportPlan emptyId = planExport(bankOf({present("", "reasampler_bank/kick.wav")})); + CHECK(emptyId.verdict == ExportVerdict::Refused); + + const ExportPlan absolute = + planExport(bankOf({present("s1", "C:/elsewhere/kick.wav")})); + CHECK(absolute.verdict == ExportVerdict::Refused); + CHECK(hasExclusion(absolute, "s1", ExclusionReason::RecordUnrepresentable)); + + // Refused outranks Incomplete: a corrupt record is not something the + // "export the present N" confirm can proceed past. + const ExportPlan both = planExport(bankOf({ + withState("s1", "reasampler_bank/gone.wav", SourceFileState::Missing), + present("s2", "reasampler_bank/../evil.wav"), + })); + CHECK(both.verdict == ExportVerdict::Refused); +} + +// --- transport names ---------------------------------------------------------- + +static void testHostileNamesAreRepairedNotRelayed() { + // Every one of these is a name the codec refuses and a filesystem somewhere + // produces honestly. + const std::vector hostile = { + "reasampler_bank/ki:ck?.wav", "reasampler_bank/a|bd\"e*f.wav", + "reasampler_bank/CON.wav", "reasampler_bank/nul", + "reasampler_bank/trailing .wav ", "reasampler_bank/dots...", + // A literal ".." COMPONENT is not a name to repair — it is a traversing + // record, and the Refused test above owns it. + "reasampler_bank/.....", "reasampler_bank/.", + std::string("reasampler_bank/bad\xC3.wav"), // truncated UTF-8 sequence + std::string("reasampler_bank/") + std::string(400, 'x') + ".wav", + }; + std::vector candidates; + for (std::size_t i = 0; i < hostile.size(); ++i) + candidates.push_back(present("s" + std::to_string(i), hostile[i])); + + const ExportPlan p = planExport(bankOf(candidates)); + CHECK(p.verdict == ExportVerdict::Ready); + CHECK(p.manifest.entries.size() == hostile.size()); + for (const PackageEntry& e : p.manifest.entries) { + CHECK(isValidEntryName(e.fileName)); + CHECK(isValidNestedSamplePath(e.sample.relativePath)); + } +} + +static void testCaseFoldedCollisionsAreDisambiguated() { + const ExportPlan p = planExport(bankOf({ + present("s1", "reasampler_bank/Kick.wav"), + present("s2", "reasampler_bank/kick.wav"), + present("s3", "reasampler_bank/KICK.wav"), + })); + CHECK(p.verdict == ExportVerdict::Ready); + CHECK(p.manifest.entries.size() == 3); + for (std::size_t i = 0; i < p.manifest.entries.size(); ++i) + for (std::size_t j = i + 1; j < p.manifest.entries.size(); ++j) + CHECK(!sameEntryName(p.manifest.entries[i].fileName, + p.manifest.entries[j].fileName)); + CHECK(p.manifest.entries[0].fileName == "Kick.wav"); + // The suffix goes before the extension, so the payload keeps its type. + CHECK(p.manifest.entries[1].fileName == "kick_2.wav"); +} + +static void testSanitizeNeverReturnsANameTheCodecRefuses() { + const std::vector raws = { + "", ".", "..", "...", " ", "com1", "LPT9.WAV", "a/b", "a\\b", "C:evil", + std::string("\x01\x02\x03"), std::string(300, 'y'), + std::string("caf\xC3\xA9.wav"), // well-formed UTF-8 must survive intact + }; + for (const std::string& raw : raws) CHECK(isValidEntryName(sanitizeEntryName(raw))); + CHECK(sanitizeEntryName("caf\xC3\xA9.wav") == "caf\xC3\xA9.wav"); + CHECK(sanitizeEntryName("kick.wav") == "kick.wav"); +} + +// --- what the plan hands the codec ------------------------------------------- + +static void testPlannedManifestSatisfiesTheCodec() { + ExportPlan p = planExport(bankOf({ + present("s1", "reasampler_bank/Kick.wav"), + present("s2", "reasampler_bank/kick.wav"), + present("s3", "reasampler_bank/CON.wav"), + present("s4", std::string("reasampler_bank/caf\xC3\xA9 mix.wav")), + })); + // byteLength/byteHash are the shell's to measure; stand them in so the encode + // path under test is the naming, not the digest. + for (PackageEntry& e : p.manifest.entries) { + e.byteLength = 44; + e.byteHash = "aaaaaaaabbbbbbbb"; + } + const std::optional json = serializeManifest(p.manifest); + CHECK(json.has_value()); + if (json) { + const std::optional back = deserializeManifest(*json); + CHECK(back.has_value()); + if (back) CHECK(*back == p.manifest); + } +} + +static void testSlotsFollowMembership() { + ExportInputs in = bankOf({ + present("s1", "reasampler_bank/a.wav"), + withState("s2", "reasampler_bank/gone.wav", SourceFileState::Missing), + present("s3", "reasampler_bank/c.wav"), + }); + const ExportPlan p = planExport(in); + CHECK(p.manifest.slots.slotOf("s2") == -1); // an excluded id keeps no display position + CHECK(p.manifest.slots.slotOf("s1") >= 0); + CHECK(p.manifest.slots.slotOf("s3") >= 0); + CHECK(p.manifest.slots.size() == 2); +} + +static void testPlanIsDeterministic() { + const ExportInputs in = bankOf({ + present("s1", "reasampler_bank/Kick.wav"), + present("s2", "reasampler_bank/kick.wav"), + withState("s3", "reasampler_bank/gone.wav", SourceFileState::Missing), + }); + const ExportPlan a = planExport(in); + const ExportPlan b = planExport(in); + CHECK(a.verdict == b.verdict); + CHECK(a.manifest == b.manifest); + CHECK(a.sourceRelativePaths == b.sourceRelativePaths); + CHECK(a.excluded.size() == b.excluded.size()); +} + +int main() { + testZeroSamplesIsAReadyEmptyPlan(); + testOneSamplePresentShips(); + testMissingFileIsIncompleteNotRefused(); + testUnreadableFileStaysDistinctFromMissing(); + testUnrepresentableRecordRefusesWholeExport(); + testHostileNamesAreRepairedNotRelayed(); + testCaseFoldedCollisionsAreDisambiguated(); + testSanitizeNeverReturnsANameTheCodecRefuses(); + testPlannedManifestSatisfiesTheCodec(); + testSlotsFollowMembership(); + testPlanIsDeterministic(); + + if (g_fail == 0) { + std::printf("export_plan_tests: all passed\n"); + return 0; + } + std::printf("export_plan_tests: %d failure(s)\n", g_fail); + return 1; +} From a927dad2f46df841b2e4922cdd372afed9db1db6 Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 13:21:48 -0400 Subject: [PATCH 13/24] import: a .rsbank lands as a new bank, whole or not at all Four collisions answered explicitly: ids reminted, names never overwritten, content deduped before the write, bank name auto-suffixed. Degraded ledger refuses before the picker. --- src/app/CMakeLists.txt | 8 + src/app/main.cpp | 4 + src/core/model/bank_book.cpp | 11 + src/core/model/bank_book.h | 11 + src/core/package/CLAUDE.md | 26 +- src/core/package/CMakeLists.txt | 7 + src/core/package/import_plan.cpp | 170 +++++++++ src/core/package/import_plan.h | 76 ++++ src/core/package/package_format.cpp | 7 + src/core/package/package_format.h | 6 + src/core/package/package_manifest.cpp | 14 +- src/shell/actions/CLAUDE.md | 4 +- src/shell/actions/package_import_action.cpp | 183 +++++++++ src/shell/actions/package_import_action.h | 19 + src/shell/package/CLAUDE.md | 22 +- src/shell/package/CMakeLists.txt | 8 + src/shell/package/import_bank.cpp | 94 +++++ src/shell/package/import_bank.h | 38 ++ src/shell/package/import_landing.cpp | 137 +++++++ src/shell/package/import_landing.h | 64 ++++ src/shell/panel/CLAUDE.md | 2 +- src/shell/panel/panel_bank_ops.cpp | 8 + src/shell/panel/panel_window.cpp | 24 +- src/shell/persist/session.h | 6 + tests/test_import_landing.cpp | 394 ++++++++++++++++++++ tests/test_import_plan.cpp | 370 ++++++++++++++++++ 26 files changed, 1689 insertions(+), 24 deletions(-) create mode 100644 src/core/package/import_plan.cpp create mode 100644 src/core/package/import_plan.h create mode 100644 src/shell/actions/package_import_action.cpp create mode 100644 src/shell/actions/package_import_action.h create mode 100644 src/shell/package/import_bank.cpp create mode 100644 src/shell/package/import_bank.h create mode 100644 src/shell/package/import_landing.cpp create mode 100644 src/shell/package/import_landing.h create mode 100644 tests/test_import_landing.cpp create mode 100644 tests/test_import_plan.cpp diff --git a/src/app/CMakeLists.txt b/src/app/CMakeLists.txt index 044ab51..32a7ede 100644 --- a/src/app/CMakeLists.txt +++ b/src/app/CMakeLists.txt @@ -58,6 +58,14 @@ target_link_libraries(reaper_reasampler PRIVATE json wire file_bytes bank_model # free of the voice engine — a link edge to it here means the design drifted. target_include_directories(reaper_reasampler PRIVATE ${SDK_INC} ${WDL_INC}) +# Bank-package import: the promptless verb plus its action skin. Kept as its own +# appended block rather than merged into the lists above, so the two package +# directions stay textually independent. +target_sources(reaper_reasampler PRIVATE + ${REASAMPLER_SRC_DIR}/shell/package/import_bank.cpp + ${REASAMPLER_SRC_DIR}/shell/actions/package_import_action.cpp) +target_link_libraries(reaper_reasampler PRIVATE import_landing package_pickers) + # OUTPUT_NAME is channel-derived; the CMake target name stays "reaper_reasampler" for both # configs, since REAPER dlopen's any reaper_* module and the two channels' artifacts load # side-by-side. LIBRARY_OUTPUT_DIRECTORY pins the module to the top of the build tree even diff --git a/src/app/main.cpp b/src/app/main.cpp index 57696ae..5592e99 100644 --- a/src/app/main.cpp +++ b/src/app/main.cpp @@ -36,6 +36,7 @@ #include "shell/panel/panel_window.h" // panel lifecycle (init/toggle/open-query/shutdown) #include "shell/persist/session.h" // ReaSamplerSession #include "shell/view/view.h" // reconcileManagedLanes / applyMode +#include "shell/actions/package_import_action.h" // bank-package import action body namespace capture = reasampler::capture; @@ -91,6 +92,7 @@ static void RunCancelRealtime(int) { capture::RunCancelRealtime(g_session); } static void RunRecaptureFromSource(int) { capture::RunRecaptureFromSource(g_session); } static void RunRenderTrackInPlace(int) { capture::RunRenderTrackInPlace(g_session); } static void RunResampleBake(int) { capture::RunResampleBake(g_session); } +static void RunImportBankPackage(int) { reasampler::doImportBankPackage(g_session); } static void RunShowVersion(int) { // On-demand only — no unconditional startup print (routine console chatter pops // the console window). @@ -148,6 +150,8 @@ static std::vector buildMainActionTable() { "land pending ReaSampler 9000 resample bake", &RunResampleBake}); rows.push_back({"SHOW_VERSION", "show version", &RunShowVersion}); + rows.push_back({"IMPORT_BANK_PACKAGE", "import bank package (.rsbank)", + &RunImportBankPackage}); return rows; } diff --git a/src/core/model/bank_book.cpp b/src/core/model/bank_book.cpp index 17b4ff5..efa5e25 100644 --- a/src/core/model/bank_book.cpp +++ b/src/core/model/bank_book.cpp @@ -117,6 +117,17 @@ bool BankBook::createBank(const std::string& id, const std::string& displayName) return true; } +// Runs behind displayNameTaken so the probe and the create/rename check can never +// disagree about what "already used" means. exceptId is deliberately "" — no bank can +// carry an empty id, so nothing is excluded from the scan. +std::string BankBook::uniqueDisplayName(const std::string& seed) const { + if (!displayNameTaken(seed, /*exceptId=*/std::string{})) return seed; + for (int n = 2;; ++n) { + std::string candidate = seed + " " + std::to_string(n); + if (!displayNameTaken(candidate, /*exceptId=*/std::string{})) return candidate; + } +} + bool BankBook::renameBank(const std::string& id, const std::string& displayName) { if (id == kPoolBankId) return false; // pool is un-renamable Bank* b = bank(id); diff --git a/src/core/model/bank_book.h b/src/core/model/bank_book.h index 284654a..3a19d65 100644 --- a/src/core/model/bank_book.h +++ b/src/core/model/bank_book.h @@ -100,6 +100,17 @@ public: // no-op success. bool renameBank(const std::string& id, const std::string& displayName); + // The first name in the sequence `seed`, "seed 2", "seed 3", … whose fold is free + // in this book — what a caller that must not be rejected (the package import) asks + // for before createBank. First-FREE-ascending, not highest-plus-one, so it fills a + // gap ("Drums" + "Drums 3" present yields "Drums 2") and is a pure function of the + // current name set. The seed is returned verbatim when free and is NEVER re-parsed: + // a bare trailing integer cannot be told from a user's own name, so "Kit 808" would + // become "Kit 2" under a stripping rule. Terminates by pigeonhole (one of the first + // N+1 candidates is free for N banks), so it needs no cap. A blank seed comes back + // blank — what a missing name should become is the caller's policy, not the model's. + std::string uniqueDisplayName(const std::string& seed) const; + // Deletes a named bank and its member entries (files untouched — a shell/prune // concern). Rejects (false, no mutation) an unknown id or the pool. Remaining // ordinals compact after; if the deleted bank was active, falls back to the pool. diff --git a/src/core/package/CLAUDE.md b/src/core/package/CLAUDE.md index 9114f83..5d80a01 100644 --- a/src/core/package/CLAUDE.md +++ b/src/core/package/CLAUDE.md @@ -73,6 +73,11 @@ landing after the format. here, computed where payloads are streamed (shell). The bank's `slot_map` rides along. Unknown keys skip at every level; duplicate entry names are rejected both ways. +- `import_plan` — the pure import decision, and the reason the whole feature is + testable without a DAW: the destination bank's display name after + `BankBook`'s own fold, the reminted sample ids and remapped parents, and the + per-entry land / collapse / rename disposition. Also `importLedgerRefusal`, + the import's ledger gate. - `bank_package` — framing and arithmetic composing the two above: `encodePackage` (prefix bytes + layout + total size, stamping this build's ladder pair and `version::stampVersion()`), `decodePackage` (prefix + observed @@ -91,6 +96,11 @@ landing after the format. deliberately NOT enforced by the codec — they are `import_plan` decisions. The codec rejects only what makes the container itself incoherent (duplicate entry names, invalid names, a non-single-sample nested index). +- **`import_plan` consults no other bank's hashes, and that is the ruling, not + an omission.** An import always creates a NEW bank, so "already present in the + destination bank by content" is exactly "already landed by this same plan". + Cross-bank dedup is not enforced anywhere (`core/model/CLAUDE.md`), so a hash + the pool already holds still lands its own file here. - `requiredPrefixSize` trusts fields beyond the frozen region only when the version pair classifies `Readable`; for `TooNew` it stops at the semver — don't "fix" it to read the manifest length there, a future structural format @@ -133,15 +143,13 @@ landing after the format. collision class as the ASCII case fold, which `sameEntryName` does catch. A table-free fix does not exist, and restricting names to ASCII would be genuinely over-strict for non-English users. Left open knowingly. -- **`duplicateName` is O(n²) over `entries` on the decode path** — pre-existing - shape (the double loop is unchanged since `af35fc5`; only the comparator - changed). Under the `kMaxManifestBytes` cap (64 MB) a minimal entry is - ~100 bytes, so a hostile package can declare ~670k entries — ~2×10¹¹ pair - comparisons, a multi-minute hang on import. It signals an error rather than - UB, so the hostile-input invariant above still holds, but it sits against - this module's "a forged header cannot demand gigabytes" posture. Forward - obligation for `import_plan`: fold this into a sorted vector or hash set - when that track lands; not changed here. +- **`duplicateName` folds through a hash set, not a pairwise scan.** Under the + `kMaxManifestBytes` cap (64 MB) a minimal entry is ~100 bytes, so a hostile + package can declare ~670k entries; the former double loop was ~2×10¹¹ pair + comparisons — a multi-minute hang on the decode path an import drives. The + set is keyed on `entryNameKey`, which is `sameEntryName`'s ASCII-case fold + made explicit, so the equivalence rule still has one home (`lowerAscii`). + Do not reintroduce the pairwise scan. - **Cross-module contract with `src/shell/package`:** a genuinely zero-length entry cannot round-trip through the filesystem seam there (`appendPayload` refuses an empty payload — an empty buffer signals an upstream read failure, diff --git a/src/core/package/CMakeLists.txt b/src/core/package/CMakeLists.txt index fc7fd2d..0a04b90 100644 --- a/src/core/package/CMakeLists.txt +++ b/src/core/package/CMakeLists.txt @@ -12,3 +12,10 @@ reasampler_pure_library(bank_package LINK PUBLIC package_format package_manifest PRIVATE app_version) # app_version: the tests pin the stamped writer semver against stampVersion(). reasampler_test(bank_package LINK bank_package app_version) + +reasampler_pure_library(import_plan + SOURCES import_plan.cpp + LINK PUBLIC package_manifest bank_book origin_ledger PRIVATE package_format capture_paths) +# tracking_authority: the ledger-gate test proves the import does NOT share prune's +# composite blocker, which needs the composite to compare against. +reasampler_test(import_plan LINK import_plan tracking_authority) diff --git a/src/core/package/import_plan.cpp b/src/core/package/import_plan.cpp new file mode 100644 index 0000000..372b2c4 --- /dev/null +++ b/src/core/package/import_plan.cpp @@ -0,0 +1,170 @@ +#include "core/package/import_plan.h" + +#include +#include +#include + +#include "core/capture/capture_paths.h" +#include "core/package/package_format.h" + +namespace reasampler::package { + +namespace { + +using capture::bankRelativeForName; +using capture::deriveBankPaths; +using capture::sanitizeStem; + +bool blankName(const std::string& s) { + for (char c : s) + if (c != ' ' && c != '\t' && c != '\n' && c != '\r') return false; + return true; +} + +// "kick.wav" -> "kick"; a name with no dot is its own stem. deriveBankPaths re-adds +// the extension, so handing it the full name would file "kick.wav" as "kick.wav.wav". +std::string stemOf(const std::string& fileName) { + const std::size_t dot = fileName.rfind('.'); + if (dot == std::string::npos || dot == 0) return fileName; + return fileName.substr(0, dot); +} + +// True when `fileName` is already spelled the way this tool spells a bank file, so a +// package landing in a fresh project keeps the names it travelled with. Anything else +// is minted through deriveBankPaths, which is also the sanitizer. +bool spelledLikeABankFile(const std::string& fileName) { + const std::string stem = stemOf(fileName); + return stem != fileName && sanitizeStem(stem) == stem && fileName == stem + ".wav"; +} + +// The bank-folder names an import must not land on: what is there already, plus what +// this import has minted so far. Case-folded, because the two filesystems this tool +// ships on would treat "Kick.wav" and "kick.wav" as one file. +class NameSet { +public: + explicit NameSet(const std::vector& present) { + keys_.reserve(present.size()); + for (const std::string& n : present) keys_.insert(entryNameKey(n)); + } + bool taken(const std::string& name) const { return keys_.count(entryNameKey(name)) != 0; } + void claim(const std::string& name) { keys_.insert(entryNameKey(name)); } + +private: + std::unordered_set keys_; +}; + +// The name this entry lands under. Terminates: each attempt carries a distinct +// counter, and the taken set is finite. +std::string mintFileName(const std::string& projectDir, const std::string& packageName, + const std::string& uniqueTag, const NameSet& taken) { + if (spelledLikeABankFile(packageName) && !taken.taken(packageName)) return packageName; + + const std::string stem = stemOf(packageName); + std::string tag = uniqueTag; + for (int n = 2;; ++n) { + const std::string candidate = deriveBankPaths(projectDir, stem, tag).fileName; + if (!taken.taken(candidate)) return candidate; + tag = uniqueTag + "-" + std::to_string(n); + } +} + +} // namespace + +LedgerRefusal importLedgerRefusal(tracking::LedgerStatus status) { + switch (status) { + case tracking::LedgerStatus::Unreadable: return LedgerRefusal::Malformed; + case tracking::LedgerStatus::FutureVersion: return LedgerRefusal::FutureVersion; + case tracking::LedgerStatus::Fresh: + case tracking::LedgerStatus::Loaded: break; + } + return LedgerRefusal::None; +} + +std::string bankFolderDir(const std::string& projectDir) { + // Only the directory half of the result is wanted; the stem is a placeholder. + return deriveBankPaths(projectDir, "bank", std::string{}).absoluteDir; +} + +ImportPlan planImport(const PackageManifest& manifest, + const BankBook& destination, + const std::string& projectDir, + const std::vector& bankFolderFileNames, + const std::string& uniqueTag) { + ImportPlan plan; + + plan.seedBankName = blankName(manifest.bankDisplayName) + ? std::string(kDefaultImportBankName) + : manifest.bankDisplayName; + plan.bankDisplayName = destination.uniqueDisplayName(plan.seedBankName); + plan.bankNameAdjusted = plan.bankDisplayName != plan.seedBankName; + + NameSet taken(bankFolderFileNames); + // The destination bank is created empty by this same import, so "already in the + // destination bank by content" is exactly "already landed by this plan" — the + // hash set below IS that bank's findByHash. Cross-bank dedup is deliberately not + // enforced (core/model/CLAUDE.md), so other banks' hashes are not consulted. + std::unordered_map landedIdForHash; + // Every package id, including a collapsed one's, so a parent link that pointed at + // a duplicate still resolves to the entry that survived it. + std::unordered_map idRemap; + + plan.entries.reserve(manifest.entries.size()); + for (std::size_t i = 0; i < manifest.entries.size(); ++i) { + const PackageEntry& src = manifest.entries[i]; + + PlannedEntry e; + e.manifestIndex = i; + + const std::string& hash = src.sample.contentHash; + if (!hash.empty()) { + const auto hit = landedIdForHash.find(hash); + if (hit != landedIdForHash.end()) { + e.action = EntryAction::Collapse; + idRemap[src.sample.id] = hit->second; + ++plan.collapseCount; + plan.entries.push_back(std::move(e)); + continue; + } + } + + e.destFileName = mintFileName(projectDir, src.fileName, uniqueTag, taken); + e.renamed = e.destFileName != src.fileName; + taken.claim(e.destFileName); + + e.sample = src.sample; + e.sample.id = std::string(kImportIdPrefix) + uniqueTag + "-" + e.destFileName; + e.sample.relativePath = bankRelativeForName(e.destFileName); + + if (!hash.empty()) landedIdForHash.emplace(hash, e.sample.id); + idRemap[src.sample.id] = e.sample.id; + + ++plan.landCount; + if (e.renamed) ++plan.renameCount; + plan.entries.push_back(std::move(e)); + } + + // Second pass: the remap must be complete before a parent is resolved, since a + // sample may precede its own parent in manifest order. + for (PlannedEntry& e : plan.entries) { + if (e.action != EntryAction::Land || !e.sample.provenance) continue; + const auto hit = idRemap.find(e.sample.provenance->parentSampleId); + e.sample.provenance->parentSampleId = + hit == idRemap.end() ? std::string{} : hit->second; + } + + // The package's display order, over the ids that actually landed. SlotMap's own + // repair rules settle the rest: two package ids collapsed onto one landed id give + // one slot (first wins), and a landed sample the package never positioned is + // appended by BankBook::reconcileSlots afterwards. + std::vector> slotPairs; + for (const std::string& oldId : manifest.slots.orderedIds()) { + const auto hit = idRemap.find(oldId); + if (hit == idRemap.end()) continue; + slotPairs.emplace_back(hit->second, manifest.slots.slotOf(oldId)); + } + plan.slots = model::SlotMap::fromEntries(slotPairs); + + return plan; +} + +} // namespace reasampler::package diff --git a/src/core/package/import_plan.h b/src/core/package/import_plan.h new file mode 100644 index 0000000..a79dd46 --- /dev/null +++ b/src/core/package/import_plan.h @@ -0,0 +1,76 @@ +#pragma once +// import_plan — the pure import decision: the destination bank's display name after +// the book's own uniqueness fold, the reminted sample ids and remapped parents, and +// the per-entry land / collapse / rename disposition. Value inputs only; no +// filesystem, no session handle, no host types. + +#include +#include +#include + +#include "core/model/bank_book.h" +#include "core/model/slot_map.h" +#include "core/package/package_manifest.h" +#include "core/tracking/origin_ledger.h" + +namespace reasampler::package { + +// The bank name a package that recorded none (or a blank one) imports under. +inline constexpr const char* kDefaultImportBankName = "Imported bank"; + +// The prefix every imported sample id is reminted under, so a package's own ids — +// unique only within the project that made them — never enter this index. +inline constexpr const char* kImportIdPrefix = "pkg-"; + +// Which of the two refusal messages the import owes the user, if either. +// +// Keyed on the LEDGER STATUS ALONE, never on prune's composite blockedByTracking: +// that flag also fires on undecodable rsusage_* keys, which govern which files a +// DELETION may touch. An import deletes nothing and computes no protected set — it +// writes birth records — so an unreadable usage key must not refuse one. +enum class LedgerRefusal { None, Malformed, FutureVersion }; + +LedgerRefusal importLedgerRefusal(tracking::LedgerStatus status); + +// What one manifest entry does when the import runs. +// - Land: write the payload under destFileName and add `sample`. +// - Collapse: an equal contentHash already lands in this same import, so the payload +// is NOT written and no entry is added. Writing it and letting +// BankModel::add collapse the entry would leave the file referenced by +// nothing — an orphan manufactured by a dedup. +enum class EntryAction { Land, Collapse }; + +struct PlannedEntry { + std::size_t manifestIndex = 0; + EntryAction action = EntryAction::Land; + std::string destFileName; // Land only — a bare name in the bank folder + model::Sample sample; // Land only — id, path and parent already remapped + bool renamed = false; // the package's own name was taken, so a fresh one was minted +}; + +struct ImportPlan { + std::string bankDisplayName; + bool bankNameAdjusted = false; // the seed was taken, so the name carries a suffix + std::string seedBankName; // the seed the probe started from + std::vector entries; // one per manifest entry, in manifest order + model::SlotMap slots; // the package's slots over the reminted ids + int landCount = 0; + int collapseCount = 0; + int renameCount = 0; +}; + +// The bank folder an import lands into — the same expression capture uses, so an +// imported file is spelled exactly like a captured one. +std::string bankFolderDir(const std::string& projectDir); + +// Decides everything about an import except the bytes. `bankFolderFileNames` are the +// bare names already present in that folder (never overwritten); `uniqueTag` is the +// shell's per-import disambiguator, extended with an ascending counter where one tag +// is not enough. Total: every manifest entry yields exactly one PlannedEntry. +ImportPlan planImport(const PackageManifest& manifest, + const BankBook& destination, + const std::string& projectDir, + const std::vector& bankFolderFileNames, + const std::string& uniqueTag); + +} // namespace reasampler::package diff --git a/src/core/package/package_format.cpp b/src/core/package/package_format.cpp index 76808ac..34af822 100644 --- a/src/core/package/package_format.cpp +++ b/src/core/package/package_format.cpp @@ -103,6 +103,13 @@ bool sameEntryName(const std::string& a, const std::string& b) { return true; } +std::string entryNameKey(const std::string& name) { + std::string key; + key.reserve(name.size()); + for (unsigned char c : name) key += lowerAscii(c); + return key; +} + bool isValidNestedSamplePath(const std::string& path) { if (util::isAbsolutePath(path)) return false; // Component-wise, not a substring scan: "take..final/a.wav" is a legal diff --git a/src/core/package/package_format.h b/src/core/package/package_format.h index fd7c7a9..346f5ff 100644 --- a/src/core/package/package_format.h +++ b/src/core/package/package_format.h @@ -76,6 +76,12 @@ bool isValidEntryName(const std::string& name); // bytes compare exactly (see this directory's CLAUDE.md on NFC/NFD). bool sameEntryName(const std::string& a, const std::string& b); +// sameEntryName's fold made explicit: the ASCII-lower-cased bytes, so +// entryNameKey(a) == entryNameKey(b) exactly when sameEntryName(a, b). For a caller +// holding many names at once — folding them into a set is what turns an O(n^2) +// pairwise scan into a linear one. +std::string entryNameKey(const std::string& name); + // The one field in the format that CAN express a path: a nested Sample's // relativePath, which is bank-relative by design. Rejects every absolute form // (the shared util::isAbsolutePath test) and any ".." component — BankModel::add diff --git a/src/core/package/package_manifest.cpp b/src/core/package/package_manifest.cpp index df2f772..b634d05 100644 --- a/src/core/package/package_manifest.cpp +++ b/src/core/package/package_manifest.cpp @@ -1,5 +1,6 @@ #include "core/package/package_manifest.h" +#include #include #include "core/json/json.h" @@ -14,11 +15,16 @@ using ObjWriter = json::Writer; // Shared by serializeManifest and deserializeManifest — see this directory's // CLAUDE.md for why duplicate names are rejected both ways. Equivalence is the -// format's, not std::string's: sameEntryName folds ASCII case. +// format's, not std::string's: entryNameKey is sameEntryName's ASCII-case fold. +// +// A set, not the pairwise scan this replaced: under kMaxManifestBytes a hostile +// package can declare hundreds of thousands of minimal entries, and O(n^2) over that +// is a multi-minute hang on the decode path an import drives. bool duplicateName(const std::vector& entries) { - for (std::size_t i = 0; i < entries.size(); ++i) - for (std::size_t j = i + 1; j < entries.size(); ++j) - if (sameEntryName(entries[i].fileName, entries[j].fileName)) return true; + std::unordered_set seen; + seen.reserve(entries.size()); + for (const auto& e : entries) + if (!seen.insert(entryNameKey(e.fileName)).second) return true; return false; } diff --git a/src/shell/actions/CLAUDE.md b/src/shell/actions/CLAUDE.md index befbaed..9f2bb9d 100644 --- a/src/shell/actions/CLAUDE.md +++ b/src/shell/actions/CLAUDE.md @@ -4,7 +4,8 @@ The bindable action families routed through REAPER's `command_id`/`gaccel`/ `hookcommand` contract (Design View toggle actions, bank actions, the prune -action, and the shared registration plumbing/table), plus the three drag-out +action, the bank-package import action, and the shared registration +plumbing/table), plus the three drag-out outcome shells (OS hand-off, instrument drop, arrange drop), plus the extension-side ingest-through-the-bank shell. This is where user-facing REAPER actions and OS-level drag/drop live; the underlying @@ -42,6 +43,7 @@ is owned by other directories and only skinned here. - `drag_out_win` — OS drag-out shell: Windows OLE `DoDragDrop`/`CF_HDROP`, copy-only (`DROPEFFECT_MOVE` not offered); macOS/Linux via `SWELL_InitiateDragDropOfFileList`. - `instrument_drop_win` — instrument-drop shell: `probeDropTarget` resolves a screen point to a track + a `ReaperSurface` (via the pure `wire::classifyReaperSurface`, whose token rules `core/wire/CLAUDE.md` owns), and the drop half adds a ReaSampler 9000 instance and applies the dragged capture's state via a transient `.vstpreset` + `TrackFX_SetPreset` (the former `TrackFX_SetNamedConfigParm` "vst_chunk" write was silently unappliable for VST3). Exposes `loadInstrumentOntoTrack` (inner half, no own undo block) and `performInstrumentDrop` (wraps in its own undo block). **Never captures, never writes the bank, never inserts a timeline item.** - `arrange_drop_win` — the drag-out gesture's arrange outcome: `arrangeTimeAtScreenX` (pointer column → time via `GetSet_ArrangeView2`'s one-pixel-span reading — inferred, not SDK-documented) and `performArrangeDrop` (snap the drop time, then one `InsertMedia` per capture on the pointer's track — assumed, not confirmed, to land end-to-end via REAPER's own cursor advance — in ONE undo block, counting only InsertMedia's reported successes, with the caller's track selection and edit cursor restored). The one timeline-placing shell here, per the invariant above; it never captures and never writes the bank. +- `package_import_action` — the bindable / bank-menu / file-drop skin over `shell/package`'s `importBankPackage`. Owns the **ledger gate**, which runs BEFORE the file picker (a refusal must not cost the user a file choice) and is keyed on the session's `LedgerStatus` alone — never on `PruneReport::blockedByTracking`, whose undecodable-`rsusage_*` arm governs deletion-time protection and would refuse an import that only writes birth records. Owns every message the import produces; the verb itself is promptless. - `ingest` — ingest-through-the-bank shell on the EXTENSION side: three surfaces — (1) arrange capture→bank→assign (bindable action), (2) Media-Explorer import→bank→instrument on the selected track, (3) file drop onto the bank panel→bank only. Only surface (1) writes the `assignment_request` ext-state wire. **ingest NEVER inserts a timeline item.** Surface (2)'s action is the one in this directory published into a NON-main action section (Media Explorer) as well as Main — two ids, one handler, two dispatch hooks; see root `CLAUDE.md` §"REAPER extension contract" for the mechanism. ## Gotchas diff --git a/src/shell/actions/package_import_action.cpp b/src/shell/actions/package_import_action.cpp new file mode 100644 index 0000000..2531858 --- /dev/null +++ b/src/shell/actions/package_import_action.cpp @@ -0,0 +1,183 @@ +// package_import_action.cpp — see package_import_action.h for the contract. +// main.cpp owns the API pointers; this TU gets them extern. + +#include "shell/actions/package_import_action.h" + +#include + +#include "core/package/import_plan.h" +#include "core/package/package_format.h" +#include "core/version/app_version.h" +#include "shell/package/import_bank.h" +#include "shell/package/package_pickers.h" +#include "shell/persist/session.h" + +#define REAPERAPI_MINIMAL +#define REAPERAPI_WANT_ShowConsoleMsg +#define REAPERAPI_WANT_ShowMessageBox +#include "reaper_plugin_functions.h" + +namespace reasampler { + +namespace { + +constexpr const char* kTitle = "ReaSampler: import bank package"; + +std::string quoted(const std::string& s) { return "\"" + s + "\""; } + +// Mirrors prune's abort block in structure and tone, because a user who has hit that +// one should recognise this one. Every recovery line names THIS build's namespace: a +// beta user handed the stable spelling clears the wrong key and is still blocked. +void reportLedgerRefusal(package::LedgerRefusal refusal) { + const std::string& ns = version::extStateNamespace(); + std::string msg = + "ReaSampler import: ABORTED -- the file-tracking ledger could not be read. " + "Nothing was imported.\n"; + if (refusal == package::LedgerRefusal::Malformed) { + msg += "The stored file-tracking ledger is malformed. It has been left intact " + "rather than overwritten, so it can be repaired or cleared:\n" + " reaper.SetProjExtState(0, \"" + ns + "\", \"owned_files\", \"\")\n" + "Clearing it makes every existing bank file un-reclaimable (they stop " + "being attributable to ReaSampler); no file is lost. Reopen the project " + "afterwards -- the block is held for the rest of this session.\n"; + } else { + msg += "The stored file-tracking ledger was written by a NEWER version of " + "ReaSampler than this one, so its records cannot be read safely. It has " + "been left intact and will NOT be overwritten. Reopen the project with " + "that newer version -- do NOT clear this key from here, that would " + "discard tracking records this build cannot see. The block is held for " + "the rest of this session.\n"; + } + msg += "An import can land hundreds of files in one gesture. With no readable " + "ledger, none of them could be given a birth record, and every one would be " + "permanently unreclaimable.\n"; + ShowConsoleMsg(msg.c_str()); +} + +// The refusal a user can act on names all three: what the package needs, what this +// build reads, and which build wrote it. Any two of them leave them stuck. +void reportTooNew(const ImportBankResult& r) { + const std::string writer = + r.header.writerVersion.empty() ? std::string("an unidentified build") + : "ReaSampler " + r.header.writerVersion; + const std::string msg = + "Cannot import this bank package.\n" + "It was written by " + writer + " and needs package format " + + std::to_string(r.header.minReaderVersion) + " or newer.\n" + "This build (" + version::appVersion() + ") reads package format " + + std::to_string(package::kPackageFormatVersion) + ".\n" + "Nothing was imported. Install " + writer + " or newer and try again."; + ShowMessageBox(msg.c_str(), kTitle, 0); +} + +void reportSuccess(const ImportBankResult& r) { + std::string detail = "ReaSampler import: imported " + std::to_string(r.landedCount) + + " sample(s) into a new bank: " + quoted(r.bankDisplayName); + if (r.bankNameAdjusted) + detail += " (a bank named " + quoted(r.seedBankName) + + " already exists in this project)"; + detail += ".\n"; + if (r.renamedCount > 0) { + detail += " " + std::to_string(r.renamedCount) + + " file(s) landed under a freshly minted name (the package's own name " + "was already taken in the bank folder, or was not spelled the way " + "this bank spells a file). An existing bank file is never " + "overwritten.\n"; + } + if (r.collapsedCount > 0) { + detail += " " + std::to_string(r.collapsedCount) + + " sample(s) were already present by content and were not written " + "again.\n"; + } + detail += "One undo removes the imported bank and its entries. It does NOT delete " + "the imported files -- they stay in the bank folder, referenced by " + "nothing, until a prune reclaims them.\n"; + ShowConsoleMsg(detail.c_str()); + + // The console carries the copyable detail; the box makes the outcome unmissable. + const std::string summary = "Imported " + std::to_string(r.landedCount) + + " sample(s) into a new bank: " + + quoted(r.bankDisplayName) + "."; + ShowMessageBox(summary.c_str(), kTitle, 0); +} + +void reportRollback(const RollbackResult& rollback, std::string& msg) { + if (rollback.failedCount > 0) { + msg += "\n" + std::to_string(rollback.failedCount) + + " partly-imported file(s) could not be removed and are still in the bank " + "folder. They are referenced by no bank; a prune will reclaim them."; + } +} + +void report(const ImportBankResult& r) { + switch (r.outcome) { + case ImportOutcome::Landed: + reportSuccess(r); + return; + case ImportOutcome::TooNew: + reportTooNew(r); + return; + case ImportOutcome::NoProject: + ShowMessageBox("Save the project before importing a bank package -- an " + "unsaved project has no bank folder to import into.", + kTitle, 0); + return; + case ImportOutcome::Unreadable: + ShowMessageBox("That file could not be opened. Nothing was imported.", + kTitle, 0); + return; + case ImportOutcome::Malformed: + // Distinct from TooNew on purpose: the recoveries are opposite -- one is + // "install a newer build", this one is "get an intact copy". + ShowMessageBox("This file is not a readable bank package (corrupt or " + "truncated). Nothing was imported.", + kTitle, 0); + return; + case ImportOutcome::IntegrityFailed: { + std::string msg = "This bank package is damaged (entry " + + quoted(r.failedEntryName) + + " failed its integrity check). Nothing was imported."; + ShowMessageBox(msg.c_str(), kTitle, 0); + return; + } + case ImportOutcome::WriteFailed: { + std::string msg = "Import failed and was rolled back. Nothing was added."; + reportRollback(r.rollback, msg); + ShowMessageBox(msg.c_str(), kTitle, 0); + return; + } + case ImportOutcome::IndexRejected: { + std::string msg = "The bank index rejected the import. Nothing was added."; + reportRollback(r.rollback, msg); + ShowMessageBox(msg.c_str(), kTitle, 0); + return; + } + } +} + +// FIRST, before the picker: making the user find and choose a file we have already +// decided to refuse is the wrong order. +bool ledgerPermits(ReaSamplerSession& session) { + const package::LedgerRefusal refusal = + package::importLedgerRefusal(session.ledgerStatus()); + if (refusal == package::LedgerRefusal::None) return true; + reportLedgerRefusal(refusal); + return false; +} + +} // namespace + +void doImportBankPackage(ReaSamplerSession& session) { + if (!ledgerPermits(session)) return; + std::string path; + if (!pickPackageForImport(path) || path.empty()) return; + report(importBankPackage(session, path)); +} + +void doImportBankPackageFile(ReaSamplerSession& session, const std::string& packageAbsPath) { + if (packageAbsPath.empty()) return; + if (!ledgerPermits(session)) return; + report(importBankPackage(session, packageAbsPath)); +} + +} // namespace reasampler diff --git a/src/shell/actions/package_import_action.h b/src/shell/actions/package_import_action.h new file mode 100644 index 0000000..719b7a5 --- /dev/null +++ b/src/shell/actions/package_import_action.h @@ -0,0 +1,19 @@ +#pragma once +// package_import_action — the bindable/menu/drop skin over importBankPackage: the +// ledger gate (which runs BEFORE the picker, so a refusal never costs the user a file +// choice), the picker itself, and every message the import produces. + +#include + +namespace reasampler { + +class ReaSamplerSession; + +// Gate, pick, import, report. The bound action and the panel's bank menu both call this. +void doImportBankPackage(ReaSamplerSession& session); + +// Same, for a .rsbank already named by the user — the panel's file-drop route. The gate +// still runs first; only the picker is skipped. +void doImportBankPackageFile(ReaSamplerSession& session, const std::string& packageAbsPath); + +} // namespace reasampler diff --git a/src/shell/package/CLAUDE.md b/src/shell/package/CLAUDE.md index b4db0c4..4d51e78 100644 --- a/src/shell/package/CLAUDE.md +++ b/src/shell/package/CLAUDE.md @@ -6,11 +6,12 @@ The filesystem and dialog acts behind bank-package export/import: streaming pack file I/O plus the file-status and exclusive-create acts (`package_io`), the UTF-8 path conversion every one of them goes through (`package_path`), the landed-file journal and its rollback delete (`package_rollback`), and the two file pickers -(`package_pickers`). This seam is bytes-only — the package format (magic, manifest, -entry layout) is `core/package`'s business, and the export/import verbs that -orchestrate both do not live here yet. No REAPER project state is touched in this -directory: no ext-state read or write, no undo block, no generation bump — those -belong to the verbs. +(`package_pickers`). Those are bytes-only — the package format (magic, manifest, entry +layout) is `core/package`'s business. Beside them sits the import verb, split so its +decisions stay testable: `import_landing` (REAPER-free) decides and writes, +`import_bank` owns the only REAPER project state this directory touches (the +ext-state persist, the undo block, the generation bump). The export verb does not +live here yet. ## Invariants @@ -65,7 +66,14 @@ belong to the verbs. landed files with no index entry and a journal that now refuses to roll them back — after which `rollback()` refuses and `writeLandedFile` refuses. (Destroying an armed journal without calling either does NOT roll it back — see - `LandedFileJournal`'s own doc comment.) + `LandedFileJournal`'s own doc comment.) `import_bank` honours it: it calls + `markIndexCommitted()` only after `persistBankOp` has returned. +- **Integrity is proven before the first byte lands, not undone after.** + `landPackage` hashes every declared payload against the manifest and only then + creates the bank folder, so a damaged package costs no rollback at all and cannot + leave debris behind a rollback that itself failed. The second read of each payload + is deliberate on a once-per-gesture path — do not fold it into one + hash-and-write pass. - **Both pickers ride `GetUserFileName`** — mode 1 for import, mode 0 for export. There is no platform split and no fallback: `main.cpp` defines `REAPERAPI_IMPLEMENT` without `REAPERAPI_MINIMAL` and aborts the extension load if any single name fails @@ -77,6 +85,8 @@ belong to the verbs. - `package_io` — every filesystem act the verbs need: `PayloadBuffer` (move-only payload + the `alive()` seam counter), `PackageFileWriter` (append-only temp+atomic-rename writer), `PackageFileReader` (seek-and-read one range per call, range-checked against the real file size), `readFilePayload` (one source file as one entry's payload), `fileStatus` (Present/Absent/Unreadable — export's refusal message must distinguish the last two, and an empty payload cannot), `writeFileExclusive` (exclusive create + write, self-cleaning on a partial write), and `listFolderFileNames` (bare UTF-8 names, sorted, non-recursive, non-throwing). REAPER-free; tested without a DAW. - `package_rollback` — `LandedFileJournal`: `writeLandedFile` (exclusive-create land, path resolved absolute, recorded on success only), `markIndexCommitted` (disarms the journal), and `rollback` (deletes exactly the recorded set, hard unlink, tolerating a vanished file; refuses once disarmed). REAPER-free; tested without a DAW. - `package_pickers` — `pickPackageForImport` and `pickPackageSavePath`, both `GetUserFileName`. Compile-only until the verbs land; nothing here can be exercised in a unit test. +- `import_landing` — the import's two halves that decide anything: `landPackage` (decode, plan, verify EVERY payload's digest, then land through the journal) and `applyImportedBank` (the new bank's entries plus a birth record per landed file, in one straight-line block). REAPER-free deliberately — all-or-nothing, integrity and birth-record behaviour are assertable without a DAW. +- `import_bank` — the promptless import verb over a live `ReaSamplerSession`: the project directory, the minted bank id, the `recordCreated` writer, and the one undo-batched persist. REAPER-facing, so it compiles into the extension module rather than into a library with a test target. ## Gotchas diff --git a/src/shell/package/CMakeLists.txt b/src/shell/package/CMakeLists.txt index aff4414..3003b1b 100644 --- a/src/shell/package/CMakeLists.txt +++ b/src/shell/package/CMakeLists.txt @@ -9,6 +9,14 @@ reasampler_test(package_io LINK package_io) reasampler_pure_library(package_rollback SOURCES package_rollback.cpp LINK PUBLIC package_io) reasampler_test(package_rollback LINK package_rollback) +# The import's decisions and its file half, both REAPER-free, so all-or-nothing, +# integrity and birth-record behaviour are assertable without a DAW. The REAPER-facing +# verb over them (import_bank.cpp) compiles into the extension module instead. +reasampler_pure_library(import_landing + SOURCES import_landing.cpp + LINK PUBLIC import_plan package_rollback bank_book PRIVATE bank_package wav_codec) +reasampler_test(import_landing LINK import_landing bank_package app_version wav_codec origin_ledger) + # The pickers call the REAPER API, so no test target can exercise them; declared as a # library so the TU stays compiled. reaper_plugin.h pulls SWELL in on non-Windows. add_library(package_pickers STATIC package_pickers.cpp) diff --git a/src/shell/package/import_bank.cpp b/src/shell/package/import_bank.cpp new file mode 100644 index 0000000..777b165 --- /dev/null +++ b/src/shell/package/import_bank.cpp @@ -0,0 +1,94 @@ +// import_bank.cpp — see import_bank.h for the contract. The REAPER-facing half of the +// import: the project directory, the minted bank id, the birth records, and the one +// undo-batched persist. Every decision it makes is in import_landing / import_plan. +// +// main.cpp owns the API pointers; this TU gets them extern. DAW-verified, not unit tested. + +#include "shell/package/import_bank.h" + +#include +#include +#include +#include + +#include "core/capture/capture_paths.h" // projectDirOfRpp +#include "shell/bank_ops/bank_ops.h" // persistBankOp — one bank op is one Ctrl-Z +#include "shell/persist/session.h" + +#define REAPERAPI_MINIMAL +#define REAPERAPI_WANT_EnumProjects +#define REAPERAPI_WANT_genGuid +#define REAPERAPI_WANT_guidToString +#include "reaper_plugin_functions.h" + +namespace reasampler { + +namespace { + +std::string activeProjectDir() { + std::vector buf(4096, '\0'); + EnumProjects(-1, buf.data(), static_cast(buf.size())); + return capture::projectDirOfRpp(std::string(buf.data())); +} + +// The model mints no ids (it stays pure and deterministic), so the shell does — the +// same GUID pair bankOpCreate uses. +std::string mintBankId() { + GUID g{}; + genGuid(&g); + char buf[64] = {0}; // guidToString needs >=64 chars (SDK contract) + guidToString(&g, buf); + return std::string(buf); +} + +void fillPlanCounts(ImportBankResult& out, const package::ImportPlan& plan) { + out.bankDisplayName = plan.bankDisplayName; + out.seedBankName = plan.seedBankName; + out.bankNameAdjusted = plan.bankNameAdjusted; + out.landedCount = plan.landCount; + out.renamedCount = plan.renameCount; + out.collapsedCount = plan.collapseCount; +} + +} // namespace + +ImportBankResult importBankPackage(ReaSamplerSession& session, + const std::string& packageAbsPath) { + ImportBankResult out; + + // A per-import disambiguator, the same shape capture and ingest file under. + const std::string uniqueTag = + std::to_string(static_cast(std::time(nullptr))); + + LandedFileJournal journal; + const ImportLanding landing = landPackage(packageAbsPath, activeProjectDir(), + session.book(), uniqueTag, journal); + out.outcome = landing.outcome; + out.header = landing.header; + out.failedEntryName = landing.failedEntryName; + out.rollback = landing.rollback; + fillPlanCounts(out, landing.plan); + if (landing.outcome != ImportOutcome::Landed) return out; + + const bool applied = applyImportedBank( + session.book(), mintBankId(), landing.plan, + [&session](const model::Sample& s) { + session.recordCreated(s, tracking::OriginKind::PackageImport); + }); + if (!applied) { + out.outcome = ImportOutcome::IndexRejected; + out.rollback = journal.rollback(); + return out; + } + + // Generation bump + persist ride inside one undo block, so a Ctrl-Z takes the whole + // import back out of the index. It does NOT un-write the files — the summary says so. + persistBankOp(session, "ReaSampler: import bank package", /*bumpGeneration=*/true); + // Only now: the files are referenced, so prune's self-cleanup carve-out no longer + // covers them (see package_rollback.h). + journal.markIndexCommitted(); + + return out; +} + +} // namespace reasampler diff --git a/src/shell/package/import_bank.h b/src/shell/package/import_bank.h new file mode 100644 index 0000000..94433cc --- /dev/null +++ b/src/shell/package/import_bank.h @@ -0,0 +1,38 @@ +#pragma once +// shell/package/import_bank — the promptless import verb: one package becomes one NEW +// bank in the live session, completely or not at all. No prompts, no message boxes, no +// picker — it reports and the action skin (shell/actions/package_import_action) speaks. +// The ledger gate is the skin's, because it must refuse BEFORE a file is even chosen. + +#include + +#include "core/package/import_plan.h" +#include "shell/package/import_landing.h" + +namespace reasampler { + +class ReaSamplerSession; + +struct ImportBankResult { + ImportOutcome outcome = ImportOutcome::Unreadable; + package::PackageHeader header; // TooNew names the writer's build from here + + std::string bankDisplayName; // the bank actually created + std::string seedBankName; // what the package asked to be called + bool bankNameAdjusted = false; + + int landedCount = 0; + int renamedCount = 0; + int collapsedCount = 0; + + std::string failedEntryName; + RollbackResult rollback; +}; + +// Lands `packageAbsPath` as a new bank in `session`, in ONE undo point, bumping the +// bank generation so live instances reload. Places no timeline item. On any failure +// nothing remains on disk and the book is untouched. +ImportBankResult importBankPackage(ReaSamplerSession& session, + const std::string& packageAbsPath); + +} // namespace reasampler diff --git a/src/shell/package/import_landing.cpp b/src/shell/package/import_landing.cpp new file mode 100644 index 0000000..f51a338 --- /dev/null +++ b/src/shell/package/import_landing.cpp @@ -0,0 +1,137 @@ +// import_landing.cpp — see import_landing.h for the contract. REAPER-free: standard +// filesystem only, so every property this file decides is unit-testable. + +#include "shell/package/import_landing.h" + +#include +#include +#include +#include +#include + +#include "core/capture/wav_codec.h" // hashBytes — the digest the manifest records +#include "core/package/bank_package.h" +#include "shell/package/package_io.h" +#include "shell/package/package_path.h" + +namespace reasampler { + +namespace fs = std::filesystem; +using package::EntryAction; +using package::PackageEntrySpan; + +namespace { + +ImportLanding refusal(ImportOutcome outcome, const package::PackageHeader& header) { + ImportLanding out; + out.outcome = outcome; + out.header = header; + return out; +} + +// Reads the package head incrementally: requiredPrefixSize may grow its answer as +// fields arrive, so ask, read to the count, ask again. False means these bytes can +// never frame a package, or the file is shorter than its own header claims. +bool readPrefix(PackageFileReader& reader, std::vector& prefix) { + for (;;) { + const auto need = package::requiredPrefixSize(prefix); + if (!need) return false; + if (prefix.size() >= *need) return true; + PayloadBuffer head = reader.readRange(0, *need); + if (head.size() != *need) return false; + prefix.assign(head.data(), head.data() + head.size()); + } +} + +} // namespace + +ImportLanding landPackage(const std::string& packageAbsPath, + const std::string& projectDir, + const BankBook& destination, + const std::string& uniqueTag, + LandedFileJournal& journal) { + const package::PackageHeader noHeader; + if (projectDir.empty()) return refusal(ImportOutcome::NoProject, noHeader); + + PackageFileReader reader(packageAbsPath); + if (!reader.ok()) return refusal(ImportOutcome::Unreadable, noHeader); + + std::vector prefix; + if (!readPrefix(reader, prefix)) return refusal(ImportOutcome::Malformed, noHeader); + + const package::DecodedPackage dec = package::decodePackage(prefix, reader.fileSize()); + if (dec.status == package::PackageReadability::TooNew) + return refusal(ImportOutcome::TooNew, dec.header); + if (dec.status != package::PackageReadability::Readable) + return refusal(ImportOutcome::Malformed, dec.header); + + const std::string bankDir = package::bankFolderDir(projectDir); + + ImportLanding out; + out.header = dec.header; + out.plan = package::planImport(dec.manifest, destination, projectDir, + listFolderFileNames(bankDir), uniqueTag); + + // Integrity first, over EVERY declared entry — including one the plan collapses, + // since a package that fails its own digest is refused whole rather than partly + // trusted. Nothing is on disk yet, so a failure here needs no rollback. + for (std::size_t i = 0; i < dec.layout.size(); ++i) { + const PackageEntrySpan& span = dec.layout[i]; + // The format refuses a zero-length entry on encode; one arriving anyway cannot + // be told from a failed read at this seam, so it is not well-formed input. + if (span.length == 0) return refusal(ImportOutcome::Malformed, dec.header); + + PayloadBuffer payload = reader.readRange(span.offset, span.length); + if (payload.size() != span.length) { + out.outcome = ImportOutcome::IntegrityFailed; + out.failedEntryName = span.name; + return out; + } + if (capture::hashBytes(payload.data(), payload.size()) != + dec.manifest.entries[i].byteHash) { + out.outcome = ImportOutcome::IntegrityFailed; + out.failedEntryName = span.name; + return out; + } + } + + std::error_code ec; + fs::create_directories(utf8Path(bankDir), ec); // idempotent; the write reports failure + + for (const package::PlannedEntry& e : out.plan.entries) { + if (e.action != EntryAction::Land) continue; + const PackageEntrySpan& span = dec.layout[e.manifestIndex]; + PayloadBuffer payload = reader.readRange(span.offset, span.length); + if (payload.size() == span.length && + journal.writeLandedFile(bankDir + "/" + e.destFileName, payload)) { + continue; + } + out.outcome = ImportOutcome::WriteFailed; + out.failedEntryName = e.destFileName; + out.rollback = journal.rollback(); + return out; + } + + out.outcome = ImportOutcome::Landed; + return out; +} + +bool applyImportedBank(BankBook& book, const std::string& bankId, + const package::ImportPlan& plan, const RecordBirth& recordBirth) { + if (!book.createBank(bankId, plan.bankDisplayName)) return false; + + BankModel* index = book.index(bankId); + for (const package::PlannedEntry& e : plan.entries) { + if (e.action != EntryAction::Land) continue; + index->add(e.sample); + // Unconditional on the add's outcome: the file exists either way, and an + // unrecorded file is permanently unreclaimable. + recordBirth(e.sample); + } + + book.bank(bankId)->slots = plan.slots; + book.reconcileSlots(); + return true; +} + +} // namespace reasampler diff --git a/src/shell/package/import_landing.h b/src/shell/package/import_landing.h new file mode 100644 index 0000000..7dc85e3 --- /dev/null +++ b/src/shell/package/import_landing.h @@ -0,0 +1,64 @@ +#pragma once +// shell/package/import_landing — the import's filesystem half and its index half, +// both REAPER-free so the all-or-nothing, integrity and birth-record properties are +// assertable without a DAW. The verb that drives them against a live session is +// import_bank; the REAPER-facing reporting is shell/actions/package_import_action. + +#include +#include + +#include "core/model/bank_book.h" +#include "core/model/bank_model.h" +#include "core/package/import_plan.h" +#include "core/package/package_format.h" +#include "shell/package/package_rollback.h" + +namespace reasampler { + +// How a landing ended. Every value but Landed means NOTHING is on disk and NO index +// was touched — the two refuse-whole failures (TooNew, Malformed) before a byte is +// written, the other two after a rollback. +enum class ImportOutcome { + Landed, + NoProject, // unsaved project: there is no bank folder to land into + Unreadable, // the package file could not be opened + Malformed, // not a well-formed RSBK: corrupt, truncated, or trailing garbage + TooNew, // minReaderVersion above this build's ladder + IntegrityFailed, // an entry's payload did not match its recorded digest + WriteFailed, // a write failed partway; the landed files were rolled back + IndexRejected, // the book refused the bank the plan minted a free name for +}; + +struct ImportLanding { + ImportOutcome outcome = ImportOutcome::Unreadable; + // Meaningful from the moment the header parsed — a TooNew refusal names the + // writer's build, which is the only part of that message a user can act on. + package::PackageHeader header; + package::ImportPlan plan; + std::string failedEntryName; // IntegrityFailed / WriteFailed + RollbackResult rollback; // IntegrityFailed / WriteFailed +}; + +// Streams `packageAbsPath` into the project's bank folder: decode, plan, verify EVERY +// payload's digest, then land. Verification runs to completion before the first write, +// so a damaged package costs no rollback at all. Mutates no index and holds at most +// one payload at a time. `journal` is left armed on success — the caller applies the +// plan to the book and only then disarms it. +ImportLanding landPackage(const std::string& packageAbsPath, + const std::string& projectDir, + const BankBook& destination, + const std::string& uniqueTag, + LandedFileJournal& journal); + +// Called for every landed file, in the same straight-line block as its bank add — +// core/tracking/CLAUDE.md's no-silent-gaps invariant, kept structural by passing the +// writer in rather than letting a caller add first and record later. +using RecordBirth = std::function; + +// Adds the plan's landed entries to a NEW bank under `bankId`. False (no mutation) +// only if the book refuses the create — the name was minted free against this same +// book, so that means the book changed underneath the plan. +bool applyImportedBank(BankBook& book, const std::string& bankId, + const package::ImportPlan& plan, const RecordBirth& recordBirth); + +} // namespace reasampler diff --git a/src/shell/panel/CLAUDE.md b/src/shell/panel/CLAUDE.md index b8da4c2..0cc9763 100644 --- a/src/shell/panel/CLAUDE.md +++ b/src/shell/panel/CLAUDE.md @@ -48,7 +48,7 @@ live in `shell/bank_ops`, a sibling directory, not here. ## Modules - `bank_panel` (`shell/panel/`: `panel_window` / `panel_layout` / `panel_render` / `panel_input` / `panel_drag` / `panel_thumbnails` / `panel_audition` / `panel_bank_ops`, sharing state via `panel_state.h` — Q-W2 split of the former god-module into eight TUs) — docked LICE-drawn grid with three-zone layout: top toolbar (Capture → Maintenance → Placement via `action_bar`, short labels, More (⋯) overflow menu via `overflow_menu`), bottom toolbar (four opposite-mode tag buttons + Show Both), and footer (`[Arrange|Design]` toggle, Tail button, Prune via `footer_bar`). Grid renders in sparse slot order with gap cells, drop dispatch, metadata overlay, and selection via `accent/tertiary` purple border. Draws through the L1 kit by palette role; OS drag-out via `drag_out` + `drag_out_win`. `panel_window` owns the SWELL dialog lifecycle + dialog proc + drop-target opt-in; `panel_layout` the toolbar/footer/menu rects + vertical-split geometry (the one geometry source both paint and hit-test read); `panel_render` the WM_PAINT draw; `panel_input` click/wheel/keyboard routing + the new-content auto-tag timer; `panel_drag` the hover + card-drag state machine + drop dispatch; `panel_thumbnails` the PCM→envelope thumbnail cache + the bank-change fingerprint pass; `panel_audition` the preview-playback engine; `panel_bank_ops` the menu/prompt UX skin over the promptless `shell/bank_ops` verbs. `draw_kit` (shared with the VST3 editor) stays a separate TU. - - `panel_window` — SWELL dialog lifecycle + dialog proc + drop-target opt-in. + - `panel_window` — SWELL dialog lifecycle + dialog proc + drop-target opt-in. The drop splits by extension: a `.rsbank` is a whole bank and routes to the package-import action (one NEW bank each), everything else keeps the audio-ingest route. - `panel_layout` — toolbar/footer/menu rects + vertical-split geometry (the one geometry source both paint and hit-test read). - `panel_render` — the WM_PAINT draw. - `panel_input` — click/wheel/keyboard routing + the new-content auto-tag timer. diff --git a/src/shell/panel/panel_bank_ops.cpp b/src/shell/panel/panel_bank_ops.cpp index 0ce2fbb..6518ee3 100644 --- a/src/shell/panel/panel_bank_ops.cpp +++ b/src/shell/panel/panel_bank_ops.cpp @@ -16,6 +16,7 @@ #include "shell/panel/panel_state.h" #include "shell/panel/panel_bank_ops.h" +#include "shell/actions/package_import_action.h" // doImportBankPackage — the menu's import row #include "shell/bank_ops/bank_ops.h" // bankOp* promptless verbs #include "shell/persist/session.h" // ReaSamplerSession — the live session the ops mutate @@ -242,6 +243,7 @@ enum : unsigned int { kMenuEvacuate, kMenuCreate, kMenuRemove, // remove selected sample(s) from the source bank + kMenuImportPackage, // land a .rsbank as a NEW bank (never merges into this one) kMenuMoveBase = 1000, // move-to-bank: kMenuMoveBase + destination index kMenuCopyBase = 2000, // copy-to-bank: kMenuCopyBase + destination index }; @@ -267,6 +269,7 @@ void showTabMenu(int screenX, int screenY, const std::string& bankId) { menuAppend(menu, kMenuDelete, "Delete..."); menuSeparator(menu); menuAppend(menu, kMenuCreate, "New bank..."); + menuAppend(menu, kMenuImportPackage, "Import bank package..."); const int cmd = TrackPopupMenu(menu, TPM_RETURNCMD, screenX, screenY, 0, g_panel.hwnd, nullptr); @@ -278,6 +281,11 @@ void showTabMenu(int screenX, int screenY, const std::string& bankId) { case kMenuEvacuate: doEvacuateBank(bankId); break; case kMenuDelete: doDeleteBank(bankId); break; case kMenuCreate: doCreateBank(); break; + // Always a NEW bank, never a merge into the right-clicked one — the row sits + // here because this is the panel's bank menu, not because it targets this bank. + case kMenuImportPackage: + if (g_panel.session) doImportBankPackage(*g_panel.session); + break; default: break; } } diff --git a/src/shell/panel/panel_window.cpp b/src/shell/panel/panel_window.cpp index 9cc90f8..a953d3d 100644 --- a/src/shell/panel/panel_window.cpp +++ b/src/shell/panel/panel_window.cpp @@ -15,6 +15,7 @@ #include "shell/panel/draw_kit.h" #include "shell/actions/ingest.h" +#include "shell/actions/package_import_action.h" #ifdef _WIN32 #include // GET_X_LPARAM / GET_Y_LPARAM (SWELL supplies them on mac/linux) @@ -40,12 +41,24 @@ PanelState g_panel; namespace { +bool isPackagePath(const std::string& path) { + static const std::string kExt = ".rsbank"; + if (path.size() <= kExt.size()) return false; + std::string tail = path.substr(path.size() - kExt.size()); + for (char& c : tail) + if (c >= 'A' && c <= 'Z') c = static_cast(c - 'A' + 'a'); + return tail == kExt; +} + // DragQueryFile(hDrop, 0xFFFFFFFF, ...) returns the file count; each path is then // queried by index (length first, excludes NUL, then a sized buffer). DragFinish -// always frees the shell-allocated drop buffer. Multi-file drop imports all into -// the active bank (bank-fill only — no assignment to any live instance). +// always frees the shell-allocated drop buffer. A .rsbank is a whole bank, not audio, +// so it routes to the import verb (one new bank each); everything else keeps the +// existing ingest route — multi-file drop imports all into the active bank (bank-fill +// only, no assignment to any live instance). void handleDropFiles(HDROP hDrop) { std::vector paths; + std::vector packages; const UINT count = DragQueryFile(hDrop, 0xFFFFFFFF, nullptr, 0); paths.reserve(count); for (UINT i = 0; i < count; ++i) { @@ -54,9 +67,14 @@ void handleDropFiles(HDROP hDrop) { std::vector buf(static_cast(len) + 1, '\0'); DragQueryFile(hDrop, i, buf.data(), static_cast(buf.size())); std::string p(buf.data()); - if (!p.empty()) paths.push_back(std::move(p)); + if (p.empty()) continue; + if (isPackagePath(p)) packages.push_back(std::move(p)); + else paths.push_back(std::move(p)); } DragFinish(hDrop); + if (g_panel.session) + for (const std::string& pkg : packages) + doImportBankPackageFile(*g_panel.session, pkg); if (!paths.empty()) ingestDroppedFiles(paths); } diff --git a/src/shell/persist/session.h b/src/shell/persist/session.h index ffbaad0..d8a808b 100644 --- a/src/shell/persist/session.h +++ b/src/shell/persist/session.h @@ -94,6 +94,12 @@ public: // treatment. void recordCreated(const model::Sample& sample, tracking::OriginKind kind); + // Whether a birth record can be written at all this session — the status WITHOUT + // the records. That is not a hole in the pairing rule above: the rule exists so an + // absent record is never read as a definite answer, and this exposes strictly less + // than the pair. The package import gates on it before it opens a file picker. + tracking::LedgerStatus ledgerStatus() const { return trackingStatus_; } + // The version that last wrote the active project: PreVersioning (no // stamp), Unknown (malformed), or Stamped. const version::WritingVersion& writingVersion() const { return writingVersion_; } diff --git a/tests/test_import_landing.cpp b/tests/test_import_landing.cpp new file mode 100644 index 0000000..e97f7f0 --- /dev/null +++ b/tests/test_import_landing.cpp @@ -0,0 +1,394 @@ +// Standalone tests for shell/package/import_landing — no REAPER, no framework, real +// package bytes on a real filesystem. Packages are FRAMED BY HAND (not by +// encodePackage) so the version-ladder suites can dial formatVersion and +// minReaderVersion independently, and so an encode-side regression cannot hide the +// import's behaviour from itself. + +#include "../src/shell/package/import_landing.h" + +#include +#include +#include +#include +#include +#include + +#include "../src/core/capture/wav_codec.h" +#include "../src/core/package/bank_package.h" +#include "../src/core/tracking/origin_ledger.h" +#include "../src/core/version/app_version.h" +#include "../src/shell/package/package_path.h" + +using namespace reasampler; +using namespace reasampler::package; +using reasampler::model::Sample; +namespace fs = std::filesystem; + +static int g_fail = 0; +#define CHECK(cond) do { if(!(cond)) { \ + std::printf("FAIL line %d: %s\n", __LINE__, #cond); ++g_fail; } } while(0) + +static const char* kTag = "1754000000"; +static const char* kBankId = "import-test-bank"; + +// --- scratch project --------------------------------------------------------- + +// One project directory per suite, torn down after, so no suite can observe another's +// bank folder in listFolderFileNames. +class Scratch { +public: + explicit Scratch(const std::string& name) + : dir_(pathToUtf8(fs::current_path() / utf8Path("import_scratch_" + name))) { + std::error_code ec; + fs::remove_all(utf8Path(dir_), ec); + fs::create_directories(utf8Path(dir_), ec); + } + ~Scratch() { + std::error_code ec; + fs::remove_all(utf8Path(dir_), ec); + } + const std::string& projectDir() const { return dir_; } + std::string bankDir() const { return bankFolderDir(dir_); } + std::string packagePath() const { return dir_ + "/bank.rsbank"; } + + std::vector bankFiles() const { + std::vector out; + std::error_code ec; + for (const auto& e : fs::directory_iterator(utf8Path(bankDir()), ec)) + if (e.is_regular_file(ec)) out.push_back(pathToUtf8(e.path().filename())); + return out; + } + +private: + std::string dir_; +}; + +static void writeBytes(const std::string& path, const std::vector& bytes) { + std::ofstream f(utf8Path(path), std::ios::binary | std::ios::trunc); + f.write(reinterpret_cast(bytes.data()), + static_cast(bytes.size())); +} + +static std::vector readBytes(const std::string& path) { + std::ifstream f(utf8Path(path), std::ios::binary); + return std::vector(std::istreambuf_iterator(f), + std::istreambuf_iterator()); +} + +// --- hand-rolled package framing -------------------------------------------- + +static void putU32(std::vector& out, std::uint32_t v) { + for (int b = 0; b < 4; ++b) out.push_back(static_cast((v >> (b * 8)) & 0xFFu)); +} + +static std::vector frame(std::uint32_t formatVersion, + std::uint32_t minReaderVersion, + const std::string& writerSemver, + const std::string& manifestJson, + const std::vector>& payloads) { + std::vector out(kPackageMagic, kPackageMagic + 4); + putU32(out, formatVersion); + putU32(out, minReaderVersion); + putU32(out, static_cast(writerSemver.size())); + out.insert(out.end(), writerSemver.begin(), writerSemver.end()); + putU32(out, static_cast(manifestJson.size())); + out.insert(out.end(), manifestJson.begin(), manifestJson.end()); + for (const auto& p : payloads) out.insert(out.end(), p.begin(), p.end()); + return out; +} + +static std::vector payloadOf(std::size_t n, std::uint8_t seed) { + std::vector v(n); + for (std::size_t i = 0; i < n; ++i) v[i] = static_cast(seed + i * 13u); + return v; +} + +struct Fixture { + PackageManifest manifest; + std::vector> payloads; +}; + +static void addEntry(Fixture& f, const std::string& fileName, const std::string& id, + const std::string& contentHash, std::uint8_t seed) { + std::vector payload = payloadOf(48 + seed, seed); + PackageEntry e; + e.fileName = fileName; + e.byteLength = payload.size(); + e.byteHash = capture::hashBytes(payload.data(), payload.size()); + e.sample.id = id; + e.sample.displayName = id; + e.sample.relativePath = "reasampler_bank/" + fileName; + e.sample.contentHash = contentHash; + f.manifest.entries.push_back(std::move(e)); + f.payloads.push_back(std::move(payload)); +} + +static Fixture twoEntryFixture(const std::string& bankName) { + Fixture f; + f.manifest.bankDisplayName = bankName; + addEntry(f, "kick.wav", "cap-kick", "h-kick", 1); + addEntry(f, "snare.wav", "cap-snare", "h-snare", 2); + return f; +} + +// Frames `f` with this build's own ladder pair unless overridden. +static std::vector packageBytes(const Fixture& f, + std::uint32_t formatVersion = kPackageFormatVersion, + std::uint32_t minReader = kPackageMinReaderVersion, + const std::string& manifestOverride = {}) { + const auto json = serializeManifest(f.manifest); + const std::string body = manifestOverride.empty() ? *json : manifestOverride; + return frame(formatVersion, minReader, version::stampVersion(), body, f.payloads); +} + +// --- the sequence the verb runs, minus REAPER -------------------------------- + +struct RunResult { + ImportLanding landing; + bool applied = false; + tracking::OriginLedger ledger; +}; + +static RunResult runImport(const Scratch& scratch, BankBook& book, + const std::string& bankId = kBankId) { + RunResult r; + LandedFileJournal journal; + r.landing = landPackage(scratch.packagePath(), scratch.projectDir(), book, kTag, journal); + if (r.landing.outcome != ImportOutcome::Landed) return r; + r.applied = applyImportedBank(book, bankId, r.landing.plan, [&r](const Sample& s) { + tracking::OriginRecord rec; + rec.relativePath = s.relativePath; + rec.kind = tracking::OriginKind::PackageImport; + rec.sampleId = s.id; + r.ledger.record(rec); + }); + if (r.applied) journal.markIndexCommitted(); + return r; +} + +// --- suites ------------------------------------------------------------------ + +static void testCleanImportLandsEveryPayloadByteExact() { + Scratch scratch("clean"); + const Fixture f = twoEntryFixture("Drums"); + writeBytes(scratch.packagePath(), packageBytes(f)); + + BankBook book; + const RunResult r = runImport(scratch, book); + + CHECK(r.landing.outcome == ImportOutcome::Landed); + CHECK(r.applied); + CHECK(scratch.bankFiles().size() == 2); + for (std::size_t i = 0; i < 2; ++i) { + const std::string name = r.landing.plan.entries[i].destFileName; + CHECK(readBytes(scratch.bankDir() + "/" + name) == f.payloads[i]); + } + const Bank* bank = book.bank(kBankId); + CHECK(bank != nullptr && bank->displayName == "Drums"); + CHECK(bank->index.size() == 2); + CHECK(PayloadBuffer::alive() == 0); +} + +static void testEveryLandedFileHasAPackageImportBirthRecord() { + Scratch scratch("births"); + const Fixture f = twoEntryFixture("Drums"); + writeBytes(scratch.packagePath(), packageBytes(f)); + + BankBook book; + const RunResult r = runImport(scratch, book); + CHECK(r.applied); + + // Read the ledger back rather than counting calls. + CHECK(r.ledger.size() == 2); + for (const std::string& name : scratch.bankFiles()) { + const tracking::OriginRecord* rec = + r.ledger.find("reasampler_bank/" + name); + CHECK(rec != nullptr); + if (rec) CHECK(rec->kind == tracking::OriginKind::PackageImport); + } +} + +static void testACollapsedEntryLeavesNoFileBehind() { + Scratch scratch("collapse"); + Fixture f; + f.manifest.bankDisplayName = "Drums"; + addEntry(f, "kick.wav", "cap-a", "same-hash", 1); + addEntry(f, "kick_copy.wav", "cap-b", "same-hash", 2); + writeBytes(scratch.packagePath(), packageBytes(f)); + + BankBook book; + const RunResult r = runImport(scratch, book); + CHECK(r.landing.outcome == ImportOutcome::Landed); + // One file, one entry, one birth record — the dedup never manufactured an orphan. + CHECK(scratch.bankFiles().size() == 1); + CHECK(book.bank(kBankId)->index.size() == 1); + CHECK(r.ledger.size() == 1); +} + +static void testHashMismatchLandsNothingAndMutatesNothing() { + Scratch scratch("integrity"); + const Fixture f = twoEntryFixture("Drums"); + std::vector bytes = packageBytes(f); + bytes.back() ^= 0xFFu; // corrupt the LAST entry's payload + writeBytes(scratch.packagePath(), bytes); + + BankBook book; + const BankBook before = book; + const RunResult r = runImport(scratch, book); + + CHECK(r.landing.outcome == ImportOutcome::IntegrityFailed); + CHECK(r.landing.failedEntryName == "snare.wav"); + // Refused BEFORE landing anything, not landed-then-rolled-back: the bank folder is + // created on the way into the write loop, so its absence dates the refusal. + CHECK(!fs::exists(utf8Path(scratch.bankDir()))); + CHECK(r.landing.rollback.deletedCount == 0); + CHECK(book == before); // zero index mutation + CHECK(!r.applied); +} + +static void testWriteFailureAtEntryTwoRollsBackEntryOne() { + Scratch scratch("rollback"); + const Fixture f = twoEntryFixture("Drums"); + writeBytes(scratch.packagePath(), packageBytes(f)); + + // Occupy the second entry's destination with a DIRECTORY: listFolderFileNames sees + // regular files only, so the plan does not rename around it, and the exclusive + // create then fails exactly where the injection wants it. + std::error_code ec; + fs::create_directories(utf8Path(scratch.bankDir()), ec); + fs::create_directory(utf8Path(scratch.bankDir() + "/snare.wav"), ec); + + BankBook book; + const BankBook before = book; + const RunResult r = runImport(scratch, book); + + CHECK(r.landing.outcome == ImportOutcome::WriteFailed); + CHECK(r.landing.failedEntryName == "snare.wav"); + CHECK(r.landing.rollback.deletedCount == 1); // entry one was rolled back + CHECK(scratch.bankFiles().empty()); // nothing from this import survives + CHECK(book == before); + CHECK(PayloadBuffer::alive() == 0); +} + +static void testTooNewRefusesWholeAndNamesTheWriter() { + Scratch scratch("toonew"); + const Fixture f = twoEntryFixture("Drums"); + writeBytes(scratch.packagePath(), + packageBytes(f, kPackageFormatVersion + 1, kPackageFormatVersion + 1)); + + BankBook book; + const BankBook before = book; + const RunResult r = runImport(scratch, book); + + CHECK(r.landing.outcome == ImportOutcome::TooNew); + // All three facts the refusal must name are available from the landing. + CHECK(r.landing.header.minReaderVersion == kPackageFormatVersion + 1); + CHECK(r.landing.header.writerVersion == version::stampVersion()); + // The refusal returns before the bank folder is even created. + CHECK(!fs::exists(utf8Path(scratch.bankDir()))); + CHECK(book == before); +} + +static void testNewerFormatVersionStillImportsWhenTheReaderIsReachable() { + Scratch scratch("additive"); + const Fixture f = twoEntryFixture("Drums"); + // An additive newer writer: formatVersion moved, minReaderVersion did not, and the + // manifest carries a key this build has never heard of. + std::string json = *serializeManifest(f.manifest); + CHECK(!json.empty() && json.front() == '{'); + json.insert(1, "\"futureKey\":{\"nested\":[1,2,3]},"); + writeBytes(scratch.packagePath(), + packageBytes(f, kPackageFormatVersion + 1, kPackageMinReaderVersion, json)); + + BankBook book; + const RunResult r = runImport(scratch, book); + + CHECK(r.landing.outcome == ImportOutcome::Landed); + CHECK(r.landing.header.formatVersion == kPackageFormatVersion + 1); + CHECK(book.bank(kBankId)->index.size() == 2); + CHECK(scratch.bankFiles().size() == 2); +} + +static void testTruncationAndGarbageReportMalformedNotTooNew() { + { + Scratch scratch("truncated"); + const Fixture f = twoEntryFixture("Drums"); + std::vector bytes = packageBytes(f); + bytes.pop_back(); + writeBytes(scratch.packagePath(), bytes); + + BankBook book; + const BankBook before = book; + const RunResult r = runImport(scratch, book); + CHECK(r.landing.outcome == ImportOutcome::Malformed); + CHECK(book == before); + } + { + Scratch scratch("garbage"); + writeBytes(scratch.packagePath(), payloadOf(200, 9)); + BankBook book; + CHECK(runImport(scratch, book).landing.outcome == ImportOutcome::Malformed); + } +} + +static void testMissingPackageFileIsUnreadableNotMalformed() { + Scratch scratch("absent"); + BankBook book; + CHECK(runImport(scratch, book).landing.outcome == ImportOutcome::Unreadable); +} + +static void testUnsavedProjectRefusesBeforeAnythingIsRead() { + Scratch scratch("noproject"); + writeBytes(scratch.packagePath(), packageBytes(twoEntryFixture("Drums"))); + BankBook book; + LandedFileJournal journal; + const ImportLanding landing = + landPackage(scratch.packagePath(), std::string{}, book, kTag, journal); + CHECK(landing.outcome == ImportOutcome::NoProject); +} + +static void testReimportingIntoTheSourceProjectLandsBesideAnUntouchedOriginal() { + Scratch scratch("roundtrip"); + const Fixture f = twoEntryFixture("B"); + writeBytes(scratch.packagePath(), packageBytes(f)); + + // A project that already holds bank "B" with the package's own entries. + BankBook book; + book.createBank("bank-b", "B"); + for (const PackageEntry& e : f.manifest.entries) book.index("bank-b")->add(e.sample); + const BankModel originalB = *book.index("bank-b"); + + const RunResult second = runImport(scratch, book, "bank-b2"); + CHECK(second.landing.outcome == ImportOutcome::Landed); + CHECK(book.bank("bank-b2")->displayName == "B 2"); + CHECK(book.bank("bank-b2")->index.size() == 2); + CHECK(*book.index("bank-b") == originalB); // B itself unmutated + for (const Sample& s : book.index("bank-b2")->all()) { + CHECK(s.id.rfind(kImportIdPrefix, 0) == 0); + CHECK(s.id != "cap-kick" && s.id != "cap-snare"); + } + + const RunResult third = runImport(scratch, book, "bank-b3"); + CHECK(third.landing.outcome == ImportOutcome::Landed); + CHECK(book.bank("bank-b3")->displayName == "B 3"); + CHECK(*book.index("bank-b") == originalB); + // Six distinct files: the original two plus two per re-import, never overwritten. + CHECK(scratch.bankFiles().size() == 4); +} + +int main() { + testCleanImportLandsEveryPayloadByteExact(); + testEveryLandedFileHasAPackageImportBirthRecord(); + testACollapsedEntryLeavesNoFileBehind(); + testHashMismatchLandsNothingAndMutatesNothing(); + testWriteFailureAtEntryTwoRollsBackEntryOne(); + testTooNewRefusesWholeAndNamesTheWriter(); + testNewerFormatVersionStillImportsWhenTheReaderIsReachable(); + testTruncationAndGarbageReportMalformedNotTooNew(); + testMissingPackageFileIsUnreadableNotMalformed(); + testUnsavedProjectRefusesBeforeAnythingIsRead(); + testReimportingIntoTheSourceProjectLandsBesideAnUntouchedOriginal(); + + if (g_fail == 0) std::printf("import_landing: all tests passed\n"); + return g_fail == 0 ? 0 : 1; +} diff --git a/tests/test_import_plan.cpp b/tests/test_import_plan.cpp new file mode 100644 index 0000000..760d650 --- /dev/null +++ b/tests/test_import_plan.cpp @@ -0,0 +1,370 @@ +// Standalone tests for reasampler::package::import_plan — no REAPER, no filesystem, +// no test framework. Every one of the four collision classes (sample id, bank-folder +// file name, content hash, bank display name) is exercised here, which is the point of +// the module: the whole collision rule set is decidable from strings and hashes. + +#include "../src/core/package/import_plan.h" + +#include +#include +#include + +#include "../src/core/tracking/tracking_authority.h" + +using namespace reasampler; +using namespace reasampler::package; +using reasampler::model::Sample; +using reasampler::model::SlotMap; + +static int g_fail = 0; +#define CHECK(cond) do { if(!(cond)) { \ + std::printf("FAIL line %d: %s\n", __LINE__, #cond); ++g_fail; } } while(0) + +static const char* kProjectDir = "/proj"; +static const char* kTag = "1754000000"; + +// --- fixtures ---------------------------------------------------------------- + +static PackageEntry entry(const std::string& fileName, const std::string& id, + const std::string& hash) { + PackageEntry e; + e.fileName = fileName; + e.byteLength = 64; + e.byteHash = "0011223344556677"; + e.sample.id = id; + e.sample.displayName = id; + e.sample.relativePath = "reasampler_bank/" + fileName; + e.sample.contentHash = hash; + return e; +} + +static PackageManifest manifestOf(std::vector entries, + const std::string& bankName) { + PackageManifest m; + m.bankDisplayName = bankName; + m.entries = std::move(entries); + return m; +} + +// A book carrying the named banks, in order, each with a caller-supplied id. +static BankBook bookWithBanks(const std::vector& names) { + BankBook book; + for (std::size_t i = 0; i < names.size(); ++i) + book.createBank("bank-" + std::to_string(i), names[i]); + return book; +} + +static const PlannedEntry& landed(const ImportPlan& plan, std::size_t manifestIndex) { + return plan.entries[manifestIndex]; +} + +// --- the bank-name probe (collision class 4) --------------------------------- + +static std::string plannedName(const std::vector& existingBanks, + const std::string& packageBankName) { + const BankBook book = bookWithBanks(existingBanks); + return planImport(manifestOf({}, packageBankName), book, kProjectDir, {}, kTag) + .bankDisplayName; +} + +static void testFreeSeedIsKeptVerbatim() { + CHECK(plannedName({"Percussion"}, "Drums") == "Drums"); + const ImportPlan plan = + planImport(manifestOf({}, "Drums"), bookWithBanks({}), kProjectDir, {}, kTag); + CHECK(!plan.bankNameAdjusted); + CHECK(plan.seedBankName == "Drums"); +} + +static void testFoldedCollisionTakesTheFirstSuffix() { + // The book's fold is case- and whitespace-insensitive, so "drums" blocks "Drums". + CHECK(plannedName({"drums"}, "Drums") == "Drums 2"); + CHECK(plannedName({" DRUMS "}, "Drums") == "Drums 2"); + + const ImportPlan plan = + planImport(manifestOf({}, "Drums"), bookWithBanks({"drums"}), kProjectDir, {}, kTag); + CHECK(plan.bankNameAdjusted); + CHECK(plan.seedBankName == "Drums"); // the message needs what was asked for +} + +static void testProbeFillsAGap() { + // First-free-ascending, not highest-plus-one: "Drums 2" is free, so it wins. + CHECK(plannedName({"Drums", "Drums 3"}, "Drums") == "Drums 2"); +} + +static void testSeedIsNeverReparsed() { + // "Drums 2" colliding lands as "Drums 2 2", NOT "Drums 3" — a bare trailing integer + // cannot be told from a name the user wrote. + CHECK(plannedName({"Drums 2"}, "Drums 2") == "Drums 2 2"); + CHECK(plannedName({"Kit 808"}, "Kit 808") == "Kit 808 2"); +} + +static void testBlankRecordedNameFallsBackToTheDefault() { + CHECK(plannedName({}, "") == kDefaultImportBankName); + CHECK(plannedName({}, " \t ") == kDefaultImportBankName); + // And the fallback is a seed like any other, so a second one suffixes. + CHECK(plannedName({kDefaultImportBankName}, "") == + std::string(kDefaultImportBankName) + " 2"); +} + +static void testPoolExportLandsAsANamedBank() { + // The destination's pool always exists and always carries the protected name + // "Pool", so a pool export imports as a NAMED bank "Pool 2" — intended, not a glitch. + CHECK(plannedName({}, "Pool") == "Pool 2"); + const ImportPlan plan = + planImport(manifestOf({}, "Pool"), bookWithBanks({}), kProjectDir, {}, kTag); + CHECK(plan.bankNameAdjusted); +} + +static void testRepeatedImportsWalkTheSuffixUpwards() { + CHECK(plannedName({"B"}, "B") == "B 2"); + CHECK(plannedName({"B", "B 2"}, "B") == "B 3"); +} + +// --- sample ids (collision class 1) ------------------------------------------ + +static void testEveryIdIsRemintedUnderTheImportPrefix() { + const PackageManifest m = manifestOf({entry("kick.wav", "cap-1-kick.wav", "h1"), + entry("snare.wav", "cap-2-snare.wav", "h2")}, + "Drums"); + const ImportPlan plan = planImport(m, bookWithBanks({}), kProjectDir, {}, kTag); + + CHECK(plan.landCount == 2); + for (const PlannedEntry& e : plan.entries) { + CHECK(e.sample.id.rfind(kImportIdPrefix, 0) == 0); + CHECK(e.sample.id != "cap-1-kick.wav"); + CHECK(e.sample.id != "cap-2-snare.wav"); + } + CHECK(plan.entries[0].sample.id != plan.entries[1].sample.id); +} + +static void testReimportingIntoTheSourceProjectRemintsRatherThanCollides() { + // The package came FROM this project, so its ids are the ones already in use. + BankBook book = bookWithBanks({"B"}); + Sample existing; + existing.id = "cap-1-kick.wav"; + existing.relativePath = "reasampler_bank/kick.wav"; + existing.contentHash = "h1"; + book.index("bank-0")->add(existing); + + const PackageManifest m = manifestOf({entry("kick.wav", "cap-1-kick.wav", "h1")}, "B"); + const ImportPlan plan = + planImport(m, book, kProjectDir, {"kick.wav"}, kTag); + + CHECK(plan.bankDisplayName == "B 2"); + CHECK(landed(plan, 0).sample.id != "cap-1-kick.wav"); + // The hash lives in another bank; cross-bank dedup is deliberately not enforced, + // so the entry still lands rather than collapsing onto B's copy. + CHECK(landed(plan, 0).action == EntryAction::Land); + CHECK(landed(plan, 0).renamed); +} + +static void testParentIsRemappedWhenItTravelledInThePackage() { + PackageEntry parent = entry("kick.wav", "cap-parent", "h1"); + PackageEntry child = entry("kick_r2.wav", "cap-child", "h2"); + child.sample.provenance = model::Provenance{"cap-parent", "fx-snapshot"}; + + const ImportPlan plan = planImport(manifestOf({parent, child}, "Drums"), + bookWithBanks({}), kProjectDir, {}, kTag); + + CHECK(landed(plan, 1).sample.provenance.has_value()); + CHECK(landed(plan, 1).sample.provenance->parentSampleId == + landed(plan, 0).sample.id); + CHECK(landed(plan, 1).sample.provenance->fxChainSnapshot == "fx-snapshot"); +} + +static void testParentIsRemappedEvenWhenItFollowsTheChild() { + // Manifest order does not constrain lineage, so the remap runs after every id is minted. + PackageEntry child = entry("kick_r2.wav", "cap-child", "h2"); + child.sample.provenance = model::Provenance{"cap-parent", ""}; + PackageEntry parent = entry("kick.wav", "cap-parent", "h1"); + + const ImportPlan plan = planImport(manifestOf({child, parent}, "Drums"), + bookWithBanks({}), kProjectDir, {}, kTag); + CHECK(landed(plan, 0).sample.provenance->parentSampleId == landed(plan, 1).sample.id); +} + +static void testForeignParentIsClearedNotCarried() { + PackageEntry child = entry("kick.wav", "cap-child", "h1"); + child.sample.provenance = model::Provenance{"cap-not-in-this-package", "fx"}; + + const ImportPlan plan = planImport(manifestOf({child}, "Drums"), + bookWithBanks({}), kProjectDir, {}, kTag); + CHECK(landed(plan, 0).sample.provenance.has_value()); + CHECK(landed(plan, 0).sample.provenance->parentSampleId.empty()); + CHECK(landed(plan, 0).sample.provenance->fxChainSnapshot == "fx"); +} + +// --- bank-folder file names (collision class 2) ------------------------------ + +static void testAFreeBankLegalNameIsKept() { + const ImportPlan plan = planImport(manifestOf({entry("kick.wav", "a", "h1")}, "Drums"), + bookWithBanks({}), kProjectDir, + {"unrelated.wav"}, kTag); + CHECK(landed(plan, 0).destFileName == "kick.wav"); + CHECK(!landed(plan, 0).renamed); + CHECK(plan.renameCount == 0); + CHECK(landed(plan, 0).sample.relativePath == "reasampler_bank/kick.wav"); +} + +static void testATakenNameIsMintedFreshAndNeverOverwritten() { + const ImportPlan plan = planImport(manifestOf({entry("kick.wav", "a", "h1")}, "Drums"), + bookWithBanks({}), kProjectDir, + {"kick.wav"}, kTag); + CHECK(landed(plan, 0).destFileName != "kick.wav"); + CHECK(landed(plan, 0).renamed); + CHECK(plan.renameCount == 1); + CHECK(landed(plan, 0).sample.relativePath == + "reasampler_bank/" + landed(plan, 0).destFileName); +} + +static void testTheFolderNameCheckFoldsAsciiCase() { + // Windows and the default APFS would land "kick.wav" onto "KICK.WAV". + const ImportPlan plan = planImport(manifestOf({entry("kick.wav", "a", "h1")}, "Drums"), + bookWithBanks({}), kProjectDir, + {"KICK.WAV"}, kTag); + CHECK(landed(plan, 0).destFileName != "kick.wav"); + CHECK(landed(plan, 0).renamed); +} + +static void testTwoEntriesNeverLandOnOneName() { + // Two package names that differ only by case are one destination file. + const ImportPlan plan = + planImport(manifestOf({entry("kick.wav", "a", "h1"), entry("Kick.wav", "b", "h2")}, + "Drums"), + bookWithBanks({}), kProjectDir, {}, kTag); + CHECK(plan.landCount == 2); + CHECK(landed(plan, 0).destFileName != landed(plan, 1).destFileName); +} + +static void testAnUnsanitaryNameIsMintedRatherThanLandedVerbatim() { + const ImportPlan plan = + planImport(manifestOf({entry("Hit One.wav", "a", "h1")}, "Drums"), + bookWithBanks({}), kProjectDir, {}, kTag); + CHECK(landed(plan, 0).destFileName.find(' ') == std::string::npos); + CHECK(landed(plan, 0).renamed); +} + +// --- content hash (collision class 3) ---------------------------------------- + +static void testAnAlreadyLandedHashCollapsesWithoutAWrite() { + const ImportPlan plan = + planImport(manifestOf({entry("kick.wav", "a", "same"), + entry("kick_copy.wav", "b", "same")}, + "Drums"), + bookWithBanks({}), kProjectDir, {}, kTag); + + CHECK(plan.landCount == 1); + CHECK(plan.collapseCount == 1); + CHECK(landed(plan, 0).action == EntryAction::Land); + CHECK(landed(plan, 1).action == EntryAction::Collapse); + // No name is claimed for it — a dedup that wrote a file would manufacture an orphan. + CHECK(landed(plan, 1).destFileName.empty()); + // Every manifest entry still yields exactly one planned entry: planImport is total. + CHECK(plan.entries.size() == 2); +} + +static void testAParentPointingAtACollapsedEntryResolvesToTheSurvivor() { + PackageEntry first = entry("kick.wav", "cap-first", "same"); + PackageEntry dupe = entry("kick_copy.wav", "cap-dupe", "same"); + PackageEntry child = entry("kick_r2.wav", "cap-child", "other"); + child.sample.provenance = model::Provenance{"cap-dupe", ""}; + + const ImportPlan plan = planImport(manifestOf({first, dupe, child}, "Drums"), + bookWithBanks({}), kProjectDir, {}, kTag); + CHECK(landed(plan, 2).sample.provenance->parentSampleId == landed(plan, 0).sample.id); +} + +static void testAnEmptyHashNeverCollapses() { + // Mirrors findByHash: an unhashable entry does not participate in dedup. + const ImportPlan plan = + planImport(manifestOf({entry("a.wav", "a", ""), entry("b.wav", "b", "")}, "Drums"), + bookWithBanks({}), kProjectDir, {}, kTag); + CHECK(plan.landCount == 2); + CHECK(plan.collapseCount == 0); +} + +// --- slots ------------------------------------------------------------------- + +static void testSlotsRideAlongOverTheRemintedIds() { + PackageManifest m = manifestOf({entry("kick.wav", "cap-a", "h1"), + entry("snare.wav", "cap-b", "h2")}, + "Drums"); + // A gap the package carried: slot 0 empty, occupants at 1 and 3. + m.slots = SlotMap::fromEntries({{"cap-a", 1}, {"cap-b", 3}}); + + const ImportPlan plan = planImport(m, bookWithBanks({}), kProjectDir, {}, kTag); + CHECK(plan.slots.slotOf(landed(plan, 0).sample.id) == 1); + CHECK(plan.slots.slotOf(landed(plan, 1).sample.id) == 3); + // The package's own ids are gone from the map — a foreign id never enters the index. + CHECK(plan.slots.slotOf("cap-a") == -1); +} + +static void testACollapsedEntryDoesNotDoubleOccupyASlot() { + PackageManifest m = manifestOf({entry("kick.wav", "cap-a", "same"), + entry("kick_copy.wav", "cap-b", "same")}, + "Drums"); + m.slots = SlotMap::fromEntries({{"cap-a", 0}, {"cap-b", 1}}); + + const ImportPlan plan = planImport(m, bookWithBanks({}), kProjectDir, {}, kTag); + CHECK(plan.slots.size() == 1); + CHECK(plan.slots.slotOf(landed(plan, 0).sample.id) == 0); +} + +// --- the ledger gate --------------------------------------------------------- + +static void testDegradedLedgerStatusesRefuseAndTheUsableOnesDoNot() { + CHECK(importLedgerRefusal(tracking::LedgerStatus::Unreadable) == + LedgerRefusal::Malformed); + CHECK(importLedgerRefusal(tracking::LedgerStatus::FutureVersion) == + LedgerRefusal::FutureVersion); + CHECK(importLedgerRefusal(tracking::LedgerStatus::Fresh) == LedgerRefusal::None); + CHECK(importLedgerRefusal(tracking::LedgerStatus::Loaded) == LedgerRefusal::None); +} + +static void testAnUndecodableUsageKeyBlocksPruneButNotImport() { + // The tempting reuse of PruneReport::blockedByTracking would silently refuse an + // import over a key that only ever governs what a DELETION may touch. + tracking::OriginLedger ledger; + wire::UsageFoldResult usage; + usage.abortPrune = true; + usage.offendingKeys = {"rsusage_{ABC}"}; + + const tracking::TrackingState state{tracking::LedgerStatus::Loaded, ledger, usage}; + CHECK(tracking::pruneProtection(state).blocked); + CHECK(importLedgerRefusal(tracking::LedgerStatus::Loaded) == LedgerRefusal::None); +} + +int main() { + testFreeSeedIsKeptVerbatim(); + testFoldedCollisionTakesTheFirstSuffix(); + testProbeFillsAGap(); + testSeedIsNeverReparsed(); + testBlankRecordedNameFallsBackToTheDefault(); + testPoolExportLandsAsANamedBank(); + testRepeatedImportsWalkTheSuffixUpwards(); + + testEveryIdIsRemintedUnderTheImportPrefix(); + testReimportingIntoTheSourceProjectRemintsRatherThanCollides(); + testParentIsRemappedWhenItTravelledInThePackage(); + testParentIsRemappedEvenWhenItFollowsTheChild(); + testForeignParentIsClearedNotCarried(); + + testAFreeBankLegalNameIsKept(); + testATakenNameIsMintedFreshAndNeverOverwritten(); + testTheFolderNameCheckFoldsAsciiCase(); + testTwoEntriesNeverLandOnOneName(); + testAnUnsanitaryNameIsMintedRatherThanLandedVerbatim(); + + testAnAlreadyLandedHashCollapsesWithoutAWrite(); + testAParentPointingAtACollapsedEntryResolvesToTheSurvivor(); + testAnEmptyHashNeverCollapses(); + + testSlotsRideAlongOverTheRemintedIds(); + testACollapsedEntryDoesNotDoubleOccupyASlot(); + + testDegradedLedgerStatusesRefuseAndTheUsableOnesDoNot(); + testAnUndecodableUsageKeyBlocksPruneButNotImport(); + + if (g_fail == 0) std::printf("import_plan: all tests passed\n"); + return g_fail == 0 ? 0 : 1; +} From 454f67b3bc6e3b6421ea15d9234bfe8414e5bed3 Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 14:02:01 -0400 Subject: [PATCH 14/24] Close bank-export review findings: name-cap underflow, double overwrite prompt, test scope Clamps insertSuffix's underflow, floors uniqueEntryName's validity guard, suppresses the redundant overwrite confirm via a picker out-param, adds a PayloadBuffer high-water mark, and corrects stale CLAUDE.md/CMake claims. --- docs/PLAN.md | 10 +- docs/product/bank-package.md | 24 ++++ src/app/main.cpp | 4 +- src/core/package/CLAUDE.md | 24 ++-- src/core/package/export_plan.cpp | 40 ++++-- src/core/package/export_plan.h | 6 +- src/shell/actions/package_export_action.cpp | 8 +- src/shell/package/CLAUDE.md | 38 ++--- src/shell/package/CMakeLists.txt | 13 +- src/shell/package/export_bank.h | 8 +- src/shell/package/package_io.cpp | 10 +- src/shell/package/package_io.h | 5 + src/shell/package/package_pickers.cpp | 7 +- src/shell/package/package_pickers.h | 9 +- src/shell/panel/panel_bank_ops.cpp | 2 +- tests/test_export_bank.cpp | 148 ++++++++++++++++---- tests/test_export_plan.cpp | 22 +++ tests/test_package_io.cpp | 4 +- 18 files changed, 291 insertions(+), 91 deletions(-) diff --git a/docs/PLAN.md b/docs/PLAN.md index 0833a9a..b2ccbbe 100644 --- a/docs/PLAN.md +++ b/docs/PLAN.md @@ -2704,10 +2704,12 @@ on the import side, `bank_ops`, or `persist`. - Exporting an empty bank produces a valid, importable package with zero entries rather than refusing. An empty bank is a legitimate thing to carry. -**Open questions.** **[propose at review]** whether the export affordance is action-only, -panel-only, or both at ship. **[propose at review]** whether the default file name is derived -from the bank's display name (recommended, sanitized through -`capture_paths::sanitizeStem`) or from the project name. +**Open questions — both answered at implementation review, recorded at +`docs/product/bank-package.md` §"Implementation decisions — Ε-W2-T1".** Affordance: +**both** the action and the panel row (the action is the only spelling that can reach +the pool; the panel row is the direct gesture on a named bank). Default file name: +the bank's **display name**, sanitized through `capture_paths::sanitizeStem`, as +recommended. #### Ε-W2-T2 — `bank-import` diff --git a/docs/product/bank-package.md b/docs/product/bank-package.md index 92c9ee4..dd51dea 100644 --- a/docs/product/bank-package.md +++ b/docs/product/bank-package.md @@ -725,6 +725,30 @@ contemplates one untracked capture, an import strands hundreds). --- +## Implementation decisions — Ε-W2-T1 + +Not [Daniel]-class forks — both were `[propose at review]` calls in `docs/PLAN.md`'s +Ε-W2-T1 track, answered at implementation review rather than by Daniel, and recorded +here per this phase's own convention for keeping such answers where the design lives +rather than only in the track's own now-stale open-questions line. + +- **Affordance: both the bindable action and the panel row.** The action targets the + **active** bank and is the only spelling that can reach the **pool** (the panel's + `showTabMenu` returns early on `isPool()` — a named-bank-tab context menu has no tab + to right-click for the pool), while the exported unit's own definition above includes + the pool. The panel row is the direct gesture on a specific named bank. Neither + subsumes the other. +- **Default file name: the bank's display name**, sanitized through + `capture_paths::sanitizeStem`, seeded into `/.rsbank`. A + project-derived name was the rejected alternative: three banks exported from one + project must produce three distinguishable files, and a project-derived name + collides on the second export. Known wart, worth recording rather than hiding: + `sanitizeStem` collapses an all-non-ASCII display name to the literal `capture`, so + two such banks still collide — the existing rename verb is the recovery, same as the + import-side auto-suffix collisions above. + +--- + ## Non-goals and guardrails - **No auto-insertion of imported audio into the arrange.** Same rule as capture. diff --git a/src/app/main.cpp b/src/app/main.cpp index 599003c..b898cd6 100644 --- a/src/app/main.cpp +++ b/src/app/main.cpp @@ -92,9 +92,7 @@ static void RunCancelRealtime(int) { capture::RunCancelRealtime(g_session); } static void RunRecaptureFromSource(int) { capture::RunRecaptureFromSource(g_session); } static void RunRenderTrackInPlace(int) { capture::RunRenderTrackInPlace(g_session); } static void RunResampleBake(int) { capture::RunResampleBake(g_session); } -static void RunExportBankPackage(int) { - reasampler::doBankPackageExport(g_session, g_session.book().activeBankId()); -} +static void RunExportBankPackage(int) { reasampler::doBankPackageExport(g_session, g_session.book().activeBankId()); } static void RunShowVersion(int) { // On-demand only — no unconditional startup print (routine console chatter pops // the console window). diff --git a/src/core/package/CLAUDE.md b/src/core/package/CLAUDE.md index 80f475f..fb9391a 100644 --- a/src/core/package/CLAUDE.md +++ b/src/core/package/CLAUDE.md @@ -6,8 +6,8 @@ The hand-rolled `RSBK` bank-package container, entirely pure (REAPER-free, unit-tested outside the DAW): the format contract and version ladder, the JSON manifest, and the framing/layout codec. No filesystem — the shell (`src/shell/package`) streams bytes against the layouts produced here. The -export/import *decisions* (`export_plan` / `import_plan`) are separate modules -landing after the format. +export/import *decisions* (`export_plan` / `import_plan`) are separate modules; +`export_plan` has landed, `import_plan` has not. ## Invariants @@ -123,12 +123,20 @@ landing after the format. corruption. - **A written package carries no path in ANY field.** `isValidNestedSamplePath` permits a relative `relativePath` because a *record* may hold one, but - `export_plan` writes each shipping entry's `relativePath` as its bare package - name, so an emitted manifest has no separator anywhere and the entry name is the - single naming authority on both sides. The directory component it drops carries - no information — the bank subfolder is a fixed `capture_paths` constant the - importer re-spells. The nested-path rule stays as the decode-side backstop for a - package this build did not write. + `export_plan` writes each shipping entry's `relativePath` as its bare, sanitized + and disambiguated transport name (`export_plan.cpp`'s `e.fileName`), so an + emitted manifest has no separator anywhere and the entry name is the single + naming authority on both sides. The directory component it drops carries no + information — the bank subfolder is a fixed `capture_paths` constant + (`capture_paths.cpp`'s `deriveBankPaths`) the importer re-spells. **The + basename spelling is dropped too, not just the directory**: `e.fileName` is + `uniqueEntryName(sanitizeEntryName(...))`, not the source basename, so a + macOS-authored `Hit?.wav` survives only in `displayName` — the transport name + itself may differ. Accepted for the same reason the directory drop is: the + transport name exists to be a valid, collision-free package entry, not a + faithful copy of the source spelling, and `displayName` is the field that + carries the original for display. The nested-path rule stays as the decode-side + backstop for a package this build did not write. - **Obligation on the export track: sanitize, don't relay the refusal.** `serializeManifest` returns one indistinguishable `nullopt` for every rejection — an unrepresentable name, a case-folded collision, a traversing nested path, a diff --git a/src/core/package/export_plan.cpp b/src/core/package/export_plan.cpp index aa4e6a0..4803cf0 100644 --- a/src/core/package/export_plan.cpp +++ b/src/core/package/export_plan.cpp @@ -17,16 +17,18 @@ std::string baseNameOf(const std::string& path) { return sep == std::string::npos ? path : path.substr(sep + 1); } -// Trailing dots and spaces are stripped at file creation on Windows, so a name -// carrying them would collide with its stripped twin (isValidEntryName refuses them -// for that reason). +// Mirrors package_format.cpp's trailing-dot/space rule so a truncated stem never +// reintroduces the collision isValidEntryName exists to prevent. std::string stripTrailingDotsAndSpaces(std::string s) { while (!s.empty() && (s.back() == '.' || s.back() == ' ')) s.pop_back(); return s; } -// Truncates to at most `max` bytes without splitting a UTF-8 sequence — a split one -// would leave the name ill-formed, which isValidEntryName refuses outright. +// Truncates to at most `max` bytes without splitting a UTF-8 sequence (the rule +// itself is package_format.cpp's isWellFormedUtf8). A sequence landing exactly on +// the cut is dropped whole, one character short of `max`, rather than checked for +// cleanliness — over-truncating by one character is cheap insurance against a +// subtly wrong boundary check. std::string truncateUtf8(std::string s, std::size_t max) { if (s.size() <= max) return s; s.resize(max); @@ -36,12 +38,19 @@ std::string truncateUtf8(std::string s, std::size_t max) { } // `name` with `suffix` inserted before its extension, trimmed so the result still -// fits the entry-name cap. +// fits the entry-name cap. `suffix.size() + ext.size()` can exceed the cap on its +// own (a long extension, a two-digit disambiguation suffix) — clamped rather than +// subtracted unchecked, which would underflow the size_t `room` below and turn +// truncateUtf8 into a silent no-op. An extension that alone leaves no room even +// after the whole stem is dropped is dropped too; uniqueEntryName's own floor +// covers what even that cannot fix. std::string insertSuffix(const std::string& name, const std::string& suffix) { const std::size_t dot = name.find_last_of('.'); const bool hasExt = dot != std::string::npos && dot > 0; std::string stem = hasExt ? name.substr(0, dot) : name; - const std::string ext = hasExt ? name.substr(dot) : std::string(); + std::string ext = hasExt ? name.substr(dot) : std::string(); + if (suffix.size() >= kMaxEntryNameBytes) return std::string(); + if (ext.size() > kMaxEntryNameBytes - suffix.size()) ext.clear(); const std::size_t room = kMaxEntryNameBytes - suffix.size() - ext.size(); stem = truncateUtf8(std::move(stem), room); return stem + suffix + ext; @@ -54,18 +63,21 @@ bool nameTaken(const std::string& candidate, const std::vector& tak } // A transport name distinct from every name already claimed, under the format's own -// case-folding equivalence (two names differing only by ASCII case would extract onto -// one file on Windows and default APFS). +// case-folding equivalence (package_format.h's sameEntryName). std::string uniqueEntryName(const std::string& base, const std::vector& taken) { if (!nameTaken(base, taken)) return base; - // Bounded by construction: each iteration either returns or collides with a - // distinct member of `taken`, and the suffixed names are pairwise distinct. - std::string candidate = base; + // Each iteration either returns a name both valid and distinct from `taken`, or + // advances to the next suffix; taken.size() + 2 attempts is enough by pigeonhole + // now that insertSuffix cannot underflow. The floor below is the residual case + // validity alone can still fail — an extension so long insertSuffix must drop it + // on every attempt tried here. for (std::size_t n = 2; n <= taken.size() + 2; ++n) { - candidate = insertSuffix(base, "_" + std::to_string(n)); + const std::string candidate = insertSuffix(base, "_" + std::to_string(n)); if (!nameTaken(candidate, taken) && isValidEntryName(candidate)) return candidate; } - return candidate; + // Floors like sanitizeEntryName's own "entry" floor: always valid, regardless of + // how base's own extension behaved. + return sanitizeEntryName("entry_" + std::to_string(taken.size() + 2)); } // What BankModel::add and the manifest's nested-path rule together accept — the pair diff --git a/src/core/package/export_plan.h b/src/core/package/export_plan.h index 77df29d..0d7505a 100644 --- a/src/core/package/export_plan.h +++ b/src/core/package/export_plan.h @@ -80,10 +80,8 @@ ExportPlan planExport(const ExportInputs& in); // separators, reserved characters and control bytes to '_', an over-long name // truncated on a UTF-8 boundary, and an underscore prefix for the reserved forms // ("." / ".." / a DOS device name). Never returns a name isValidEntryName refuses. -// -// A bank ingested on macOS/Linux legitimately holds names Windows cannot spell, and -// relaying the codec's one indistinguishable refusal would make a single such file -// an unactionable total failure of the whole export. +// Why sanitize rather than relay the codec's refusal: this directory's own +// CLAUDE.md, "Obligation on the export track." std::string sanitizeEntryName(const std::string& rawFileName); } // namespace reasampler::package diff --git a/src/shell/actions/package_export_action.cpp b/src/shell/actions/package_export_action.cpp index 9b83c6c..fcaebe3 100644 --- a/src/shell/actions/package_export_action.cpp +++ b/src/shell/actions/package_export_action.cpp @@ -164,7 +164,8 @@ void doBankPackageExport(const ReaSamplerSession& session, const std::string& ba const std::string suggested = projectDir + "/" + capture::sanitizeStem(bankName) + ".rsbank"; std::string dest; - if (!pickPackageSavePath(suggested, dest)) return; // user cancelled the picker + bool appended = false; + if (!pickPackageSavePath(suggested, dest, &appended)) return; // user cancelled the picker ExportRequest req; req.projectDir = projectDir; @@ -172,6 +173,11 @@ void doBankPackageExport(const ReaSamplerSession& session, const std::string& ba req.destAbsPath = dest; req.exportTimestamp = static_cast(std::time(nullptr)); req.allowIncomplete = allowIncomplete; + // The dialog's own overwrite confirm covered exactly this path when the picker + // did not need to append `.rsbank` to reach it — asking again would be a second + // prompt for the same consent. An appended path is one the dialog never saw, so + // that case still falls through to exportBank's own refusal and the confirm below. + req.allowOverwrite = !appended; ExportOutcome out = exportBank(session, req); if (out.status == ExportStatus::RefusedDestinationExists) { diff --git a/src/shell/package/CLAUDE.md b/src/shell/package/CLAUDE.md index 0ab8adc..a24737f 100644 --- a/src/shell/package/CLAUDE.md +++ b/src/shell/package/CLAUDE.md @@ -5,12 +5,12 @@ The filesystem and dialog acts behind bank-package export/import: streaming package file I/O plus the file-status and exclusive-create acts (`package_io`), the UTF-8 path conversion every one of them goes through (`package_path`), the landed-file -journal and its rollback delete (`package_rollback`), and the two file pickers -(`package_pickers`). This seam is bytes-only — the package format (magic, manifest, -entry layout) is `core/package`'s business, and the export/import verbs that -orchestrate both do not live here yet. No REAPER project state is touched in this -directory: no ext-state read or write, no undo block, no generation bump — those -belong to the verbs. +journal and its rollback delete (`package_rollback`), the two file pickers +(`package_pickers`), and the promptless export verb (`export_bank`) — the import verb +does not live here yet. The package format itself (magic, manifest, entry layout) +stays `core/package`'s business. No REAPER project state is touched in this directory: +no ext-state read or write, no undo block, no generation bump — those belong to the +prompting skin (`shell/actions/package_export_action`), not this seam. ## Invariants @@ -47,17 +47,17 @@ belong to the verbs. writer could win the race against. Collision handling (auto-rename) remains the import plan's job upstream. The package writer itself DOES replace an existing destination — the export save dialog's own overwrite confirm is the consent — and - that asymmetry is deliberate. **Known gap, obligation on the export verb:** - `pickPackageSavePath`'s own `.rsbank` re-append (see its Gotcha below) can turn a - confirmed path `X` into a write target `X.rsbank` that the dialog never asked about. - The export verb MUST re-check `fileStatus()` on the path actually handed to - `PackageFileWriter` — after any extension append — and get its own consent if that - re-checked path is `Present`; the dialog's confirm only ever covered the pre-append - path. Not fixed at this seam: prompting is verb-level UX, and `pickPackageSavePath` - has no caller yet, so the gap is latent, not live. **Closed on the export side:** - `exportBank` re-checks `fileStatus()` on the post-append path and refuses - `RefusedDestinationExists` until the caller sets `allowOverwrite`, which - `package_export_action` does only after its own confirm naming that exact path. + that asymmetry is deliberate. **Closed, both halves.** `pickPackageSavePath`'s own + `.rsbank` re-append (see its Gotcha below) can turn a confirmed path `X` into a write + target `X.rsbank` that the dialog never asked about, so `exportBank` re-checks + `fileStatus()` on the path actually handed to `PackageFileWriter` — after any + extension append — and refuses `RefusedDestinationExists` until the caller sets + `allowOverwrite`. The caller does not always re-prompt to get there: + `pickPackageSavePath` reports whether it appended (`outAppended`), and + `package_export_action` pre-grants `allowOverwrite` whenever it did NOT — an + unappended path is exactly what the dialog's own confirm already covered, so asking + again would be a second prompt for the same consent. Only an appended path, one the + dialog never saw, still costs the verb's own confirm naming that exact path. - **The rollback delete is prune's ONE carve-out, and only HALF of it is structural.** The citation and the full discriminator live at `package_rollback.cpp`'s header. "Did this call create it" is structural: only exclusively-created paths are @@ -79,8 +79,8 @@ belong to the verbs. - `package_path` — header-only; the ONE UTF-8-narrow → `fs::path` conversion, so the encoding contract has a single enforcement point. - `package_io` — every filesystem act the verbs need: `PayloadBuffer` (move-only payload + the `alive()` seam counter), `PackageFileWriter` (append-only temp+atomic-rename writer), `PackageFileReader` (seek-and-read one range per call, range-checked against the real file size), `readFilePayload` (one source file as one entry's payload), `fileStatus` (Present/Absent/Unreadable — export's refusal message must distinguish the last two, and an empty payload cannot), `writeFileExclusive` (exclusive create + write, self-cleaning on a partial write), and `listFolderFileNames` (bare UTF-8 names, sorted, non-recursive, non-throwing). REAPER-free; tested without a DAW. - `package_rollback` — `LandedFileJournal`: `writeLandedFile` (exclusive-create land, path resolved absolute, recorded on success only), `markIndexCommitted` (disarms the journal), and `rollback` (deletes exactly the recorded set, hard unlink, tolerating a vanished file; refuses once disarmed). REAPER-free; tested without a DAW. -- `package_pickers` — `pickPackageForImport` and `pickPackageSavePath`, both `GetUserFileName`. Compile-only until the verbs land; nothing here can be exercised in a unit test. -- `export_bank` — the promptless export verb, in three composable public steps: `surveyBankExport` (the read-only plan, report-before-acting), `digestSources` (measures each entry's length + `hashBytes` digest, one payload at a time), and `writePackageFile` (prefix, then each payload re-read and re-verified against that digest before it is appended, then commit). `exportBank` composes the three and gates on the plan verdict, the incomplete confirm and the destination confirm. The session arrives **const** — every mutator on it is non-const, so "an export writes no ext state, opens no undo point and never bumps the generation" is enforced by the type rather than remembered. Reads the session through inline accessors only, which is why its tests link and run without a DAW. +- `package_pickers` — `pickPackageForImport` and `pickPackageSavePath`, both `GetUserFileName`; `pickPackageSavePath` also reports whether it appended `.rsbank` (`outAppended`), the signal `package_export_action` uses to skip a redundant overwrite confirm. `pickPackageForImport` stays compile-only until the import verb lands; neither picker can be exercised in a unit test. +- `export_bank` — the promptless export verb, in three composable public steps: `surveyBankExport` (the read-only plan, report-before-acting), `digestSources` (measures each entry's length + `hashBytes` digest, one payload at a time), and `writePackageFile` (prefix, then each payload re-read and re-verified against that digest before it is appended, then commit). `exportBank` composes the three and gates on the plan verdict, the incomplete confirm and the destination confirm. The session arrives **const**: `saveToActiveProject`, `bumpBankGeneration` and `writeAssignmentRequest` are the session's only non-const acts, so a const session cannot reach them and "an export writes no ext state, opens no undo point and never bumps the generation" holds by the type rather than by memory (`pruneReclaim`, the sole file-deletion path, is const too and sits outside this claim). Reads the session through inline accessors only, which is why its tests link and run without a DAW. ## Gotchas diff --git a/src/shell/package/CMakeLists.txt b/src/shell/package/CMakeLists.txt index caa87da..6f2f148 100644 --- a/src/shell/package/CMakeLists.txt +++ b/src/shell/package/CMakeLists.txt @@ -1,7 +1,8 @@ # The filesystem + dialog seam for bank packages. package_io / package_rollback are # REAPER-free (standard filesystem only), so the pure-library/test helpers fit and -# their tests run without a DAW. The export/import verbs that drive all three targets -# are not in this directory yet. +# their tests run without a DAW. export_bank, the promptless export verb, lives here +# too for the same reason (ReaSamplerSession's inline accessors keep it REAPER-free); +# the import verb does not live here yet. reasampler_pure_library(package_io SOURCES package_io.cpp) reasampler_test(package_io LINK package_io) @@ -11,9 +12,15 @@ reasampler_test(package_rollback LINK package_rollback) # export_bank reads the live session through ReaSamplerSession's INLINE accessors only, # so it pulls in no REAPER-facing TU and its tests link (and run) without a DAW. +# bank_book / tail_control / origin_ledger / tracking_authority / prune_reconcile / +# app_version / view_mode_model are session.h's own transitive includes (BankBook::bank() +# in particular is out-of-line, in bank_book.cpp) — declared here, on the library that +# actually needs them, rather than left for every consumer to enumerate. reasampler_pure_library(export_bank SOURCES export_bank.cpp - LINK PUBLIC export_plan bank_package package_io PRIVATE capture_paths wav_codec) + LINK PUBLIC export_plan bank_package package_io + PRIVATE capture_paths wav_codec bank_book tail_control origin_ledger + tracking_authority prune_reconcile app_version view_mode_model) reasampler_test(export_bank LINK export_bank bank_book slot_map view_mode_model tail_control origin_ledger tracking_authority prune_reconcile app_version capture_paths) diff --git a/src/shell/package/export_bank.h b/src/shell/package/export_bank.h index a6deae8..d78ee3f 100644 --- a/src/shell/package/export_bank.h +++ b/src/shell/package/export_bank.h @@ -2,8 +2,12 @@ // stream, commit. No prompts and no message boxes (shell/actions/ // package_export_action is the skin). The session arrives CONST, which is how "an // export writes no ext state, opens no undo point and never bumps the bank -// generation" is enforced rather than remembered — every mutator on the session is -// non-const. Blocking I/O: UI-thread actions only. +// generation" is enforced rather than remembered — those three acts +// (saveToActiveProject, bumpBankGeneration, writeAssignmentRequest) are exactly the +// session members that are non-const (session.h:113,108,147). Constness does not +// block every mutation, though: pruneReclaim (session.h:140-141) is const and is +// the system's sole file-deletion path — irrelevant to export, but not something a +// const session forbids in general. Blocking I/O: UI-thread actions only. #pragma once diff --git a/src/shell/package/package_io.cpp b/src/shell/package/package_io.cpp index c268742..f057f03 100644 --- a/src/shell/package/package_io.cpp +++ b/src/shell/package/package_io.cpp @@ -29,6 +29,7 @@ namespace fs = std::filesystem; namespace { std::atomic g_alivePayloads{0}; +std::atomic g_peakAlivePayloads{0}; } // --------------------------------------------------------------------------- @@ -36,7 +37,13 @@ std::atomic g_alivePayloads{0}; PayloadBuffer::PayloadBuffer(std::vector bytes) : bytes_(std::move(bytes)), counted_(!bytes_.empty()) { - if (counted_) g_alivePayloads.fetch_add(1, std::memory_order_relaxed); + if (counted_) { + const int now = g_alivePayloads.fetch_add(1, std::memory_order_relaxed) + 1; + int peak = g_peakAlivePayloads.load(std::memory_order_relaxed); + while (now > peak && !g_peakAlivePayloads.compare_exchange_weak( + peak, now, std::memory_order_relaxed)) { + } + } } PayloadBuffer::~PayloadBuffer() { release(); } @@ -60,6 +67,7 @@ PayloadBuffer& PayloadBuffer::operator=(PayloadBuffer&& other) noexcept { } int PayloadBuffer::alive() { return g_alivePayloads.load(std::memory_order_relaxed); } +int PayloadBuffer::highWaterMark() { return g_peakAlivePayloads.load(std::memory_order_relaxed); } void PayloadBuffer::release() { if (counted_) g_alivePayloads.fetch_sub(1, std::memory_order_relaxed); diff --git a/src/shell/package/package_io.h b/src/shell/package/package_io.h index adf60ec..384a06b 100644 --- a/src/shell/package/package_io.h +++ b/src/shell/package/package_io.h @@ -33,6 +33,11 @@ public: // Buffers currently holding at least one byte, process-wide. static int alive(); + // The largest alive() has ever been, process-wide. A point-in-time alive() == 0 + // check after a call returns cannot fail on a whole-package-in-memory shape that + // allocated N buffers and freed them all one at a time — highWaterMark() can, + // since it is never reset. + static int highWaterMark(); private: void release(); diff --git a/src/shell/package/package_pickers.cpp b/src/shell/package/package_pickers.cpp index 16395a6..da04894 100644 --- a/src/shell/package/package_pickers.cpp +++ b/src/shell/package/package_pickers.cpp @@ -47,7 +47,8 @@ bool pickPackageForImport(std::string& outAbsPath) { return runPicker(1, "Import bank package", "", outAbsPath); } -bool pickPackageSavePath(const std::string& suggestedPath, std::string& outAbsPath) { +bool pickPackageSavePath(const std::string& suggestedPath, std::string& outAbsPath, + bool* outAppended) { // [verify — DAW] GetUserFileName takes no owner window, so the dialog's parenting // is REAPER's to do; the previous Win32 path passed GetMainHwnd() explicitly. if (!runPicker(0, "Export bank package", suggestedPath.c_str(), outAbsPath)) { @@ -56,7 +57,9 @@ bool pickPackageSavePath(const std::string& suggestedPath, std::string& outAbsPa // GetUserFileName has no lpstrDefExt equivalent (the old Win32 picker's // ofn.lpstrDefExt = L"rsbank"); whether mode 0 appends one itself from // kExtList is [verify — DAW], so append it ourselves whenever it's missing. - if (!hasCaseInsensitiveSuffix(outAbsPath, ".rsbank")) outAbsPath += ".rsbank"; + const bool appended = !hasCaseInsensitiveSuffix(outAbsPath, ".rsbank"); + if (appended) outAbsPath += ".rsbank"; + if (outAppended) *outAppended = appended; return true; } diff --git a/src/shell/package/package_pickers.h b/src/shell/package/package_pickers.h index 9f60512..136a6d9 100644 --- a/src/shell/package/package_pickers.h +++ b/src/shell/package/package_pickers.h @@ -18,7 +18,12 @@ bool pickPackageForImport(std::string& outAbsPath); // suggestedPath is a bare file name ("MyBank.rsbank") or a full path — a full one // also seeds the dialog's starting directory, which is how a caller keeps the picker // off REAPER's process working directory. True with outAbsPath set iff the user chose -// a destination; the dialog's own overwrite confirm has already run by then. -bool pickPackageSavePath(const std::string& suggestedPath, std::string& outAbsPath); +// a destination; the dialog's own overwrite confirm has already run by then, against +// the path the user actually chose — NOT necessarily outAbsPath, if the `.rsbank` +// re-append below fires. outAppended, when non-null, is set to whether it fired: the +// caller's signal that its own overwrite consent may not cover the returned path +// (see this directory's CLAUDE.md). +bool pickPackageSavePath(const std::string& suggestedPath, std::string& outAbsPath, + bool* outAppended = nullptr); } // namespace reasampler diff --git a/src/shell/panel/panel_bank_ops.cpp b/src/shell/panel/panel_bank_ops.cpp index f946c4f..a45cb99 100644 --- a/src/shell/panel/panel_bank_ops.cpp +++ b/src/shell/panel/panel_bank_ops.cpp @@ -251,7 +251,7 @@ enum : unsigned int { } // namespace // Shows the right-click context menu for a named-bank TAB: activate / rename / delete -// / evacuate that bank, plus a create entry. Drives the id-keyed ops. +// / evacuate / export that bank, plus a create entry. Drives the id-keyed ops. void showTabMenu(int screenX, int screenY, const std::string& bankId) { if (!book()) return; const Bank* bk = book()->bank(bankId); diff --git a/tests/test_export_bank.cpp b/tests/test_export_bank.cpp index 25e1b23..33a6c1b 100644 --- a/tests/test_export_bank.cpp +++ b/tests/test_export_bank.cpp @@ -86,10 +86,11 @@ struct Fixture { } void addSample(const std::string& id, const std::string& fileName, - std::size_t bytes, std::uint8_t seed, bool writeToDisk = true) { + std::size_t bytes, std::uint8_t seed, bool writeToDisk = true, + const std::string& displayName = "") { model::Sample s; s.id = id; - s.displayName = id; + s.displayName = displayName.empty() ? id : displayName; s.relativePath = std::string("reasampler_bank/") + fileName; s.sampleRate = 48000; s.channelCount = 2; @@ -146,6 +147,11 @@ static void testHealthyExportCarriesEveryPayloadByteExact() { CHECK(out.entriesWritten == 3); CHECK(out.excluded.empty()); CHECK(PayloadBuffer::alive() == 0); + // Point-in-time alive() == 0 alone cannot fail on a whole-package-in-memory + // shape (N buffers allocated and freed one at a time still ends at 0); the + // high-water mark can, across this three-entry export and everything the test + // binary ran before it — it must never exceed the "at most one payload" claim. + CHECK(PayloadBuffer::highWaterMark() == 1); const package::DecodedPackage decoded = decodeFromDisk(fx.destPath()); CHECK(decoded.status == package::PackageReadability::Readable); @@ -176,17 +182,51 @@ static void testHealthyExportCarriesEveryPayloadByteExact() { CHECK(PayloadBuffer::alive() == 0); } +// Every occurrence of `"key":"value"` in `json`, value returned raw (escape-aware +// only enough to not stop early on an escaped quote — this file's own writer output +// never nests an unescaped quote, so that is sufficient here). +static std::vector jsonStringValuesForKey(const std::string& json, + const std::string& key) { + std::vector values; + const std::string marker = "\"" + key + "\":\""; + std::size_t pos = 0; + while ((pos = json.find(marker, pos)) != std::string::npos) { + std::size_t i = pos + marker.size(); + while (i < json.size() && json[i] != '"') { + if (json[i] == '\\') ++i; // skip the escaped char too + ++i; + } + values.push_back(json.substr(pos + marker.size(), i - (pos + marker.size()))); + pos = i; + } + return values; +} + +// True for a value opening with an ':' drive-relative prefix (":" alone is +// JSON's own key separator, so this is checked on isolated VALUES, never on raw text). +static bool looksLikeDriveForm(const std::string& value) { + return value.size() >= 2 && std::isalpha(static_cast(value[0])) && + value[1] == ':'; +} + static void testEmittedManifestBytesCarryNoPath() { Fixture fx("nopath"); + // The bank's own display name AND a sample's displayName each carry a literal + // '/' — free text, unlike the entry `name` / nested `relativePath` fields this + // test actually polices (docs/product/bank-package.md:282-289: the destination is + // derived from the entry name, never from free text). Present in the fixture so + // the scan below proves it is scoped correctly rather than merely holding by + // accident on names that happen not to collide with the rule. + CHECK(fx.session.book().renameBank(fx.bankId, "Drums/Bus")); // Both source records carry a directory component; neither may reach the file. - fx.addSample("s1", "kick.wav", 200, 3); + fx.addSample("s1", "kick.wav", 200, 3, /*writeToDisk=*/true, "Kick / alt take"); fx.addSample("s2", "snare take 2.wav", 200, 9); CHECK(exportBank(fx.session, fx.request()).status == ExportStatus::Written); - // Scan the MANIFEST REGION of the emitted file, located by walking the frozen - // header the way a reader does: magic | fv | minReader | len+semver | len+JSON. - // The binary length fields are deliberately excluded — a length whose byte - // happens to be 0x2F is not a separator. + // Locate the MANIFEST REGION of the emitted file by walking the frozen header the + // way a reader does: magic | fv | minReader | len+semver | len+JSON. The binary + // length fields are deliberately excluded — a length whose byte happens to be + // 0x2F is not a separator. const std::vector file = readFile(fx.destPath()); CHECK(file.size() > 20); auto le32 = [&](std::size_t at) { @@ -202,22 +242,31 @@ static void testEmittedManifestBytesCarryNoPath() { CHECK(manifestAt + manifestLen <= file.size()); const std::string manifest(reinterpret_cast(file.data() + manifestAt), manifestLen); - CHECK(manifest.find("kick.wav") != std::string::npos); // the scan is looking at the manifest - CHECK(manifest.find('/') == std::string::npos); - CHECK(manifest.find('\\') == std::string::npos); - CHECK(manifest.find("..") == std::string::npos); - CHECK(manifest.find("reasampler_bank") == std::string::npos); - // ':' cannot be banned outright — it is JSON's own key separator — so the check - // is for the drive form specifically: a string value opening with ':'. - // With '/' and '\\' already absent, that covers the drive-relative spelling too. - bool driveForm = false; - for (std::size_t i = 0; i + 2 < manifest.size(); ++i) - if (manifest[i] == '"' && - std::isalpha(static_cast(manifest[i + 1])) && - manifest[i + 2] == ':') - driveForm = true; - CHECK(!driveForm); + + // The boundary, pinned rather than assumed: free text legitimately carries '/'. + CHECK(manifest.find("Drums/Bus") != std::string::npos); + CHECK(manifest.find("Kick / alt take") != std::string::npos); + + // The rule itself: scoped to the two fields the importer derives a destination + // from — the entry `name` and the nested Sample's own `relativePath` — never to + // `displayName` or the manifest's `bankDisplayName`. + const std::vector names = jsonStringValuesForKey(manifest, "name"); + const std::vector relPaths = jsonStringValuesForKey(manifest, "relativePath"); + CHECK(!names.empty()); + CHECK(!relPaths.empty()); + for (const std::string& v : names) { + CHECK(v.find('/') == std::string::npos); + CHECK(v.find('\\') == std::string::npos); + CHECK(v.find("..") == std::string::npos); + CHECK(!looksLikeDriveForm(v)); + } + for (const std::string& v : relPaths) { + CHECK(v.find('/') == std::string::npos); + CHECK(v.find('\\') == std::string::npos); + CHECK(v.find("..") == std::string::npos); + CHECK(!looksLikeDriveForm(v)); + } } static void testFailureInjectedMidWriteLeavesNoPackageAndSparesThePriorFile() { @@ -252,6 +301,39 @@ static void testFailureInjectedMidWriteLeavesNoPackageAndSparesThePriorFile() { CHECK(PayloadBuffer::alive() == 0); } +static void testCommitFailureIsAGenuineMidWriteAbandon() { + // The mid-READ injection above (a source vanishing between digest and stream) is + // not what "mid-write" names in export_bank.cpp:115-118/122-125 — those guard a + // failure IN the write itself: appendPayload's stream going bad, or commit's + // rename failing. A directory squatting on the destination (test_package_io.cpp's + // own precedent for a real, not simulated, commit failure) makes every payload + // stream fine and only the final rename fail. + Fixture fx("commitfail"); + fx.addSample("s1", "kick.wav", 300, 1); + fx.addSample("s2", "snare.wav", 300, 2); + + std::error_code ec; + fs::create_directory(utf8Path(fx.destPath()), ec); + CHECK(!ec); + + ExportSurvey survey = surveyBankExport(fx.session, fx.projectDir, fx.bankId); + CHECK(survey.plan.verdict == package::ExportVerdict::Ready); + package::PackageManifest manifest = survey.plan.manifest; + const std::vector sources = {fx.absPathOf("kick.wav"), fx.absPathOf("snare.wav")}; + std::string failed; + CHECK(digestSources(manifest, sources, failed)); + const std::optional encoded = package::encodePackage(manifest); + CHECK(encoded.has_value()); + + const ExportOutcome out = writePackageFile(*encoded, manifest, sources, fx.destPath()); + CHECK(out.status == ExportStatus::WriteFailed); + CHECK(fs::is_directory(utf8Path(fx.destPath()))); // the squatting dir is untouched + CHECK(!exists(fx.destPath() + ".rsbanktmp")); // commit()'s own self-clean ran + CHECK(PayloadBuffer::alive() == 0); + + fs::remove(utf8Path(fx.destPath()), ec); +} + static void testPayloadChangedBetweenDigestAndStreamAborts() { Fixture fx("changed"); fx.addSample("s1", "kick.wav", 500, 1); @@ -280,14 +362,27 @@ static void testExportTouchesNoProjectState() { fx.addSample("s1", "kick.wav", 400, 5); fx.addSample("s2", "snare.wav", 400, 6); - // The ext-state blob IS the serialized book (shell/persist/ext_state_io), so - // byte-identity of that string is byte-identity of what a persist would write. - const std::string extStateBefore = fx.session.book().serialize(); + // saveToActiveProject writes seven keys (shell/persist/ext_state_io.cpp): `banks`, + // the legacy-key clear, `view_state`, the tail setting, the tracking ledger, the + // version stamp, and the bank-generation counter. This asserts byte-identity of + // the three that have an in-memory string to diff (`banks`, `view_state`, the tail + // setting) plus bankGeneration() (the bank-generation-counter key IS its + // serialization). The legacy-key clear and the version stamp are session-external, + // nothing here to diff against. The tracking ledger has no public accessor to diff + // either, but needs none: exportBank/digestSources/writePackageFile all take the + // session by `const&`, and ReaSamplerSession::recordCreated — the ledger's one + // writer (session.h) — is non-const, so it is not reachable through this call at + // all; the compiler enforces "untouched" here rather than a runtime check proving it. + const std::string bankBookBefore = fx.session.book().serialize(); + const std::string viewBefore = fx.session.view().serialize(); + const std::string tailBefore = capture::serializeTailSetting(fx.session.tail()); const std::int64_t generationBefore = fx.session.bankGeneration(); CHECK(exportBank(fx.session, fx.request()).status == ExportStatus::Written); - CHECK(fx.session.book().serialize() == extStateBefore); + CHECK(fx.session.book().serialize() == bankBookBefore); + CHECK(fx.session.view().serialize() == viewBefore); + CHECK(capture::serializeTailSetting(fx.session.tail()) == tailBefore); CHECK(fx.session.bankGeneration() == generationBefore); } @@ -359,6 +454,7 @@ int main() { testHealthyExportCarriesEveryPayloadByteExact(); testEmittedManifestBytesCarryNoPath(); testFailureInjectedMidWriteLeavesNoPackageAndSparesThePriorFile(); + testCommitFailureIsAGenuineMidWriteAbandon(); testPayloadChangedBetweenDigestAndStreamAborts(); testExportTouchesNoProjectState(); testEmptyBankExportsAsAValidZeroEntryPackage(); diff --git a/tests/test_export_plan.cpp b/tests/test_export_plan.cpp index f29c356..3ccd784 100644 --- a/tests/test_export_plan.cpp +++ b/tests/test_export_plan.cpp @@ -167,6 +167,27 @@ static void testHostileNamesAreRepairedNotRelayed() { } } +static void testUniqueNameSurvivesLongExtensionUnderflow() { + // insertSuffix computes room = kMaxEntryNameBytes - suffix.size() - ext.size() in + // size_t; an extension long enough that even a two-digit "_10" suffix pushes the + // sum past the cap must not wrap that subtraction. Ten same-named entries force + // the tenth collision into double digits against a 253-byte extension (253 + 3 = + // 256, one over kMaxEntryNameBytes). + const std::string hostileName = "a." + std::string(252, 'x'); // 254 bytes, otherwise valid + std::vector candidates; + for (int i = 0; i < 10; ++i) + candidates.push_back(present("s" + std::to_string(i), "reasampler_bank/" + hostileName)); + + const ExportPlan p = planExport(bankOf(candidates)); + CHECK(p.verdict == ExportVerdict::Ready); + CHECK(p.manifest.entries.size() == 10); + for (const PackageEntry& e : p.manifest.entries) CHECK(isValidEntryName(e.fileName)); + for (std::size_t i = 0; i < p.manifest.entries.size(); ++i) + for (std::size_t j = i + 1; j < p.manifest.entries.size(); ++j) + CHECK(!sameEntryName(p.manifest.entries[i].fileName, + p.manifest.entries[j].fileName)); +} + static void testCaseFoldedCollisionsAreDisambiguated() { const ExportPlan p = planExport(bankOf({ present("s1", "reasampler_bank/Kick.wav"), @@ -253,6 +274,7 @@ int main() { testUnreadableFileStaysDistinctFromMissing(); testUnrepresentableRecordRefusesWholeExport(); testHostileNamesAreRepairedNotRelayed(); + testUniqueNameSurvivesLongExtensionUnderflow(); testCaseFoldedCollisionsAreDisambiguated(); testSanitizeNeverReturnsANameTheCodecRefuses(); testPlannedManifestSatisfiesTheCodec(); diff --git a/tests/test_package_io.cpp b/tests/test_package_io.cpp index d6237f6..eaddba9 100644 --- a/tests/test_package_io.cpp +++ b/tests/test_package_io.cpp @@ -72,7 +72,9 @@ static void testPayloadCounterTracksMovesNotCopies() { } static void testStreamingRoundTripHoldsOnePayload() { - const std::string dest = "pkg_io_scratch.rsbank"; + std::error_code destEc; + const std::string dest = + (fs::temp_directory_path(destEc) / "pkg_io_scratch.rsbank").generic_string(); const std::vector header = patternBytes(16, 0xA0); const std::vector> entries = { patternBytes(1000, 1), patternBytes(500, 2), patternBytes(1, 3)}; From 33ea95078dc907a8512cf906840041f7479cd0c5 Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 11:46:05 -0400 Subject: [PATCH 15/24] docs: add the two package directories to the architecture table Also adds the missing core/instrument/engine/loop row and corrects the per-directory CLAUDE.md count from twenty-three to twenty-six. --- CLAUDE.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 6d5a3d6..d45415f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co **ReaSampler** is a per-project audio sample-bank capture tool that builds two artifacts: the REAPER extension (`reaper_reasampler`) and **ReaSampler 9000**, a Windows-only VST3 sampler instrument (`reasampler_9000.vst3`, `core/instrument/` + `shell/instrument/`, second CMake target `reasampler_vst`, gated on the vendored `vendor/vst3sdk` submodule slice). The pure-testable-core / REAPER-facing-shell discipline is preserved throughout: `core/` never includes REAPER or VST3 SDK types, `shell/` is where those hosts are actually touched, `app/` is the extension entry point. Every REAPER API name cited in project docs is correct-by-intent; verify argument order, types, and flag values against `vendor/reaper-sdk/sdk/reaper_plugin_functions.h` before use. -Per-module detail — what each file owns, its invariants — lives in the twenty-three per-directory `src/**/CLAUDE.md` files; see the compact map in "Architecture: the load-bearing split" below to find the right one. Landed-phase history lives in `docs/ARCHIVE.md`; current work lives in `docs/COMPLETED.md`, `docs/TODO.md`, and `docs/TODO-1.0.md` — see "Project docs" below. +Per-module detail — what each file owns, its invariants — lives in the twenty-six per-directory `src/**/CLAUDE.md` files; see the compact map in "Architecture: the load-bearing split" below to find the right one. Landed-phase history lives in `docs/ARCHIVE.md`; current work lives in `docs/COMPLETED.md`, `docs/TODO.md`, and `docs/TODO-1.0.md` — see "Project docs" below. ## Settled decisions @@ -84,7 +84,7 @@ There is no hot-reload. Copy the **Release** build's binary (`build/Release/` on ## Architecture: the load-bearing split -`core/` holds pure, unit-testable logic — no REAPER or VST3 SDK types, each with a corresponding `_tests` target that runs without a DAW. `shell/` holds the REAPER/host-facing shells — where those SDK types are actually touched. `app/` is the extension entry point. Each of the twenty-three directories below carries its own `CLAUDE.md` with the full module list and that area's invariants — open the relevant one for detail; this file states only repo-wide truth. +`core/` holds pure, unit-testable logic — no REAPER or VST3 SDK types, each with a corresponding `_tests` target that runs without a DAW. `shell/` holds the REAPER/host-facing shells — where those SDK types are actually touched. `app/` is the extension entry point. Each of the twenty-six directories below carries its own `CLAUDE.md` with the full module list and that area's invariants — open the relevant one for detail; this file states only repo-wide truth. | Directory | Scope | |---|---| @@ -94,9 +94,11 @@ There is no hot-reload. Copy the **Release** build's binary (`build/Release/` on | `src/core/instrument/` | pure VST3-instrument core (bake / engine / map / note / ui) | | `src/core/instrument/bake/` | the resample bake's pure half — the programmed note resolved to a frame window, the offline render over a bake-only voice engine, and the post-bake reset | | `src/core/instrument/engine/filter/` | pure per-voice resonant TPT/SVF filter (HP→BP→LP / HP→notch→LP morph, drive stage), run by each `Voice` between the pitch and amp stages | +| `src/core/instrument/engine/loop/` | the sustain loop's ONE validity/clamp fold plus its pre-seam crossfade geometry and the editor's default handle span | | `src/core/instrument/note/` | the programmed capture-signal model — musical divisions, tempo resolution, anchored offsets | | `src/core/json/` | the hand-rolled JSON lexical layer | | `src/core/model/` | the pure bank/sample index and its multi-bank container | +| `src/core/package/` | the pure RSBK bank-package codec — format contract, version ladder, JSON manifest, framing/layout codec | | `src/core/reclaim/` | pure prune orphan computation | | `src/core/tracking/` | the consolidated file-tracking system — birth/lineage records and the one authority answering prune's protected set and the resample's replace-vs-add | | `src/core/ui/` | pure UI geometry, palette, and interaction-decision modules | @@ -108,6 +110,7 @@ There is no hot-reload. Copy the **Release** build's binary (`build/Release/` on | `src/shell/bank_ops/` | promptless bank-mutation verbs | | `src/shell/capture/` | REAPER-facing capture backends and action bodies | | `src/shell/instrument/` | ReaSampler 9000 VST3 shells | +| `src/shell/package/` | package filesystem I/O (streaming atomic read/write, exclusive-create landing), the rollback journal, and the REAPER file pickers | | `src/shell/panel/` | the docked bank-panel shell + the shared LICE draw kit | | `src/shell/persist/` | project ext-state persistence, prune filesystem I/O, usage scan | | `src/shell/view/` | Design View mode application shell | From f8dde16a7ef3f313946bfe612cde50d1fa1e1f0c Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 14:02:05 -0400 Subject: [PATCH 16/24] =?UTF-8?q?import:=20remediate=20review=20findings?= =?UTF-8?q?=20=E2=80=94=20ledger=20gate,=20docs,=20message=20split?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Delegates the refuse-gate to ledgerDegraded(), lifts its console message into a pure testable fold, fixes stale doc line citations and an inaccurate outcome-enum comment, and splits the rename counter into collision-vs-sanitize. --- docs/PLAN.md | 15 ++-- docs/product/bank-package.md | 6 +- src/core/model/bank_book.cpp | 7 +- src/core/package/CLAUDE.md | 18 +++- src/core/package/import_plan.cpp | 67 +++++++++++++-- src/core/package/import_plan.h | 14 ++- src/core/util/CLAUDE.md | 5 +- src/core/util/ascii_ws.h | 10 +++ src/shell/actions/CLAUDE.md | 2 +- src/shell/actions/package_import_action.cpp | 95 +++++++++++---------- src/shell/actions/package_import_action.h | 11 ++- src/shell/package/import_bank.cpp | 7 +- src/shell/package/import_bank.h | 4 +- src/shell/package/import_landing.cpp | 14 ++- src/shell/package/import_landing.h | 16 ++-- src/shell/panel/panel_bank_ops.cpp | 9 +- src/shell/panel/panel_window.cpp | 22 ++++- src/shell/persist/session.h | 7 ++ tests/test_import_landing.cpp | 4 +- tests/test_import_plan.cpp | 58 ++++++++++++- 20 files changed, 299 insertions(+), 92 deletions(-) create mode 100644 src/core/util/ascii_ws.h diff --git a/docs/PLAN.md b/docs/PLAN.md index 0833a9a..b6f2e5a 100644 --- a/docs/PLAN.md +++ b/docs/PLAN.md @@ -2366,7 +2366,7 @@ These bind every track in this phase, in addition to the plan-wide set above. touching it must **cite, not restate**. - **Export is read-only against the project.** No ext-state write, no `bumpBankGeneration()`, no undo point. Import does the opposite: it bumps the generation - (`src/shell/persist/session.h:108`) so live ReaSampler 9000 instances reload, and batches + (`src/shell/persist/session.h:114`) so live ReaSampler 9000 instances reload, and batches its index mutation into one Ctrl-Z through `persistBankOp`. - **All-or-nothing on both sides.** No partial export, no partial import. A truncated `.rsbank` must never exist on disk (temp file + atomic rename, the Ψ-W2-T2 precedent); a @@ -2396,10 +2396,13 @@ new directories** (`src/core/package/`, `src/shell/package/`) that no other phas not read here. The only pre-existing files any Ε track edits are named per track below — `core/tracking/origin_ledger` (W1-T3, exclusively), the root `CMakeLists.txt` `add_subdirectory` list (W1-T1 and W1-T2, one line each), `src/app/main.cpp` and the panel's -bank menu (W2-T1 and W2-T2, one registration line and one menu row each), and +bank menu (W2-T1 and W2-T2, one registration line and one menu row each), `core/model/bank_book.{h,cpp}` (W2-T2 only — **one additive public `const` member**, required -by the Ε-F2 auto-suffix rule so the name fold keeps its single home). **No Ε track -touches `core/instrument/`, `shell/instrument/`, or any capture backend.** +by the Ε-F2 auto-suffix rule so the name fold keeps its single home), and +`shell/persist/session.h` (W2-T2 only — one additive public accessor, `ledgerStatus()`, so the +import gate can key on `LedgerStatus` alone without going through `pruneDryRun()`'s +enumeration+scan). **No Ε track touches `core/instrument/`, `shell/instrument/`, or any +capture backend.** --- @@ -2764,7 +2767,7 @@ fold, it does not add a second one), `origin_ledger` (Ε-W1-T3's). `"Drums 3"` — a bare trailing integer is indistinguishable from `"Kit 808"`); the probe **fills gaps** (first-free, not highest-plus-one, so it is a pure function of the current name set); the probe **terminates** by pigeonhole within `B + 1` candidates for `B` banks, - so **no arbitrary cap**; and the fold is `BankBook`'s own (`bank_book.h:252-258`), reached + so **no arbitrary cap**; and the fold is `BankBook`'s own (`bank_book.h:263-269`), reached through the new public member, never re-implemented in `import_plan`. Sample display names are **not** suffixed, and `slot_map` positions are untouched. - **Always a new bank; never a merge (Ε-F2, ruled).** The import creates a bank — it never @@ -2787,7 +2790,7 @@ fold, it does not add a second one), `origin_ledger` (Ε-W1-T3's). remain as orphans until a prune reclaims them — the same designed window a non-empty bank delete already produces (`core/model/CLAUDE.md`'s sample-removal section). The user-facing summary says so. -- **`bumpBankGeneration()` on success** (`session.h:108`), so live instances reload. +- **`bumpBankGeneration()` on success** (`session.h:114`), so live instances reload. - **No timeline item is placed. Ever.** - **A new FOREVER-STABLE command id**, minted the same way T1's is. diff --git a/docs/product/bank-package.md b/docs/product/bank-package.md index 92c9ee4..5f4ad9b 100644 --- a/docs/product/bank-package.md +++ b/docs/product/bank-package.md @@ -381,7 +381,7 @@ trim, the seed is the literal `Imported bank`. **The probe.** Let `seed` be that string and `fold(x)` be `BankBook`'s own uniqueness key — strip leading/trailing ASCII whitespace, lower-case ASCII letters -(`bank_book.h:252-258`). Take the **first** name in this sequence whose fold is not +(`bank_book.h:263-269`). Take the **first** name in this sequence whose fold is not already carried by a bank in the destination book: seed, seed + " 2", seed + " 3", seed + " 4", … @@ -410,7 +410,7 @@ implementations diverge:** `B + 1` candidates is free by pigeonhole, so no cap is needed and none should be added. 4. **The fold has exactly one home.** `import_plan` must **not** re-implement - `nameKey` — `bank_book.h:252-258` says in as many words that a drifted second copy + `nameKey` — `bank_book.h:263-269` says in as many words that a drifted second copy would let the uniqueness invariant be violated. The probe therefore runs behind `BankBook`'s own folding, which means Ε-W2-T2 adds **one additive public `const` member** to `BankBook` (recommended: `std::string uniqueDisplayName(const @@ -690,7 +690,7 @@ constructors. freshly-generated pair. - **Bank generation.** Import mutates bank content that live ReaSampler 9000 instances may play, so it must `bumpBankGeneration()` - (`src/shell/persist/session.h:108`, whose own comment says call sites "err toward + (`src/shell/persist/session.h:114`, whose own comment says call sites "err toward bumping"). Export mutates nothing and must bump nothing, write no ext state, and open no undo point. - **Beta/stable channel isolation.** Packages are channel-**agnostic** and this is diff --git a/src/core/model/bank_book.cpp b/src/core/model/bank_book.cpp index efa5e25..3ecb817 100644 --- a/src/core/model/bank_book.cpp +++ b/src/core/model/bank_book.cpp @@ -3,6 +3,8 @@ #include #include +#include "core/util/ascii_ws.h" + // bank_book implementation — the registry RULES half: construction, pool // privileges, bank lifecycle, active bank, sample movement/removal, slot order, // and the reference queries. The JSON round-trip half lives in bank_book_json.cpp, @@ -76,9 +78,8 @@ void BankBook::normalizeOrdinals() { // one folding rule shared with bank_book_json.cpp's parse-time coalesce. std::string BankBook::nameKey(const std::string& s) { std::size_t b = 0, e = s.size(); - auto isWs = [](char c) { return c == ' ' || c == '\t' || c == '\n' || c == '\r'; }; - while (b < e && isWs(s[b])) ++b; - while (e > b && isWs(s[e - 1])) --e; + while (b < e && util::isAsciiWs(s[b])) ++b; + while (e > b && util::isAsciiWs(s[e - 1])) --e; std::string out; out.reserve(e - b); for (std::size_t i = b; i < e; ++i) { diff --git a/src/core/package/CLAUDE.md b/src/core/package/CLAUDE.md index 5d80a01..6e5d445 100644 --- a/src/core/package/CLAUDE.md +++ b/src/core/package/CLAUDE.md @@ -76,8 +76,11 @@ landing after the format. - `import_plan` — the pure import decision, and the reason the whole feature is testable without a DAW: the destination bank's display name after `BankBook`'s own fold, the reminted sample ids and remapped parents, and the - per-entry land / collapse / rename disposition. Also `importLedgerRefusal`, - the import's ledger gate. + per-entry land / collapse / rename disposition. Also `importLedgerRefusal` (the + import's ledger gate, delegating entirely to `tracking::ledgerDegraded`) and + `ledgerRefusalMessage` (the gate's console-block body, a pure + `(LedgerRefusal, namespace) -> string` fold the shell only supplies the + channel-correct namespace to). - `bank_package` — framing and arithmetic composing the two above: `encodePackage` (prefix bytes + layout + total size, stamping this build's ladder pair and `version::stampVersion()`), `decodePackage` (prefix + observed @@ -156,3 +159,14 @@ landing after the format. not a real entry). `serializeManifest` refuses a zero-length `PackageEntry` at encode so this layer never produces one; decode does not enforce it (a hostile/older package declaring one is not this track's concern). +- **`import_plan`'s `spelledLikeABankFile` mints a fresh name even with NO + collision, and that third condition is a deliberate decision, not spec-derived.** + `docs/product/bank-package.md:447` ties the auto-rename mint to a *collision* + only; `spelledLikeABankFile` additionally mints whenever the package's own name + isn't spelled the way `deriveBankPaths` spells one (extension, sanitized stem). + Kept for two reasons: uniform folder spelling for every landed file regardless of + origin, and — the sharper one — a hostile entry name that isn't a legal Windows + filename or carries an unexpected extension (e.g. `evil.exe`) lands sanitized + (`evil_.wav`) rather than verbatim. `ImportPlan` counts this separately from a + genuine folder-name collision (`sanitizeRenameCount` vs `collisionRenameCount`) so + the summary line means what `bank-package.md:447` says it means. diff --git a/src/core/package/import_plan.cpp b/src/core/package/import_plan.cpp index 372b2c4..1c9c38a 100644 --- a/src/core/package/import_plan.cpp +++ b/src/core/package/import_plan.cpp @@ -6,6 +6,7 @@ #include "core/capture/capture_paths.h" #include "core/package/package_format.h" +#include "core/util/ascii_ws.h" namespace reasampler::package { @@ -14,10 +15,13 @@ namespace { using capture::bankRelativeForName; using capture::deriveBankPaths; using capture::sanitizeStem; +using util::isAsciiWs; +// Shares BankBook::nameKey's whitespace set (core/util/ascii_ws.h) so a name nameKey +// would fold to empty is never treated as recorded here. bool blankName(const std::string& s) { for (char c : s) - if (c != ' ' && c != '\t' && c != '\n' && c != '\r') return false; + if (!isAsciiWs(c)) return false; return true; } @@ -40,6 +44,14 @@ bool spelledLikeABankFile(const std::string& fileName) { // The bank-folder names an import must not land on: what is there already, plus what // this import has minted so far. Case-folded, because the two filesystems this tool // ships on would treat "Kick.wav" and "kick.wav" as one file. +// +// `bankFolderFileNames` comes from `listFolderFileNames` (shell/package/package_io), +// which skips non-regular files — so a DIRECTORY sharing a bank file's name is +// invisible here. The plan then never mints around it, and the later exclusive-create +// land fails on that one entry (WriteFailed, rolled back). Safe direction ("never +// overwrite" still holds) but worth knowing before chasing a WriteFailed report that +// traces back to a same-named folder in the bank directory; test_import_landing's +// rollback suite deliberately exploits this to exercise the rollback path. class NameSet { public: explicit NameSet(const std::vector& present) { @@ -71,13 +83,46 @@ std::string mintFileName(const std::string& projectDir, const std::string& packa } // namespace LedgerRefusal importLedgerRefusal(tracking::LedgerStatus status) { - switch (status) { - case tracking::LedgerStatus::Unreadable: return LedgerRefusal::Malformed; - case tracking::LedgerStatus::FutureVersion: return LedgerRefusal::FutureVersion; - case tracking::LedgerStatus::Fresh: - case tracking::LedgerStatus::Loaded: break; + // Delegates the refuse/proceed decision entirely to ledgerDegraded() rather than + // re-deriving it from the two named statuses, so a future degraded status added + // there is refused here too rather than silently falling through to None. + if (!tracking::ledgerDegraded(status)) return LedgerRefusal::None; + // Below this point status is known degraded; only the message variant is picked. + // Unreadable gets its own "corrupt, may be cleared" wording; every other degraded + // status (today only FutureVersion) gets the "written by a newer build" wording. + return status == tracking::LedgerStatus::Unreadable ? LedgerRefusal::Malformed + : LedgerRefusal::FutureVersion; +} + +// Mirrors prune's abort block in structure and tone (shell/actions/prune_action.cpp), +// because a user who has hit that one should recognise this one. Every recovery line +// names THIS build's namespace: a beta user handed the stable spelling clears the wrong +// key and is still blocked. +std::string ledgerRefusalMessage(LedgerRefusal refusal, const std::string& extStateNamespace) { + if (refusal == LedgerRefusal::None) return {}; + + std::string msg = + "ReaSampler import: ABORTED -- the file-tracking ledger could not be read. " + "Nothing was imported.\n"; + if (refusal == LedgerRefusal::Malformed) { + msg += "The stored file-tracking ledger is malformed. It has been left intact " + "rather than overwritten, so it can be repaired or cleared:\n" + " reaper.SetProjExtState(0, \"" + extStateNamespace + "\", \"owned_files\", \"\")\n" + "Clearing it makes every existing bank file un-reclaimable (they stop " + "being attributable to ReaSampler); no file is lost. Reopen the project " + "afterwards -- the block is held for the rest of this session.\n"; + } else { + msg += "The stored file-tracking ledger was written by a NEWER version of " + "ReaSampler than this one, so its records cannot be read safely. It has " + "been left intact and will NOT be overwritten. Reopen the project with " + "that newer version -- do NOT clear this key from here, that would " + "discard tracking records this build cannot see. The block is held for " + "the rest of this session.\n"; } - return LedgerRefusal::None; + msg += "An import can land hundreds of files in one gesture. With no readable " + "ledger, none of them could be given a birth record, and every one would be " + "permanently unreclaimable.\n"; + return msg; } std::string bankFolderDir(const std::string& projectDir) { @@ -139,7 +184,13 @@ ImportPlan planImport(const PackageManifest& manifest, idRemap[src.sample.id] = e.sample.id; ++plan.landCount; - if (e.renamed) ++plan.renameCount; + // A rename happens for one of two reasons: the package's own name was already + // taken (spelledLikeABankFile true but the mint's fast path lost the race to + // `taken`), or the name never qualified for that fast path at all (sanitize). + if (e.renamed) { + if (spelledLikeABankFile(src.fileName)) ++plan.collisionRenameCount; + else ++plan.sanitizeRenameCount; + } plan.entries.push_back(std::move(e)); } diff --git a/src/core/package/import_plan.h b/src/core/package/import_plan.h index a79dd46..104df91 100644 --- a/src/core/package/import_plan.h +++ b/src/core/package/import_plan.h @@ -32,6 +32,13 @@ enum class LedgerRefusal { None, Malformed, FutureVersion }; LedgerRefusal importLedgerRefusal(tracking::LedgerStatus status); +// The console-block body for a refusal — a pure (LedgerRefusal, namespace) -> string +// fold, so the wording is assertable without a DAW. `extStateNamespace` is the +// channel-correct namespace (`version::extStateNamespace()`) every recovery line must +// name, so a beta user is never handed the stable spelling. Empty string for None — +// callers only reach this once `importLedgerRefusal` has already returned a refusal. +std::string ledgerRefusalMessage(LedgerRefusal refusal, const std::string& extStateNamespace); + // What one manifest entry does when the import runs. // - Land: write the payload under destFileName and add `sample`. // - Collapse: an equal contentHash already lands in this same import, so the payload @@ -56,7 +63,12 @@ struct ImportPlan { model::SlotMap slots; // the package's slots over the reminted ids int landCount = 0; int collapseCount = 0; - int renameCount = 0; + // Two distinct triggers, counted separately (bank-package.md:447 defines the first + // as THE collision counter; conflating the second into it would misreport a mint + // that never collided as a collision). + int collisionRenameCount = 0; // the package's own name was already taken in the bank folder + int sanitizeRenameCount = 0; // the package's name was not spelled the way this tool spells + // a bank file (see spelledLikeABankFile, core/package/CLAUDE.md) }; // The bank folder an import lands into — the same expression capture uses, so an diff --git a/src/core/util/CLAUDE.md b/src/core/util/CLAUDE.md index 94ae77e..de8663f 100644 --- a/src/core/util/CLAUDE.md +++ b/src/core/util/CLAUDE.md @@ -3,8 +3,8 @@ ## Scope Tiny, dependency-free pure helpers linked by both artifacts: whole-file byte -loading, unit-interval clamping, the absolute-path rejection test, and the -per-segment envelope curve law. +loading, unit-interval clamping, the absolute-path rejection test, the +per-segment envelope curve law, and the ASCII-whitespace fold test. ## Modules @@ -19,6 +19,7 @@ per-segment envelope curve law. before curves existed play unchanged, and what the knob law's centre detent exists to keep reachable from the dial. - `relative_path` (`core/util`, header-only) — the ONE absolute-path rejection test behind the relative-paths-only invariant, shared by `bank_model` (`Sample.relativePath`) and `core/tracking/origin_ledger` (`OriginRecord.relativePath`). The two must reject identically or a path one accepts could be smuggled past the other; that is why it is one function and not two. +- `ascii_ws` (`core/util`, header-only) — the ONE ASCII-whitespace test (space/tab/CR/LF) behind `BankBook::nameKey`'s trim, shared by `core/package/import_plan`'s blank-bank-name fallback. Same rationale as `relative_path`: two independently-maintained copies could drift on what counts as blank. ## Gotchas diff --git a/src/core/util/ascii_ws.h b/src/core/util/ascii_ws.h new file mode 100644 index 0000000..a8e0c72 --- /dev/null +++ b/src/core/util/ascii_ws.h @@ -0,0 +1,10 @@ +#pragma once +// ascii_ws — the ONE ASCII-whitespace test shared by every fold that must agree with +// BankBook::nameKey's trim (space/tab/CR/LF): a drifted second copy could accept a +// package bank name nameKey would treat as blank, or vice versa. + +namespace reasampler::util { + +inline bool isAsciiWs(char c) { return c == ' ' || c == '\t' || c == '\n' || c == '\r'; } + +} // namespace reasampler::util diff --git a/src/shell/actions/CLAUDE.md b/src/shell/actions/CLAUDE.md index 9f2bb9d..d620190 100644 --- a/src/shell/actions/CLAUDE.md +++ b/src/shell/actions/CLAUDE.md @@ -43,7 +43,7 @@ is owned by other directories and only skinned here. - `drag_out_win` — OS drag-out shell: Windows OLE `DoDragDrop`/`CF_HDROP`, copy-only (`DROPEFFECT_MOVE` not offered); macOS/Linux via `SWELL_InitiateDragDropOfFileList`. - `instrument_drop_win` — instrument-drop shell: `probeDropTarget` resolves a screen point to a track + a `ReaperSurface` (via the pure `wire::classifyReaperSurface`, whose token rules `core/wire/CLAUDE.md` owns), and the drop half adds a ReaSampler 9000 instance and applies the dragged capture's state via a transient `.vstpreset` + `TrackFX_SetPreset` (the former `TrackFX_SetNamedConfigParm` "vst_chunk" write was silently unappliable for VST3). Exposes `loadInstrumentOntoTrack` (inner half, no own undo block) and `performInstrumentDrop` (wraps in its own undo block). **Never captures, never writes the bank, never inserts a timeline item.** - `arrange_drop_win` — the drag-out gesture's arrange outcome: `arrangeTimeAtScreenX` (pointer column → time via `GetSet_ArrangeView2`'s one-pixel-span reading — inferred, not SDK-documented) and `performArrangeDrop` (snap the drop time, then one `InsertMedia` per capture on the pointer's track — assumed, not confirmed, to land end-to-end via REAPER's own cursor advance — in ONE undo block, counting only InsertMedia's reported successes, with the caller's track selection and edit cursor restored). The one timeline-placing shell here, per the invariant above; it never captures and never writes the bank. -- `package_import_action` — the bindable / bank-menu / file-drop skin over `shell/package`'s `importBankPackage`. Owns the **ledger gate**, which runs BEFORE the file picker (a refusal must not cost the user a file choice) and is keyed on the session's `LedgerStatus` alone — never on `PruneReport::blockedByTracking`, whose undecodable-`rsusage_*` arm governs deletion-time protection and would refuse an import that only writes birth records. Owns every message the import produces; the verb itself is promptless. +- `package_import_action` — the bindable / bank-menu / file-drop skin over `shell/package`'s `importBankPackage`. Owns the **ledger gate**, which runs BEFORE the file picker (a refusal must not cost the user a file choice) and is keyed on the session's `LedgerStatus` alone — never on `PruneReport::blockedByTracking`, whose undecodable-`rsusage_*` arm governs deletion-time protection and would refuse an import that only writes birth records. Builds and shows every message the import produces, but the ledger-refusal body itself is `core/package::ledgerRefusalMessage` — a pure fold this TU only supplies the channel-correct namespace to — so the wording is assertable without a DAW. `doImportBankPackage`/`doImportBankPackageFile` return the minted bank id on a landed import (empty otherwise) so a caller can focus it; the verb itself is promptless. - `ingest` — ingest-through-the-bank shell on the EXTENSION side: three surfaces — (1) arrange capture→bank→assign (bindable action), (2) Media-Explorer import→bank→instrument on the selected track, (3) file drop onto the bank panel→bank only. Only surface (1) writes the `assignment_request` ext-state wire. **ingest NEVER inserts a timeline item.** Surface (2)'s action is the one in this directory published into a NON-main action section (Media Explorer) as well as Main — two ids, one handler, two dispatch hooks; see root `CLAUDE.md` §"REAPER extension contract" for the mechanism. ## Gotchas diff --git a/src/shell/actions/package_import_action.cpp b/src/shell/actions/package_import_action.cpp index 2531858..c8ca753 100644 --- a/src/shell/actions/package_import_action.cpp +++ b/src/shell/actions/package_import_action.cpp @@ -25,48 +25,31 @@ constexpr const char* kTitle = "ReaSampler: import bank package"; std::string quoted(const std::string& s) { return "\"" + s + "\""; } -// Mirrors prune's abort block in structure and tone, because a user who has hit that -// one should recognise this one. Every recovery line names THIS build's namespace: a -// beta user handed the stable spelling clears the wrong key and is still blocked. +// The message body itself is core/package::ledgerRefusalMessage — a pure +// (LedgerRefusal, namespace) -> string fold, testable without a DAW. This TU only +// supplies the channel-correct namespace and the console call. void reportLedgerRefusal(package::LedgerRefusal refusal) { - const std::string& ns = version::extStateNamespace(); - std::string msg = - "ReaSampler import: ABORTED -- the file-tracking ledger could not be read. " - "Nothing was imported.\n"; - if (refusal == package::LedgerRefusal::Malformed) { - msg += "The stored file-tracking ledger is malformed. It has been left intact " - "rather than overwritten, so it can be repaired or cleared:\n" - " reaper.SetProjExtState(0, \"" + ns + "\", \"owned_files\", \"\")\n" - "Clearing it makes every existing bank file un-reclaimable (they stop " - "being attributable to ReaSampler); no file is lost. Reopen the project " - "afterwards -- the block is held for the rest of this session.\n"; - } else { - msg += "The stored file-tracking ledger was written by a NEWER version of " - "ReaSampler than this one, so its records cannot be read safely. It has " - "been left intact and will NOT be overwritten. Reopen the project with " - "that newer version -- do NOT clear this key from here, that would " - "discard tracking records this build cannot see. The block is held for " - "the rest of this session.\n"; - } - msg += "An import can land hundreds of files in one gesture. With no readable " - "ledger, none of them could be given a birth record, and every one would be " - "permanently unreclaimable.\n"; - ShowConsoleMsg(msg.c_str()); + ShowConsoleMsg( + package::ledgerRefusalMessage(refusal, version::extStateNamespace()).c_str()); } // The refusal a user can act on names all three: what the package needs, what this // build reads, and which build wrote it. Any two of them leave them stuck. void reportTooNew(const ImportBankResult& r) { + const bool knownWriter = !r.header.writerVersion.empty(); const std::string writer = - r.header.writerVersion.empty() ? std::string("an unidentified build") - : "ReaSampler " + r.header.writerVersion; - const std::string msg = + knownWriter ? "ReaSampler " + r.header.writerVersion : std::string("an unidentified build"); + std::string msg = "Cannot import this bank package.\n" "It was written by " + writer + " and needs package format " + std::to_string(r.header.minReaderVersion) + " or newer.\n" "This build (" + version::appVersion() + ") reads package format " + std::to_string(package::kPackageFormatVersion) + ".\n" - "Nothing was imported. Install " + writer + " or newer and try again."; + "Nothing was imported. "; + // "Install or newer" reads fine when writer is a real semver; it does not + // when writer is the "unidentified build" filler, so that case gets its own sentence. + msg += knownWriter ? "Install " + writer + " or newer and try again." + : "Install a newer version of ReaSampler and try again."; ShowMessageBox(msg.c_str(), kTitle, 0); } @@ -77,17 +60,27 @@ void reportSuccess(const ImportBankResult& r) { detail += " (a bank named " + quoted(r.seedBankName) + " already exists in this project)"; detail += ".\n"; - if (r.renamedCount > 0) { - detail += " " + std::to_string(r.renamedCount) + + // Two distinct triggers (core/package::ImportPlan), reported as two counts rather + // than folded into one ambiguous "already taken, or not spelled right" line. + if (r.collisionRenameCount > 0) { + detail += " " + std::to_string(r.collisionRenameCount) + " file(s) landed under a freshly minted name (the package's own name " - "was already taken in the bank folder, or was not spelled the way " - "this bank spells a file). An existing bank file is never " - "overwritten.\n"; + "was already taken in the bank folder). An existing bank file is " + "never overwritten.\n"; + } + if (r.sanitizeRenameCount > 0) { + detail += " " + std::to_string(r.sanitizeRenameCount) + + " file(s) landed under a freshly minted name (not spelled the way " + "this bank spells a file).\n"; } if (r.collapsedCount > 0) { + // "Already present" here can only mean a duplicate BY CONTENT inside this same + // package (Ε-F2: import never consults another bank's hashes) — deliberately + // reworded from bank-package.md:448's "already present" phrasing, which reads + // as "already in your project" and is misleading in this direction. detail += " " + std::to_string(r.collapsedCount) + - " sample(s) were already present by content and were not written " - "again.\n"; + " sample(s) duplicated another entry in this same package by content " + "and were written once.\n"; } detail += "One undo removes the imported bank and its entries. It does NOT delete " "the imported files -- they stay in the bank folder, referenced by " @@ -129,6 +122,14 @@ void report(const ImportBankResult& r) { case ImportOutcome::Malformed: // Distinct from TooNew on purpose: the recoveries are opposite -- one is // "install a newer build", this one is "get an intact copy". + // + // bank-package.md:443 asks for a separate "This package is not well-formed" + // message when an entry name carries a separator / ".." / an absolute form. + // Not implemented: deserializeManifest returns one indistinguishable nullopt + // for that and for ordinary corruption, so it folds into this generic box. + // The binding spec (PLAN.md:2678) only requires Malformed != TooNew, which + // this still satisfies -- that product-doc row is knowingly left open, not + // silently missed. ShowMessageBox("This file is not a readable bank package (corrupt or " "truncated). Nothing was imported.", kTitle, 0); @@ -167,17 +168,21 @@ bool ledgerPermits(ReaSamplerSession& session) { } // namespace -void doImportBankPackage(ReaSamplerSession& session) { - if (!ledgerPermits(session)) return; +std::string doImportBankPackage(ReaSamplerSession& session) { + if (!ledgerPermits(session)) return {}; std::string path; - if (!pickPackageForImport(path) || path.empty()) return; - report(importBankPackage(session, path)); + if (!pickPackageForImport(path) || path.empty()) return {}; + const ImportBankResult r = importBankPackage(session, path); + report(r); + return r.outcome == ImportOutcome::Landed ? r.bankId : std::string{}; } -void doImportBankPackageFile(ReaSamplerSession& session, const std::string& packageAbsPath) { - if (packageAbsPath.empty()) return; - if (!ledgerPermits(session)) return; - report(importBankPackage(session, packageAbsPath)); +std::string doImportBankPackageFile(ReaSamplerSession& session, const std::string& packageAbsPath) { + if (packageAbsPath.empty()) return {}; + if (!ledgerPermits(session)) return {}; + const ImportBankResult r = importBankPackage(session, packageAbsPath); + report(r); + return r.outcome == ImportOutcome::Landed ? r.bankId : std::string{}; } } // namespace reasampler diff --git a/src/shell/actions/package_import_action.h b/src/shell/actions/package_import_action.h index 719b7a5..34f705e 100644 --- a/src/shell/actions/package_import_action.h +++ b/src/shell/actions/package_import_action.h @@ -9,11 +9,14 @@ namespace reasampler { class ReaSamplerSession; -// Gate, pick, import, report. The bound action and the panel's bank menu both call this. -void doImportBankPackage(ReaSamplerSession& session); +// Gate, pick, import, report. The bound action and the panel's bank menu both call +// this. Returns the minted bank id on a landed import, "" otherwise (cancelled, +// refused, or failed) — a caller that wants to focus the new bank (mirroring +// doCreateBank) checks the return rather than reaching back into ImportBankResult. +std::string doImportBankPackage(ReaSamplerSession& session); // Same, for a .rsbank already named by the user — the panel's file-drop route. The gate -// still runs first; only the picker is skipped. -void doImportBankPackageFile(ReaSamplerSession& session, const std::string& packageAbsPath); +// still runs first; only the picker is skipped. Same return contract as doImportBankPackage. +std::string doImportBankPackageFile(ReaSamplerSession& session, const std::string& packageAbsPath); } // namespace reasampler diff --git a/src/shell/package/import_bank.cpp b/src/shell/package/import_bank.cpp index 777b165..2261361 100644 --- a/src/shell/package/import_bank.cpp +++ b/src/shell/package/import_bank.cpp @@ -46,7 +46,8 @@ void fillPlanCounts(ImportBankResult& out, const package::ImportPlan& plan) { out.seedBankName = plan.seedBankName; out.bankNameAdjusted = plan.bankNameAdjusted; out.landedCount = plan.landCount; - out.renamedCount = plan.renameCount; + out.collisionRenameCount = plan.collisionRenameCount; + out.sanitizeRenameCount = plan.sanitizeRenameCount; out.collapsedCount = plan.collapseCount; } @@ -70,8 +71,9 @@ ImportBankResult importBankPackage(ReaSamplerSession& session, fillPlanCounts(out, landing.plan); if (landing.outcome != ImportOutcome::Landed) return out; + const std::string bankId = mintBankId(); const bool applied = applyImportedBank( - session.book(), mintBankId(), landing.plan, + session.book(), bankId, landing.plan, [&session](const model::Sample& s) { session.recordCreated(s, tracking::OriginKind::PackageImport); }); @@ -80,6 +82,7 @@ ImportBankResult importBankPackage(ReaSamplerSession& session, out.rollback = journal.rollback(); return out; } + out.bankId = bankId; // Generation bump + persist ride inside one undo block, so a Ctrl-Z takes the whole // import back out of the index. It does NOT un-write the files — the summary says so. diff --git a/src/shell/package/import_bank.h b/src/shell/package/import_bank.h index 94433cc..018ac9d 100644 --- a/src/shell/package/import_bank.h +++ b/src/shell/package/import_bank.h @@ -17,12 +17,14 @@ struct ImportBankResult { ImportOutcome outcome = ImportOutcome::Unreadable; package::PackageHeader header; // TooNew names the writer's build from here + std::string bankId; // the minted id — meaningful only when outcome == Landed std::string bankDisplayName; // the bank actually created std::string seedBankName; // what the package asked to be called bool bankNameAdjusted = false; int landedCount = 0; - int renamedCount = 0; + int collisionRenameCount = 0; // renamed: the package's own name was already taken + int sanitizeRenameCount = 0; // renamed: not spelled the way this tool spells a bank file int collapsedCount = 0; std::string failedEntryName; diff --git a/src/shell/package/import_landing.cpp b/src/shell/package/import_landing.cpp index f51a338..d4c1a26 100644 --- a/src/shell/package/import_landing.cpp +++ b/src/shell/package/import_landing.cpp @@ -3,6 +3,7 @@ #include "shell/package/import_landing.h" +#include #include #include #include @@ -118,12 +119,23 @@ ImportLanding landPackage(const std::string& packageAbsPath, bool applyImportedBank(BankBook& book, const std::string& bankId, const package::ImportPlan& plan, const RecordBirth& recordBirth) { + // An empty std::function throws std::bad_function_call on invoke; every real caller + // supplies one, so an empty one here is a caller bug, not a runtime condition to + // recover from — enforce the contract rather than let it surface as an uncaught + // exception out of an extension action. + assert(recordBirth && "applyImportedBank: RecordBirth must not be empty"); if (!book.createBank(bankId, plan.bankDisplayName)) return false; BankModel* index = book.index(bankId); for (const package::PlannedEntry& e : plan.entries) { if (e.action != EntryAction::Land) continue; - index->add(e.sample); + const AddResult added = index->add(e.sample); + // planImport already deduped Land entries by hash against an empty destination + // bank (this same freshly-created one), so a Collapsed add here would mean the + // plan and the book disagree — that would silently undercount reportSuccess's + // landedCount rather than fail loudly. + assert(added == AddResult::Added && "planImport's Land entries must not collapse"); + (void)added; // Unconditional on the add's outcome: the file exists either way, and an // unrecorded file is permanently unreclaimable. recordBirth(e.sample); diff --git a/src/shell/package/import_landing.h b/src/shell/package/import_landing.h index 7dc85e3..0731892 100644 --- a/src/shell/package/import_landing.h +++ b/src/shell/package/import_landing.h @@ -15,18 +15,22 @@ namespace reasampler { -// How a landing ended. Every value but Landed means NOTHING is on disk and NO index -// was touched — the two refuse-whole failures (TooNew, Malformed) before a byte is -// written, the other two after a rollback. +// How a landing ended. Every value but Landed means NOTHING is on disk and NO index was +// touched. NoProject/Unreadable/Malformed/TooNew refuse before a byte is written. +// IntegrityFailed also refuses before any write — the full-package digest verification +// runs to completion first (landPackage) — so it needs no rollback either. WriteFailed +// is the only outcome that actually wrote and then rolled back. IndexRejected is never +// returned by landPackage/this struct — it is import_bank's own outcome, minted after a +// successful landing when the book itself refuses the create. enum class ImportOutcome { Landed, NoProject, // unsaved project: there is no bank folder to land into Unreadable, // the package file could not be opened Malformed, // not a well-formed RSBK: corrupt, truncated, or trailing garbage TooNew, // minReaderVersion above this build's ladder - IntegrityFailed, // an entry's payload did not match its recorded digest + IntegrityFailed, // an entry's payload did not match its recorded digest; pre-write refusal WriteFailed, // a write failed partway; the landed files were rolled back - IndexRejected, // the book refused the bank the plan minted a free name for + IndexRejected, // never set here — see the comment above; import_bank's outcome only }; struct ImportLanding { @@ -36,7 +40,7 @@ struct ImportLanding { package::PackageHeader header; package::ImportPlan plan; std::string failedEntryName; // IntegrityFailed / WriteFailed - RollbackResult rollback; // IntegrityFailed / WriteFailed + RollbackResult rollback; // WriteFailed only — IntegrityFailed leaves it default }; // Streams `packageAbsPath` into the project's bank folder: decode, plan, verify EVERY diff --git a/src/shell/panel/panel_bank_ops.cpp b/src/shell/panel/panel_bank_ops.cpp index 6518ee3..7cee606 100644 --- a/src/shell/panel/panel_bank_ops.cpp +++ b/src/shell/panel/panel_bank_ops.cpp @@ -284,7 +284,14 @@ void showTabMenu(int screenX, int screenY, const std::string& bankId) { // Always a NEW bank, never a merge into the right-clicked one — the row sits // here because this is the panel's bank menu, not because it targets this bank. case kMenuImportPackage: - if (g_panel.session) doImportBankPackage(*g_panel.session); + if (g_panel.session) { + const std::string id = doImportBankPackage(*g_panel.session); + if (!id.empty()) { // landed — show the freshly-imported bank + g_panel.shownBankId = id; + g_panel.focusedRegion = Region::Banks; + invalidatePanel(); + } + } break; default: break; } diff --git a/src/shell/panel/panel_window.cpp b/src/shell/panel/panel_window.cpp index a953d3d..12a188d 100644 --- a/src/shell/panel/panel_window.cpp +++ b/src/shell/panel/panel_window.cpp @@ -16,6 +16,9 @@ #include "shell/panel/draw_kit.h" #include "shell/actions/ingest.h" #include "shell/actions/package_import_action.h" +#include "core/package/import_plan.h" +#include "core/version/app_version.h" +#include "shell/persist/session.h" // ReaSamplerSession::ledgerStatus() — panel_state.h only forward-declares it #ifdef _WIN32 #include // GET_X_LPARAM / GET_Y_LPARAM (SWELL supplies them on mac/linux) @@ -29,6 +32,7 @@ #define REAPERAPI_WANT_DockWindowActivate #define REAPERAPI_WANT_DockWindowRemove #define REAPERAPI_WANT_GetMainHwnd +#define REAPERAPI_WANT_ShowConsoleMsg #include "reaper_plugin_functions.h" // main.cpp owns the module instance handle. @@ -72,9 +76,21 @@ void handleDropFiles(HDROP hDrop) { else paths.push_back(std::move(p)); } DragFinish(hDrop); - if (g_panel.session) - for (const std::string& pkg : packages) - doImportBankPackageFile(*g_panel.session, pkg); + if (g_panel.session && !packages.empty()) { + // One refusal block for the whole drop, not one per dropped .rsbank: the gate + // decision is the same for all N (session state does not change mid-drop), so + // checking it here first avoids doImportBankPackageFile's own per-file gate + // check printing the identical console block N times. + const package::LedgerRefusal refusal = + package::importLedgerRefusal(g_panel.session->ledgerStatus()); + if (refusal != package::LedgerRefusal::None) { + ShowConsoleMsg( + package::ledgerRefusalMessage(refusal, version::extStateNamespace()).c_str()); + } else { + for (const std::string& pkg : packages) + doImportBankPackageFile(*g_panel.session, pkg); + } + } if (!paths.empty()) ingestDroppedFiles(paths); } diff --git a/src/shell/persist/session.h b/src/shell/persist/session.h index d8a808b..0bc1967 100644 --- a/src/shell/persist/session.h +++ b/src/shell/persist/session.h @@ -98,6 +98,13 @@ public: // the records. That is not a hole in the pairing rule above: the rule exists so an // absent record is never read as a definite answer, and this exposes strictly less // than the pair. The package import gates on it before it opens a file picker. + // + // A tradeoff, not the only route: `pruneDryRun()` already exposes the same degraded + // pair via `PruneReport::ledgerUnreadable`/`ledgerFutureVersion`, with no new + // accessor needed. Rejected because that route is genuinely worse for a gate: it + // drags a full bank-folder enumeration and every live instance's FX scan onto a + // check that only needs to know "can I write a record", and it shapes an import + // decision as an answer borrowed from prune's report rather than the session's own. tracking::LedgerStatus ledgerStatus() const { return trackingStatus_; } // The version that last wrote the active project: PreVersioning (no diff --git a/tests/test_import_landing.cpp b/tests/test_import_landing.cpp index e97f7f0..441002c 100644 --- a/tests/test_import_landing.cpp +++ b/tests/test_import_landing.cpp @@ -372,7 +372,9 @@ static void testReimportingIntoTheSourceProjectLandsBesideAnUntouchedOriginal() CHECK(third.landing.outcome == ImportOutcome::Landed); CHECK(book.bank("bank-b3")->displayName == "B 3"); CHECK(*book.index("bank-b") == originalB); - // Six distinct files: the original two plus two per re-import, never overwritten. + // Four distinct files: two per re-import, never overwritten. Bank "B"'s own two + // entries were seeded index-only above (book.index("bank-b")->add), never written + // to disk, so they don't add to this count. CHECK(scratch.bankFiles().size() == 4); } diff --git a/tests/test_import_plan.cpp b/tests/test_import_plan.cpp index 760d650..2f60977 100644 --- a/tests/test_import_plan.cpp +++ b/tests/test_import_plan.cpp @@ -202,7 +202,8 @@ static void testAFreeBankLegalNameIsKept() { {"unrelated.wav"}, kTag); CHECK(landed(plan, 0).destFileName == "kick.wav"); CHECK(!landed(plan, 0).renamed); - CHECK(plan.renameCount == 0); + CHECK(plan.collisionRenameCount == 0); + CHECK(plan.sanitizeRenameCount == 0); CHECK(landed(plan, 0).sample.relativePath == "reasampler_bank/kick.wav"); } @@ -212,7 +213,9 @@ static void testATakenNameIsMintedFreshAndNeverOverwritten() { {"kick.wav"}, kTag); CHECK(landed(plan, 0).destFileName != "kick.wav"); CHECK(landed(plan, 0).renamed); - CHECK(plan.renameCount == 1); + // A genuine folder-name collision, not a spelling mint. + CHECK(plan.collisionRenameCount == 1); + CHECK(plan.sanitizeRenameCount == 0); CHECK(landed(plan, 0).sample.relativePath == "reasampler_bank/" + landed(plan, 0).destFileName); } @@ -224,6 +227,9 @@ static void testTheFolderNameCheckFoldsAsciiCase() { {"KICK.WAV"}, kTag); CHECK(landed(plan, 0).destFileName != "kick.wav"); CHECK(landed(plan, 0).renamed); + // The case-fold hit is still a collision, not a spelling mint. + CHECK(plan.collisionRenameCount == 1); + CHECK(plan.sanitizeRenameCount == 0); } static void testTwoEntriesNeverLandOnOneName() { @@ -242,6 +248,9 @@ static void testAnUnsanitaryNameIsMintedRatherThanLandedVerbatim() { bookWithBanks({}), kProjectDir, {}, kTag); CHECK(landed(plan, 0).destFileName.find(' ') == std::string::npos); CHECK(landed(plan, 0).renamed); + // No collision here (the bank folder is empty) — this is a sanitize mint. + CHECK(plan.sanitizeRenameCount == 1); + CHECK(plan.collisionRenameCount == 0); } // --- content hash (collision class 3) ---------------------------------------- @@ -334,6 +343,46 @@ static void testAnUndecodableUsageKeyBlocksPruneButNotImport() { CHECK(importLedgerRefusal(tracking::LedgerStatus::Loaded) == LedgerRefusal::None); } +// Pins the delegation itself: importLedgerRefusal must refuse EXACTLY the statuses +// ledgerDegraded() calls degraded, over every value the enum has today. A gate that +// re-derived its own notion of "degraded" could silently diverge from this the moment +// either side changes without the other. +static void testImportLedgerRefusalDelegatesToLedgerDegraded() { + const tracking::LedgerStatus all[] = { + tracking::LedgerStatus::Fresh, + tracking::LedgerStatus::Loaded, + tracking::LedgerStatus::Unreadable, + tracking::LedgerStatus::FutureVersion, + }; + for (tracking::LedgerStatus s : all) + CHECK((importLedgerRefusal(s) != LedgerRefusal::None) == tracking::ledgerDegraded(s)); +} + +// --- the ledger-refusal message (pure, so both channels are assertable without a DAW) -- + +static void testLedgerRefusalMessageIsEmptyForNone() { + CHECK(ledgerRefusalMessage(LedgerRefusal::None, "reasampler").empty()); +} + +static void testLedgerRefusalMessageNamesTheChannelCorrectNamespace() { + // The two real namespaces (app_version.h): stable "reasampler", beta "reasampler_beta". + const std::string stable = ledgerRefusalMessage(LedgerRefusal::Malformed, "reasampler"); + CHECK(stable.find("\"reasampler\"") != std::string::npos); + CHECK(stable.find("reasampler_beta") == std::string::npos); + + const std::string beta = ledgerRefusalMessage(LedgerRefusal::Malformed, "reasampler_beta"); + CHECK(beta.find("\"reasampler_beta\"") != std::string::npos); +} + +static void testLedgerRefusalMessageDistinguishesMalformedFromFutureVersion() { + const std::string malformed = ledgerRefusalMessage(LedgerRefusal::Malformed, "reasampler"); + const std::string futureVersion = + ledgerRefusalMessage(LedgerRefusal::FutureVersion, "reasampler"); + CHECK(malformed != futureVersion); + CHECK(malformed.find("malformed") != std::string::npos); + CHECK(futureVersion.find("NEWER version") != std::string::npos); +} + int main() { testFreeSeedIsKeptVerbatim(); testFoldedCollisionTakesTheFirstSuffix(); @@ -364,6 +413,11 @@ int main() { testDegradedLedgerStatusesRefuseAndTheUsableOnesDoNot(); testAnUndecodableUsageKeyBlocksPruneButNotImport(); + testImportLedgerRefusalDelegatesToLedgerDegraded(); + + testLedgerRefusalMessageIsEmptyForNone(); + testLedgerRefusalMessageNamesTheChannelCorrectNamespace(); + testLedgerRefusalMessageDistinguishesMalformedFromFutureVersion(); if (g_fail == 0) std::printf("import_plan: all tests passed\n"); return g_fail == 0 ? 0 : 1; From 2069ae80862924c6c07d47d492ff4e479ec5738a Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 14:23:03 -0400 Subject: [PATCH 17/24] docs: point the three bumpBankGeneration citations at the right line A prior pass corrected session.h:108 to :114, but :114 is the read accessor; the bump is at :121. Also corrects a module count in the package doc. --- docs/PLAN.md | 4 ++-- docs/product/bank-package.md | 2 +- src/core/package/CLAUDE.md | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/PLAN.md b/docs/PLAN.md index ce4b7cb..3a2883d 100644 --- a/docs/PLAN.md +++ b/docs/PLAN.md @@ -2366,7 +2366,7 @@ These bind every track in this phase, in addition to the plan-wide set above. touching it must **cite, not restate**. - **Export is read-only against the project.** No ext-state write, no `bumpBankGeneration()`, no undo point. Import does the opposite: it bumps the generation - (`src/shell/persist/session.h:114`) so live ReaSampler 9000 instances reload, and batches + (`src/shell/persist/session.h:121`) so live ReaSampler 9000 instances reload, and batches its index mutation into one Ctrl-Z through `persistBankOp`. - **All-or-nothing on both sides.** No partial export, no partial import. A truncated `.rsbank` must never exist on disk (temp file + atomic rename, the Ψ-W2-T2 precedent); a @@ -2792,7 +2792,7 @@ fold, it does not add a second one), `origin_ledger` (Ε-W1-T3's). remain as orphans until a prune reclaims them — the same designed window a non-empty bank delete already produces (`core/model/CLAUDE.md`'s sample-removal section). The user-facing summary says so. -- **`bumpBankGeneration()` on success** (`session.h:114`), so live instances reload. +- **`bumpBankGeneration()` on success** (`session.h:121`), so live instances reload. - **No timeline item is placed. Ever.** - **A new FOREVER-STABLE command id**, minted the same way T1's is. diff --git a/docs/product/bank-package.md b/docs/product/bank-package.md index 733b1d7..fca7e79 100644 --- a/docs/product/bank-package.md +++ b/docs/product/bank-package.md @@ -690,7 +690,7 @@ constructors. freshly-generated pair. - **Bank generation.** Import mutates bank content that live ReaSampler 9000 instances may play, so it must `bumpBankGeneration()` - (`src/shell/persist/session.h:114`, whose own comment says call sites "err toward + (`src/shell/persist/session.h:121`, whose own comment says call sites "err toward bumping"). Export mutates nothing and must bump nothing, write no ext state, and open no undo point. - **Beta/stable channel isolation.** Packages are channel-**agnostic** and this is diff --git a/src/core/package/CLAUDE.md b/src/core/package/CLAUDE.md index 2ce0c62..c24ffcb 100644 --- a/src/core/package/CLAUDE.md +++ b/src/core/package/CLAUDE.md @@ -87,7 +87,7 @@ both have landed. `ledgerRefusalMessage` (the gate's console-block body, a pure `(LedgerRefusal, namespace) -> string` fold the shell only supplies the channel-correct namespace to). -- `bank_package` — framing and arithmetic composing the two above: +- `bank_package` — framing and arithmetic composing the four above: `encodePackage` (prefix bytes + layout + total size, stamping this build's ladder pair and `version::stampVersion()`), `decodePackage` (prefix + observed file size in; header/manifest/layout out), and `requiredPrefixSize` (the From 9521b5339fa5c043e4a474af3264dc624f61d7a8 Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 15:16:55 -0400 Subject: [PATCH 18/24] Freeze the package compatibility corpus: real .rsbank bytes proving both ladder directions, every truncation site, and the round trip --- docs/verify-package-transfer.md | 172 +++++++++ src/core/package/CLAUDE.md | 4 + src/core/package/CMakeLists.txt | 6 + src/shell/package/CLAUDE.md | 4 + src/shell/package/CMakeLists.txt | 10 + tests/fixtures/package_compat/README.md | 101 ++++++ .../package_compat/additive_forward.rsbank | Bin 0 -> 1283 bytes .../package_compat/hostile_name_dotdot.rsbank | Bin 0 -> 624 bytes .../hostile_name_drive_absolute.rsbank | Bin 0 -> 643 bytes .../hostile_name_parent_backslash.rsbank | Bin 0 -> 634 bytes .../hostile_name_parent_slash.rsbank | Bin 0 -> 633 bytes .../hostile_name_subdir_slash.rsbank | Bin 0 -> 634 bytes .../hostile_name_unc_absolute.rsbank | Bin 0 -> 646 bytes .../hostile_path_dotdot_backslash.rsbank | Bin 0 -> 640 bytes .../hostile_path_dotdot_slash.rsbank | Bin 0 -> 641 bytes .../hostile_path_drive_absolute.rsbank | Bin 0 -> 643 bytes .../package_compat/hostile_path_rooted.rsbank | Bin 0 -> 635 bytes .../hostile_path_unc_absolute.rsbank | Bin 0 -> 646 bytes .../package_compat/refuse_structural.rsbank | Bin 0 -> 1207 bytes .../package_compat/trunc_magic.rsbank | 1 + .../package_compat/trunc_manifest_body.rsbank | Bin 0 -> 466 bytes .../trunc_manifest_length.rsbank | Bin 0 -> 23 bytes .../package_compat/trunc_one_short.rsbank | Bin 0 -> 1206 bytes .../trunc_payload_middle.rsbank | Bin 0 -> 1057 bytes .../package_compat/trunc_payload_start.rsbank | Bin 0 -> 907 bytes .../package_compat/trunc_version_pair.rsbank | Bin 0 -> 10 bytes .../package_compat/trunc_writer_semver.rsbank | Bin 0 -> 18 bytes .../package_compat/v1_shipping.rsbank | Bin 0 -> 1207 bytes tests/package_fixtures.h | 25 ++ tests/test_package_compat.cpp | 334 ++++++++++++++++++ tests/test_package_round_trip.cpp | 250 +++++++++++++ 31 files changed, 907 insertions(+) create mode 100644 docs/verify-package-transfer.md create mode 100644 tests/fixtures/package_compat/README.md create mode 100644 tests/fixtures/package_compat/additive_forward.rsbank create mode 100644 tests/fixtures/package_compat/hostile_name_dotdot.rsbank create mode 100644 tests/fixtures/package_compat/hostile_name_drive_absolute.rsbank create mode 100644 tests/fixtures/package_compat/hostile_name_parent_backslash.rsbank create mode 100644 tests/fixtures/package_compat/hostile_name_parent_slash.rsbank create mode 100644 tests/fixtures/package_compat/hostile_name_subdir_slash.rsbank create mode 100644 tests/fixtures/package_compat/hostile_name_unc_absolute.rsbank create mode 100644 tests/fixtures/package_compat/hostile_path_dotdot_backslash.rsbank create mode 100644 tests/fixtures/package_compat/hostile_path_dotdot_slash.rsbank create mode 100644 tests/fixtures/package_compat/hostile_path_drive_absolute.rsbank create mode 100644 tests/fixtures/package_compat/hostile_path_rooted.rsbank create mode 100644 tests/fixtures/package_compat/hostile_path_unc_absolute.rsbank create mode 100644 tests/fixtures/package_compat/refuse_structural.rsbank create mode 100644 tests/fixtures/package_compat/trunc_magic.rsbank create mode 100644 tests/fixtures/package_compat/trunc_manifest_body.rsbank create mode 100644 tests/fixtures/package_compat/trunc_manifest_length.rsbank create mode 100644 tests/fixtures/package_compat/trunc_one_short.rsbank create mode 100644 tests/fixtures/package_compat/trunc_payload_middle.rsbank create mode 100644 tests/fixtures/package_compat/trunc_payload_start.rsbank create mode 100644 tests/fixtures/package_compat/trunc_version_pair.rsbank create mode 100644 tests/fixtures/package_compat/trunc_writer_semver.rsbank create mode 100644 tests/fixtures/package_compat/v1_shipping.rsbank create mode 100644 tests/package_fixtures.h create mode 100644 tests/test_package_compat.cpp create mode 100644 tests/test_package_round_trip.cpp diff --git a/docs/verify-package-transfer.md b/docs/verify-package-transfer.md new file mode 100644 index 0000000..d701cc9 --- /dev/null +++ b/docs/verify-package-transfer.md @@ -0,0 +1,172 @@ +# DAW verification — bank-package transfer across machines + +What a DAW pass must establish for `.rsbank` export and import, and the exact strings or +counts to read off. The unit corpus (`tests/fixtures/package_compat/`) already proves the +version ladder, the truncation verdicts and the hostile-name refusals against frozen +bytes. **Nothing below is covered by it**: every cell here depends on a real REAPER +session, a real file dialog, or a genuine second machine. + +**Build to use.** Release, installed into `UserPlugins/`, REAPER restarted — extensions +load at startup only. Note the version the *About*/version action reports; §5 needs it. + +**Machines to use.** Two: **A** (the source) and **B** (the destination). B must be a +different machine, or at minimum a different user account with its own REAPER resource +path and its own projects folder — the point is that no absolute path from A can resolve +on B. A USB stick, a network share, or a cloud folder are all acceptable transports. + +**Projects to use.** On A: one **saved** project with a bank holding at least **three** +samples, at least one of them audibly distinct from the others, and at least one whose +display name carries a non-ASCII character (e.g. `Café hit`). On B: one **saved**, +otherwise empty project. + +--- + +## 1. Export writes one file and touches nothing else + +On A, right-click the bank's header in the docked panel → **Export as package...** (or +run *ReaSampler: export active bank as package*). Accept the suggested file name. + +Read off: + +- The console shows `ReaSampler export: wrote 3 entry/entries (N bytes) to `, with + the entry count matching the bank. +- A single `.rsbank` file exists at that path. **No `.rsbanktmp` sibling remains** — a + leftover temp file means the atomic rename did not complete. +- The bank's card count, the bank folder's file count, and the project's dirty flag are + all **unchanged**. An export writes no ext state and opens no undo point, so REAPER + must not consider the project modified by it alone. +- Nothing was added to the arrange view. + +## 2. The transfer itself — the claim no unit test can make + +Copy the `.rsbank` to B by whatever transport you chose. Do **not** copy the project, the +bank folder, or anything else. + +On B, open the empty saved project. Panel bank menu → **Import bank package...** (or run +*ReaSampler: import bank package (.rsbank)*), and choose the transferred file. + +Read off: + +- A message box: `Imported 3 sample(s) into a new bank: "".` +- The console block repeats that line and ends with `One undo removes the imported bank + and its entries. It does NOT delete the imported files ...`. +- The panel shows a **new** bank with the same display name and the same number of cards, + **in the same order** as on A. +- B's bank folder holds three new files. The non-ASCII display name from A renders + correctly on the card — a mangled name here means the UTF-8 path/name conversion broke + in transit. +- **Audition each card.** They must sound like their counterparts on A. This is the whole + claim: the audio survived a machine boundary with no shared path. +- Press **Ctrl-Z once**. The imported bank and its entries disappear in one step. The + three files remain in B's bank folder (that is stated in the console block above, and is + the designed behaviour — a prune reclaims them). Redo to continue. + +## 3. Re-importing the same package never overwrites + +Still on B, import the **same** file a second time. + +Read off: + +- A second new bank appears, named with a suffix (` 2`), and the box's + `(a bank named "" already exists in this project)` clause appears in the + console block. +- B's bank folder now holds **six** files, not three. The console reports + `3 file(s) landed under a freshly minted name (the package's own name was already taken + in the bank folder). An existing bank file is never overwritten.` +- The first imported bank's cards still audition correctly — nothing was replaced under it. + +## 4. Round trip back to the source + +On B, export the imported bank (§1) to a second `.rsbank`. Carry it back to A and import +it into A's original project. + +Read off: + +- The import succeeds and lands as a new bank beside the original. +- The original bank on A is untouched: same card count, same names, same audio. +- Compare the two `.rsbank` files' **sizes**. They will usually differ — entry names, + sample ids and the export timestamp are all legitimately re-minted across a trip. The + payload bytes are what must survive, and that half is closed by + `tests/test_package_round_trip.cpp` against frozen bytes; do **not** treat a size + difference here as a defect. + +## 5. The too-new refusal, with the message read verbatim + +This is the direction a user hits when a collaborator is ahead of them, and the message is +the only actionable output. Produce it by hand: + +1. Copy the `.rsbank` from §1 to a scratch name. +2. Open the copy in a hex editor. Bytes 0–3 are `RSBK`; bytes 4–7 are `formatVersion` + little-endian; bytes **8–11** are `minReaderVersion` little-endian. +3. Change byte **8** from `01` to `02`, and byte **4** from `01` to `02` (a writer cannot + require a reader newer than the format it wrote — leaving `formatVersion` at 1 makes + the file incoherent and it will be refused as malformed instead, which is a different + cell). Save. +4. Import the edited copy. + +Read off — the message box, all four lines: + +``` +Cannot import this bank package. +It was written by ReaSampler and needs package format 2 or newer. +This build () reads package format 1. +Nothing was imported. Install ReaSampler or newer and try again. +``` + +- The writer version named is the one **this** build stamped in §1 (the hex edit does not + touch the semver), so the second and fourth lines will name your own version. That is + expected — what is being verified is that all three facts are present and the box + appears at all. +- **No** new bank, **no** new files in the bank folder, **no** undo point. + +## 6. The truncated-download refusal is a different message + +Copy the §1 package again and delete the last few hundred bytes (any hex editor, or +`head -c` / `fsutil` — the exact count does not matter as long as the file is shorter). +Import it. + +Read off: + +- The message box reads exactly: `This file is not a readable bank package (corrupt or + truncated). Nothing was imported.` +- It is **not** the §5 message. Crossing these two is the failure this cell exists to + catch — "install a newer build" does not fix a partial download. +- No new bank, no new files. + +## 7. Corruption in the middle is caught before anything lands + +Copy the §1 package again and flip a single byte **well past the halfway point** (inside a +payload, not the header). Import it. + +Read off: + +- The message box names the offending entry: + `This bank package is damaged (entry "" failed its integrity check). Nothing was + imported.` +- The bank folder gained **no** files at all — not even the entries before the damaged + one. Verification runs to completion before the first write, so a damaged package costs + no rollback. + +## 8. The unsaved-project refusals + +- On B, File → New Project (do not save). Try to import. Read off: + `Save the project before importing a bank package -- an unsaved project has no bank + folder to import into.` The file picker must **not** have opened first. +- On A, in an unsaved project with no bank, try to export. Read off the console: + `ReaSampler export: save the project first -- an unsaved project has no bank folder to + read from.` + +## 9. Drag-and-drop reaches the same verb + +On B, drag a `.rsbank` from the file manager onto the docked ReaSampler panel. + +Read off: the same import box as §2, and the same new bank. A `.rsbank` is a whole bank, +not audio — it must never land as an item in the arrange view. + +--- + +## Recording the result + +For each section, record **pass**, **fail with the string actually seen**, or **not +exercised**. §2 and §4 are the load-bearing ones: they are the only cells in this document +that involve a real machine boundary, and no unit test can stand in for them. diff --git a/src/core/package/CLAUDE.md b/src/core/package/CLAUDE.md index c24ffcb..4733c76 100644 --- a/src/core/package/CLAUDE.md +++ b/src/core/package/CLAUDE.md @@ -93,6 +93,10 @@ both have landed. file size in; header/manifest/layout out), and `requiredPrefixSize` (the incremental-read seam for the shell). Framing rides `core/wire/bytes.h`. +`package_compat_tests` is declared here with no library of its own: it decodes the +frozen `.rsbank` corpus at `tests/fixtures/package_compat/`, whose README owns the +append-only rule and the per-fixture inventory. + ## Gotchas - Enums nested inside the `BankModel` blob follow `bank_model`'s own rule — an diff --git a/src/core/package/CMakeLists.txt b/src/core/package/CMakeLists.txt index 3cdc30e..0af4166 100644 --- a/src/core/package/CMakeLists.txt +++ b/src/core/package/CMakeLists.txt @@ -18,6 +18,12 @@ reasampler_pure_library(bank_package # app_version: the tests pin the stamped writer semver against stampVersion(). reasampler_test(bank_package LINK bank_package app_version) +# The frozen compatibility corpus, decoded rather than regenerated. Fixture path: see +# tests/package_fixtures.h. +reasampler_test(package_compat LINK bank_package) +target_compile_definitions(package_compat_tests PRIVATE + REASAMPLER_PACKAGE_FIXTURE_DIR="${REASAMPLER_TESTS_DIR}/fixtures/package_compat") + reasampler_pure_library(import_plan SOURCES import_plan.cpp LINK PUBLIC package_manifest bank_book origin_ledger PRIVATE package_format capture_paths) diff --git a/src/shell/package/CLAUDE.md b/src/shell/package/CLAUDE.md index bed91ee..dede777 100644 --- a/src/shell/package/CLAUDE.md +++ b/src/shell/package/CLAUDE.md @@ -94,6 +94,10 @@ belongs to its skin (`shell/actions/package_export_action`), not this seam. - `import_landing` — the import's two halves that decide anything: `landPackage` (decode, plan, verify EVERY payload's digest, then land through the journal) and `applyImportedBank` (the new bank's entries plus a birth record per landed file, in one straight-line block). REAPER-free deliberately — all-or-nothing, integrity and birth-record behaviour are assertable without a DAW. - `import_bank` — the promptless import verb over a live `ReaSamplerSession`: the project directory, the minted bank id, the `recordCreated` writer, and the one undo-batched persist. REAPER-facing, so it compiles into the extension module rather than into a library with a test target. +`package_round_trip_tests` is declared here with no library of its own: it drives the +same frozen corpus (`tests/fixtures/package_compat/`) through both verbs, which is where +export → import → export payload identity is proven. + ## Gotchas - A crash mid-export strands the `.rsbanktmp` sibling. It is not a `.rsbank` (no diff --git a/src/shell/package/CMakeLists.txt b/src/shell/package/CMakeLists.txt index 399b32d..8ea6b60 100644 --- a/src/shell/package/CMakeLists.txt +++ b/src/shell/package/CMakeLists.txt @@ -33,6 +33,16 @@ reasampler_pure_library(import_landing LINK PUBLIC import_plan package_rollback bank_book PRIVATE bank_package wav_codec) reasampler_test(import_landing LINK import_landing bank_package app_version wav_codec origin_ledger) +# The frozen compatibility corpus driven through both verbs in one process — the round +# trip is export -> import -> export, so its link set is export_bank's plus the import +# half. Fixture path: see tests/package_fixtures.h. +reasampler_test(package_round_trip + LINK import_landing export_bank bank_package bank_book slot_map view_mode_model + tail_control origin_ledger tracking_authority prune_reconcile app_version + capture_paths wav_codec) +target_compile_definitions(package_round_trip_tests PRIVATE + REASAMPLER_PACKAGE_FIXTURE_DIR="${REASAMPLER_TESTS_DIR}/fixtures/package_compat") + # The pickers call the REAPER API, so no test target can exercise them; declared as a # library so the TU stays compiled. reaper_plugin.h pulls SWELL in on non-Windows. add_library(package_pickers STATIC package_pickers.cpp) diff --git a/tests/fixtures/package_compat/README.md b/tests/fixtures/package_compat/README.md new file mode 100644 index 0000000..c63252d --- /dev/null +++ b/tests/fixtures/package_compat/README.md @@ -0,0 +1,101 @@ +# The frozen `.rsbank` compatibility corpus + +Real RSBK bytes, committed. `tests/test_package_compat.cpp` decodes them; +`tests/test_package_round_trip.cpp` drives them through the import and export verbs. + +## THE RULE: this corpus is append-only + +**No file here is ever regenerated or edited.** When a future format version ships, add +its fixture beside these and leave every existing one alone. + +The reason is the whole point of the corpus. These bytes exist to catch a format change +that quietly breaks a compatibility direction. A fixture regenerated by the build that +broke it agrees with that build by construction and catches nothing — which is exactly +the failure mode a version ladder exists to prevent. The same argument forbids a test +that builds its own fixture at run time. + +If a fixture stops decoding, the answer is never to re-cut the fixture. Either the format +change was structural (bump `minReaderVersion`, add a new fixture, and leave the old one +asserting the refusal) or it is a regression. + +## Provenance + +`v1_shipping.rsbank` was produced by running this repo's own export verb (`exportBank`) +at version **1.4.0** over a one-sample bank, and copying the emitted file here verbatim. +Every other fixture is derived from those bytes: the truncations are prefixes of them, +and the synthetic packages reuse their manifest region under different version integers +or a hand-written hostile manifest (the encoder refuses to write one, which is why those +could not come from the verb). + +Payloads are one 300-byte 16-bit mono WAV. The properties under test are structural — +version integers, framing arithmetic, name validation — so a larger payload proves +nothing extra and costs the repo bytes forever. Whole corpus: ~14 KB. + +Adding a fixture for a future version means writing it with **that** version's shipping +build, exactly as this one was, and recording the build's version here. + +## What each fixture proves + +### The three version fixtures + +| File | `formatVersion` / `minReaderVersion` | Verdict | Proves | +|---|---|---|---| +| `v1_shipping.rsbank` | 1 / 1 | `Readable` | This build reads what it wrote: header, one manifest entry, the entry digest, and every `Sample` field with every optional present. Writer semver `1.4.0` is asserted **literally**, not against `stampVersion()` — comparing against the running build would let a version bump re-anchor the fixture silently. | +| `additive_forward.rsbank` | 2 / 1 | `Readable` | An additive newer writer still reads. Carries three keys this build has never heard of — `exportTool` at the manifest root, `futureEntryKey` on the entry, `futureSampleKey` inside the nested `Sample` blob — and decodes to *exactly* the manifest `v1_shipping.rsbank` decodes to. Writer semver `1.9.0`. | +| `refuse_structural.rsbank` | 2 / 2 | `TooNew` | A structural newer writer is refused whole. The header through the writer semver still reads, so the refusal can name all three facts (`1.9.0`, needs format 2, this build reads 1); no manifest, no layout, no partial success. Its body is `v1_shipping.rsbank`'s own manifest, which parses — so the refusal is a **decision**, not an inability. | + +### Truncation — one file per distinct decode failure site + +Each is a prefix of `v1_shipping.rsbank` (907-byte prefix + 300-byte payload = 1207 +bytes). All classify `Malformed`; none may classify `TooNew`, since "install a newer +build" does not fix a partial download. + +| File | Bytes | Site the cut lands in | +|---|---|---| +| `trunc_magic.rsbank` | 2 | Inside the 4-byte magic. | +| `trunc_version_pair.rsbank` | 10 | Inside the frozen header's `minReaderVersion` u32. | +| `trunc_writer_semver.rsbank` | 18 | Inside the frozen header's writer semver. | +| `trunc_manifest_length.rsbank` | 23 | Inside the manifest-length u32. | +| `trunc_manifest_body.rsbank` | 466 | Inside the manifest JSON. | +| `trunc_payload_start.rsbank` | 907 | At the payload boundary. RSBK stores no layout section — the layout is derived from the manifest's entries — so this is the cut that exercises "manifest parses, layout computes, exact-size proof fails". | +| `trunc_payload_middle.rsbank` | 1057 | Inside the first payload. | +| `trunc_one_short.rsbank` | 1206 | One byte short of the total. | + +### Hostile names — refused at decode, before any planner + +The two naming fields carry different rules (`src/core/package/CLAUDE.md`), so each +fixture keeps the other field spelled cleanly (`kick.wav`) and the refusal is +attributable to the field under test. + +Entry name — a bare file name, no path expression possible (`isValidEntryName`): + +| File | Entry name | +|---|---| +| `hostile_name_dotdot.rsbank` | `..` | +| `hostile_name_parent_slash.rsbank` | `../evil.wav` | +| `hostile_name_parent_backslash.rsbank` | `..\evil.wav` | +| `hostile_name_subdir_slash.rsbank` | `sub/evil.wav` | +| `hostile_name_drive_absolute.rsbank` | `C:\Windows\evil.wav` | +| `hostile_name_unc_absolute.rsbank` | `\\srv\share\evil.wav` | + +Nested `Sample::relativePath` — a path by design, refused only for traversal and +absolute forms (`isValidNestedSamplePath`): + +| File | `relativePath` | Guard that fires first inside the codec | +|---|---|---| +| `hostile_path_dotdot_slash.rsbank` | `bank/../../evil.wav` | `isValidNestedSamplePath` | +| `hostile_path_dotdot_backslash.rsbank` | `bank\..\evil.wav` | `isValidNestedSamplePath` | +| `hostile_path_rooted.rsbank` | `/etc/evil.wav` | `BankModel::add`'s absolute-path rejection, which drops the record and leaves the nested blob holding zero samples | +| `hostile_path_drive_absolute.rsbank` | `C:\Windows\evil.wav` | as above | +| `hostile_path_unc_absolute.rsbank` | `\\srv\share\evil.wav` | as above | + +Both guards are inside the codec and both refuse the whole package, so the security +property is the same either way; the split is recorded because a change to either guard +alone would still leave these fixtures passing. + +### The round-trip anchor + +`v1_shipping.rsbank` doubles as it: the file **is** a real export, so importing it and +exporting the resulting bank closes export → import → export over frozen bytes. Entry +names may legally change across the trip (the importer re-spells a bank file, the +exporter mints its own transport name); the payload bytes may not. diff --git a/tests/fixtures/package_compat/additive_forward.rsbank b/tests/fixtures/package_compat/additive_forward.rsbank new file mode 100644 index 0000000000000000000000000000000000000000..656d9dd16efdb3e689eee997b95c6e23445c68cd GIT binary patch literal 1283 zcmc&z-%Aux6dtuGBEq1+2O+siSnLin`$N|)DAyk;rii;_U`E%OyY8?v_ugjitjf0F zLk~R!1w}*@6!;(%6cJJM10RHfA|eV3d=LtXh$txV!H3SBRayN9x-c;JyXSuAyWcry zD4pu7RurWQzRmFIYMZJ(uThjDB9olCBEy)Ck|^s7-z6x7#xTtfV21$K!5of7LkDxY z@4->|0x?liZ;i)vO)fyGa4q7Y2-9)c7m+8hE7HVZ)C3Q7H4>MWm}^K>1t16-+3A?28O>ID>2vO}=Iov>cMQ#s_={B`ZS9XeXfz@I(mZ zWh0>OMn{^=m>6JEX$#n3Gsf2}3}V5!APbmclLE0pSP^l8^POWT8P?SG`ammCI$m(7 zN7R-|@$$^}v%nD6W58$F7U!~eh3gaW+#+sZ9~4eN3-^>eaM%cX-UwPAywpK7cUgf@ zOpT!49J>%+?zi66_9}xAo1N^)VT-0I=3b6LfdC16t2-&f1;2@z0hgdD9C*K+B{@l& zRvImppx&O%WJO?{Hny)kfl@)-P;Ym46U_8*`?0R9BN}Rh5wB9FpTN&Vl~S$LDkc_K z&i(X-O_$DH*?#Tfja|2{&eYG|yxnwn_I~K$y~nXe+ksl}d*dH=!E!+P)|37{Iwz2>K literal 0 HcmV?d00001 diff --git a/tests/fixtures/package_compat/hostile_name_dotdot.rsbank b/tests/fixtures/package_compat/hostile_name_dotdot.rsbank new file mode 100644 index 0000000000000000000000000000000000000000..30af026d053a84217514e7c0e7caa7fa32cbb20c GIT binary patch literal 624 zcmWFvcJgLqU|;}YRv{StFim8_IJ@{3C{b5fOblu|1S@{3AR zQ%}q@V4L|@WnpaYknOdx56} zrCJ!6ni?CXnk1$Hm1X9oq*f?dRV$UH78Pga=K&4ZQ7TT%Ey#hInF+K{DLJt~Hv{gF zl+5CSoWx2D2NtE~B$i~Br3NH|ELO_SOwQITPb>r4Qk-8}l$`3Dp91s|&^gKZc_l!1 zdq7+j4mBC*9Ydhnpf5u*b5n~;fDVCp7wlVu+SuCKSfFt^`6VD9$JPdUy1A(U zgDl)J%rz~yL_q)?o{S6)yMc^yMg}GZ0fv;sl0;A-H}tbi?wH0iYrnvG}*i3fX$Hu$6ZdJIOlWe!qt$QH}1qdc>Ai`TFBW%^wDa z7G@6i9xegCDS{HBb0idGmdI-;uTeA5+M;V=xW~l7;)u0}{TZhKw=13zzIOr=LY{{StFim8_IJ@{3C{b5fOblu|1S@{3AR zQ%}q@V4L|@WnpaYknOdx56H8xB&NlXJO1L{nzP_n94DoZUY&dkpP+NYyb zoS0jX1GOd-=me$Y!~)$6xZ6@PiwklRD>2+$l$w)Rl3A7-kO;C^DLXSc8|s+i{L-T2 zRNwp*peKRONzTtJ0eZy);;L|{$v__(0$r3`l$r?ict~b$YHOefU`N^~aBzKMV{l%pB}JTmpPk1SLf0NGQlGk=IaOqh_GBMc2Y`kBNiD5o-_o iGfn|+S3Dzp?*t@-JPFT;eiK)a^d+?-^H0t|+YbQC&hE_s literal 0 HcmV?d00001 diff --git a/tests/fixtures/package_compat/hostile_name_parent_backslash.rsbank b/tests/fixtures/package_compat/hostile_name_parent_backslash.rsbank new file mode 100644 index 0000000000000000000000000000000000000000..bf100cb66f871076dae8c3fbe8f37363cde773e1 GIT binary patch literal 634 zcmWFvcJgLqU|;}YRv{StFim8_IJ@{3C{b5fOblu|1S@{3AR zQ%}q@V4L|@WnpaYknOdx56&Idmy?>8UXr0?WeifD zkys35Dp{nY8JneA7?_$G8>X5hrU8{@=B1=oC|Ol2m8BLHXXfVtti_@% literal 0 HcmV?d00001 diff --git a/tests/fixtures/package_compat/hostile_name_parent_slash.rsbank b/tests/fixtures/package_compat/hostile_name_parent_slash.rsbank new file mode 100644 index 0000000000000000000000000000000000000000..c8145cb391f0f54b0f35189f0f7aacaaf6f36d1d GIT binary patch literal 633 zcmWFvcJgLqU|;}YRv{StFim8_IJ@{3C{b5fOblu|1S@{3AR zQ%}q@V4L|@WnpaYknOdx56jRI;J?kv?w{%H$MgFHK22n z^Yco89`Jy;DjaGu&|ii?7bO>^CIY=1l9`)YTmp0m#NS|V8`Q?u*2V(emy=%t@_KA- zkf)oQ3NY-#9m8DHa!V8hzyZt1(6AfGC}(6~Vh~_RNi0bOg?2+f%jAw}JhLXu6{StFim8_IJ@{3C{b5fOblu|1S@{3AR zQ%}q@V4L|@WnpaYknOdx56&Idmy?>8UXr0?WeifD zkys35Dp{nY8JneA7?_$G8>X5hrU8{@=B1=oC|Ol2m8BLHXXfVtttz4{ literal 0 HcmV?d00001 diff --git a/tests/fixtures/package_compat/hostile_name_unc_absolute.rsbank b/tests/fixtures/package_compat/hostile_name_unc_absolute.rsbank new file mode 100644 index 0000000000000000000000000000000000000000..d50717ce47a2286bf550d7728359b062f84750ce GIT binary patch literal 646 zcmWFvcJgLqU|;}YRv{StFim8_IJ@{3C{b5fOblu|1S@{3AR zQ%}q@V4L|@WnpaYknOdx56sWAO~twCeRg1$%zHJ8F1&NWEL0XBvxWLyC^j$u_UuBH6RgWu~K$sayHa4#rdU0 z$*I2iDL{V$os*oOR|52n2gFt3P?Le4Gz7XRxhORe==YG!+|=R{phF;m01g9#+SuCK zSfKlI@=HMB5L+AM>E@;a49IZDFxRx)5(NQpC^IrN>;^K*85x)u1Q=2hOA1^Au(`4_C12#tv9CtZ=;+)T=3s*yK-nbL<;KAdR z=PzF6y!-I6|+^%>=_}&Rf2ze5o5&b5vAn8kLMdqKJfwmt2LS*np literal 0 HcmV?d00001 diff --git a/tests/fixtures/package_compat/hostile_path_dotdot_backslash.rsbank b/tests/fixtures/package_compat/hostile_path_dotdot_backslash.rsbank new file mode 100644 index 0000000000000000000000000000000000000000..700a42cad9f98a7a1d38ad5919b81bd87ec2121c GIT binary patch literal 640 zcmWFvcJgLqU|;}YRv{StFim8_IJ@{3C{b5fOblu|1S@{3AR zQ%}q@V4L|@WnpaYknOdx56&Idlard4UXr0?Weif9kys35 zDp{nY8JneA7?_$G8>X5hrU8{@=B1=oC|Ol2m8BLHXXfVtta61Y(xT*4-~1Gy z4}lI#&d)0W`oja_zHq1=K<^m>-IiRGnh5lDNM>$oaS6~#5buM1Z%`XsTN?{>X-<9# zCt#U`C?8+MxP-EqL?$bsW7r%#;oxpd)b$juvfVjetrobvp|tDJWq zK9+p_@uTJs149cl2YU~f0N)fr3DG$c3NlONHI&z=8E9?MwJ_Xc;$U&a+Qa^gQ-IqQ d&j{Z;0SO^b!ZV`Z#1$lcNv+8IlQYov0{}{StFim8_IJ@{3C{b5fOblu|1S@{3AR zQ%}q@V4L|@WnpaYknOdx56&Idlard4UXr0?Weif9kys35 zDp{nY8JneA7?_$G8>X5hrU8{@=B1=oC|Ol2m8BLHXXfVtt267PXT%n z=(6Peyb_>CJRlAXhgt&kpCQn3$wjG&KyQa+=B5^x0Nn)fKiK;QwXwCeu|TKh41FVXkSpB?q2DSrg`p zES#}KX2pWlD(hEl(%H6Qr^((O2W*ZUIPP-##5tc!7p{ifym2Sy!Gp&s&tJUCdH3OC z$=4r0YW^@Vv@mn9_izdDO%aq3og<+jvqWA)d5xNZ))rk0!#ySr7Duc-?9Vs_xLxs# b@Vyg|5b`8EBl=BTLDHAhip)Pb18qM5yF%>L literal 0 HcmV?d00001 diff --git a/tests/fixtures/package_compat/hostile_path_drive_absolute.rsbank b/tests/fixtures/package_compat/hostile_path_drive_absolute.rsbank new file mode 100644 index 0000000000000000000000000000000000000000..08fead86286e59e252fa218c12783f1ad0051b62 GIT binary patch literal 643 zcmWFvcJgLqU|;}YRv{StFim8_IJ@{3C{b5fOblu|1S@{3AR zQ%}q@V4L|@WnpaYknOdx56&Idlard4UXr0?Weif9kys35 zDp{nY8JneA7?_$G8>X5hrU8{@=B1=oC|Ol2m8BLHXXfVttOz-)&_aHxv2m{G2AiCH7&P9K>!@Yj0_FCfsArS1||jphLpsTL{PXl^s`Lv zn8q_}!d#JsGnUA#Sg=}U{fbRG+cxYp*}LO_&5;AgT~41k=X2@8)sUMv?!-KJ@Hpl9 zi&r`CK71_s`r}8<9|ndNW)AiqE&;wNf)b*0Bot(p$ZIICQ8UomqHAHe$Hc+nh_#3P i8K(fZE1nU)cLEYZo`h#azlkeI`jT3a`6p+f?FRs4`0mXB literal 0 HcmV?d00001 diff --git a/tests/fixtures/package_compat/hostile_path_rooted.rsbank b/tests/fixtures/package_compat/hostile_path_rooted.rsbank new file mode 100644 index 0000000000000000000000000000000000000000..359c3d1f4bab8c162de0bbd3130cc2fcacdcd457 GIT binary patch literal 635 zcmWFvcJgLqU|;}YRv{StFim8_IJ@{3C{b5fOblu|1S@{3AR zQ%}q@V4L|@WnpaYknOdx56&Idlard4UXr0?Weif9kys35 zDp{nY8JneA7?_$G8>X5hrU8{@=B1=oC|Ol2m8BLHXXfVttNr#@5!x0v(x?Ujp)c zY;BOIo0|$S^uis(T+?z(6a>IQ%gE5M8^|bUWME@<+~xF%b3T_YTn)K-<4(+j2ai*pzj&4N?!(8D zuRng&{9#~dVdh}(;S%7RA}Ap`M?yhniM)pL8Z`s0ExHzldrTZGj#zuxpK%IsyW$z) adnX_v{StFim8_IJ@{3C{b5fOblu|1S@{3AR zQ%}q@V4L|@WnpaYknOdx56&Idlard4UXr0?Weif9kys35 zDp{nY8JneA7?_$G8>X5hrU8{@=B1=oC|Ol2m8BLHXXfVttmL?0dZwG)Fz-O4S_C9E=o-V`aL8wH?_C~=rBkifWyF`Hnz4l z7UnvG}*i3fX$Hu$6ZdJIOlWe!qt$QH}1qdc>Ai`TFBW%^wDa7G@6i9xegCDS{HBb0idGmdI-;uTeA5+M;V=xW~l7;)u0} j{TZhKw=13zzIOr=LY{ literal 0 HcmV?d00001 diff --git a/tests/fixtures/package_compat/refuse_structural.rsbank b/tests/fixtures/package_compat/refuse_structural.rsbank new file mode 100644 index 0000000000000000000000000000000000000000..5e751b34975ac8bac171236b4a1108d5f936638e GIT binary patch literal 1207 zcmc&z&ubGw6y91z5s?ZCJrpd{B2|)QH@})jL7T+bQd>$=tWYaXcPGt~-JNlElD5P^ z4<0-S3W|s*DD+TJP((ygp@)KkA|eV3Jrooa5m8X+p$Fe=+7kN@ILpG!H*eni-uK?i z=8}CI6h)E$Cir!=OVxOzqEyfrc8depAt-@T)L~e(rYL764+RjJWRweHqJ-WVjp&+O zfVzTP#6yXZ3UX@?iC9TPEZvJb}5$5reuWWTUI0s7%gWL&7QmzR@5%4G!Xd@}_^# zyFywufCR@z@kq&%4n|O=ySuvzgeu8ovWi9n2neS66iR4nC%7^S*ma4Wq9s?<<(tJq z(u@1z*5q5GpsAV`imh9#A}0wJ5Lenn!=bMF)d|rs6D3ZDh{GsE3u!W&RnT%s&Ke&m zIVe$QT0_&srGUo+s7Qc2%CtFBWJ<*Vl}uwGgH0)0lhC&X;ru8dicQMI24WS&2rj0_ zP$HdDJHyOakHK|KQg4BKK%cA?-U0-9Td`|?5I1TR1Wn;U{&FhhOlVqlv|5FFdp4C7fx%eczVZf21$NoqOs2g-QHHyZ9nL$VwF$4t249yt(A4R7PvfwmHo8{JYutV7>?Xm2j#cpeoI+^YCgV6szIW473Ah4dL zr}wm2=CcRS^ISerhCv)K{pfie{Gckoa3>&xX}xRE4yLtkS_3|aV^eD@IAq~Ci9_as zP-V3g2C`KL>M~f$q6`kWr4nP+;@wxM_e((^Po6DxDEHcs>aGLSI@`Xysna(Jh8MF{W;HAwFwP z?qcHa%LS?p)g2g%+Ac*=D^700V3;^_{DwpU0e#cF`6Yt1H_!Vbd>gW}4_0GQezejN zte~4rCT9}V<#~RFZy!kDyFNk2f-zAQTU1I^)4Emm&hG-Plb80#y`1kRfdz~WNB^Qf F@*hUkj6(nb literal 0 HcmV?d00001 diff --git a/tests/fixtures/package_compat/trunc_manifest_length.rsbank b/tests/fixtures/package_compat/trunc_manifest_length.rsbank new file mode 100644 index 0000000000000000000000000000000000000000..3ab399a432a65b3efd737e41450bec688900edd4 GIT binary patch literal 23 acmWFvcJgLqU|;}YRv*o)B>OY literal 0 HcmV?d00001 diff --git a/tests/fixtures/package_compat/trunc_one_short.rsbank b/tests/fixtures/package_compat/trunc_one_short.rsbank new file mode 100644 index 0000000000000000000000000000000000000000..79b4f3bdd40d5db188265100388d556239086724 GIT binary patch literal 1206 zcmc&z-%C_M6uxFrM1(4?c9}uFB>=(7?dVnKS2n=R4oY zWK;chilWH93AV1rRgKpxN*Rq|uP}&Rf(+C}U4})di*i=O#iBO@kV^Vo+YN?N&C8%b)3L`)}foC7!8v&bYe z%BV!RZ&MFagprS3=Bz}vAuBR5i^$mU!Lohk;Hed){a_5WP7tvdo;Y!^uuCM3WnxqV z#-~MYk^yQ#b#aIvTr=1khkF_M0&|fiCiN_+MpvT=S)8?IiK$=+oYL$x1V{$OTfs%| zh-#HE5*(X_BSl*V7(wOE&dv$|DyLGZ3K|U~0NBl^kfEvV5X#JB&m&G3EqbD+-V7EJ zUNRuJO5Yj;P1Uq$+q$(%a*AL9d8JPx7VW5A?T`&KQRHNZxQs%!s3xmfHQFx8+T(*o z7a29KRckl#DAR3LyI^)$PZWJ<*#l}Ouw28U9%Mq%I!#06P^6o-_E1Hj6N5nSjV zLq?XvS%Yc-S+8elyI$JsXIuZ&hAZBlzYfyYkrfvL0rBo1lyE9PT`JIOmE`Lok+g z%JdUBOw=hGlm^AZ0?S#QzOeDqxhvbQUA(dL*43G&*_*dp?#|v1KfL!i{_N4*f%&=T z-7jCfKKk~}`=O5?7EUZKezLxN{pS4m`Ahs+S~|0=C}-=pY(BrWVf*DBjk~TlhMHz} zw>IC|qwRYTN$OAIhm!OAdk?-!rF-7>9UoXopBnl+oFDmql8yZ)llHRnpY#6($oHG0 literal 0 HcmV?d00001 diff --git a/tests/fixtures/package_compat/trunc_payload_middle.rsbank b/tests/fixtures/package_compat/trunc_payload_middle.rsbank new file mode 100644 index 0000000000000000000000000000000000000000..ac9ddd28189fbb8eb5b3f465871057acbef8f7bd GIT binary patch literal 1057 zcmZ8gO=uHA6y91z5s?ZCJroSnB2|)QH-Al6LGxp5sg}|dd#F8(yOUr|9FI*Vy_6uSf&D9>DuMI_HW9{4DV$Q);$AQm$8fmBl0eyrMNRh`8(_~Q^dBPz&9NY@JX;fOL;IgutAEtN{>B#F!5W>bsIA)}6J z#Pe-NAw?AV*yYZ8WE-*~6Z4pi4IeDq=MJ7+z=NdGJm2jR&h4ip#1F6In9jhxj(mYVQ6eUz7F46F@supi+OQ;4FoaHH{2T`gTl}khzqj-DGsR-2Y}TPC%9TTi;S42 z{*Mo^LP*ySAN9DJZU-;loS+O5VtNwdnU2l5RH5(!0-W2#3+aQ#2@vB@Q)H87s9rM^ zFL0>?XzsBZp_rOszd826y%dUUxFxTq5q!&Y`3kmaiDK?o7&HhhF~4<_Fg)O!SQEe! zh{A>X|`=y;-yRYr(-gm1z(zCd~zxVzD W?a<>`Mt_k!mRUJGbo6a*toRRd_EEtA literal 0 HcmV?d00001 diff --git a/tests/fixtures/package_compat/trunc_payload_start.rsbank b/tests/fixtures/package_compat/trunc_payload_start.rsbank new file mode 100644 index 0000000000000000000000000000000000000000..c7a3a720e6fa7d7b666736500457c104d8832f5f GIT binary patch literal 907 zcmZ8g+iK%55Y4jef0WsmT}X_bd;OLRd!aOxq_7XA52`&$L~Kbb%V~*#3a%$d=0HNE@PYPJ0L1z!^Nqu73KwHkU6YPS$2Q%W;k8sTnc+M2qeHu1q|>o})0 z84dbL>=#hwtmKN)r-sz&;!b9}=vC}-r(m^lTS|MLx)l|EX`HWnLmtP2K`-IHSc4{2 z&Id{x+Ow^sR!GsNN|dH}k1UauGGRJCHZJ6{GKF}1S9%BV=G!Z~?>Mqt2q*XaAbeuF z5>x5gW_+)6RM$uRuw4qZ!M>r&3F}szX_cdzBt4bGnq_|5Y$Yk(<42%2?FQ#jnX(8Ayl>%N^vpMwQ^VWE`{^*#=+c^ zeHRIF6vy4+hjli2%EBS9=L~w?+v_z#Hep_xa-4gE2LT}VXCc0gMf6Z^Ulk#@SF2TxP+l7QNMo`{!PU3 X@%eZ}zpFy~j^JT@9AAHeR)O~qU_lnw literal 0 HcmV?d00001 diff --git a/tests/fixtures/package_compat/trunc_version_pair.rsbank b/tests/fixtures/package_compat/trunc_version_pair.rsbank new file mode 100644 index 0000000000000000000000000000000000000000..c66f6dba61f1ff8d2b822b5776d6a12453a09422 GIT binary patch literal 10 RcmWFvcJgLqU|?Wm000V30W|;s literal 0 HcmV?d00001 diff --git a/tests/fixtures/package_compat/trunc_writer_semver.rsbank b/tests/fixtures/package_compat/trunc_writer_semver.rsbank new file mode 100644 index 0000000000000000000000000000000000000000..430300ebb7bc08fe2ec682b0422f90f8e45637ef GIT binary patch literal 18 VcmWFvcJgLqU|;}YRv4?c9}uFB>=(7?dVnKS2n=R4oY zWK;chilWH93AV1rRgKpxN*Rq|uP}&Rf(+C}U4})di*i=O#iBO@kV^Vo+YN?N&C8%b)3L`)}foC7!8v&bYe z%BV!RZ&MFagprS3=Bz}vAuBR5i^$mU!Lohk;Hed){a_5WP7tvdo;Y!^uuCM3WnxqV z#-~MYk^yQ#b#aIvTr=1khkF_M0&|fiCiN_+MpvT=S)8?IiK$=+oYL$x1V{$OTfs%| zh-#HE5*(X_BSl*V7(wOE&dv$|DyLGZ3K|U~0NBl^kfEvV5X#JB&m&G3EqbD+-V7EJ zUNRuJO5Yj;P1Uq$+q$(%a*AL9d8JPx7VW5A?T`&KQRHNZxQs%!s3xmfHQFx8+T(*o z7a29KRckl#DAR3LyI^)$PZWJ<*#l}Ouw28U9%Mq%I!#06P^6o-_E1Hj6N5nSjV zLq?XvS%Yc-S+8elyI$JsXIuZ&hAZBlzYfyYkrfvL0rBo1lyE9PT`JIOmE`Lok+g z%JdUBOw=hGlm^AZ0?S#QzOeDqxhvbQUA(dL*43G&*_*dp?#|v1KfL!i{_N4*f%&=T z-7jCfKKk~}`=O5?7EUZKezLxN{pS4m`Ahs+S~|0=C}-=pY(BrWVf*DBjk~TlhMHz} zw>IC|qwRYTN$OAIhm!OAdk?-!rF-7>9UoXopBnl+oFDmql8yZ)llHRnzw`eEWfhzc literal 0 HcmV?d00001 diff --git a/tests/package_fixtures.h b/tests/package_fixtures.h new file mode 100644 index 0000000..b7cab67 --- /dev/null +++ b/tests/package_fixtures.h @@ -0,0 +1,25 @@ +#pragma once +// The one accessor over the frozen package-compat corpus (tests/fixtures/package_compat). +// REASAMPLER_PACKAGE_FIXTURE_DIR is a compile-time absolute path defined by each +// consuming test target: the corpus is source-tree data, and a test's working directory +// under ctest differs between single- and multi-config generators, so no relative +// spelling reaches it from both. +// +// Every caller must check the returned size: a fixture that failed to open reads as an +// empty buffer, which a "this must be Malformed" assertion would otherwise pass. + +#include +#include +#include +#include +#include + +inline std::string packageFixturePath(const std::string& name) { + return std::string(REASAMPLER_PACKAGE_FIXTURE_DIR) + "/" + name; +} + +inline std::vector packageFixtureBytes(const std::string& name) { + std::ifstream f(packageFixturePath(name), std::ios::binary); + return std::vector(std::istreambuf_iterator(f), + std::istreambuf_iterator()); +} diff --git a/tests/test_package_compat.cpp b/tests/test_package_compat.cpp new file mode 100644 index 0000000..c262a06 --- /dev/null +++ b/tests/test_package_compat.cpp @@ -0,0 +1,334 @@ +// Standalone tests over the FROZEN package-compat corpus — no REAPER, no framework. +// Nothing here builds a package: every byte comes off disk exactly as committed, which +// is the only shape in which a later format change can be caught breaking a +// compatibility direction. A test that re-derived its own fixture would prove only that +// the codec agrees with itself. See tests/fixtures/package_compat/README.md. +// +// The values pinned below are the FIXTURE's facts, not this build's — the writer semver +// especially must never be compared against version::stampVersion(), or a version bump +// would silently re-anchor the corpus. + +#include "../src/core/package/bank_package.h" + +#include +#include +#include +#include + +#include "../src/core/package/package_format.h" +#include "../src/core/package/package_manifest.h" +#include "package_fixtures.h" + +using namespace reasampler::package; +using namespace reasampler::model; + +static int g_fail = 0; +#define CHECK(cond) do { if(!(cond)) { \ + std::printf("FAIL line %d: %s\n", __LINE__, #cond); ++g_fail; } } while(0) + +// --- what the v1 fixture's bytes say ----------------------------------------- + +static constexpr const char* kV1File = "v1_shipping.rsbank"; +static constexpr const char* kV1WriterSemver = "1.4.0"; +static constexpr const char* kV1EntryName = "kick.wav"; +static constexpr const char* kV1EntryHash = "8df36e805531e4af"; +static constexpr std::uint64_t kV1PrefixSize = 907; +static constexpr std::uint64_t kV1PayloadLength = 300; +static constexpr std::uint64_t kV1TotalSize = 1207; + +// Every Sample field the shipping build wrote, with every optional PRESENT — so +// "every known field intact" is a claim about the whole record, not a sampled few. +// relativePath is the BARE transport name: export_plan normalizes it (see +// src/core/package/CLAUDE.md), so a package carrying a directory component here would +// mean the fixture predates that rule. +static Sample expectedV1Sample() { + Sample s; + s.id = "cap-kick"; + s.displayName = "Kick (wet)"; + s.relativePath = "kick.wav"; + s.sourceMode = SourceMode::RazorArea; + s.sourceRange = {1.25, 3.5, 480.0, 1920.0}; + s.trackGuids = {"{AAA}", "{BBB}"}; + s.wetDry = 0.75; + s.channelCount = 1; + s.sampleRate = 48000; + s.lengthSeconds = 0.0026666666666666666; + s.lengthBeats = 0.00533; + s.captureTempo = 120.5; + s.captureTimeSigNum = 7; + s.captureTimeSigDenom = 8; + s.key = "F#m"; + s.rootNote = 60; + s.loop = LoopPoints{8, 120}; + s.levels = {-0.3, -12.7, -14.0}; + s.clipped = true; + s.tier = Tier::Archive; + s.contentHash = "Wcompatcorpus0001"; + s.provenance = Provenance{"cap-parent", "fx-snapshot"}; + s.createdTimestamp = 1754000000; + return s; +} + +static PackageManifest expectedV1Manifest() { + PackageManifest m; + m.bankDisplayName = "Compat Corpus"; + m.exportTimestamp = 1754100000; + m.entries.push_back({kV1EntryName, kV1PayloadLength, kV1EntryHash, expectedV1Sample()}); + m.slots.append("cap-kick"); + return m; +} + +// A fixture that failed to open reads as an empty buffer, and an empty buffer decodes +// Malformed — which would let half this file pass vacuously. Every suite loads through +// here. +static std::vector load(const char* name, std::size_t expectedSize) { + std::vector bytes = packageFixtureBytes(name); + if (bytes.size() != expectedSize) { + std::printf("FAIL: fixture %s is %zu bytes, expected %zu (path: %s)\n", name, + bytes.size(), expectedSize, packageFixturePath(name).c_str()); + ++g_fail; + } + return bytes; +} + +static bool containsToken(const std::vector& bytes, const std::string& token) { + const std::string text(reinterpret_cast(bytes.data()), bytes.size()); + return text.find(token) != std::string::npos; +} + +static std::uint32_t le32At(const std::vector& bytes, std::size_t at) { + return static_cast(bytes[at]) | + (static_cast(bytes[at + 1]) << 8) | + (static_cast(bytes[at + 2]) << 16) | + (static_cast(bytes[at + 3]) << 24); +} + +// --- direction 1: the shipping build reads what it wrote --------------------- + +// The header pair as the bytes carry it, classified without going through decode — the +// ladder rule stated against the file rather than against the decoder's reading of it. +static PackageReadability classifyFixture(const std::vector& bytes) { + return classifyPackageVersion(le32At(bytes, 4), le32At(bytes, 8)); +} + +static void testV1FixtureDecodesToTheRecordTheShippingBuildWrote() { + const std::vector bytes = load(kV1File, kV1TotalSize); + CHECK(classifyFixture(bytes) == PackageReadability::Readable); + + const DecodedPackage dec = decodePackage(bytes, bytes.size()); + CHECK(dec.status == PackageReadability::Readable); + CHECK(dec.header.formatVersion == 1); + CHECK(dec.header.minReaderVersion == 1); + CHECK(dec.header.writerVersion == kV1WriterSemver); + + CHECK(dec.manifest.entries.size() == 1); + CHECK(dec.manifest.bankDisplayName == "Compat Corpus"); + if (dec.manifest.entries.size() == 1) { + CHECK(dec.manifest.entries[0].fileName == kV1EntryName); + CHECK(dec.manifest.entries[0].byteLength == kV1PayloadLength); + CHECK(dec.manifest.entries[0].byteHash == kV1EntryHash); + CHECK(dec.manifest.entries[0].sample == expectedV1Sample()); + } + CHECK(dec.manifest == expectedV1Manifest()); + + CHECK(dec.prefixSize == kV1PrefixSize); + CHECK(dec.layout.size() == 1); + if (dec.layout.size() == 1) { + CHECK(dec.layout[0].name == kV1EntryName); + CHECK(dec.layout[0].offset == kV1PrefixSize); + CHECK(dec.layout[0].length == kV1PayloadLength); + } +} + +// --- direction 1: a NEWER additive writer still reads ------------------------ + +// formatVersion N+1, minReaderVersion unchanged: the whole reason two integers exist. +// The fixture carries three keys this build has never heard of — one at the manifest +// root, one on the entry, one inside the nested Sample blob — and must still decode to +// exactly what the v1 fixture decodes to. +static void testAdditiveForwardFixtureReadsWithEveryKnownFieldIntact() { + const std::vector bytes = load("additive_forward.rsbank", 1283); + + // Non-vacuity: the unknown keys are genuinely in the bytes, so the equality below + // is "skipped without error", not "there was nothing to skip". + CHECK(containsToken(bytes, "\"exportTool\"")); + CHECK(containsToken(bytes, "\"futureEntryKey\"")); + CHECK(containsToken(bytes, "\"futureSampleKey\"")); + + CHECK(classifyFixture(bytes) == PackageReadability::Readable); + + const DecodedPackage dec = decodePackage(bytes, bytes.size()); + CHECK(dec.status == PackageReadability::Readable); + CHECK(dec.header.formatVersion == kPackageFormatVersion + 1); + CHECK(dec.header.minReaderVersion == kPackageMinReaderVersion); + CHECK(dec.header.writerVersion == "1.9.0"); + + // Every known field, end to end: same manifest the v1 fixture yields. + CHECK(dec.manifest == expectedV1Manifest()); + CHECK(dec.layout.size() == 1); + if (dec.layout.size() == 1) CHECK(dec.layout[0].length == kV1PayloadLength); +} + +// --- direction 2: a structural newer writer is refused whole ----------------- + +static void testRefuseFixtureIsTooNewAndStillNamesTheWriter() { + const std::vector bytes = load("refuse_structural.rsbank", kV1TotalSize); + CHECK(classifyFixture(bytes) == PackageReadability::TooNew); + + const DecodedPackage dec = decodePackage(bytes, bytes.size()); + CHECK(dec.status == PackageReadability::TooNew); + // The three facts the refusal message owes the user. + CHECK(dec.header.formatVersion == kPackageFormatVersion + 1); + CHECK(dec.header.minReaderVersion == kPackageFormatVersion + 1); + CHECK(dec.header.writerVersion == "1.9.0"); + + // Nothing else: no manifest, no layout, no partial success. + CHECK(dec.manifest == PackageManifest{}); + CHECK(dec.layout.empty()); + CHECK(dec.prefixSize == 0); + + // The refusal is a DECISION, not an inability: this fixture's body is the v1 + // fixture's own manifest, which parses. Walk the frozen header by hand to lift it + // out — a reader at this version is forbidden from doing so, which is the point. + const std::size_t manifestLenAt = 16 + le32At(bytes, 12); + const std::uint32_t manifestLen = le32At(bytes, manifestLenAt); + const std::string body(reinterpret_cast(bytes.data() + manifestLenAt + 4), + manifestLen); + CHECK(deserializeManifest(body).has_value()); +} + +// --- truncation: Malformed at every site, never TooNew ----------------------- + +// One fixture per DISTINCT decode failure site rather than an arithmetic spread. The +// RSBK layout is derived from the manifest's entries, not stored as its own section, so +// the site a "mid-layout" cut maps to is the payload boundary: the manifest parses, the +// layout computes, and the exact-size proof is what fails. +struct Truncation { + const char* file; + std::size_t size; // the cut offset — the committed file IS this many bytes + const char* site; +}; + +static const Truncation kTruncations[] = { + {"trunc_magic.rsbank", 2, "inside the 4-byte magic"}, + {"trunc_version_pair.rsbank", 10, "inside the frozen header's minReaderVersion u32"}, + {"trunc_writer_semver.rsbank", 18, "inside the frozen header's writer semver"}, + {"trunc_manifest_length.rsbank", 23, "inside the manifest-length u32"}, + {"trunc_manifest_body.rsbank", 466, "inside the manifest JSON"}, + {"trunc_payload_start.rsbank", 907, + "at the payload boundary — manifest parses, layout computes, exact-size proof fails"}, + {"trunc_payload_middle.rsbank", 1057, "inside the first payload"}, + {"trunc_one_short.rsbank", 1206, "one byte short of the total"}, +}; + +static void testEveryTruncationIsMalformedNeverTooNew() { + for (const Truncation& t : kTruncations) { + const std::vector bytes = load(t.file, t.size); + const DecodedPackage dec = decodePackage(bytes, bytes.size()); + if (dec.status != PackageReadability::Malformed) { + std::printf("FAIL: %s (%s) classified %s, expected Malformed\n", t.file, t.site, + dec.status == PackageReadability::TooNew ? "TooNew" : "Readable"); + ++g_fail; + } + // A refusal never half-succeeds, at any site. + CHECK(dec.manifest.entries.empty()); + CHECK(dec.layout.empty()); + CHECK(dec.prefixSize == 0); + + // The shell hands decode a PREFIX plus the observed file size, not the whole + // file — so the verdict has to survive that call shape too, at every cut the + // incremental seam can actually satisfy. + const auto need = requiredPrefixSize(bytes); + if (need && *need <= bytes.size()) { + const std::vector head(bytes.begin(), + bytes.begin() + static_cast(*need)); + CHECK(decodePackage(head, bytes.size()).status == PackageReadability::Malformed); + } + } +} + +// --- hostile names: refused at decode, before any planner exists ------------- + +// The two fields carry DIFFERENT rules (src/core/package/CLAUDE.md): the entry name may +// not express a path at all, while the nested relativePath is a path and is refused only +// for traversal and absolute forms. Each fixture keeps the other field clean, so the +// refusal is attributable to the field under test. +struct HostileFixture { + const char* file; + std::size_t size; + const char* offending; // the form as it reads once JSON-unescaped + bool inEntryName; // false: the nested Sample's relativePath +}; + +static const HostileFixture kHostiles[] = { + {"hostile_name_dotdot.rsbank", 624, "..", true}, + {"hostile_name_parent_slash.rsbank", 633, "../evil.wav", true}, + {"hostile_name_parent_backslash.rsbank", 634, "..\\evil.wav", true}, + {"hostile_name_subdir_slash.rsbank", 634, "sub/evil.wav", true}, + {"hostile_name_drive_absolute.rsbank", 643, "C:\\Windows\\evil.wav", true}, + {"hostile_name_unc_absolute.rsbank", 646, "\\\\srv\\share\\evil.wav", true}, + {"hostile_path_dotdot_slash.rsbank", 641, "bank/../../evil.wav", false}, + {"hostile_path_dotdot_backslash.rsbank", 640, "bank\\..\\evil.wav", false}, + {"hostile_path_rooted.rsbank", 635, "/etc/evil.wav", false}, + {"hostile_path_drive_absolute.rsbank", 643, "C:\\Windows\\evil.wav", false}, + {"hostile_path_unc_absolute.rsbank", 646, "\\\\srv\\share\\evil.wav", false}, +}; + +// A backslash rides the manifest JSON doubled; the corpus table above spells the +// unescaped form, since that is what the naming rules are asked about. +static std::string jsonEscaped(const std::string& s) { + std::string out; + for (char c : s) { + if (c == '\\') out += '\\'; + out += c; + } + return out; +} + +static void testEveryHostileNameIsRefusedAtDecode() { + // The clean spelling both fields hold in the fixture they are NOT under test in — + // without this, a refusal could be coming from the wrong field. + CHECK(isValidEntryName("kick.wav")); + CHECK(isValidNestedSamplePath("kick.wav")); + + for (const HostileFixture& h : kHostiles) { + const std::vector bytes = load(h.file, h.size); + if (!containsToken(bytes, jsonEscaped(h.offending))) { + std::printf("FAIL: %s does not carry the form it is named for (%s)\n", h.file, + h.offending); + ++g_fail; + } + + // The format's own rule on the offending field, stated directly. + if (h.inEntryName) CHECK(!isValidEntryName(h.offending)); + else CHECK(!isValidNestedSamplePath(h.offending)); + + // Refused by the CODEC: decode yields no manifest at all, so there is nothing + // for a planner to have been handed. planImport takes a PackageManifest, and + // decode produced none. + const DecodedPackage dec = decodePackage(bytes, bytes.size()); + if (dec.status != PackageReadability::Malformed) { + std::printf("FAIL: %s classified %d, expected Malformed\n", h.file, + static_cast(dec.status)); + ++g_fail; + } + CHECK(dec.manifest == PackageManifest{}); + CHECK(dec.layout.empty()); + CHECK(dec.prefixSize == 0); + } +} + +int main() { + testV1FixtureDecodesToTheRecordTheShippingBuildWrote(); + testAdditiveForwardFixtureReadsWithEveryKnownFieldIntact(); + testRefuseFixtureIsTooNewAndStillNamesTheWriter(); + testEveryTruncationIsMalformedNeverTooNew(); + testEveryHostileNameIsRefusedAtDecode(); + + if (g_fail == 0) { + std::printf("package_compat_tests: all passed\n"); + return 0; + } + std::printf("package_compat_tests: %d failure(s)\n", g_fail); + return 1; +} diff --git a/tests/test_package_round_trip.cpp b/tests/test_package_round_trip.cpp new file mode 100644 index 0000000..3143a46 --- /dev/null +++ b/tests/test_package_round_trip.cpp @@ -0,0 +1,250 @@ +// The corpus driven through both verbs — no REAPER, no framework. Where +// test_package_compat asserts what the frozen bytes DECODE to, this file asserts what +// the import and export verbs DO with them: the payload bytes survive a full +// export -> import -> export, and every refusal in the corpus refuses before a planner +// or a filesystem write is reached. + +#include "../src/shell/package/import_landing.h" + +#include +#include +#include +#include +#include +#include + +#include "../src/core/package/bank_package.h" +#include "../src/shell/package/export_bank.h" +#include "../src/shell/package/package_path.h" +#include "../src/shell/persist/session.h" +#include "package_fixtures.h" + +using namespace reasampler; +namespace fs = std::filesystem; + +static int g_fail = 0; +#define CHECK(cond) do { if(!(cond)) { \ + std::printf("FAIL line %d: %s\n", __LINE__, #cond); ++g_fail; } } while(0) + +static const char* kTag = "1754000000"; +static const char* kImportedBankId = "bank-imported"; + +// --- scratch project --------------------------------------------------------- + +// One project directory per suite, torn down after, so no suite observes another's +// bank folder. +class Scratch { +public: + explicit Scratch(const std::string& name) { + std::error_code ec; + dir_ = pathToUtf8(fs::temp_directory_path(ec) / + utf8Path("reasampler_compat_" + name)); + fs::remove_all(utf8Path(dir_), ec); + fs::create_directories(utf8Path(dir_), ec); + } + ~Scratch() { + std::error_code ec; + fs::remove_all(utf8Path(dir_), ec); + } + const std::string& projectDir() const { return dir_; } + std::string bankDir() const { return package::bankFolderDir(dir_); } + std::string importPath() const { return dir_ + "/in.rsbank"; } + std::string exportPath() const { return dir_ + "/out.rsbank"; } + +private: + std::string dir_; +}; + +static std::vector readBytes(const std::string& path) { + std::ifstream f(utf8Path(path), std::ios::binary); + return std::vector(std::istreambuf_iterator(f), + std::istreambuf_iterator()); +} + +static void writeBytes(const std::string& path, const std::vector& bytes) { + std::ofstream f(utf8Path(path), std::ios::binary | std::ios::trunc); + f.write(reinterpret_cast(bytes.data()), + static_cast(bytes.size())); +} + +// Copies a committed fixture to the path the verb will be pointed at. Fails loudly on an +// empty read: an unreadable corpus would otherwise let every "must refuse" suite pass. +static bool stageFixture(const Scratch& scratch, const char* fixture) { + const std::vector bytes = packageFixtureBytes(fixture); + if (bytes.empty()) { + std::printf("FAIL: fixture %s read as 0 bytes (path: %s)\n", fixture, + packageFixturePath(fixture).c_str()); + ++g_fail; + return false; + } + writeBytes(scratch.importPath(), bytes); + return true; +} + +// Each entry's payload bytes, sliced out of a whole package file by its own layout. +static std::vector> payloadsOf(const std::vector& file) { + std::vector> out; + const package::DecodedPackage dec = package::decodePackage(file, file.size()); + if (dec.status != package::PackageReadability::Readable) return out; + for (const package::PackageEntrySpan& span : dec.layout) { + const auto begin = file.begin() + static_cast(span.offset); + out.emplace_back(begin, begin + static_cast(span.length)); + } + return out; +} + +// --- the sequence the import verb runs, minus REAPER ------------------------- + +static ImportLanding runImport(const Scratch& scratch, ReaSamplerSession& session, + int* outBirths = nullptr) { + LandedFileJournal journal; + ImportLanding landing = landPackage(scratch.importPath(), scratch.projectDir(), + session.book(), kTag, journal); + if (landing.outcome != ImportOutcome::Landed) return landing; + int births = 0; + const bool applied = applyImportedBank(session.book(), kImportedBankId, landing.plan, + [&births](const model::Sample&) { ++births; }); + CHECK(applied); + if (applied) journal.markIndexCommitted(); + if (outBirths) *outBirths = births; + return landing; +} + +// --- the round-trip anchor --------------------------------------------------- + +// The fixture IS the first export (written by the shipping build's own export verb), so +// importing and re-exporting it closes export -> import -> export over frozen bytes. +static void testV1FixtureReExportsByteIdenticalPayloads() { + Scratch scratch("roundtrip"); + if (!stageFixture(scratch, "v1_shipping.rsbank")) return; + + ReaSamplerSession session; + int births = 0; + const ImportLanding landing = runImport(scratch, session, &births); + CHECK(landing.outcome == ImportOutcome::Landed); + CHECK(births == 1); + CHECK(landing.plan.landCount == 1); + if (landing.plan.entries.size() != 1) { CHECK(false); return; } + + const std::vector> sourcePayloads = + payloadsOf(packageFixtureBytes("v1_shipping.rsbank")); + CHECK(sourcePayloads.size() == 1); + + // The landed file is the package's payload verbatim — the first half of the claim. + const std::string landed = scratch.bankDir() + "/" + landing.plan.entries[0].destFileName; + CHECK(readBytes(landed) == sourcePayloads[0]); + + ExportRequest req; + req.projectDir = scratch.projectDir(); + req.bankId = kImportedBankId; + req.destAbsPath = scratch.exportPath(); + req.exportTimestamp = 1754200000; + const ExportOutcome out = exportBank(session, req); + CHECK(out.status == ExportStatus::Written); + CHECK(out.entriesWritten == 1); + + // The second half: the re-export's payloads, byte for byte. Entry NAMES may legally + // differ across the trip — the importer re-spells a bank file and the exporter mints + // its own transport name (src/core/package/CLAUDE.md) — the payload bytes may not. + CHECK(payloadsOf(readBytes(scratch.exportPath())) == sourcePayloads); +} + +// --- the refusals, at the verb rather than the codec ------------------------- + +static void testRefuseFixtureRefusesTheWholeImportAndNamesTheWriter() { + Scratch scratch("refuse"); + if (!stageFixture(scratch, "refuse_structural.rsbank")) return; + + ReaSamplerSession session; + const BankBook before = session.book(); + LandedFileJournal journal; + const ImportLanding landing = landPackage(scratch.importPath(), scratch.projectDir(), + session.book(), kTag, journal); + + CHECK(landing.outcome == ImportOutcome::TooNew); + CHECK(landing.header.formatVersion == package::kPackageFormatVersion + 1); + CHECK(landing.header.minReaderVersion == package::kPackageFormatVersion + 1); + CHECK(landing.header.writerVersion == "1.9.0"); + // Nothing planned, nothing on disk, nothing in the index. + CHECK(landing.plan.entries.empty()); + CHECK(!fs::exists(utf8Path(scratch.bankDir()))); + CHECK(session.book() == before); +} + +// The same eight cuts test_package_compat classifies, driven through the verb's +// incremental prefix reader — the one caller that can ask requiredPrefixSize for more +// bytes than the file holds. +static const char* kTruncationFixtures[] = { + "trunc_magic.rsbank", "trunc_version_pair.rsbank", + "trunc_writer_semver.rsbank", "trunc_manifest_length.rsbank", + "trunc_manifest_body.rsbank", "trunc_payload_start.rsbank", + "trunc_payload_middle.rsbank", "trunc_one_short.rsbank", +}; + +static void testEveryTruncationRefusesTheImportAsMalformed() { + for (const char* fixture : kTruncationFixtures) { + Scratch scratch(std::string("trunc_") + fixture); + if (!stageFixture(scratch, fixture)) continue; + + ReaSamplerSession session; + const BankBook before = session.book(); + LandedFileJournal journal; + const ImportLanding landing = landPackage(scratch.importPath(), scratch.projectDir(), + session.book(), kTag, journal); + if (landing.outcome != ImportOutcome::Malformed) { + std::printf("FAIL: %s imported as outcome %d, expected Malformed\n", fixture, + static_cast(landing.outcome)); + ++g_fail; + } + CHECK(landing.plan.entries.empty()); + CHECK(!fs::exists(utf8Path(scratch.bankDir()))); + CHECK(session.book() == before); + } +} + +static const char* kHostileFixtures[] = { + "hostile_name_dotdot.rsbank", "hostile_name_parent_slash.rsbank", + "hostile_name_parent_backslash.rsbank", "hostile_name_subdir_slash.rsbank", + "hostile_name_drive_absolute.rsbank", "hostile_name_unc_absolute.rsbank", + "hostile_path_dotdot_slash.rsbank", "hostile_path_dotdot_backslash.rsbank", + "hostile_path_rooted.rsbank", "hostile_path_drive_absolute.rsbank", + "hostile_path_unc_absolute.rsbank", +}; + +static void testEveryHostileNameIsRefusedBeforeThePlannerRuns() { + for (const char* fixture : kHostileFixtures) { + Scratch scratch(std::string("hostile_") + fixture); + if (!stageFixture(scratch, fixture)) continue; + + ReaSamplerSession session; + const BankBook before = session.book(); + LandedFileJournal journal; + const ImportLanding landing = landPackage(scratch.importPath(), scratch.projectDir(), + session.book(), kTag, journal); + if (landing.outcome != ImportOutcome::Malformed) { + std::printf("FAIL: %s imported as outcome %d, expected Malformed\n", fixture, + static_cast(landing.outcome)); + ++g_fail; + } + // planImport is the ONLY producer of a non-empty plan and it runs after the + // decode — an empty one is how "refused before any planner" is observed here. + CHECK(landing.plan.entries.empty()); + CHECK(landing.plan.bankDisplayName.empty()); + CHECK(!fs::exists(utf8Path(scratch.bankDir()))); + CHECK(session.book() == before); + } +} + +int main() { + testV1FixtureReExportsByteIdenticalPayloads(); + testRefuseFixtureRefusesTheWholeImportAndNamesTheWriter(); + testEveryTruncationRefusesTheImportAsMalformed(); + testEveryHostileNameIsRefusedBeforeThePlannerRuns(); + + if (g_fail == 0) { + std::printf("package_round_trip_tests: all passed\n"); + return 0; + } + std::printf("package_round_trip_tests: %d failure(s)\n", g_fail); + return 1; +} From 1005c943a210a934cbc2fcbed5456ae2e8243acf Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 14:27:20 -0400 Subject: [PATCH 19/24] docs: record Phase E waves 1 and 2 as landed Backfills W1, which was skipped when it merged, and adds W2's two verbs. Notes the picker deviation: GetUserFileName both directions, not the spec'd Win32/SWELL split. --- docs/COMPLETED.md | 99 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 99 insertions(+) diff --git a/docs/COMPLETED.md b/docs/COMPLETED.md index f7f86bc..7cda133 100644 --- a/docs/COMPLETED.md +++ b/docs/COMPLETED.md @@ -942,3 +942,102 @@ own output, the three folder cases, collapsed-mono placement and summing, both m transitions waited out past a panel timer tick, undo, name/colour clone, and `GetProjectPathEx` against a non-default recording path — none of it is unit-testable and none has been run. + +### Ε-W1 — The contract, the filesystem, and the ledger's new kind + +Phase Ε's first wave: the `.rsbank` package contract, the filesystem/dialog seam +behind it, and a new tracking-ledger origin kind for package-sourced files — three +tracks, disjoint by directory, dispatched in parallel. + +**Ε-W1-T1 — `package-format`.** The pure `src/core/package/` codec for the +hand-rolled `RSBK` container (Ε-F1, ruled — no ZIP, no compressor, no link edge to +`vendor/WDL/WDL/zlib/`): a fixed little-endian header carrying two version +integers — `formatVersion` (what the writer emitted) and `minReaderVersion` (the +oldest reader that can read it safely) — a length-prefixed JSON manifest, and +payloads concatenated in manifest order. `classifyPackageVersion` answers +`Readable`/`TooNew`/`Malformed`; a `TooNew` header refuses whole, producing no +manifest, so the refusal can still name the writer's semver rather than +half-succeeding. Landed as three modules: `package_format` (the contract, the +version ladder, and three name-validation rules — `isValidEntryName`, +`sameEntryName`'s ASCII-case fold, `isValidNestedSamplePath`), `package_manifest` +(the manifest model + JSON codec, carrying the bank's `slot_map` and a whole-file +`hashBytes` digest per entry — deliberately not `hashWavContent`, which skips +chunks and so cannot answer "did these bytes survive"), and `bank_package` +(framing/layout arithmetic: `encodePackage`/`decodePackage`/`requiredPrefixSize`, +never holding or hashing a payload itself). Hostile input is refused, never UB, +at every byte offset. + +**Ε-W1-T2 — `package-fs-shell`.** `src/shell/package/`: streaming, atomic package +filesystem I/O (`package_io`'s `PackageFileWriter`/`PackageFileReader`, at most one +entry's payload materialized at a time, backed by a `.rsbanktmp` sibling that +reaches the destination only through a `commit()` rename — process-crash atomic, +not power-loss atomic, deliberately, since an `fsync` over a whole sample bank is a +real stall) and the rollback journal (`package_rollback`'s `LandedFileJournal`, +citing the `prune_fs.cpp` carve-out rather than restating it, disarmed only after +the caller's own write has returned success). **Deviation from spec:** the plan +called for asymmetric pickers — REAPER's `GetUserFileNameForRead` for import, Win32 +`GetSaveFileNameW`/SWELL `BrowseForSaveFile` for export, reasoning that the REAPER +API has no save picker. The landed `package_pickers` instead rides `GetUserFileName` +for both directions (mode 1 import, mode 0 export) — no platform split, since +`main.cpp` already aborts extension load if any needed API pointer fails to +resolve. Both pickers are `[verify — DAW]`, never exercised in a live REAPER +session. + +**Ε-W1-T3 — `import-origin-kind`.** `OriginKind::PackageImport` appended to the +tracking ledger as value 5 — package-sourced vs `Ingest`'s user-picked. Append-only, +per `core/tracking/CLAUDE.md`'s persisted-integer rule; an unrecognized kind +degrades to `Unknown` rather than failing the parse, and `kLedgerVersion` stays at +2 — a vocabulary addition, not a document-version bump. No decision surface +changed: `pruneProtection`'s output is unaffected for every existing kind. + +### Ε-W2 — The two verbs + +Two tracks landed on Ε-W1's contract: a bank leaves the project as one `.rsbank` +file, or the export refuses and says why; a `.rsbank` becomes a **new** bank, +completely or not at all. Both tracks were code-reviewed and remediated before +merging; the merged tree (Ε-W1 + Ε-W2) builds clean and passes 100/100 tests. + +**Ε-W2-T1 — `bank-export`.** New `core/package/export_plan` (pure: which entries, +what names, what is missing, and therefore whether the export may proceed — verdict +`Ready`/`Incomplete`/`Refused`) and `shell/package/export_bank` (the promptless +verb, in three composable public steps — `surveyBankExport`, `digestSources`, +`writePackageFile` — arriving with a **const** `ReaSamplerSession&`, so "writes no +ext state, opens no undo point, never bumps the generation" holds by the type +rather than by memory), plus `shell/actions/package_export_action`, one +`main.cpp` action-table row, and one panel bank-menu row. Nothing is re-encoded; +payloads are copied and hashed. The exported unit is one bank — the pool included, +since the pool is structurally one `BankIndex` among many — and whole-book export +stays out of scope for the phase. Both open questions were answered at review: +affordance ships as **both** the bindable action and the panel row, and the +default file name derives from the bank's display name through +`capture_paths::sanitizeStem`. + +**Ε-W2-T2 — `bank-import`.** New `core/package/import_plan` (pure: the id remap +table, the parent remap, the per-entry land/skip-already-present/rename +disposition, and the destination bank's display name after `BankBook`'s own +uniqueness fold — reached through a new additive `BankBook::uniqueDisplayName` +member, the only `core/model/` edit in the phase), and on the shell side a +REAPER-free `import_landing` (decode, verify every payload's `hashBytes` digest +against the manifest BEFORE the bank folder is created, then land through the +rollback journal) plus a REAPER-facing `import_bank` (the only piece touching the +extension's project state — the undo-batched persist and the generation bump), +`shell/actions/package_import_action`, the panel's `.rsbank` drop route, one +`main.cpp` row, one panel menu row, and a new `src/core/util/ascii_ws.h`. The +tracking-ledger guard runs before the file picker opens (Ε-F3, ruled: refuse +outright on `Unreadable`/`FutureVersion`, no confirm-and-proceed); the version +gate runs before any byte is written; all four collision classes — sample id, file +name, content hash, bank display name — are answered explicitly, with the +display-name collision auto-suffixed and never prompted (Ε-F2, ruled: always a new +bank, never a merge); birth records land via +`recordCreated(sample, OriginKind::PackageImport)` in the same straight-line block +as the index add; the index mutation is one Ctrl-Z, and the landed files' +survival as orphans until the next prune is stated in the user-facing summary, not +left implicit. **Beyond spec:** `import_plan`'s `spelledLikeABankFile` mints a +fresh name even absent a collision, whenever the package's own entry name isn't +spelled the way `deriveBankPaths` would spell it — counted separately from a +genuine folder-name collision (`sanitizeRenameCount` vs `collisionRenameCount`) so +a hostile or foreign-spelled entry name (e.g. an unexpected extension) always +lands sanitized rather than verbatim. + +**Ε-W3 (`package-compat-fixtures`) has not landed** and is the only remaining +wave of Phase Ε. From 3fd3214ff8bd3ff6148fd4dd3cd52b2549bf25af Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 15:39:04 -0400 Subject: [PATCH 20/24] =?UTF-8?q?Remediate=20=CE=95-W3-T1=20package-compat?= =?UTF-8?q?-fixtures=20review=20findings?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Freeze *.rsbank as binary via .gitattributes; add a truncated additive_forward fixture proving the exact-size proof beats TooNew; enumerate the fixture dir to catch orphaned files; make fixture-size checks fatal instead of just logged; pin fixture version asserts as literals, not build-relative. --- .gitattributes | 1 + CMakeLists.txt | 1 + src/core/package/CMakeLists.txt | 2 +- src/shell/package/CMakeLists.txt | 2 +- tests/fixtures/package_compat/README.md | 39 +++++-- .../trunc_additive_forward.rsbank | Bin 0 -> 983 bytes tests/test_package_compat.cpp | 100 +++++++++++++++--- tests/test_package_round_trip.cpp | 89 ++++++++++------ 8 files changed, 173 insertions(+), 61 deletions(-) create mode 100644 .gitattributes create mode 100644 tests/fixtures/package_compat/trunc_additive_forward.rsbank diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..6974000 --- /dev/null +++ b/.gitattributes @@ -0,0 +1 @@ +*.rsbank binary diff --git a/CMakeLists.txt b/CMakeLists.txt index 431b569..07354b0 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -85,6 +85,7 @@ set(LICE_SRC # --------------------------------------------------------------------------- set(REASAMPLER_SRC_DIR ${CMAKE_CURRENT_SOURCE_DIR}/src) set(REASAMPLER_TESTS_DIR ${CMAKE_CURRENT_SOURCE_DIR}/tests) +set(REASAMPLER_PACKAGE_FIXTURE_DIR ${REASAMPLER_TESTS_DIR}/fixtures/package_compat) include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/reasampler_targets.cmake) enable_testing() diff --git a/src/core/package/CMakeLists.txt b/src/core/package/CMakeLists.txt index 0af4166..8102725 100644 --- a/src/core/package/CMakeLists.txt +++ b/src/core/package/CMakeLists.txt @@ -22,7 +22,7 @@ reasampler_test(bank_package LINK bank_package app_version) # tests/package_fixtures.h. reasampler_test(package_compat LINK bank_package) target_compile_definitions(package_compat_tests PRIVATE - REASAMPLER_PACKAGE_FIXTURE_DIR="${REASAMPLER_TESTS_DIR}/fixtures/package_compat") + REASAMPLER_PACKAGE_FIXTURE_DIR="${REASAMPLER_PACKAGE_FIXTURE_DIR}") reasampler_pure_library(import_plan SOURCES import_plan.cpp diff --git a/src/shell/package/CMakeLists.txt b/src/shell/package/CMakeLists.txt index 8ea6b60..497f8e5 100644 --- a/src/shell/package/CMakeLists.txt +++ b/src/shell/package/CMakeLists.txt @@ -41,7 +41,7 @@ reasampler_test(package_round_trip tail_control origin_ledger tracking_authority prune_reconcile app_version capture_paths wav_codec) target_compile_definitions(package_round_trip_tests PRIVATE - REASAMPLER_PACKAGE_FIXTURE_DIR="${REASAMPLER_TESTS_DIR}/fixtures/package_compat") + REASAMPLER_PACKAGE_FIXTURE_DIR="${REASAMPLER_PACKAGE_FIXTURE_DIR}") # The pickers call the REAPER API, so no test target can exercise them; declared as a # library so the TU stays compiled. reaper_plugin.h pulls SWELL in on non-Windows. diff --git a/tests/fixtures/package_compat/README.md b/tests/fixtures/package_compat/README.md index c63252d..837d7ec 100644 --- a/tests/fixtures/package_compat/README.md +++ b/tests/fixtures/package_compat/README.md @@ -12,20 +12,29 @@ The reason is the whole point of the corpus. These bytes exist to catch a format that quietly breaks a compatibility direction. A fixture regenerated by the build that broke it agrees with that build by construction and catches nothing — which is exactly the failure mode a version ladder exists to prevent. The same argument forbids a test -that builds its own fixture at run time. +that builds its own fixture at run time. The repo-root `.gitattributes` (`*.rsbank +binary`) keeps this mechanical: without it, git's NUL-sniffing heuristic could +text-classify a future short/ASCII fixture and CRLF-mangle a line ending on a Windows +checkout, silently breaking the frozen-bytes premise. -If a fixture stops decoding, the answer is never to re-cut the fixture. Either the format -change was structural (bump `minReaderVersion`, add a new fixture, and leave the old one -asserting the refusal) or it is a regression. +A fixture's BYTES are frozen forever; a fixture's ASSERTION is not. `additive_forward.rsbank` +and `refuse_structural.rsbank` carry version pairs one step past THIS build's ladder (2/1 +and 2/2). When a future build's own `kPackageFormatVersion` reaches 2, `refuse_structural.rsbank` +classifies `Readable` under the new ladder — its bytes never claimed to need more than +format 2 — so that build re-aims the assertion (and adds a new synthetic pair one step +past the NEW ladder); it never re-cuts the fixture. If a truncation or hostile-name +fixture ever changes classification, that is a regression, never a ladder consequence. ## Provenance `v1_shipping.rsbank` was produced by running this repo's own export verb (`exportBank`) at version **1.4.0** over a one-sample bank, and copying the emitted file here verbatim. -Every other fixture is derived from those bytes: the truncations are prefixes of them, -and the synthetic packages reuse their manifest region under different version integers -or a hand-written hostile manifest (the encoder refuses to write one, which is why those -could not come from the verb). +Every other fixture is derived from those bytes: eight of the nine truncations are +prefixes of `v1_shipping.rsbank` (the ninth, `trunc_additive_forward.rsbank`, is a prefix +of `additive_forward.rsbank` itself — a prefix of a prefix, still frozen bytes, never +regenerated), and the synthetic packages reuse their manifest region under different +version integers or a hand-written hostile manifest (the encoder refuses to write one, +which is why those could not come from the verb). Payloads are one 300-byte 16-bit mono WAV. The properties under test are structural — version integers, framing arithmetic, name validation — so a larger payload proves @@ -46,9 +55,10 @@ build, exactly as this one was, and recording the build's version here. ### Truncation — one file per distinct decode failure site -Each is a prefix of `v1_shipping.rsbank` (907-byte prefix + 300-byte payload = 1207 -bytes). All classify `Malformed`; none may classify `TooNew`, since "install a newer -build" does not fix a partial download. +The first eight are prefixes of `v1_shipping.rsbank` (907-byte prefix + 300-byte payload += 1207 bytes), so `formatVersion` never exceeds this build's on that path. All classify +`Malformed`; none may classify `TooNew`, since "install a newer build" does not fix a +partial download. | File | Bytes | Site the cut lands in | |---|---|---| @@ -61,6 +71,13 @@ build" does not fix a partial download. | `trunc_payload_middle.rsbank` | 1057 | Inside the first payload. | | `trunc_one_short.rsbank` | 1206 | One byte short of the total. | +`trunc_additive_forward.rsbank` is the ninth: a 983-byte prefix of `additive_forward.rsbank` +(25-byte frozen header/manifest-length region + 958-byte manifest = 983), cut exactly at +ITS payload boundary. `formatVersion` here is 2 — one past this build's — so this is the +one truncation that proves the exact-size-proof failure stays `Malformed` even when +`formatVersion > kPackageFormatVersion`, rather than relabeling to `TooNew` (the parse +branch is the only one that relabels — see `src/core/package/CLAUDE.md`). + ### Hostile names — refused at decode, before any planner The two naming fields carry different rules (`src/core/package/CLAUDE.md`), so each diff --git a/tests/fixtures/package_compat/trunc_additive_forward.rsbank b/tests/fixtures/package_compat/trunc_additive_forward.rsbank new file mode 100644 index 0000000000000000000000000000000000000000..7e5e50d98adb199cd0fed25fce86c8f359b21c49 GIT binary patch literal 983 zcmZ8gT~FIE6zw+cf1taUO=_bg{h;KHmTgRE2(?IiKs=b_mPU1K&v6PU>R;P$+p$Bd zLZnFX@y)r%=N>PXv#XE2Uhf0m&v=vQEQh)UUXQQpV)ml%Xc6zSNsAf>8JhSeVKhrgkfa$yF(G1!W-Hd4I2p6?3a$=4@#->_v_3RiCVMtD_pq_ozJ&H1-3 z;K>Nh!*U_iBi1d|PFT0(T&n^eBvCr{mJ|nM7y$@HH|8%0roplzERwS{hDv8ezWLdd zp5Tt!^Yil^1huo-Y)9}80xvAgjH6RfEZBk8<7NQk32PABi4 z{RCz#9Qyj2@i09*eooO%7@vzx+zU&hCCl8s=L*(IQ0@23mEi0k&FmEF)%DcS% z+Q+K{QM-DcuSKbrN|<`BF(4p=dcX2AEd1u;7A_$a70&PPlD|oDyu07+Fz;GwKM`D~ M_wn^R7!~UN13*?WdH?_b literal 0 HcmV?d00001 diff --git a/tests/test_package_compat.cpp b/tests/test_package_compat.cpp index c262a06..38cd5b7 100644 --- a/tests/test_package_compat.cpp +++ b/tests/test_package_compat.cpp @@ -10,8 +10,10 @@ #include "../src/core/package/bank_package.h" +#include #include #include +#include #include #include @@ -80,15 +82,18 @@ static PackageManifest expectedV1Manifest() { // A fixture that failed to open reads as an empty buffer, and an empty buffer decodes // Malformed — which would let half this file pass vacuously. Every suite loads through -// here. -static std::vector load(const char* name, std::size_t expectedSize) { - std::vector bytes = packageFixtureBytes(name); - if (bytes.size() != expectedSize) { +// here. A size mismatch is fatal to the caller (false), not just recorded: proceeding +// with a short or empty buffer would let classifyFixture/le32At index out of range and +// the TooNew test's hand-lifted manifest slice overread the heap, rather than fail clean. +static bool load(const char* name, std::size_t expectedSize, std::vector& out) { + out = packageFixtureBytes(name); + if (out.size() != expectedSize) { std::printf("FAIL: fixture %s is %zu bytes, expected %zu (path: %s)\n", name, - bytes.size(), expectedSize, packageFixturePath(name).c_str()); + out.size(), expectedSize, packageFixturePath(name).c_str()); ++g_fail; + return false; } - return bytes; + return true; } static bool containsToken(const std::vector& bytes, const std::string& token) { @@ -112,7 +117,8 @@ static PackageReadability classifyFixture(const std::vector& bytes } static void testV1FixtureDecodesToTheRecordTheShippingBuildWrote() { - const std::vector bytes = load(kV1File, kV1TotalSize); + std::vector bytes; + if (!load(kV1File, kV1TotalSize, bytes)) return; CHECK(classifyFixture(bytes) == PackageReadability::Readable); const DecodedPackage dec = decodePackage(bytes, bytes.size()); @@ -147,7 +153,8 @@ static void testV1FixtureDecodesToTheRecordTheShippingBuildWrote() { // root, one on the entry, one inside the nested Sample blob — and must still decode to // exactly what the v1 fixture decodes to. static void testAdditiveForwardFixtureReadsWithEveryKnownFieldIntact() { - const std::vector bytes = load("additive_forward.rsbank", 1283); + std::vector bytes; + if (!load("additive_forward.rsbank", 1283, bytes)) return; // Non-vacuity: the unknown keys are genuinely in the bytes, so the equality below // is "skipped without error", not "there was nothing to skip". @@ -159,8 +166,10 @@ static void testAdditiveForwardFixtureReadsWithEveryKnownFieldIntact() { const DecodedPackage dec = decodePackage(bytes, bytes.size()); CHECK(dec.status == PackageReadability::Readable); - CHECK(dec.header.formatVersion == kPackageFormatVersion + 1); - CHECK(dec.header.minReaderVersion == kPackageMinReaderVersion); + // Literal, not kPackageFormatVersion-relative: these are the FIXTURE's frozen + // version pair (2/1), not this build's — see the file header note. + CHECK(dec.header.formatVersion == 2); + CHECK(dec.header.minReaderVersion == 1); CHECK(dec.header.writerVersion == "1.9.0"); // Every known field, end to end: same manifest the v1 fixture yields. @@ -172,14 +181,16 @@ static void testAdditiveForwardFixtureReadsWithEveryKnownFieldIntact() { // --- direction 2: a structural newer writer is refused whole ----------------- static void testRefuseFixtureIsTooNewAndStillNamesTheWriter() { - const std::vector bytes = load("refuse_structural.rsbank", kV1TotalSize); + std::vector bytes; + if (!load("refuse_structural.rsbank", kV1TotalSize, bytes)) return; CHECK(classifyFixture(bytes) == PackageReadability::TooNew); const DecodedPackage dec = decodePackage(bytes, bytes.size()); CHECK(dec.status == PackageReadability::TooNew); - // The three facts the refusal message owes the user. - CHECK(dec.header.formatVersion == kPackageFormatVersion + 1); - CHECK(dec.header.minReaderVersion == kPackageFormatVersion + 1); + // The three facts the refusal message owes the user. Literal, not + // kPackageFormatVersion-relative — this is the FIXTURE's frozen pair (2/2). + CHECK(dec.header.formatVersion == 2); + CHECK(dec.header.minReaderVersion == 2); CHECK(dec.header.writerVersion == "1.9.0"); // Nothing else: no manifest, no layout, no partial success. @@ -223,7 +234,8 @@ static const Truncation kTruncations[] = { static void testEveryTruncationIsMalformedNeverTooNew() { for (const Truncation& t : kTruncations) { - const std::vector bytes = load(t.file, t.size); + std::vector bytes; + if (!load(t.file, t.size, bytes)) continue; const DecodedPackage dec = decodePackage(bytes, bytes.size()); if (dec.status != PackageReadability::Malformed) { std::printf("FAIL: %s (%s) classified %s, expected Malformed\n", t.file, t.site, @@ -247,6 +259,27 @@ static void testEveryTruncationIsMalformedNeverTooNew() { } } +// --- truncation: the additive-forward ladder direction, not just v1 --------- + +// Every truncation above is a prefix of v1_shipping.rsbank (formatVersion == ours), so +// none of them ever puts formatVersion > kPackageFormatVersion on the exact-size-proof +// failure path — the one relabeling branch (src/core/package/CLAUDE.md: "the parse +// branch is the ONLY one that relabels") never gets exercised from the TooNew side. +// This fixture is additive_forward.rsbank (formatVersion 2, one past ours) cut exactly +// at its manifest/payload boundary: the manifest parses whole, so the failure is the +// exact-size proof, not a parse failure — it must stay Malformed, not relabel to TooNew. +static void testAdditiveForwardTruncatedAtPayloadBoundaryStaysMalformed() { + std::vector bytes; + if (!load("trunc_additive_forward.rsbank", 983, bytes)) return; + CHECK(classifyFixture(bytes) == PackageReadability::Readable); + + const DecodedPackage dec = decodePackage(bytes, bytes.size()); + CHECK(dec.status == PackageReadability::Malformed); + CHECK(dec.manifest.entries.empty()); + CHECK(dec.layout.empty()); + CHECK(dec.prefixSize == 0); +} + // --- hostile names: refused at decode, before any planner exists ------------- // The two fields carry DIFFERENT rules (src/core/package/CLAUDE.md): the entry name may @@ -292,7 +325,8 @@ static void testEveryHostileNameIsRefusedAtDecode() { CHECK(isValidNestedSamplePath("kick.wav")); for (const HostileFixture& h : kHostiles) { - const std::vector bytes = load(h.file, h.size); + std::vector bytes; + if (!load(h.file, h.size, bytes)) continue; if (!containsToken(bytes, jsonEscaped(h.offending))) { std::printf("FAIL: %s does not carry the form it is named for (%s)\n", h.file, h.offending); @@ -318,12 +352,46 @@ static void testEveryHostileNameIsRefusedAtDecode() { } } +// --- inventory: every fixture on disk is exercised by SOME test above -------- + +// This TU's own tables (the three named fixtures plus kTruncations and kHostiles) are +// the fullest account of the corpus in the tree — every other consumer (the round-trip +// harness, the README) tests a subset of these same files. Enumerating the fixture +// directory here and failing on anything absent from this list is the one check that +// catches a fixture added to disk but never wired into a table: a silent coverage drop +// that would otherwise leave a green suite. +static void testEveryFixtureOnDiskIsInSomeTable() { + std::vector known = {kV1File, "additive_forward.rsbank", + "refuse_structural.rsbank", + "trunc_additive_forward.rsbank"}; + for (const Truncation& t : kTruncations) known.push_back(t.file); + for (const HostileFixture& h : kHostiles) known.push_back(h.file); + + std::error_code ec; + for (const auto& entry : + std::filesystem::directory_iterator(REASAMPLER_PACKAGE_FIXTURE_DIR, ec)) { + if (entry.path().extension() != ".rsbank") continue; + const std::string name = entry.path().filename().string(); + if (std::find(known.begin(), known.end(), name) == known.end()) { + std::printf("FAIL: fixture %s exists on disk but is in no table in this file\n", + name.c_str()); + ++g_fail; + } + } + if (ec) { + std::printf("FAIL: could not list fixture directory: %s\n", ec.message().c_str()); + ++g_fail; + } +} + int main() { testV1FixtureDecodesToTheRecordTheShippingBuildWrote(); testAdditiveForwardFixtureReadsWithEveryKnownFieldIntact(); testRefuseFixtureIsTooNewAndStillNamesTheWriter(); testEveryTruncationIsMalformedNeverTooNew(); + testAdditiveForwardTruncatedAtPayloadBoundaryStaysMalformed(); testEveryHostileNameIsRefusedAtDecode(); + testEveryFixtureOnDiskIsInSomeTable(); if (g_fail == 0) { std::printf("package_compat_tests: all passed\n"); diff --git a/tests/test_package_round_trip.cpp b/tests/test_package_round_trip.cpp index 3143a46..79f365b 100644 --- a/tests/test_package_round_trip.cpp +++ b/tests/test_package_round_trip.cpp @@ -67,13 +67,16 @@ static void writeBytes(const std::string& path, const std::vector& static_cast(bytes.size())); } -// Copies a committed fixture to the path the verb will be pointed at. Fails loudly on an -// empty read: an unreadable corpus would otherwise let every "must refuse" suite pass. -static bool stageFixture(const Scratch& scratch, const char* fixture) { +// Copies a committed fixture to the path the verb will be pointed at. Fails loudly on a +// size mismatch (empty included): an unreadable OR mangled corpus would otherwise let +// every "must refuse" suite pass, since a garbled fixture still refuses, just not for +// the reason under test. +static bool stageFixture(const Scratch& scratch, const char* fixture, + std::size_t expectedSize) { const std::vector bytes = packageFixtureBytes(fixture); - if (bytes.empty()) { - std::printf("FAIL: fixture %s read as 0 bytes (path: %s)\n", fixture, - packageFixturePath(fixture).c_str()); + if (bytes.size() != expectedSize) { + std::printf("FAIL: fixture %s is %zu bytes, expected %zu (path: %s)\n", fixture, + bytes.size(), expectedSize, packageFixturePath(fixture).c_str()); ++g_fail; return false; } @@ -116,7 +119,7 @@ static ImportLanding runImport(const Scratch& scratch, ReaSamplerSession& sessio // importing and re-exporting it closes export -> import -> export over frozen bytes. static void testV1FixtureReExportsByteIdenticalPayloads() { Scratch scratch("roundtrip"); - if (!stageFixture(scratch, "v1_shipping.rsbank")) return; + if (!stageFixture(scratch, "v1_shipping.rsbank", 1207)) return; ReaSamplerSession session; int births = 0; @@ -128,7 +131,7 @@ static void testV1FixtureReExportsByteIdenticalPayloads() { const std::vector> sourcePayloads = payloadsOf(packageFixtureBytes("v1_shipping.rsbank")); - CHECK(sourcePayloads.size() == 1); + if (sourcePayloads.size() != 1) { CHECK(false); return; } // The landed file is the package's payload verbatim — the first half of the claim. const std::string landed = scratch.bankDir() + "/" + landing.plan.entries[0].destFileName; @@ -153,7 +156,7 @@ static void testV1FixtureReExportsByteIdenticalPayloads() { static void testRefuseFixtureRefusesTheWholeImportAndNamesTheWriter() { Scratch scratch("refuse"); - if (!stageFixture(scratch, "refuse_structural.rsbank")) return; + if (!stageFixture(scratch, "refuse_structural.rsbank", 1207)) return; ReaSamplerSession session; const BankBook before = session.book(); @@ -162,8 +165,10 @@ static void testRefuseFixtureRefusesTheWholeImportAndNamesTheWriter() { session.book(), kTag, journal); CHECK(landing.outcome == ImportOutcome::TooNew); - CHECK(landing.header.formatVersion == package::kPackageFormatVersion + 1); - CHECK(landing.header.minReaderVersion == package::kPackageFormatVersion + 1); + // Literal, not kPackageFormatVersion-relative: this is the FIXTURE's frozen pair + // (2/2), not this build's (see test_package_compat.cpp's file header note). + CHECK(landing.header.formatVersion == 2); + CHECK(landing.header.minReaderVersion == 2); CHECK(landing.header.writerVersion == "1.9.0"); // Nothing planned, nothing on disk, nothing in the index. CHECK(landing.plan.entries.empty()); @@ -173,18 +178,27 @@ static void testRefuseFixtureRefusesTheWholeImportAndNamesTheWriter() { // The same eight cuts test_package_compat classifies, driven through the verb's // incremental prefix reader — the one caller that can ask requiredPrefixSize for more -// bytes than the file holds. -static const char* kTruncationFixtures[] = { - "trunc_magic.rsbank", "trunc_version_pair.rsbank", - "trunc_writer_semver.rsbank", "trunc_manifest_length.rsbank", - "trunc_manifest_body.rsbank", "trunc_payload_start.rsbank", - "trunc_payload_middle.rsbank", "trunc_one_short.rsbank", +// bytes than the file holds. Sizes match test_package_compat.cpp's kTruncations. +struct TruncationFixture { + const char* file; + std::size_t size; +}; + +static const TruncationFixture kTruncationFixtures[] = { + {"trunc_magic.rsbank", 2}, + {"trunc_version_pair.rsbank", 10}, + {"trunc_writer_semver.rsbank", 18}, + {"trunc_manifest_length.rsbank", 23}, + {"trunc_manifest_body.rsbank", 466}, + {"trunc_payload_start.rsbank", 907}, + {"trunc_payload_middle.rsbank", 1057}, + {"trunc_one_short.rsbank", 1206}, }; static void testEveryTruncationRefusesTheImportAsMalformed() { - for (const char* fixture : kTruncationFixtures) { - Scratch scratch(std::string("trunc_") + fixture); - if (!stageFixture(scratch, fixture)) continue; + for (const TruncationFixture& fixture : kTruncationFixtures) { + Scratch scratch(std::string("trunc_") + fixture.file); + if (!stageFixture(scratch, fixture.file, fixture.size)) continue; ReaSamplerSession session; const BankBook before = session.book(); @@ -192,7 +206,7 @@ static void testEveryTruncationRefusesTheImportAsMalformed() { const ImportLanding landing = landPackage(scratch.importPath(), scratch.projectDir(), session.book(), kTag, journal); if (landing.outcome != ImportOutcome::Malformed) { - std::printf("FAIL: %s imported as outcome %d, expected Malformed\n", fixture, + std::printf("FAIL: %s imported as outcome %d, expected Malformed\n", fixture.file, static_cast(landing.outcome)); ++g_fail; } @@ -202,19 +216,30 @@ static void testEveryTruncationRefusesTheImportAsMalformed() { } } -static const char* kHostileFixtures[] = { - "hostile_name_dotdot.rsbank", "hostile_name_parent_slash.rsbank", - "hostile_name_parent_backslash.rsbank", "hostile_name_subdir_slash.rsbank", - "hostile_name_drive_absolute.rsbank", "hostile_name_unc_absolute.rsbank", - "hostile_path_dotdot_slash.rsbank", "hostile_path_dotdot_backslash.rsbank", - "hostile_path_rooted.rsbank", "hostile_path_drive_absolute.rsbank", - "hostile_path_unc_absolute.rsbank", +// Sizes match test_package_compat.cpp's kHostiles. +struct HostileFixtureFile { + const char* file; + std::size_t size; +}; + +static const HostileFixtureFile kHostileFixtures[] = { + {"hostile_name_dotdot.rsbank", 624}, + {"hostile_name_parent_slash.rsbank", 633}, + {"hostile_name_parent_backslash.rsbank", 634}, + {"hostile_name_subdir_slash.rsbank", 634}, + {"hostile_name_drive_absolute.rsbank", 643}, + {"hostile_name_unc_absolute.rsbank", 646}, + {"hostile_path_dotdot_slash.rsbank", 641}, + {"hostile_path_dotdot_backslash.rsbank", 640}, + {"hostile_path_rooted.rsbank", 635}, + {"hostile_path_drive_absolute.rsbank", 643}, + {"hostile_path_unc_absolute.rsbank", 646}, }; static void testEveryHostileNameIsRefusedBeforeThePlannerRuns() { - for (const char* fixture : kHostileFixtures) { - Scratch scratch(std::string("hostile_") + fixture); - if (!stageFixture(scratch, fixture)) continue; + for (const HostileFixtureFile& fixture : kHostileFixtures) { + Scratch scratch(std::string("hostile_") + fixture.file); + if (!stageFixture(scratch, fixture.file, fixture.size)) continue; ReaSamplerSession session; const BankBook before = session.book(); @@ -222,7 +247,7 @@ static void testEveryHostileNameIsRefusedBeforeThePlannerRuns() { const ImportLanding landing = landPackage(scratch.importPath(), scratch.projectDir(), session.book(), kTag, journal); if (landing.outcome != ImportOutcome::Malformed) { - std::printf("FAIL: %s imported as outcome %d, expected Malformed\n", fixture, + std::printf("FAIL: %s imported as outcome %d, expected Malformed\n", fixture.file, static_cast(landing.outcome)); ++g_fail; } From b99027bc4ca5d5a03543b1f923141cb5174ee765 Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 14:36:13 -0400 Subject: [PATCH 21/24] docs: collapse the landed Phase E track specs to Landed form Five tracks across W1 and W2 now point at COMPLETED.md. W3 and the phase header stay live. --- docs/PLAN.md | 456 ++++++++++----------------------------------------- 1 file changed, 88 insertions(+), 368 deletions(-) diff --git a/docs/PLAN.md b/docs/PLAN.md index 3a2883d..0583928 100644 --- a/docs/PLAN.md +++ b/docs/PLAN.md @@ -2408,6 +2408,10 @@ capture backend.** ### Ε-W1 — The contract, the filesystem, and the ledger's new kind +**All three tracks have landed** — Ε-W1-T1 (`package-format`), Ε-W1-T2 +(`package-fs-shell`), and Ε-W1-T3 (`import-origin-kind`) — see `docs/COMPLETED.md` for +the full narrative of each. + **Depends on:** nothing in this phase. **Three tracks, disjoint by directory** — the split is by *what each track's inputs are*, which is why they genuinely parallelize: T1 knows only bytes and structs, T2 knows only paths and bytes, T3 knows only the ledger. @@ -2430,212 +2434,63 @@ T3 touches no package code at all. #### Ε-W1-T1 — `package-format` -**Goal.** The container and its version ladder, entirely pure — the contract every later -track consumes, landed once so nothing downstream re-litigates the shape. - -**Spec:** `docs/product/bank-package.md` §"The container", §"Version tagging", §"What a -package carries", §"What a package deliberately does NOT carry", §"Memory". - -**Surface boundary — owns:** new `src/core/package/package_format` (the magic, the header -layout, `kPackageFormatVersion`, `kPackageMinReaderVersion`, and -`classifyPackageVersion(formatVersion, minReader) -> Readable | TooNew | Malformed`), new -`src/core/package/package_manifest` (the manifest model + its JSON codec), new -`src/core/package/bank_package` (header encode, prefix decode, entry-layout arithmetic), the -directory's `CMakeLists.txt` and `CLAUDE.md`, and one appended `add_subdirectory` line in the -root `CMakeLists.txt`. **Does not own:** `export_plan` / `import_plan` (Ε-W2), anything under -`shell/`, `core/model`, or `core/tracking`. - -**Behavior.** -- **The container is the hand-rolled `RSBK` (Ε-F1, ruled).** Magic `RSBK`, a fixed - little-endian header carrying the two version fields, a length-prefixed JSON manifest, then - each entry's payload concatenated in manifest order. Framing is built on `core/wire/bytes.h` - (`putLE` / `ByteReader`) and the manifest on `core/json` — both already owned and tested - here. **No ZIP, no compressor, no new third-party source in the build**; a link edge to - `vendor/WDL/WDL/zlib/` means the ruling was misread. -- **Two version integers, not one.** `formatVersion` = what this writer emitted; - `minReaderVersion` = the oldest reader that can read it safely. The reader's whole rule is - `minReaderVersion <= kPackageFormatVersion`. An **additive** change (a new optional - manifest key, a new `Sample` field with a defined absent-value) bumps `formatVersion` - only; a **structural** change bumps both. **Growing a persisted enum's vocabulary is - structural here, not additive** — `BankModel::deserialize` *rejects* an out-of-range - `SourceMode` or `Tier` rather than degrading it (`bank_model.cpp:232-239`, `:339-346`), - and every enum a package carries rides inside the nested `BankModel` blob, so a new - `SourceMode` or `Tier` value bumps both integers. The header carries the writer's semver - (`version::stampVersion()`) alongside them, informational, so a refusal message can name - what to install. -- **The ladder is documented the way `origin_ledger.cpp:8-21` documents its own** — a header - comment listing every shipped version and what changed, with the read-and-validate rule - stated, not implied. -- **Unknown manifest keys are skipped** (the `bank_book_json.cpp:182` behaviour), and - **unknown persisted enum integers — the manifest's own, not `BankModel`'s nested ones, which - reject per the bullet above — degrade to their defined `Unknown` equivalent**, never to - the numeric default and never to a parse failure (`core/wire/CLAUDE.md`'s `BakeStatus` rule, - verbatim). Both are pinned by tests, not left to inheritance. -- **The manifest nests `BankModel`'s own serialization verbatim**, exactly as - `bank_book_json.cpp:15-20` nests it, so per-sample shape has one owner and a future - `Sample` field reaches packages for free. Per entry the manifest adds only: the bare file - name, the byte length, and a `hashBytes` digest (`core/capture/wav_codec.h:143`) — - `hashBytes`, **not** `hashWavContent`, because the latter deliberately skips chunks - (`wav_codec.h:145-151`) and so cannot answer "did these bytes survive." -- **The bank's `slot_map` rides along** — display positions are part of what the user built. -- **Framing only, never a payload.** `bank_package` produces the header bytes and an ordered - `[{ name, offset, length }]` layout; it never holds, copies, or hashes an entry's audio. - Decode is symmetric: prefix in, manifest + layout out. -- **Path expression is structurally impossible.** Entry names are validated to contain no - `/`, `\`, `:`, no leading separator, and no `..` component, on both encode and decode. - -**Acceptance criteria.** -- `decodePackage(encodePackage(x)) == x` over a manifest fixture exercising every field, - including every `Sample` optional in both present and absent states. -- A synthetic header with `minReaderVersion` above this build classifies `TooNew` and **no - manifest is produced** — the decode does not half-succeed. -- A synthetic header with `formatVersion` above this build but `minReaderVersion` at or below - it classifies `Readable`, and its unknown manifest keys are skipped without error. This is - the additive-forward-compatibility claim, and it is the reason the two-integer design - exists; a test that does not exercise it leaves the design unproven. -- Truncated input at every byte offset in a valid package returns `Malformed` — never UB, - never a partial manifest, never a read past the buffer. Hostile-input hardening at the - `bank_model.h:204-206` standard. -- Entry names containing `..`, a separator, or an absolute prefix are rejected on encode - *and* rejected on decode. Both directions, because a package can arrive from anywhere. -- No file in the new directory exceeds ~600 lines; the three-module split above is the - responsibility seam, and a fourth module is preferred over a bisection if one is needed. -- `package_format_tests`, `package_manifest_tests`, `bank_package_tests` all run without - REAPER or a DAW. - -**Open questions.** **No [Daniel] questions — Ε-F1 is RULED** (proprietary `RSBK`), so this -track is dispatchable as written. **[propose at review]** whether `package_format` and -`bank_package` are genuinely two modules or one — the split is proposed on responsibility -grounds (constants and classification vs. offset arithmetic) and may collapse if the -arithmetic turns out to be twenty lines. +**Landed** — see `docs/COMPLETED.md` for the full narrative. The pure `src/core/package/` +codec for the hand-rolled `RSBK` container (Ε-F1, ruled — no ZIP, no compressor, no link +edge to `vendor/WDL/WDL/zlib/`): a fixed little-endian header carrying two version +integers — `formatVersion` (what the writer emitted) and `minReaderVersion` (the oldest +reader that can read it safely) — a length-prefixed JSON manifest, and payloads +concatenated in manifest order. `classifyPackageVersion` answers `Readable`/`TooNew`/ +`Malformed`; a `TooNew` header refuses whole, producing no manifest, so the refusal can +still name the writer's semver rather than half-succeeding. The ladder's one +counterintuitive rule rides with the contract: **growing a persisted enum's vocabulary is +structural, not additive** — `BankModel::deserialize` *rejects* an out-of-range `SourceMode` +or `Tier` rather than degrading it (`bank_model.cpp:232-239`, `:339-346`), and every enum a +package carries rides inside the nested `BankModel` blob, so a new value bumps both +integers, where a new `Sample` field with a defined absent-value bumps `formatVersion` +alone. Landed as three modules: `package_format` (the contract, the version ladder, and +three name-validation rules — +`isValidEntryName`, `sameEntryName`'s ASCII-case fold, `isValidNestedSamplePath`), +`package_manifest` (the manifest model + JSON codec, carrying the bank's `slot_map` and a +whole-file `hashBytes` digest per entry — deliberately not `hashWavContent`, which skips +chunks and so cannot answer "did these bytes survive"), and `bank_package` +(framing/layout arithmetic: `encodePackage`/`decodePackage`/`requiredPrefixSize`, never +holding or hashing a payload itself). Hostile input is refused, never UB, at every byte +offset. #### Ε-W1-T2 — `package-fs-shell` -**Goal.** Every filesystem and dialog act the two verbs need, landed behind an API that knows -nothing about what a package contains — so it can be authored, reviewed, and tested in -parallel with the format it will carry. - -**Spec:** `docs/product/bank-package.md` §"Where it lives", §"Failure modes", §"Memory". - -**Surface boundary — owns:** new `src/shell/package/package_io` (read a file's bytes, write -bytes through temp + atomic rename, read one bank file, write one landed file, enumerate the -bank folder's existing names, and the rollback delete), the file-picker seam for both verbs — -**one picker, REAPER's own, on every platform**, so this track carries **no** -`#ifdef _WIN32` / `#else swell/swell.h` split. That split is a real pattern in this codebase -(`src/shell/panel/draw_kit.cpp:11-15`, `src/shell/persist/prune_fs.cpp:35-38`); it is simply -not this track's shape, because REAPER owns the dialog. Also owns the -directory's `CMakeLists.txt` and `CLAUDE.md`, and one appended `add_subdirectory` line in the -root `CMakeLists.txt`. **Does not own:** `export_bank` / `import_bank` (Ε-W2), anything under -`core/`, and — emphatically — `prune_fs`, which stays the deletion authority. - -**Behavior.** -- **Atomic write.** A package is written to a temp path in the destination directory and - renamed on complete success. A failed or interrupted write leaves no `.rsbank` behind. This - is the mono-collapse precedent (Ψ-W2-T2, temp file + atomic rename) applied to a much - larger file. -- **Streaming, both ways.** Append one payload at a time on write; seek and read one payload - at a time on read. The API must make holding the whole package awkward, not merely - discouraged. -- **The rollback delete is the carve-out, cited.** Its TU header cites - `src/shell/persist/prune_fs.cpp:5-11` and states the discriminator it satisfies — this call - created the file, and no index ever referenced it — rather than restating the carve-out's - text. Anything that does not satisfy that discriminator is not this function's business. -- **Both pickers are REAPER's own, and they are symmetric.** `GetUserFileName(int mode, - const char* caption, const char* initial_file_or_path, const char* extension_list, char* - fnOutNeedBig, int fnOutNeedBig_sz)` — **verified**, - `vendor/reaper-sdk/sdk/reaper_plugin_functions.h:3790`, documented at `:3788` — serves - both directions: `mode=0` ("choose a new file") is export's destination picker, `mode=1` - ("existing file") is import's source picker. `extension_list` takes the - `'ReaSampler banks|*.rsbank|All files|*.*'` form, and `initial_file_or_path` may be a bare - `'.rsbank'` to set the default extension. There is **no `#ifdef _WIN32` / SWELL split and - no wide-char round trip here** — REAPER owns the dialog on every platform, so no - `GetSaveFileNameW` and no `BrowseForSaveFile`. `GetUserFileNameForRead` is explicitly - "Superseded, see GetUserFileName" (`:3796`) and is not used. -- **No fallback path.** `src/app/main.cpp:15` defines `REAPERAPI_IMPLEMENT` *without* - `REAPERAPI_MINIMAL`, so the resolver walks the full table — `GetUserFileName` included - (`reaper_plugin_functions.h:9084`) — and `main.cpp:292-293` aborts the extension load if - any single function fails to resolve. No REAPER build that loads this extension can lack - `GetUserFileName`, which makes a fallback unreachable code. -- **No REAPER project state is touched here.** No ext-state read or write, no undo block, no - generation bump; those belong to the verbs in Ε-W2. - -**Acceptance criteria.** -- A write interrupted before completion leaves the destination path absent or holding its - prior contents — never a partial new file. Tested by injecting a failure at the writer seam. -- Reading and writing a multi-entry package never holds more than one entry's payload; the - test asserts against a seam counter, not against a memory measurement. -- The rollback deletes exactly the files it was given and nothing else, and is a no-op on a - path it did not write. -- **There is exactly one picker call site and it is REAPER's.** Both verbs reach - `GetUserFileName` — export with `mode=0`, import with `mode=1` — and no symbol named - `GetSaveFileNameW`, `BrowseForSaveFile`, or `GetUserFileNameForRead` appears anywhere in - `src/shell/package/`, nor any platform `#ifdef` in the picker's TU. Greppable, so it stays - true. `[verify — DAW]` — the picker is not exercised in a live REAPER session by this track. -- No file exceeds ~600 lines; the picker lives in its own TU with its own header. **Not** for - the `drag_out` / `drag_out_win` reason — that precedent isolates a *Win32-only* TU, and - there is no platform split here — but because the picker is the only REAPER-facing part of - an otherwise REAPER-free, unit-tested module: folding it into a shared header would drag - `reaper_plugin_functions.h` into the testable seam's include graph. - -**Open questions.** **[propose at review]** where the `extension_list` and default-extension -strings live — this track's picker TU, or the Ε-W2 verbs that call it. They are user-facing -text, and the verbs own the rest of the user-facing text; the counter-argument is that they -are picker plumbing and only one picker exists. **[verify — DAW]** two things the header does -not answer: (1) whether the `mode=0` picker **appends** an extension when the user types a -bare name — `:3788` documents that `initial_file_or_path` may be `'.rsbank'` "to set the -default extension," but not that the dialog enforces it on return, so the verb may still have -to append `.rsbank` itself; (2) **dialog parenting** — `GetUserFileName`'s signature -(`:3790`) takes no owner window, where the abandoned Win32 `OPENFILENAME` path would have -passed `GetMainHwnd()`, so modality against the REAPER main window is unobserved. +**Landed** — see `docs/COMPLETED.md` for the full narrative. `src/shell/package/`: +streaming, atomic package filesystem I/O (`package_io`'s `PackageFileWriter`/ +`PackageFileReader`, at most one entry's payload materialized at a time, backed by a +`.rsbanktmp` sibling that reaches the destination only through a `commit()` rename — +process-crash atomic, not power-loss atomic, deliberately, since an `fsync` over a whole +sample bank is a real stall) and the rollback journal (`package_rollback`'s +`LandedFileJournal`, citing the `prune_fs.cpp` carve-out rather than restating it, +disarmed only after the caller's own write has returned success). `package_pickers` is the +one picker seam, REAPER's own in both directions as specified — `GetUserFileName` with +`mode=0` for export's destination and `mode=1` for import's source, so no platform `#ifdef`, +no SWELL `BrowseForSaveFile`, no Win32 `GetSaveFileNameW`, and no `GetUserFileNameForRead`, +which the SDK header marks superseded by `GetUserFileName`. No fallback path was needed +either: `main.cpp` aborts the extension load if any API pointer fails to resolve, so no +REAPER build that loads the extension can lack it. The picker is `[verify — DAW]` in both +directions, never exercised in a live REAPER session. #### Ε-W1-T3 — `import-origin-kind` -**Goal.** Give the ledger a birth-record kind for a package import, so an imported file is -tracked from the moment it lands rather than becoming a permanently unreclaimable foreign -file — landed as its own track, with its own review, because it edits safety-critical -territory that nothing else in this phase touches. - -**Spec:** `docs/product/bank-package.md` §"What a package deliberately does NOT carry" (the -origin-ledger bullet); `src/core/tracking/CLAUDE.md` for the invariants it must not weaken. - -**Surface boundary — owns:** `src/core/tracking/origin_ledger.{h,cpp}` and its tests, -exclusively. **Does not own:** `tracking_authority` (no decision changes), `shell/persist`, -or any consumer. - -**Behavior.** -- **Append `OriginKind::PackageImport` as value 5.** Append only — `Capture`=1, `Ingest`=2, - `Recapture`=3, `Resample`=4 keep their integers, per `core/tracking/CLAUDE.md`'s - "PERSISTED INTEGERS — never renumber, only append". -- **A build that does not know value 5 degrades it to `Unknown`**, which is the existing - `kindFromInt` behaviour and is the safe direction: the path is still owned, so still - protected; only the kind detail is lost. This is the *field-vocabulary* rule, and it must - stay distinct from the *document-version* rule right beside it, which blocks - (`origin_ledger.cpp:18-21`). -- **Nothing else changes.** No new field, no version bump, no lineage semantics. A new enum - value in an append-only vocabulary is precisely the change that does **not** need `"v"` to - move, and demonstrating that is part of the point. -- **`recordCreated` needs no signature change** — it already takes an `OriginKind` - (`src/shell/persist/session.h:95`). Confirm that in the same pass; if it turns out - otherwise, that discovery is this track's, not Ε-W2's. - -**Acceptance criteria.** -- A ledger containing a kind-5 record round-trips through serialize/deserialize unchanged. -- A record carrying an *unrecognized* kind integer (6, 99, negative) loads as `Unknown` and - the ledger loads `Loaded`, not `Unreadable` — the vocabulary gap does not halt prune. -- `kLedgerVersion` is **unchanged** at 2, and a test asserts it, so the append-vs-bump - distinction is pinned rather than assumed. -- `pruneProtection`'s output is unchanged for every existing kind — this track alters no - decision. - -**Open questions.** **[propose at review]** whether the kind is named `PackageImport` or -folded onto the existing `Ingest`. The plan's recommendation is a distinct value: `Ingest` -means "the user brought in a file," which is close, but losing the distinction makes a future -"where did this bank come from" question unanswerable, and an appended integer costs nothing. +**Landed** — see `docs/COMPLETED.md` for the full narrative. `OriginKind::PackageImport` +appended to the tracking ledger as value 5 — package-sourced vs `Ingest`'s user-picked. +Append-only, per `core/tracking/CLAUDE.md`'s persisted-integer rule; an unrecognized kind +degrades to `Unknown` rather than failing the parse, and `kLedgerVersion` stays at 2 — a +vocabulary addition, not a document-version bump. No decision surface changed: +`pruneProtection`'s output is unaffected for every existing kind. --- ### Ε-W2 — The two verbs +**Both tracks have landed** — Ε-W2-T1 (`bank-export`) and Ε-W2-T2 (`bank-import`) — see +`docs/COMPLETED.md` for the full narrative of each. + **Depends on Ε-W1 — all three tracks.** T1 for the format the verbs speak, T2 for every filesystem act they perform, T3 for the kind their birth records carry. No part of either verb is authorable against a format that has not settled. @@ -2660,182 +2515,47 @@ second. #### Ε-W2-T1 — `bank-export` -**Goal.** One bank leaves the project as one file, or the export refuses and says why. - -**Spec:** `docs/product/bank-package.md` §"Failure modes" (export rows), §"What a package -carries". - -**Surface boundary — owns:** new `core/package/export_plan` (pure: which entries, what -names, what is missing, and therefore whether the export may proceed), new -`shell/package/export_bank` (the promptless verb — takes a `ReaSamplerSession&`, returns an -outcome, **no prompts and no message boxes**, mirroring `src/shell/bank_ops/`), new -`shell/actions/package_export_action` (the bindable-action skin, mirroring `prune_action`), -one registration line in `src/app/main.cpp`, one panel menu row. **Does not own:** anything -on the import side, `bank_ops`, or `persist`. - -**Behavior.** -- **The exported unit is one bank** — the pool included, since the pool is structurally a bank - (`core/model/CLAUDE.md`'s pool-privileges section). Whole-book export is an explicit - non-goal of this phase and is preserved as an additive future by the manifest's shape, not - by a promise. -- **Refuse-if-incomplete, report-before-acting.** An index entry whose file is missing or - unreadable stops the export by default; the "export the N present entries" path exists only - behind an explicit confirm that lists what is absent, distinguishing missing from - unreadable. This is prune's dry-run-then-confirm discipline applied to a non-destructive - act, and it is deliberate: a silently-incomplete package is discovered on the far side, in - another project, weeks later. -- **The project is not touched.** No ext-state write, no generation bump, no undo point. An - export that mutates project state is a defect, and the acceptance criteria name it as one. -- **Nothing is re-encoded.** Payload bytes are copied and hashed. `wav_codec` is not asked to - rebuild anything. -- **A new FOREVER-STABLE command id** is minted through `version::channelCommandId(suffix)` - per the root `CLAUDE.md` action contract, with its per-channel display name through - `channelActionName`. - -**Acceptance criteria.** -- `planExport` is pure and total over its inputs: a bank with a missing file, an unreadable - file, zero samples, and one sample all classify without touching a filesystem. -- Exporting a bank and re-reading the package yields, for every entry, a `hashBytes` digest - equal to the source file's — asserted per entry, not in aggregate. -- The exported manifest contains no absolute path and no path separator, asserted by a test - that scans the emitted bytes rather than by inspecting the model. -- A failure injected mid-write leaves no `.rsbank` at the destination and the prior file, if - any, intact. -- Project ext state is byte-identical before and after an export, and `bankGeneration()` is - unchanged — a direct assertion, because "we did not mean to write anything" is not a - property that survives without one. -- Exporting an empty bank produces a valid, importable package with zero entries rather than - refusing. An empty bank is a legitimate thing to carry. - -**Open questions — both answered at implementation review, recorded at -`docs/product/bank-package.md` §"Implementation decisions — Ε-W2-T1".** Affordance: -**both** the action and the panel row (the action is the only spelling that can reach -the pool; the panel row is the direct gesture on a named bank). Default file name: -the bank's **display name**, sanitized through `capture_paths::sanitizeStem`, as -recommended. +**Landed** — see `docs/COMPLETED.md` for the full narrative. New `core/package/export_plan` +(pure: which entries, what names, what is missing, and therefore whether the export may +proceed — verdict `Ready`/`Incomplete`/`Refused`) and `shell/package/export_bank` (the +promptless verb, in three composable public steps — `surveyBankExport`, `digestSources`, +`writePackageFile` — arriving with a **const** `ReaSamplerSession&`, so "writes no ext +state, opens no undo point, never bumps the generation" holds by the type rather than by +memory), plus `shell/actions/package_export_action`, one `main.cpp` action-table row, and +one panel bank-menu row. Nothing is re-encoded; payloads are copied and hashed. The +exported unit is one bank — the pool included, since the pool is structurally one +`BankIndex` among many — and whole-book export stays out of scope for the phase. Both open +questions were answered at review: affordance ships as **both** the bindable action and +the panel row, and the default file name derives from the bank's display name through +`capture_paths::sanitizeStem`. #### Ε-W2-T2 — `bank-import` -**Goal.** A package becomes a **new** bank in this project — completely, or not at all — -with every one of the four collision classes answered explicitly rather than by whatever the -model happens to do. - -**Spec:** `docs/product/bank-package.md` §"Identity and collision on import" (including the -auto-suffix rule), §"Failure modes" (import rows), §"Import under a degraded tracking -ledger", §"Version tagging: both directions". - -**Surface boundary — owns:** new `core/package/import_plan` (pure: the id remap table, the -parent remap, the per-entry write / skip-already-present / rename disposition, the -destination bank name after uniqueness folding), new `shell/package/import_bank` (the -promptless verb), new `shell/actions/package_import_action`, the panel's `WM_DROPFILES` route -for a `.rsbank` (routing only — the existing ingest route for audio files is untouched), one -registration line in `src/app/main.cpp`, one panel menu row, and — the **only** `core/model/` -edit in the phase — **one additive public `const` member on `BankBook`** (recommended -`std::string uniqueDisplayName(const std::string& seed) const`), so the auto-suffix probe runs -behind the model's own name fold. **Does not own:** anything on the export side, -`bank_book`'s *rules* (consumed, never re-implemented — the new member exposes the existing -fold, it does not add a second one), `origin_ledger` (Ε-W1-T3's). - -**Behavior.** -- **The ledger guard runs FIRST — before the file picker opens (Ε-F3, ruled: refuse).** If - `tracking::ledgerDegraded(status)` holds for the project's loaded ledger status - (`Unreadable` or `FutureVersion`; `core/tracking/origin_ledger.h:94`, `:100-101`), the - import **refuses outright** — no picker, no bytes read, no confirm-and-proceed path, no - opt-out. `Fresh` and `Loaded` both proceed. **Do not key this on - `PruneReport::blockedByTracking`**: that flag also fires on undecodable `rsusage_*` keys, - which govern deletion-time protection and have nothing to do with writing birth records. - The refusal is a `ShowConsoleMsg` block mirroring `prune_action.cpp:30-69` in structure and - tone, with two variants (malformed / newer-build) and every recovery line naming **this - build's** namespace through `version::extStateNamespace()`. Exact wording in the spec doc. - **Export is deliberately not gated this way** — that is Ε-W2-T1's, and it stays ungated. -- **Version gate second, before any byte is written.** `minReaderVersion` above this build - refuses the whole package and reports through `ShowMessageBox` - (**verified**, `reaper_plugin_functions.h:6546`) naming three things: the package's - requirement, this build's ceiling, and the writer's semver. Two of the three is not enough - to act on. A malformed or truncated package reports **distinctly** — the two failures have - opposite recoveries, which is exactly why `origin_ledger.cpp:178-185` separates them. -- **Four collisions, four answers.** (1) **Sample id** — remint every id and remap - `Provenance::parentSampleId` (`bank_model.h:45-50`) through the same map, to the reminted - parent when it came in the same package and cleared otherwise; a foreign id never enters - the index. (2) **File name** — never overwrite; mint a fresh unique name through - `capture_paths::deriveBankPaths`, silently, counted in the summary. (3) **Content hash** — - consult `BankModel::findByHash` **before writing the payload**; on a hit, skip the write - entirely and let the entry collapse, so a dedup never manufactures an orphan. (4) **Bank - display name** — **auto-suffix, no prompt** (Ε-F2, ruled). Seed = the package's recorded - source bank name **verbatim** (or the literal `Imported bank` if absent/blank); take the - first of `seed`, `seed + " 2"`, `seed + " 3"`, … whose fold is free in the destination - book, ascending from 2. Four points that decide the behaviour and must not be re-invented: - the seed is **never re-parsed** (`"Drums 2"` colliding lands as `"Drums 2 2"`, not - `"Drums 3"` — a bare trailing integer is indistinguishable from `"Kit 808"`); the probe - **fills gaps** (first-free, not highest-plus-one, so it is a pure function of the current - name set); the probe **terminates** by pigeonhole within `B + 1` candidates for `B` banks, - so **no arbitrary cap**; and the fold is `BankBook`'s own (`bank_book.h:263-269`), reached - through the new public member, never re-implemented in `import_plan`. Sample display names - are **not** suffixed, and `slot_map` positions are untouched. -- **Always a new bank; never a merge (Ε-F2, ruled).** The import creates a bank — it never - merges into an existing one, never lands into the pool, and offers no target picker. A - pool export therefore lands as a **named** bank `"Pool 2"`, which is correct, not a glitch. - This track ships **one** action, not two; merge-into-existing is out of scope for the - phase, and move/copy already cover the after-the-fact case. -- **Birth records at landing.** Every landed file goes through - `ReaSamplerSession::recordCreated(sample, OriginKind::PackageImport)` at the same point the - `Sample` is added, in the same straight-line block, per `core/tracking/CLAUDE.md`'s - no-silent-gaps invariant. An import that lands a file without a record is the exact failure - that section exists to prevent. -- **All-or-nothing, with rollback.** Any failure after the first write deletes the files - *this call wrote* and abandons the index mutation. The rollback cites the - `prune_fs.cpp:5-11` carve-out; it does not restate it, and it does not reach outside the set - it wrote. -- **One Ctrl-Z for the index, and the file residue is stated, not implied.** The index - mutation batches through `persistBankOp` (`Undo_BeginBlock2` / `Undo_EndBlock2`, verified at - `reaper_plugin_functions.h:7758` / `:7806`). Undo does **not** un-write the files; they - remain as orphans until a prune reclaims them — the same designed window a non-empty bank - delete already produces (`core/model/CLAUDE.md`'s sample-removal section). The user-facing - summary says so. -- **`bumpBankGeneration()` on success** (`session.h:121`), so live instances reload. -- **No timeline item is placed. Ever.** -- **A new FOREVER-STABLE command id**, minted the same way T1's is. - -**Acceptance criteria.** -- `planImport` is pure and total, and every one of the four collision classes has a test that - exercises it without a filesystem: colliding ids, colliding file names, a hash already - present, and a colliding bank name. -- Importing a package built from bank B back into the project that already contains B lands - a **new** bank named `"B 2"`, with every id reminted, no entry lost, and B itself - unmutated. Importing it a third time lands `"B 3"`. -- The suffix probe is pinned by pure tests over a name set, covering at minimum: a free seed - (no suffix applied), a case/whitespace-folded collision (`"drums"` blocks `"Drums"`), a gap - (`"Drums"` + `"Drums 3"` present ⇒ `"Drums 2"`), a seed that already ends in a number - (`"Drums 2"` colliding ⇒ `"Drums 2 2"`), an absent/blank recorded name (⇒ `Imported bank`), - and a package whose source bank was the pool (⇒ `"Pool 2"`, a named bank). -- A degraded ledger (`Unreadable` and `FutureVersion`, both asserted) refuses the import with - **no picker shown, zero files written, zero index mutation**, and the message names the - channel-correct ext-state namespace. An undecodable `rsusage_*` key with an otherwise - `Loaded` ledger **does not** block — asserted, because the tempting reuse of - `blockedByTracking` would silently make it. -- An entry whose payload fails its `hashBytes` check aborts the import with **zero** files - landed and **zero** index mutation — asserted on both, since either alone would pass a - weaker test. -- A write failure injected at entry k of n leaves exactly zero files from this import on - disk and the index unchanged. -- Every landed file has a ledger birth record with kind `PackageImport`, asserted by reading - the ledger after the import, not by counting calls. -- `minReaderVersion` above the build: nothing written, message names all three facts. - `formatVersion` above the build with `minReaderVersion` at or below it: **imports cleanly**, - unknown keys skipped. Both directions asserted, in this track, against real package bytes. -- No timeline item exists after an import; the arrange is byte-identical. -- **DAW-verification obligation** (to be discharged by Daniel, not by this track): import a - package produced on another machine, confirm the panel shows every sample with its - metadata, confirm a live ReaSampler 9000 instance picks up the new bank content on the - generation bump, and confirm one Ctrl-Z removes the index entries. - -**Open questions.** **No [Daniel] questions — Ε-F2 and Ε-F3 are both RULED** (new bank -always with an auto suffix; refuse on a degraded ledger), so this track is dispatchable as -written and ships one action. **[propose at review]** whether the import summary is a console -block, a message box, or both; the recommendation is a console block plus a one-line message -box, so the detail is copyable and the outcome is unmissable. Note the refusal path is -already fixed at a console block by the Ε-F3 spec, so this call is about the *success* -summary only. +**Landed** — see `docs/COMPLETED.md` for the full narrative. New +`core/package/import_plan` (pure: the id remap table, the parent remap, the per-entry +land/skip-already-present/rename disposition, and the destination bank's display name +after `BankBook`'s own uniqueness fold — reached through a new additive +`BankBook::uniqueDisplayName` member, the only `core/model/` edit in the phase), and on +the shell side a REAPER-free `import_landing` (decode, verify every payload's +`hashBytes` digest against the manifest BEFORE the bank folder is created, then land +through the rollback journal) plus a REAPER-facing `import_bank` (the only piece +touching the extension's project state — the undo-batched persist and the generation +bump), `shell/actions/package_import_action`, the panel's `.rsbank` drop route, one +`main.cpp` row, one panel menu row, and a new `src/core/util/ascii_ws.h`. The +tracking-ledger guard runs before the file picker opens (Ε-F3, ruled: refuse outright on +`Unreadable`/`FutureVersion`, no confirm-and-proceed); the version gate runs before any +byte is written; all four collision classes — sample id, file name, content hash, bank +display name — are answered explicitly, with the display-name collision auto-suffixed +and never prompted (Ε-F2, ruled: always a new bank, never a merge); birth records land +via `recordCreated(sample, OriginKind::PackageImport)` in the same straight-line block +as the index add; the index mutation is one Ctrl-Z, and the landed files' survival as +orphans until the next prune is stated in the user-facing summary, not left implicit. +**Beyond spec:** `import_plan`'s `spelledLikeABankFile` mints a fresh name even absent a +collision, whenever the package's own entry name isn't spelled the way +`deriveBankPaths` would spell it — counted separately from a genuine folder-name +collision (`sanitizeRenameCount` vs `collisionRenameCount`) so a hostile or +foreign-spelled entry name (e.g. an unexpected extension) always lands sanitized rather +than verbatim. --- From 24569956cf656b0c62b5fdf5d542681a1fa21b3d Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 15:57:06 -0400 Subject: [PATCH 22/24] =?UTF-8?q?docs:=20record=20=CE=95-W3=20as=20landed?= =?UTF-8?q?=20and=20Phase=20=CE=95=20as=20complete?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/COMPLETED.md | 38 ++++++++++++++++++++++++++++-- docs/PLAN.md | 60 +++++++++++------------------------------------ 2 files changed, 50 insertions(+), 48 deletions(-) diff --git a/docs/COMPLETED.md b/docs/COMPLETED.md index 7cda133..14e6574 100644 --- a/docs/COMPLETED.md +++ b/docs/COMPLETED.md @@ -1039,5 +1039,39 @@ genuine folder-name collision (`sanitizeRenameCount` vs `collisionRenameCount`) a hostile or foreign-spelled entry name (e.g. an unexpected extension) always lands sanitized rather than verbatim. -**Ε-W3 (`package-compat-fixtures`) has not landed** and is the only remaining -wave of Phase Ε. +### Ε-W3 — The compatibility fixtures + +The phase's third and final wave, and with it Phase Ε's implementation is complete: the +version-compatibility policy stated in `docs/product/bank-package.md` is now a property +proven against frozen bytes rather than an assertion in a doc. + +**Ε-W3-T1 — `package-compat-fixtures`.** A new checked-in corpus of 23 frozen `.rsbank` +fixtures under `tests/fixtures/package_compat/` — one v1 package written by the shipping +build (`1.4.0`), a synthetic additive-forward package (`formatVersion` 2 / +`minReaderVersion` 1) carrying three keys this build has never heard of, a synthetic +structural-refusal package (2/2), nine truncations (one per distinct decode failure +site, including one cut at `additive_forward.rsbank`'s own payload boundary), and eleven +hostile-name packages (six bad entry names, five bad nested `relativePath` values) — +every payload a single 300-byte 16-bit mono WAV, ~15 KB for the whole corpus. Two new +test targets decode and exercise it: `package_compat_tests` (frozen bytes decode to +exactly what the shipping build wrote, the additive fixture reads with every unknown key +skipped, every truncation classifies `Malformed` and never `TooNew`, every hostile name +is refused before any planner runs) and `package_round_trip_tests` (the same corpus +driven through the actual verbs — export → import → export over `v1_shipping.rsbank` +yields byte-identical payloads, and every refusal fixture refuses the whole import with +nothing landed and nothing in the index). A new repo-root `.gitattributes` (`*.rsbank +binary`) is load-bearing, not decoration: under `core.autocrlf = true`, git's NUL-sniffing +heuristic would text-classify a future short, ASCII-heavy fixture and CRLF-mangle it on a +Windows checkout, silently breaking the frozen-bytes premise the whole corpus rests on. A +standalone DAW verification script, `docs/verify-package-transfer.md`, covers the one +claim no unit test can make — a real cross-machine transfer, including the too-new +refusal, the truncated-download refusal, and mid-payload corruption, each read off as an +exact message string. **Open question resolved:** the recommendation (one-sample +packages, a few hundred bytes of payload each) was followed — the corpus holds +one-sample packages with a 300-byte payload each. **Deviation from spec:** the plan +called for a truncation cut mid-layout; RSBK stores no layout section (the layout is +derived from the manifest's entries, not stored as its own section), so the fixture that +exercises "the manifest parses, the layout computes, the exact-size proof fails" lands at +the payload boundary instead. No production module was touched — the wave adds test-tree +files, the corpus, its README, the verification script, and one path variable in the root +`CMakeLists.txt`. diff --git a/docs/PLAN.md b/docs/PLAN.md index 0583928..54b61c4 100644 --- a/docs/PLAN.md +++ b/docs/PLAN.md @@ -2404,6 +2404,9 @@ import gate can key on `LedgerStatus` alone without going through `pruneDryRun() enumeration+scan). **No Ε track touches `core/instrument/`, `shell/instrument/`, or any capture backend.** +**All three waves have landed — Phase Ε is complete.** W1 through W3 each carry their own +landed note below; see `docs/COMPLETED.md` for every track's full narrative. + --- ### Ε-W1 — The contract, the filesystem, and the ledger's new kind @@ -2568,52 +2571,17 @@ against a package this build is incapable of writing. **One track.** The whole deliverable is one corpus and the harness over it; splitting it would mean two tracks writing two halves of one fixture set. -#### Ε-W3-T1 — `package-compat-fixtures` - -**Goal.** Turn the version-compatibility policy from an assertion in a doc into a property -proven against **frozen bytes**, so a later format change cannot silently break either -direction. - -**Spec:** `docs/product/bank-package.md` §"Version tagging: both directions". - -**Surface boundary — owns:** a new checked-in fixture corpus under the package modules' test -tree, the harness that decodes it, and one `docs/` verification script for the DAW half. -**Does not own:** any production module — if a fixture reveals a defect, the fix is filed -against the owning track's module and this track carries the failing test, not the patch. - -**Behavior.** -- **Frozen bytes, not regenerated ones.** The corpus holds real `.rsbank` bytes committed to - the repo: a v1 package written by the shipping build, a synthetic - `formatVersion` = N+1 / `minReaderVersion` = current package (the additive-forward case), - and a synthetic `formatVersion` = N+1 / `minReaderVersion` = N+1 package (the refuse case). - A test that regenerates its own fixture proves only that the code agrees with itself — - which is precisely the failure mode a format ladder exists to catch. -- **A truncation corpus.** The valid package truncated at a spread of offsets, each asserted - `Malformed` rather than `TooNew`, so the two recoveries never get crossed. -- **A hostile-name corpus.** Packages whose entry names carry `..`, separators, and absolute - prefixes, each refused. -- **The round-trip anchor.** Export → import → export over the v1 fixture yields - byte-identical payloads. -- **A standalone DAW verification script** in `docs/`, following the - `docs/verify-track-scope-multitrack.md` precedent — the cross-machine transfer is the one - claim no unit test can make. -- **The corpus is append-only.** When a future format version ships, its fixture is added; - no existing fixture is ever regenerated or edited. Stated in the corpus's own README so the - rule survives the person who wrote it. - -**Acceptance criteria.** -- All three version fixtures classify as specified, and the additive-forward one imports with - every known field intact and every unknown key skipped. -- Every truncation offset classifies `Malformed`; none classifies `TooNew`, `Readable`, or - crashes. -- Every hostile-name fixture is refused at decode, before any planner runs. -- The round-trip fixture's payloads are byte-identical after export → import → export. -- The DAW script exists and names its steps concretely enough to run without reading this - plan. - -**Open questions.** **[propose at review]** how large the committed corpus is allowed to be — -the recommendation is one-sample packages with a few hundred bytes of payload each, since the -properties under test are structural and a large payload proves nothing extra. +**Ε-W3-T1 has landed** — `package-compat-fixtures` — see `docs/COMPLETED.md` for the full +narrative. 23 frozen `.rsbank` fixtures now live under `tests/fixtures/package_compat/` +(three version fixtures, nine truncations, eleven hostile-name packages, ~15 KB total), +decoded by a new `package_compat_tests` and driven through export → import → export by a +new `package_round_trip_tests`, plus the DAW script `docs/verify-package-transfer.md` for +the one claim no unit test can make. A repo-root `.gitattributes` (`*.rsbank binary`) +keeps the frozen bytes frozen under `core.autocrlf`. No production module was touched. +**Open question resolved:** the recommendation was followed — the corpus is one-sample +packages with a 300-byte payload each. **Deviation from spec:** RSBK stores no layout +section (it is derived from the manifest), so the planned "mid-layout" truncation lands +at the payload boundary instead, where the manifest parses but the exact-size proof fails. --- From a2ee5ff69d71aa01f18cb8bd6d8a9e33a9c2a582 Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 16:19:22 -0400 Subject: [PATCH 23/24] docs: the picker was never a deviation, it was a regex miss in the plan --- docs/COMPLETED.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/docs/COMPLETED.md b/docs/COMPLETED.md index 14e6574..dbbf5b9 100644 --- a/docs/COMPLETED.md +++ b/docs/COMPLETED.md @@ -974,13 +974,15 @@ reaches the destination only through a `commit()` rename — process-crash atomi not power-loss atomic, deliberately, since an `fsync` over a whole sample bank is a real stall) and the rollback journal (`package_rollback`'s `LandedFileJournal`, citing the `prune_fs.cpp` carve-out rather than restating it, disarmed only after -the caller's own write has returned success). **Deviation from spec:** the plan -called for asymmetric pickers — REAPER's `GetUserFileNameForRead` for import, Win32 -`GetSaveFileNameW`/SWELL `BrowseForSaveFile` for export, reasoning that the REAPER -API has no save picker. The landed `package_pickers` instead rides `GetUserFileName` -for both directions (mode 1 import, mode 0 export) — no platform split, since -`main.cpp` already aborts extension load if any needed API pointer fails to -resolve. Both pickers are `[verify — DAW]`, never exercised in a live REAPER +the caller's own write has returned success). `package_pickers` rides REAPER's own +`GetUserFileName` for both directions, as specified (mode 1 import, mode 0 export) +— the plan's "REAPER has no save picker" finding was a regex miss in the original +research, not a real gap, so there was no asymmetric-picker deviation to land: no +SWELL `BrowseForSaveFile`, no Win32 `GetSaveFileNameW`, no `GetUserFileNameForRead` +(the SDK header marks it superseded). REAPER owning the dialog on every platform is +why there's no platform split; that's separate from `main.cpp` already aborting +extension load if any needed API pointer fails to resolve, which is why no fallback +path is needed. Both pickers are `[verify — DAW]`, never exercised in a live REAPER session. **Ε-W1-T3 — `import-origin-kind`.** `OriginKind::PackageImport` appended to the From f33ef9af371807c3b52115423055f2830ca2a134 Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Sun, 2 Aug 2026 16:31:17 -0400 Subject: [PATCH 24/24] docs: give Phase E's DAW obligations a home in VERIFICATION.md --- docs/VERIFICATION.md | 40 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 39 insertions(+), 1 deletion(-) diff --git a/docs/VERIFICATION.md b/docs/VERIFICATION.md index 02cdf28..7b59278 100644 --- a/docs/VERIFICATION.md +++ b/docs/VERIFICATION.md @@ -1,6 +1,6 @@ # DAW verification — post-1.0 work on `dev` -Checks for Θ, Ξ, and Ψ work that no unit test can close. Build **Release**, install into +Checks for Θ, Ξ, Ψ, and Ε work that no unit test can close. Build **Release**, install into `UserPlugins/`, restart REAPER. Panel tail toggle = **None**, project rate 48000, unless a check says otherwise. ## Precision invariants @@ -59,6 +59,44 @@ Checks for Θ, Ξ, and Ψ work that no unit test can close. Build **Release**, i - [ ] Drag-out to an external app twenty-plus times in a row — audio arrives every time; this is a soak, a single pass is not a gate (`docs/COMPLETED.md:109`) - [ ] Drop a capture onto an FX container — the instrument loads with that capture (`docs/COMPLETED.md:110`) +## Bank packages + +- [ ] **Run in full.** `docs/verify-package-transfer.md` — the whole cross-machine + export/import round trip: writes-one-file, the transfer itself, re-importing the + same file never overwrites, the round trip back to the source, the too-new / + truncated / mid-payload-corruption refusals (each an exact string), the + unsaved-project refusals, and drag-and-drop (`docs/COMPLETED.md:1068`, `PLAN.md:2578`) +- [ ] Force a degraded tracking ledger and confirm the import refuses **before the + file picker opens**: save a project with a bank, close REAPER, edit the saved + `.rpp`'s `owned_files` ext-state value inside its `` block — corrupt + the JSON for the `Unreadable` case, or bump `"v":2` to `"v":3` for the + `FutureVersion` case — reopen the project, then run *ReaSampler: import bank + package (.rsbank)*. Read off: the console prints the ledger-refusal block and no + file dialog ever appears (`src/core/tracking/origin_ledger.h:95,101`, + `src/shell/actions/package_import_action.cpp:161-167`) +- [ ] Export dialog: type a destination name with no extension, then again over a + name that already carries a different one (e.g. `mybank.bak`) — read off whether + `GetUserFileName` appended `.rsbank` itself or ReaSampler's own re-append produced + the double-extension result (`mybank.bak.rsbank`) the code expects + (`src/shell/package/CLAUDE.md:112-122`) +- [ ] Both the export and the import file dialogs open in front of REAPER's main + window, not behind it — `GetUserFileName` takes no owner window + (`src/shell/package/CLAUDE.md:110-112`) +- [ ] With a ReaSampler 9000 instance's editor open on the destination project + (Browse view visible), import a `.rsbank` from the docked panel — the browser + reflects the new bank without closing or reopening the editor (the bank-generation + bump, `src/shell/persist/session.h:114-121`, polled by the instrument at + `src/shell/instrument/processor_reload.cpp:444-475`) +- [ ] Drag two or more `.rsbank` files onto the docked panel in one drop — each lands + as its OWN new bank, never merged into one, and if the tracking ledger is degraded + the refusal prints ONCE for the whole drop rather than once per file + (`src/shell/panel/panel_window.cpp:63-94`) +- [ ] Kill REAPER (or the process) partway through an import so a partial bank file + is stranded under its real name in the bank folder, then re-run the same import + into the same project — read off what happens. Whether the import verb should + pre-clean that stale debris is an open question, not yet decided + (`src/shell/package/CLAUDE.md:102-108`) + ## The resample bake - [ ] Bake a dialed sound — the banked file sounds like what the editor was playing (`docs/COMPLETED.md:737`)