reasampler/src/core/package/CLAUDE.md

src/core/package — the pure RSBK bank-package codec

Scope

The hand-rolled RSBK bank-package container, entirely pure (REAPER-free, unit-tested outside the DAW): the format contract and version ladder, the JSON manifest, and the framing/layout codec. No filesystem — the shell (src/shell/package) streams bytes against the layouts produced here. The export/import decisions (export_plan / import_plan) are separate modules; both have landed.

Invariants

• The container is the proprietary RSBK — ruled, not revisitable here. No ZIP, no compressor, no link edge to vendor/WDL/WDL/zlib/. The version ladder, not a format swap, is how the format moves.
• Two version integers, two jobs. formatVersion = what the writer emitted; minReaderVersion = the oldest reader that can read it safely. The reader's whole rule is minReaderVersion <= kPackageFormatVersion. Additive changes (a new optional manifest key, a new enum value with a defined degrade) bump formatVersion only; structural changes bump both. The full ladder lives as a comment in package_format.h and is READ and validated, never merely written.
• TooNew refuses whole. A minReaderVersion above this build yields the header (so the refusal can name the writer's semver and version) and nothing else — no manifest, no layout, no half-success. The fields through the writer semver are FROZEN for all future versions to keep that refusal producible.
• Every name and path in the format is validated on encode AND decode, to the extent stated below, because a package can arrive from anywhere. Three rules, all in package_format, whose doc comments are the itemized authority:
  • isValidEntryName — a payload's name is a bare file name (no separators, no .. component, no drive/UNC/rooted form, no control bytes, no Windows-reserved character, no trailing dot/space, no DOS device name, well-formed UTF-8 only). Path expression is impossible in this field.
  • sameEntryName — two entry names differing only by ASCII case are ONE name. Windows and macOS's default APFS would extract them onto a single file, and a bank authored on a case-sensitive filesystem produces the pair honestly.
  • isValidNestedSamplePath — the nested Sample::relativePath IS a path by design, and is the one field here that can express one. It refuses a .. component and every absolute form; BankModel::add checks only the latter, so traversal would otherwise reach a future import_plan inside a record the format vouched for. Scope is traversal and absolute-form only — no UTF-8 well-formedness check (unlike isValidEntryName), no device-name check, no case-fold dedup on relativePath (unlike sameEntryName on the entry name). Correct for what this field is — a record field, not a filesystem destination; BankModel::add owns the rest. Forward contract for import_plan: the destination file is derived from the entry name, never from relativePath.
• Framing only, never a payload. bank_package produces header bytes and an ordered {name, offset, length} layout; it never holds, copies, or hashes an entry's audio. decodePackage proves prefix + payload lengths equal the observed file size exactly, so truncation and trailing garbage are Malformed without any payload being read.
• Per-sample shape has one owner. Each entry nests a one-sample BankModel blob emitted/parsed by bank_model's own codec (the bank_book_json precedent), so a future Sample field reaches packages with no change here.
• Hostile input: error signaled, never UB — the bank_model.h deserialize standard, plus allocation caps on every length field so a forged header cannot demand gigabytes.

Modules

• package_format — the contract: magic, kPackageFormatVersion / kPackageMinReaderVersion, the ladder comment, the three-way classifyPackageVersion (Readable / TooNew / Malformed), the three naming rules above, and PackageHeader.
• package_manifest — the manifest model (PackageEntry / PackageManifest) and its JSON codec. Per entry: bare name, byte length, and a whole-file capture::hashBytes digest (deliberately NOT hashWavContent, which skips chunks and cannot answer "did these bytes survive") — the digest is carried here, computed where payloads are streamed (shell). The bank's slot_map rides along. Unknown keys skip at every level; duplicate entry names are rejected both ways.
• export_plan — the pure export decision over value inputs (the bank's members plus the shell's per-file probe result): the verdict (Ready / Incomplete / Refused), the transport name per shipping entry, and what is excluded and why (missing / unreadable / an index record the format cannot represent). Owns the name repair the codec's refusal backstops, and normalizes each shipping record's relativePath to the bare package name — see the transport-name gotcha below.
• import_plan — the pure import decision, and the reason the whole feature is testable without a DAW: the destination bank's display name after BankBook's own fold, the reminted sample ids and remapped parents, and the per-entry land / collapse / rename disposition. Also importLedgerRefusal (the import's ledger gate, delegating entirely to tracking::ledgerDegraded) and ledgerRefusalMessage (the gate's console-block body, a pure (LedgerRefusal, namespace) -> string fold the shell only supplies the channel-correct namespace to).
• bank_package — framing and arithmetic composing the four above: encodePackage (prefix bytes + layout + total size, stamping this build's ladder pair and version::stampVersion()), decodePackage (prefix + observed file size in; header/manifest/layout out), and requiredPrefixSize (the incremental-read seam for the shell). Framing rides core/wire/bytes.h.

package_compat_tests is declared here with no library of its own: it decodes the frozen .rsbank corpus at tests/fixtures/package_compat/, whose README owns the append-only rule and the per-fixture inventory.

Gotchas

• Enums nested inside the BankModel blob follow bank_model's own rule — an out-of-range sourceMode/tier REJECTS the parse — so growing one of those vocabularies is a minReaderVersion bump for packages, not an additive change. Any enum integer the manifest itself ever adds must instead follow the degrade-to-Unknown rule (core/wire's BakeStatus precedent) to stay additive. The manifest carries no enum of its own today.
• Sample-id rules (remap, collision, dedup across the destination) are deliberately NOT enforced by the codec — they are import_plan decisions. The codec rejects only what makes the container itself incoherent (duplicate entry names, invalid names, a non-single-sample nested index).
• import_plan consults no other bank's hashes, and that is the ruling, not an omission. An import always creates a NEW bank, so "already present in the destination bank by content" is exactly "already landed by this same plan". Cross-bank dedup is not enforced anywhere (core/model/CLAUDE.md), so a hash the pool already holds still lands its own file here.
• requiredPrefixSize trusts fields beyond the frozen region only when the version pair classifies Readable; for TooNew it stops at the semver — don't "fix" it to read the manifest length there, a future structural format may have moved it.
• A package whose header classifies Readable (fv > ours, minReader still within reach — the additive case) but whose manifest fails to parse is reported TooNew, not Malformed: the header is valid and already carries the writer's semver, so the refusal can still name what to install. This widens TooNew to cover "read and failed" as well as "stopped at the frozen region" — both refuse whole and write nothing, so the safety property is unchanged, only the message. classifyPackageVersion and the frozen-region TooNew path are unaffected; this is the post-manifest-parse branch only.
• The parse branch is the ONLY one that relabels, deliberately: a newer package that trips the manifest cap, a short manifest read, the layout overflow, or the exact-size proof still reports Malformed even with formatVersion above ours. The size proof clearly should — "install 1.9.0" does not fix a truncated download — and the other three are indistinguishable from ordinary corruption at the point they fail. Don't "complete" the relabel across them for symmetry; the split is the answer, not an omission.
• The format carries no algorithm tag for byteHash — it is FNV-1a (capture::hashBytes) implicitly. Changing the digest algorithm is a minReaderVersion bump, not additive: an old reader would otherwise compare a stored digest against bytes hashed the new way and silently misjudge corruption.
• A written package carries no path in ANY field. isValidNestedSamplePath permits a relative relativePath because a record may hold one, but export_plan writes each shipping entry's relativePath as its bare, sanitized and disambiguated transport name (export_plan.cpp's e.fileName), so an emitted manifest has no separator anywhere and the entry name is the single naming authority on both sides. The directory component it drops carries no information — the bank subfolder is a fixed capture_paths constant (capture_paths.cpp's deriveBankPaths) the importer re-spells. The basename spelling is dropped too, not just the directory: e.fileName is uniqueEntryName(sanitizeEntryName(...)), not the source basename, so a macOS-authored Hit?.wav survives only in displayName — the transport name itself may differ. Accepted for the same reason the directory drop is: the transport name exists to be a valid, collision-free package entry, not a faithful copy of the source spelling, and displayName is the field that carries the original for display. The nested-path rule stays as the decode-side backstop for a package this build did not write.
• Obligation on the export track: sanitize, don't relay the refusal. serializeManifest returns one indistinguishable nullopt for every rejection — an unrepresentable name, a case-folded collision, a traversing nested path, a zero-length entry, a record BankModel::add refuses — and most of the naming rules are Windows'. A bank ingested on macOS/Linux legitimately holds Hit?.wav, snare .wav, or two names differing only by case, and a nested relativePath is only checked for the absolute forms where it is written. Relaying the nullopt makes ONE such file an unactionable total failure of the whole export. export_plan must map bank entries to package names that satisfy these rules (and disambiguate case-folded collisions) before calling this layer; the codec's refusal is the backstop, not the user-facing behaviour.
• NFC/NFD normalization collisions are accepted, not solved. macOS compares file names normalization-insensitively, so the NFC and NFD spellings of one accented name are two manifest entries that extract onto one file — the same collision class as the ASCII case fold, which sameEntryName does catch. A table-free fix does not exist, and restricting names to ASCII would be genuinely over-strict for non-English users. Left open knowingly.
• duplicateName folds through a hash set, not a pairwise scan. Under the kMaxManifestBytes cap (64 MB) a minimal entry is ~100 bytes, so a hostile package can declare ~670k entries; the former double loop was ~2×10¹¹ pair comparisons — a multi-minute hang on the decode path an import drives. The set is keyed on entryNameKey, which is sameEntryName's ASCII-case fold made explicit, so the equivalence rule still has one home (lowerAscii). Do not reintroduce the pairwise scan.
• Cross-module contract with src/shell/package: a genuinely zero-length entry cannot round-trip through the filesystem seam there (appendPayload refuses an empty payload — an empty buffer signals an upstream read failure, not a real entry). serializeManifest refuses a zero-length PackageEntry at encode so this layer never produces one; decode does not enforce it (a hostile/older package declaring one is not this track's concern).
• import_plan's spelledLikeABankFile mints a fresh name even with NO collision, and that third condition is a deliberate decision, not spec-derived. docs/product/bank-package.md §"Identity and collision on import" (collision rule 2) ties the auto-rename mint to a collision only; spelledLikeABankFile additionally mints whenever the package's own name isn't spelled the way deriveBankPaths spells one (extension, sanitized stem). Kept for two reasons: uniform folder spelling for every landed file regardless of origin, and — the sharper one — a hostile entry name that isn't a legal Windows filename or carries an unexpected extension (e.g. evil.exe) lands sanitized (evil_<tag>.wav) rather than verbatim. ImportPlan counts this separately from a genuine folder-name collision (sanitizeRenameCount vs collisionRenameCount) so the summary line means what bank-package.md §"Identity and collision on import" (collision rule 2) says it means.