package: one bank leaves the project as one .rsbank, or the export refuses and says why
Pure planner classifies missing/unreadable/unrepresentable and repairs transport names; the verb digests, streams and commits atomically over a const session.
This commit is contained in:
@@ -73,6 +73,12 @@ landing after the format.
|
||||
here, computed where payloads are streamed (shell). The bank's `slot_map`
|
||||
rides along. Unknown keys skip at every level; duplicate entry names are
|
||||
rejected both ways.
|
||||
- `export_plan` — the pure export decision over value inputs (the bank's members
|
||||
plus the shell's per-file probe result): the verdict (`Ready` / `Incomplete` /
|
||||
`Refused`), the transport name per shipping entry, and what is excluded and why
|
||||
(missing / unreadable / an index record the format cannot represent). Owns the
|
||||
name repair the codec's refusal backstops, and normalizes each shipping record's
|
||||
`relativePath` to the bare package name — see the transport-name gotcha below.
|
||||
- `bank_package` — framing and arithmetic composing the two above:
|
||||
`encodePackage` (prefix bytes + layout + total size, stamping this build's
|
||||
ladder pair and `version::stampVersion()`), `decodePackage` (prefix + observed
|
||||
@@ -115,6 +121,14 @@ landing after the format.
|
||||
`minReaderVersion` bump, not additive: an old reader would otherwise compare
|
||||
a stored digest against bytes hashed the new way and silently misjudge
|
||||
corruption.
|
||||
- **A written package carries no path in ANY field.** `isValidNestedSamplePath`
|
||||
permits a relative `relativePath` because a *record* may hold one, but
|
||||
`export_plan` writes each shipping entry's `relativePath` as its bare package
|
||||
name, so an emitted manifest has no separator anywhere and the entry name is the
|
||||
single naming authority on both sides. The directory component it drops carries
|
||||
no information — the bank subfolder is a fixed `capture_paths` constant the
|
||||
importer re-spells. The nested-path rule stays as the decode-side backstop for a
|
||||
package this build did not write.
|
||||
- **Obligation on the export track: sanitize, don't relay the refusal.**
|
||||
`serializeManifest` returns one indistinguishable `nullopt` for every rejection
|
||||
— an unrepresentable name, a case-folded collision, a traversing nested path, a
|
||||
|
||||
@@ -6,6 +6,11 @@ reasampler_pure_library(package_manifest
|
||||
LINK PUBLIC bank_model slot_map PRIVATE package_format json)
|
||||
reasampler_test(package_manifest LINK package_manifest)
|
||||
|
||||
reasampler_pure_library(export_plan
|
||||
SOURCES export_plan.cpp
|
||||
LINK PUBLIC package_manifest PRIVATE package_format)
|
||||
reasampler_test(export_plan LINK export_plan package_format)
|
||||
|
||||
# bytes.h is header-only (see src/core/wire/CLAUDE.md) — no wire link edge needed.
|
||||
reasampler_pure_library(bank_package
|
||||
SOURCES bank_package.cpp
|
||||
|
||||
@@ -0,0 +1,165 @@
|
||||
// export_plan.cpp — see export_plan.h for the contract.
|
||||
|
||||
#include "core/package/export_plan.h"
|
||||
|
||||
#include <cstddef>
|
||||
|
||||
#include "core/package/package_format.h"
|
||||
|
||||
namespace reasampler::package {
|
||||
|
||||
namespace {
|
||||
|
||||
// The bare file name a bank-relative path ends in. Separators are matched in both
|
||||
// spellings: a persisted index may hold either on Windows.
|
||||
std::string baseNameOf(const std::string& path) {
|
||||
const std::size_t sep = path.find_last_of("/\\");
|
||||
return sep == std::string::npos ? path : path.substr(sep + 1);
|
||||
}
|
||||
|
||||
// Trailing dots and spaces are stripped at file creation on Windows, so a name
|
||||
// carrying them would collide with its stripped twin (isValidEntryName refuses them
|
||||
// for that reason).
|
||||
std::string stripTrailingDotsAndSpaces(std::string s) {
|
||||
while (!s.empty() && (s.back() == '.' || s.back() == ' ')) s.pop_back();
|
||||
return s;
|
||||
}
|
||||
|
||||
// Truncates to at most `max` bytes without splitting a UTF-8 sequence — a split one
|
||||
// would leave the name ill-formed, which isValidEntryName refuses outright.
|
||||
std::string truncateUtf8(std::string s, std::size_t max) {
|
||||
if (s.size() <= max) return s;
|
||||
s.resize(max);
|
||||
while (!s.empty() && (static_cast<unsigned char>(s.back()) & 0xC0) == 0x80) s.pop_back();
|
||||
if (!s.empty() && static_cast<unsigned char>(s.back()) >= 0xC0) s.pop_back();
|
||||
return s;
|
||||
}
|
||||
|
||||
// `name` with `suffix` inserted before its extension, trimmed so the result still
|
||||
// fits the entry-name cap.
|
||||
std::string insertSuffix(const std::string& name, const std::string& suffix) {
|
||||
const std::size_t dot = name.find_last_of('.');
|
||||
const bool hasExt = dot != std::string::npos && dot > 0;
|
||||
std::string stem = hasExt ? name.substr(0, dot) : name;
|
||||
const std::string ext = hasExt ? name.substr(dot) : std::string();
|
||||
const std::size_t room = kMaxEntryNameBytes - suffix.size() - ext.size();
|
||||
stem = truncateUtf8(std::move(stem), room);
|
||||
return stem + suffix + ext;
|
||||
}
|
||||
|
||||
bool nameTaken(const std::string& candidate, const std::vector<std::string>& taken) {
|
||||
for (const std::string& t : taken)
|
||||
if (sameEntryName(candidate, t)) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
// A transport name distinct from every name already claimed, under the format's own
|
||||
// case-folding equivalence (two names differing only by ASCII case would extract onto
|
||||
// one file on Windows and default APFS).
|
||||
std::string uniqueEntryName(const std::string& base, const std::vector<std::string>& taken) {
|
||||
if (!nameTaken(base, taken)) return base;
|
||||
// Bounded by construction: each iteration either returns or collides with a
|
||||
// distinct member of `taken`, and the suffixed names are pairwise distinct.
|
||||
std::string candidate = base;
|
||||
for (std::size_t n = 2; n <= taken.size() + 2; ++n) {
|
||||
candidate = insertSuffix(base, "_" + std::to_string(n));
|
||||
if (!nameTaken(candidate, taken) && isValidEntryName(candidate)) return candidate;
|
||||
}
|
||||
return candidate;
|
||||
}
|
||||
|
||||
// What BankModel::add and the manifest's nested-path rule together accept — the pair
|
||||
// package_manifest::serializeManifest checks per entry. The codec's refusal is the
|
||||
// backstop; classifying here is what lets the export name the offending entry.
|
||||
bool recordRepresentable(const model::Sample& s) {
|
||||
return !s.id.empty() && isValidNestedSamplePath(s.relativePath);
|
||||
}
|
||||
|
||||
ExcludedEntry excludedFrom(const model::Sample& s, ExclusionReason reason) {
|
||||
ExcludedEntry e;
|
||||
e.sampleId = s.id;
|
||||
e.displayName = s.displayName;
|
||||
e.relativePath = s.relativePath;
|
||||
e.reason = reason;
|
||||
return e;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
std::string sanitizeEntryName(const std::string& rawFileName) {
|
||||
std::string n = rawFileName;
|
||||
for (char& c : n) {
|
||||
const unsigned char u = static_cast<unsigned char>(c);
|
||||
if (u < 0x20 || u == 0x7F || u == '/' || u == '\\' || u == ':' || u == '*' ||
|
||||
u == '?' || u == '|' || u == '<' || u == '>' || u == '"')
|
||||
c = '_';
|
||||
}
|
||||
// One byte of headroom so the prefix repair below still fits the cap.
|
||||
n = stripTrailingDotsAndSpaces(truncateUtf8(std::move(n), kMaxEntryNameBytes - 1));
|
||||
if (isValidEntryName(n)) return n;
|
||||
|
||||
// One prefix answers every remaining reserved form at once: "." / "..", a DOS
|
||||
// device name, and a name the strips emptied.
|
||||
std::string prefixed = stripTrailingDotsAndSpaces("_" + n);
|
||||
if (isValidEntryName(prefixed)) return prefixed;
|
||||
|
||||
// Ill-formed UTF-8 is what is left, and isValidEntryName is the only authority on
|
||||
// it here, so fold the whole non-ASCII range rather than re-deriving the scanner.
|
||||
for (char& c : prefixed)
|
||||
if (static_cast<unsigned char>(c) >= 0x80) c = '_';
|
||||
prefixed = stripTrailingDotsAndSpaces(prefixed);
|
||||
return isValidEntryName(prefixed) ? prefixed : std::string("entry");
|
||||
}
|
||||
|
||||
ExportPlan planExport(const ExportInputs& in) {
|
||||
ExportPlan plan;
|
||||
plan.manifest.bankDisplayName = in.bankDisplayName;
|
||||
|
||||
bool anyAbsent = false;
|
||||
bool anyUnrepresentable = false;
|
||||
std::vector<std::string> takenNames;
|
||||
std::vector<std::string> shippedIds;
|
||||
|
||||
for (const ExportCandidate& c : in.candidates) {
|
||||
if (!recordRepresentable(c.sample)) {
|
||||
plan.excluded.push_back(
|
||||
excludedFrom(c.sample, ExclusionReason::RecordUnrepresentable));
|
||||
anyUnrepresentable = true;
|
||||
continue;
|
||||
}
|
||||
if (c.fileState != SourceFileState::Present) {
|
||||
plan.excluded.push_back(excludedFrom(
|
||||
c.sample, c.fileState == SourceFileState::Unreadable
|
||||
? ExclusionReason::FileUnreadable
|
||||
: ExclusionReason::FileMissing));
|
||||
anyAbsent = true;
|
||||
continue;
|
||||
}
|
||||
|
||||
PackageEntry e;
|
||||
e.sample = c.sample;
|
||||
e.fileName = uniqueEntryName(sanitizeEntryName(baseNameOf(c.sample.relativePath)),
|
||||
takenNames);
|
||||
// The transport record names its payload by the package name and nothing
|
||||
// else, so the manifest carries no path at all — the bank subfolder is a
|
||||
// fixed constant the importer re-spells through capture_paths.
|
||||
e.sample.relativePath = e.fileName;
|
||||
|
||||
takenNames.push_back(e.fileName);
|
||||
shippedIds.push_back(c.sample.id);
|
||||
plan.sourceRelativePaths.push_back(c.sample.relativePath);
|
||||
plan.manifest.entries.push_back(std::move(e));
|
||||
}
|
||||
|
||||
// Display positions follow membership: an excluded entry's slot marker would name
|
||||
// a sample the package does not carry.
|
||||
plan.manifest.slots = in.slots;
|
||||
plan.manifest.slots.reconcile(shippedIds);
|
||||
|
||||
plan.verdict = anyUnrepresentable ? ExportVerdict::Refused
|
||||
: anyAbsent ? ExportVerdict::Incomplete
|
||||
: ExportVerdict::Ready;
|
||||
return plan;
|
||||
}
|
||||
|
||||
} // namespace reasampler::package
|
||||
@@ -0,0 +1,89 @@
|
||||
#pragma once
|
||||
// export_plan — the pure export decision: which bank entries ship, what each one is
|
||||
// named inside the package, what is absent, and therefore whether the export may
|
||||
// proceed at all. Values in, verdict out — the shell probes the filesystem and hands
|
||||
// the results here. Pure: no filesystem, no host types.
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "core/model/bank_model.h"
|
||||
#include "core/model/slot_map.h"
|
||||
#include "core/package/package_manifest.h"
|
||||
|
||||
namespace reasampler::package {
|
||||
|
||||
// What the shell's filesystem probe found for one indexed entry. Missing and
|
||||
// Unreadable stay distinct all the way to the refusal message: the file is gone vs.
|
||||
// the file is there and will not open, which have opposite recoveries.
|
||||
enum class SourceFileState {
|
||||
Present,
|
||||
Missing,
|
||||
Unreadable,
|
||||
};
|
||||
|
||||
struct ExportCandidate {
|
||||
model::Sample sample;
|
||||
SourceFileState fileState = SourceFileState::Missing;
|
||||
};
|
||||
|
||||
// One bank as the planner sees it: the display name that rides in the manifest
|
||||
// envelope, the members in bank insertion order, and the bank's display positions.
|
||||
struct ExportInputs {
|
||||
std::string bankDisplayName;
|
||||
std::vector<ExportCandidate> candidates;
|
||||
model::SlotMap slots;
|
||||
};
|
||||
|
||||
// Why an indexed entry cannot ship.
|
||||
enum class ExclusionReason {
|
||||
FileMissing,
|
||||
FileUnreadable,
|
||||
// The index record itself cannot be written: an empty id, or a relativePath the
|
||||
// format's nested-path rule refuses. Not something a confirm can proceed past.
|
||||
RecordUnrepresentable,
|
||||
};
|
||||
|
||||
struct ExcludedEntry {
|
||||
std::string sampleId;
|
||||
std::string displayName;
|
||||
std::string relativePath;
|
||||
ExclusionReason reason = ExclusionReason::FileMissing;
|
||||
};
|
||||
|
||||
enum class ExportVerdict {
|
||||
Ready, // every candidate ships
|
||||
Incomplete, // a file is absent or unreadable; the rest may ship behind an explicit confirm
|
||||
Refused, // an index record the format cannot represent — no confirm path
|
||||
};
|
||||
|
||||
struct ExportPlan {
|
||||
ExportVerdict verdict = ExportVerdict::Ready;
|
||||
|
||||
// Entries in bank order, each carrying its transport name and its record. The
|
||||
// shell measures `byteLength`/`byteHash` from the payload, so they are 0/"" here;
|
||||
// `exportTimestamp` is the shell's clock read and is 0 here too.
|
||||
PackageManifest manifest;
|
||||
|
||||
// Where each shipping entry's bytes are read from, parallel to
|
||||
// `manifest.entries` — the record's own relativePath is normalized to the bare
|
||||
// package name (see the transport-name note in this directory's CLAUDE.md), so
|
||||
// the source spelling has to survive separately.
|
||||
std::vector<std::string> sourceRelativePaths;
|
||||
|
||||
std::vector<ExcludedEntry> excluded;
|
||||
};
|
||||
|
||||
ExportPlan planExport(const ExportInputs& in);
|
||||
|
||||
// The smallest repair of one bare file name that satisfies isValidEntryName —
|
||||
// separators, reserved characters and control bytes to '_', an over-long name
|
||||
// truncated on a UTF-8 boundary, and an underscore prefix for the reserved forms
|
||||
// ("." / ".." / a DOS device name). Never returns a name isValidEntryName refuses.
|
||||
//
|
||||
// A bank ingested on macOS/Linux legitimately holds names Windows cannot spell, and
|
||||
// relaying the codec's one indistinguishable refusal would make a single such file
|
||||
// an unactionable total failure of the whole export.
|
||||
std::string sanitizeEntryName(const std::string& rawFileName);
|
||||
|
||||
} // namespace reasampler::package
|
||||
Reference in New Issue
Block a user