daniel/reasampler
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(product): lock D1/D5/D6 for MIDI-playback instrument; add bare-VST3-vs-JUCE assessment

Browse Source 
Native VST3, Windows-only, two-products-but-integrated. Retire the JSFX fork
in place, add grounded assessment of the raw Steinberg VST3 surface without JUCE,
and re-present the residual forks in priority order.
This commit is contained in:
Daniel Harvey
2026-07-26 07:10:04 -04:00
parent 1bba95a0e8
commit 0d07d3cbfe
1 changed files with 414 additions and 237 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/product/midi-playback.md
+414 -237
View File
@@ -2,22 +2,49 @@
Framing for a **MIDI-triggered audio sampler** that plays back ReaSampler's captured
banks. This is a **discussion-shaping doc, not a build plan** — no phase, no PLAN.md
points, no settled forks yet. It exists so Daniel and the-boss can react to an honest
map of the option space before anything is scoped.
points. Several forks that were open in prior drafts are now **locked** (see below);
what remains open is re-presented in §4 so Daniel can make the residual calls before
anything is scoped.
Status: framed by product-designer (2026-07-26), **revised 2026-07-26** to correct a
material omission — the earlier draft missed REAPER's **VST-host bridge**: a VST/VST3
plugin *hosted inside REAPER* can call back into REAPER's own API from within the
plugin, resolving API function pointers by name over the host `audioMaster` callback.
That single fact changes the integration seam (§2), the "one product or two" question
(D6), and the JSFX-vs-native calculus (D1/D2). The corrections are made plainly
in-place, not appended as a footnote; where the prior draft was too pessimistic about
what a native instrument can reach, it is fixed and the correction is flagged. Grounded
in the vendored REAPER SDK headers (`vendor/reaper-sdk/sdk/reaper_plugin.h`,
`reaper_plugin_functions.h`, `video_processor.h`, `reaper_plugin_fx_embed.h`) and
REAPER's published VST-extensions SDK page (`reaper.fm/sdk/vst/vst_ext.php`) — the
plugin-format claims below are checked against those, not asserted. The genuine forks
are flagged as **Daniel's to decide**; nothing here pre-decides them.
Status: framed by product-designer (2026-07-26), **revised 2026-07-26 (r3)**. Prior
revisions (a) established that a REAPER *extension* cannot be a MIDI instrument, and (b)
corrected a material omission — REAPER's **VST-host bridge**, which lets a VST3 plugin
*hosted inside REAPER* call back into REAPER's own API by resolving function pointers by
name over the host callback. This revision folds in **Daniel's locked decisions** (D1,
D5, D6 — see the box below), retires the now-settled forks in place (marked **DECIDED**,
with the reasoning preserved, not deleted), and adds a substantive, honest assessment of
**what complying with the VST3 surface actually costs without JUCE** (§1a). Grounded in
the vendored REAPER SDK headers (`vendor/reaper-sdk/sdk/reaper_plugin.h`,
`reaper_plugin_functions.h`, `video_processor.h`, `reaper_plugin_fx_embed.h`), REAPER's
published VST-extensions SDK page (`reaper.fm/sdk/vst/vst_ext.php`), and the Steinberg
VST3 SDK documentation (portal + class reference — cited inline in §1a). Where a claim is
**experienced estimate** rather than a **verified-from-source** fact, it is flagged as
such in §1a. The genuine remaining forks are flagged as **Daniel's to decide**; nothing
here pre-decides them.
> ## Locked decisions (SETTLED — do not re-present as open)
>
> Daniel has nailed these down. Everything downstream assumes them.
>
> - **D6 → DECIDED: two products, but integrated.** The instrument is a *separate*
> product/artifact from the ReaSampler extension, but *tightly integrated* via the
> VST-host bridge — it reads live `"reasampler"` project ext-state and is project-aware.
> **Not** a divorced, file-only companion. (The old §4 D6 reasoning is preserved below,
> marked decided.)
> - **D1 → DECIDED: native VST3.** JSFX is off the table. The instrument is a native VST3
> plugin. (The old JSFX option and its whole-doc entanglement are retired below, marked
> decided; the JSFX reasoning is kept as the record of *why* it was considered and set
> aside.)
> - **D5 → DECIDED: no cross-platform, no multiformat. REAPER-specific, Windows-only.**
> The existing extension is Windows-only; Daniel does not work on other platforms. So:
> **no mac/Linux DSP/build/signing matrix, no CLAP-for-portability argument, no
> "runs standalone in other hosts" concern.** REAPER-coupling via the bridge is fine and
> intended. This collapses several costs the prior draft carried (§1a, §4).
>
> **What these lock-downs do to the shape:** the doc is no longer weighing "extension vs.
> plugin," "JSFX vs. native," or "portable vs. coupled." It is weighing **how to build one
> native, Windows-only, REAPER-coupled VST3 instrument** — and the single biggest live
> question inside that is now **bare Steinberg VST3 SDK vs. JUCE** (§1a, D-A below).
> **The bridge, stated once, up front (the correction).** A hosted VST is not limited to
> scraping bank WAVs + JSON off disk. REAPER hands the plugin its host callback; calling
@@ -109,94 +136,239 @@ the playback engine must be a **standard instrument plugin** (VST3 / CLAP / JSFX
*not* the extension. The extension SDK is the wrong format for that job, and this is
the single most important thing for Daniel to internalize before scoping anything.
### The three honest options
### The shape, now that D1/D6 are locked
**Option A — a real VSTi/instrument plugin (JUCE or bare VST3/CLAP SDK) that reads
ReaSampler's banks.** A separate build artifact: a VST3 (and/or CLAP) sampler plugin
that the user instantiates on an instrument track. It maps samples across the keyboard
and plays them MIDI-triggered with a real voice engine. This is the "sophisticated
sampler" answer.
- *Gives you:* everything an instrument is — polyphony, velocity, envelopes, the works,
fully integrated into REAPER's routing/render/record path like any VSTi. **And,
because it's hosted in REAPER, it reaches ReaSampler's project state directly via the
VST-host bridge** — it doesn't merely parse bank JSON off disk, it can read the live
`"reasampler"` ext-state `persist` writes, know its own host project, and follow the
active project. The prior draft treated Option A as a divorced file-reader; that was
the omission. A native ReaSampler instrument can be **natively, tightly integrated with
the extension's project state**, not loosely coupled through a file format alone.
- *Costs:* a **second codebase in a second plugin format**, almost certainly a new
dependency (JUCE is the pragmatic choice; bare VST3 SDK is more code, CLAP is leaner
but younger). It is a real DSP/voice-engine build, not a weekend. Cross-platform DSP,
its own build/release/signing story, its own UI toolkit. This is a **product-sized
commitment**, not a feature. And leaning on the bridge for state means the instrument
is **REAPER-coupled** — it stops being a portable VST that runs in any host and becomes
a REAPER companion. That may be exactly right (see D6), but it is a real narrowing and
must be a conscious choice, not a side effect.
The old three-way option set (native VSTi / JSFX / hybrid) has collapsed to a single
resolved shape:
**Option B — a JSFX sampler.** JSFX is REAPER's built-in scriptable plugin format
(text `.jsfx` files, JIT-compiled by REAPER, hostable as an instrument). A JSFX
instrument *can* receive live MIDI and emit audio on a track. It can load samples
(`Xen`-style file reads / the JSFX file/serialize API) and play them back.
- *Gives you:* a real in-track instrument with **zero new binary, zero new SDK, zero
JUCE** — ships as a text file alongside the extension, cross-platform for free
(REAPER runs the JIT everywhere it runs).
- *Costs:* JSFX is a constrained DSP scripting language, not C++. A polyphonic
multisample engine with round-robin/velocity-layers/streaming is *doable* but you're
writing DSP in JSFX's idiom, and large-sample streaming / disk I/O is more awkward
than in a native plugin. Reading ReaSampler's JSON index from JSFX is friction (JSFX
is not a general-purpose file parser). Best fit for a **minimal-to-mid** sampler, a
real ceiling for a **sophisticated** one.
- *What JSFX can't match, now that the bridge is on the table:* the VST-host bridge is a
**VST/VST3/CLAP capability** — it is how a *hosted plugin* resolves REAPER API pointers.
JSFX is not a hosted VST; it has REAPER integration of its own kind (`ext_noinit`,
`file_*` ops, the shared `gmem[]`, `slider`/parameter plumbing), but it does **not** get
the same string-keyed REAPER-API surface a native VST does, and in particular has no
clean, native path to read `persist`'s `"reasampler"` project ext-state the way a native
VST can. On the JSFX path, the seam is genuinely "read a file ReaSampler wrote next to
the bank" (hence the sidecar note in §2). This asymmetry is new information: the bridge
makes **native meaningfully more integratable** than the prior draft assumed — it's not
just "native is more powerful DSP," it's "native can be *part of ReaSampler's state
model*, JSFX stays a file-coupled consumer."
**A separate native VST3 instrument that consumes ReaSampler's banks, tightly integrated
via the bridge.** The extension stays the sole owner of **capture + organization** (its
whole existing identity and the load-bearing "capture and placement are separate acts"
principle). The instrument is a *separate build artifact* — a native VST3 sampler the user
instantiates on an instrument track — that maps samples across the keyboard and plays them
MIDI-triggered with a real voice engine. It reaches ReaSampler's project state directly via
the VST-host bridge: it reads the live `"reasampler"` ext-state `persist` writes, knows its
own host project, and follows the active project. This is the "two products, but
integrated" shape D6 locked in. The extension never becomes an instrument; the instrument
never captures.
**Option C — the hybrid (recommended framing to explore first).** Keep the extension
as the sole owner of **capture + organization** (which is its whole existing identity
and the load-bearing "capture and placement are separate acts" principle). Add a
**separate instrument** (Option A *or* B) that **consumes the banks** as a
shared-artifact contract. The extension never becomes an instrument; the instrument
never captures. Each does what its format is good at. This is the honest shape of the
whole thing — the two options above are really "which instrument technology" *within*
the hybrid, because the extension is staying regardless.
*What this gives:* everything an instrument is — polyphony, velocity, envelopes, the works,
fully in REAPER's routing/render/record path like any VSTi — **plus** live, project-aware
integration with the extension's state, not a divorced file-reader.
The real fork, then, is **not** "extension vs. plugin" (the extension stays either
way) — it is **"which instrument format consumes the banks: JSFX or native VSTi/CLAP,"**
and **"how much sampler do we actually want."** Those are §3 and §4.
*What it costs:* a **second codebase**, the Steinberg VST3 SDK (or JUCE) as a dependency,
and a real DSP/voice-engine build — a product-sized commitment, not a feature. The
Windows-only + VST3-only + REAPER-only lock-downs (D5) *remove* the costs the prior draft
carried around cross-platform DSP, code-signing, and multi-format wrappers — a material
simplification. The remaining cost question is almost entirely **"how much scaffolding does
the VST3 surface demand, and do we take JUCE to get it"** — answered honestly in §1a.
> **JSFX — retired (D1 DECIDED: native VST3).** Prior drafts weighed a JSFX sampler
> (REAPER's built-in scriptable format: zero new binary, zero SDK, cross-platform free) as
> the cheap-prototype path. It is off the table. The reasons it lost, for the record: (1)
> JSFX is a constrained DSP scripting language — a polyphonic multisample engine with
> round-robin/velocity-layers/streaming is doable but fights the idiom, and large-sample
> disk streaming is awkward; (2) JSFX gets **no VST-host bridge**, so it could never read
> `persist`'s live `"reasampler"` ext-state — it would be a permanently file-coupled
> consumer needing a sidecar seam, which is exactly the *loose* companion D6 rejected; (3)
> its "pure core" would be JSFX code, outside the CTest harness (see D3). With D5 locking
> Windows-only, JSFX's one real edge — cross-platform-for-free — is worth nothing here.
> Native VST3 wins cleanly given the locks.
---
## 1a. How crazy is bare VST3 without JUCE? (the honest assessment)
Daniel's question, directly: *how crazy is it to comply with the VST3 surface without
something like JUCE?* Short answer: **not crazy — the audio-processing side is a
few-hundred-lines-of-ceremony-you-write-once problem, not a tar pit. The one genuine
question is the editor UI, and ReaSampler is unusually well-positioned to answer it
without JUCE.** The detail, honestly, with estimate-vs-verified flagged.
### What the raw Steinberg VST3 SDK actually demands
A working VST3 instrument must present these interfaces (all VST3 interfaces descend from
`FUnknown`, a COM-like base with `queryInterface` / `addRef` / `release` — **verified**,
Steinberg VST3 SDK class reference):
- **`IComponent`** — the plugin's identity and bus/state setup: `initialize`,
`setActive`, `getBusCount` / `getBusInfo`, `activateBus`, `setState` / `getState`.
- **`IAudioProcessor`** — the DSP contract: `setBusArrangements`, `setupProcessing`
(`ProcessSetup`: sample rate, block size, symbolic sample size), `setProcessing`, and
the hot path **`process(ProcessData&)`** — where you read MIDI events off the event
input bus and write audio to the output bus. For an *instrument* you declare an **event
input bus** (MIDI in) and an **audio output bus**, no audio input. (**Verified**:
`IAudioProcessor` reference; the instrument bus topology is standard.)
- **`IEditController`** — parameter model + editor: `getParameterCount` /
`getParameterInfo`, `getState` / `setState`, `setComponentState`, normalized↔plain
parameter conversion, and `createView("editor")` returning an `IPlugView` if you have a
GUI.
- **`IPluginFactory`** (via the module's exported **`GetPluginFactory`**) — enumerates the
classes the module offers (the processor and, in the two-component model, the
controller), keyed by class UIDs. On Windows the module also exports **`InitDll` /
`ExitDll`** (bundle entry points differ per-OS, but D5 makes Windows the only target, so
it's just these two plus `GetPluginFactory`). (**Experienced estimate** on the exact
Windows export names — the SDK's `dllmain.cpp` / `public.sdk` main glue provides these;
I have not re-read the header this session, so treat the exact symbol names as
to-verify-against-`public.sdk/source/main/` before build, not as a load-bearing claim.)
**The COM plumbing is real but bounded.** `queryInterface`/`addRef`/`release` plus the
class-factory macros (`BEGIN_FACTORY` / `DEF_CLASS2` / `END_FACTORY`, and the
`DECLARE_FUNKNOWN_METHODS` / `IMPLEMENT_REFCOUNT` helper macros) are **provided by the
SDK's `pluginterfaces` and `public.sdk` layers** — you do not hand-write refcounting; you
invoke macros. (**Experienced estimate** on the exact macro names — these are the
long-standing VST3 SDK factory macros; verify spelling against the vendored SDK headers at
build time.) This is the "ceremony you write once" — it is copy-adapt-from-the-example
work, not design work.
**And the SDK hands you base classes that absorb most of it.** The critical fact for
Daniel's question: you do **not** implement those four interfaces from scratch. The SDK's
`public.sdk` layer provides:
- **`AudioEffect`** (with `Component` / `AudioEffect` base) — implements `IComponent` +
`IAudioProcessor` boilerplate; you override `initialize` (declare busses), `setupProcessing`,
`setActive`, `setState`/`getState`, and `process`.
- **`EditControllerEx1`** — implements `IEditController` boilerplate; you override
parameter registration and state.
- **`SingleComponentEffect`** — **combines processor and controller into one class**
(descends from `EditControllerEx1` and the component hierarchy). You override
`initialize` (call `addAudioOutput` + `addEventInput`), `setupProcessing`, and
`process`. (**Verified**: Steinberg SDK `SingleComponentEffect` class reference — "default
implementation for a non-distributable Plug-in that combines processor and edit
controller in one component." The SDK cautions to prefer the two-component split for
distributable plugins, but for a **REAPER-only, non-distributable** instrument (D5/D6),
`SingleComponentEffect` is exactly the sanctioned shortcut and cuts the interface surface
roughly in half.)
**Honest quantification of the audio side:** with `SingleComponentEffect` + the factory
macros, a *silent-but-loading* VST3 instrument skeleton — factory, class registration,
module entry, bus setup, empty `process` — is on the order of **~200–400 lines of
adapt-from-example ceremony**, written once, then largely untouched. The AGain / Note
Expression Synth examples that ship with the SDK are exactly this skeleton and are the
copy-source. This is **not** a tar pit. The tar-pit reputation VST3 has comes from (a)
multi-format wrappers (AU/AAX/VST2 — **not our problem**, D5) and (b) the GUI, addressed
next. (**Experienced estimate** on the line-count band — grounded in the shape of the SDK
examples, not a line-counted measurement; treat as an order-of-magnitude honest estimate,
not a promise.)
### What JUCE buys — and whether ReaSampler needs it
JUCE exists largely to solve problems D5 has already deleted for us. Weighed against the
locked constraints:
| What JUCE provides | Do we need it, given the locks? |
|---|---|
| **Multi-format wrapper** (one codebase → VST3/AU/AAX/VST2/standalone) | **No.** Single format, VST3 only, REAPER only. This is JUCE's biggest reason to exist and it's moot here. |
| **Plugin boilerplate / `AudioProcessor`** | Marginal. The Steinberg SDK's `SingleComponentEffect` already absorbs the VST3 boilerplate; JUCE's abstraction sits *on top of* the same SDK. |
| **Parameter management** (`AudioProcessorValueTreeState`) | Nice-to-have. Genuinely convenient, but a sampler's parameter set (envelope, gain, tuning) is small; hand-rolling over `IEditController` is bounded. |
| **DSP utilities** (`juce::dsp`, filters, oscillators, interpolators) | Nice-to-have for Tier 3 (filters/LFOs). For Tier 0–2 the sampler DSP — repitch interpolation, envelopes, voice allocation — is exactly the pure core we *want* to write and test ourselves (D3). |
| **Editor/UI framework** (`juce::Component`, graphics, widgets) | **The one real pull.** VST3 gives you `IPlugView` and *no toolkit whatsoever*. Something has to draw the editor. This is the crux — see below. |
**Costs of JUCE, honestly:** it is a large dependency (a whole framework, not a library);
it brings its own build system (Projucer / CMake integration) and idioms that would sit
oddly beside ReaSampler's lean two-submodule CMake discipline; and its **licensing** is a
real standing commitment. JUCE 8 is **dual-licensed AGPLv3 or commercial**. The free tier
(Starter) is usable below **~$20K/yr revenue** and — notably in JUCE 8 — **no longer
requires a splash screen**. Above that, **Indie (~$200K/yr revenue limit) is a paid
license (~$3,500 as of JUCE 8)**, and Pro above that. (**Verified via web**, JUCE forum +
license pages — see Sources.) For a personal/ReaSampler-scale tool the free Starter tier
likely applies today, but taking JUCE means accepting AGPL-or-pay as a permanent posture
on the instrument. That's a bigger standing commitment than the Steinberg SDK, which is a
**permissive** (proprietary-but-royalty-free, GPLv3-optional) license with no revenue
gate.
### The UI is the real question — and ReaSampler is unusually well-armed
Strip everything else away and the honest "reach for JUCE?" decision reduces to one thing:
**who draws the editor?** VST3 gives you an `IPlugView` seat and nothing to fill it with.
The options on Windows-only (D5):
1. **JUCE** — for its `Component` graphics stack alone. This is the *usual* reason indie
devs take JUCE, and if ReaSampler had no UI competence it'd be the default.
2. **VSTGUI** — the UI toolkit that **ships with the Steinberg SDK itself**. Lighter than
JUCE, purpose-built for VST editors, no extra dependency beyond the SDK you already
took. A real middle option. (**Experienced estimate** — VSTGUI is bundled with the VST3
SDK; verify the vendored SDK includes it before relying on it.)
3. **Win32 / GDI / Direct2D directly** — Windows-only makes this viable; you own an HWND in
the `IPlugView`.
4. **The same LICE/SWELL stack the extension already uses.** — **This is the one worth
staring at for Daniel specifically.**
**Why option 4 changes the calculus for *this* project.** ReaSampler's `bank_panel` is
**already a LICE-drawn UI** — a real, working, docked, custom-drawn panel (grid,
thumbnails, tab strip, hit-testing) built on LICE/SWELL, the same stack REAPER itself and
SWS use. That's **67 LICE call-sites in `bank_panel.cpp`** today. The team (Daniel + the
implementers) has already paid the learning cost of drawing a custom audio-tool UI in
LICE. The usual "take JUCE because hand-rolling a plugin GUI from nothing is miserable"
argument is **much weaker here than for a typical indie dev**, because ReaSampler is not
starting from nothing — it has demonstrated LICE UI competence and a house style. A VST3
`IPlugView` that hosts a LICE-drawn surface would (a) reuse existing UI muscle, (b) keep
the instrument's look consistent with the panel, and (c) avoid the JUCE dependency and its
license posture entirely. There's even the **embedded-UI affordance** (D7 below) that lets
a REAPER-hosted plugin draw inline in the TCP/MCP using this same Cockos surface.
*The honest caveat:* wiring LICE into a VST3 `IPlugView` (window creation, sizing,
event routing from the host into your draw/hit-test loop) is **integration work with real
edges** — you're bridging the SDK's view lifecycle to a LICE `HWND`/bitmap. It's not free,
and it's less trodden than "drop in a JUCE editor." But it is *the same class of work
ReaSampler already did* to dock `bank_panel`, not a new competence. (**Experienced
estimate** on the difficulty band — grounded in how `IPlugView` and LICE each work, not a
built prototype. Flag: the exact `IPlugView`↔LICE bridge should be spiked before it's
promised in a plan.)
### The pure-core discipline (D3) lands cleanly here
**Confirmed: the discipline holds, and JUCE-vs-bare doesn't change it.** The sampler's
voice engine, envelope math, key/velocity mapping, repitch/interpolation, and keymap
resolution are pure DSP + data — **exactly the kind of REAPER-free, DAW-free, unit-tested
core** this project already excels at (`bank_model` / `peaks` / `view_mode_model` are the
template). They live in a pure module, tested in the CTest harness outside any host. The
VST3 wrapper — `SingleComponentEffect` subclass, bus setup, `process` marshalling MIDI→core
and core→audio-buffer, the `IPlugView` editor, and the bridge calls that read
`"reasampler"` ext-state — is the **thin shell**, the only part that touches VST3 or REAPER
at all. This is precisely ReaSampler's load-bearing split, transplanted to a new format.
Native VST3 makes this *cleaner* than the retired JSFX path would have (JSFX's "core" would
be JSFX script, un-unit-testable in CTest). And **JUCE-vs-bare doesn't move it**: whether
the shell is a bare-SDK `SingleComponentEffect` or a `juce::AudioProcessor`, the pure core
underneath is identical, REAPER-free, and tested the same way. The format choice is a
*shell* choice; the core is invariant. That's a reassuring result — it means D-A (bare vs.
JUCE, below) can be decided on shell ergonomics and dependency posture alone, without
risking the part of the architecture Daniel most cares about.
### Bottom line for Daniel's question
Complying with the VST3 surface without JUCE is **reasonable, not crazy** — the
audio-processing scaffolding is bounded, example-driven ceremony that
`SingleComponentEffect` cuts down further, and the D5 lock-downs delete JUCE's biggest
justification (multi-format). The **only** place JUCE earns its weight is the editor UI —
and ReaSampler's existing LICE competence makes even *that* argument weaker than it would
be for a typical indie. My honest lean (D-A below): **bare Steinberg SDK + LICE editor**,
with JUCE as the fallback if the `IPlugView`↔LICE bridge proves gnarlier than the panel
work suggests. But it's a genuine fork and it's Daniel's — laid out in §4.
---
## 2. The integration seam
The prior draft assumed one seam: **the bank folder (project-relative WAVs) + the
bank/index JSON on disk**, with the playback engine as a file-reader. That's still the
seam for *sample audio* (WAVs are files; there's no getting a live PCM stream across the
bridge, nor would you want to). But for **everything that isn't the raw audio** — the
index, the mapping data, which project's bank is active — the bridge opens a **second,
richer seam** the prior draft missed:
There are two seams, and with native VST3 locked (D1) the instrument gets **both**:
- **File seam (audio + a portable snapshot).** WAVs on disk; index/mapping as JSON or a
sidecar. Tool-agnostic, host-agnostic, portable. Any instrument (JSFX or native, in
REAPER or any other host) can consume it. This is the *only* seam JSFX gets, and the
*fallback/export* seam for native.
- **Live-state seam (native-in-REAPER only).** A native VST instance reads the
`"reasampler"` project ext-state directly via the bridge (`GetProjExtState` /
`EnumProjExtState`, resolved by name over `hostcb`), and knows its own host project via
the context callback. It sees what `persist` last wrote, follows the active project, and
needs no "point me at the right bank folder" wiring — it *asks REAPER* which project it's
in. This is strictly more than the file seam and it's REAPER-coupled by construction.
- **File seam (audio, always).** The sample **audio** is WAVs on disk — there's no getting
a live PCM stream across the bridge, nor would you want to. WAVs are project-relative and
travel with the `.rpp` (M4 machinery). This is the permanent seam for sample audio.
- **Live-state seam (the mapping / index, via the bridge).** For **everything that isn't
the raw audio** — the index, the mapping data, which project's bank is active — a native
VST instance reads the `"reasampler"` project ext-state directly via the bridge
(`GetProjExtState` / `EnumProjExtState`, resolved by name over `hostcb`), and knows its
own host project via the context callback. It sees what `persist` last wrote, follows the
active project, and needs no "point me at the right bank folder" wiring — it *asks REAPER*
which project it's in. This is the tight integration D6 locked in.
The design question below — "is the JSON sufficient, or does playback need mapping the
index doesn't carry" — is unchanged. What the bridge changes is *where that mapping can
live and how the instrument gets it*: for native, the mapping can be **live shared state**
between extension and instrument, not just a file handed across.
So: **audio comes across as files; the mapping comes across as live shared state.** The
design question below — "is the ext-state/index sufficient, or does playback need mapping
it doesn't carry" — is unchanged. What the bridge settles is *where that mapping lives*:
between extension and instrument as **live shared `"reasampler"` state**, not a file one
writes and the other re-parses.
**What the current index carries** (from `bank_model`'s `Sample`, per CONTEXT.md §Data
model): id, display name, relative path, source range, channel count, sample rate,
@@ -242,29 +414,16 @@ opinionated part. It also means **the bank index grows only by file-intrinsic fi
seam ReaSampler already knows how to add (mirror of how `provenance` was added as a
field in M1 and populated later).
**A note on format (revised for the bridge):** the friction is asymmetric, and more so
than the prior draft said.
- *Native (Option A):* doesn't just "read the bank JSON off disk" — it can read the map
out of **live project ext-state** via the bridge (whichever of i/ii/iii you choose, the
extension and instrument can share the *same* `"reasampler"` state rather than one
writing a file the other re-parses). The `.wav` audio is still files; the mapping need
not be. Cleanest possible integration.
- *JSFX (Option B):* no bridge — reading arbitrary JSON is friction, so the seam likely
needs a **simpler sidecar** (a flat `.txt`/key-value map ReaSampler writes next to the
bank, JSFX-parseable). On the JSFX path the seam is unavoidably file-based. That's a
concrete cost of JSFX and a reason the seam design and the format choice stay coupled.
**Portability caveat (partly dissolved on the native path).** The bank is project-relative
and travels with the `.rpp` (M4 machinery). Any *file-seam* consumer must resolve paths
the same way — so a JSFX instrument still needs to be told the current project bank folder
(saved in its own state), a real wiring detail, don't hand-wave it. But a **native**
instance sidesteps this: via the context callback it asks REAPER which project it's in,
then reads that project's bank location straight from ext-state — no "point me at the
folder" step. What was a shared wiring cost is now a **cost only on the JSFX path.** Note
the flip side: a native instrument that resolves its bank *only* through the bridge won't
work outside REAPER at all — so if a portable/exportable instrument is ever a goal, keep
the file seam as the source of truth and treat the live-state seam as an accelerator, not
the sole path.
**Wiring note (native, the locked path).** The native instance sidesteps the "point me at
the right bank folder" problem entirely: via the context callback it asks REAPER which
project it's in, then reads that project's bank location straight from `"reasampler"`
ext-state — no user wiring, no saved folder path in the instrument's own state. This is the
cleanest possible integration and it's the one D6 chose. The one thing to keep deliberate:
the WAV **audio** still resolves via the project-relative path machinery (M4), so the
instrument must resolve those paths the same way `persist` does — a shared convention, not a
hand-wave. Since D5 locks REAPER-only/Windows-only, the old "but it won't run in other
hosts" concern is **moot by design** — the bridge dependency is intended, not a narrowing to
regret.
---
@@ -277,8 +436,9 @@ tier above is optional depth, not a prerequisite for the one below.
**Tier 0 — "the bank plays."** One sample mapped chromatically across the keyboard from
a root note; monophonic-or-basic-polyphony; a simple amp envelope; velocity → volume.
Point one bank sample at a MIDI track and play it repitched. *This is the smallest
thing that delivers the promise* and is the honest MVP. On the JSFX path this is a
genuinely small build; on the native path it's the skeleton of the plugin.
thing that delivers the promise* and is the honest MVP. On the native path this is the
skeleton of the VST3 plugin — the `SingleComponentEffect` shell (§1a) plus a single-voice
core, with the editor deferrable behind a parameters-only default view.
**Tier 1 — "a keymap."** Multiple samples zoned across the keyboard (key ranges), each
with its own root note. Now a captured *kit* (several one-shots) or a *multisampled
@@ -313,143 +473,160 @@ instrument keeps its own copy of the samples").
---
## 4. Risks & open decisions — Daniel's to call
## 4. Decisions — settled and residual
None of these are pre-decided here. Each is a genuine fork.
### Settled (DECIDED — reasoning preserved, not re-opened)
**D1 — Instrument format: JSFX vs. native VSTi (VST3/CLAP, likely via JUCE).** The
central fork. JSFX = no new binary, no JUCE, cross-platform free, real ceiling on
sophistication, awkward bank-JSON reading, **and no VST-host bridge** (file-seam only).
Native = full sophistication, clean integration, **plus the bridge**: live `"reasampler"`
ext-state, project-awareness, embedded TCP/MCP UI (see below) — at the cost of a whole
second codebase in a second format with its own build/release/dependency/signing story,
**and REAPER-coupling if it leans on the bridge.**
These are locked. The reasoning is kept as the record of *why*, so the choices don't get
silently re-litigated.
*What the bridge does to the native column (the honest update).* The prior draft's native
pitch was "more DSP power + parses JSON cleanly." The bridge adds a category the prior
draft didn't weigh: native can be **part of ReaSampler's live state model**, not a
file-coupled consumer. That is a real pull toward native beyond raw DSP — it's the
difference between "an instrument that reads a ReaSampler export" and "an instrument that
*is* ReaSampler's playback surface." It doesn't make JSFX wrong; it makes native's ceiling
higher and its integration tighter than I credited. Weigh it against the coupling cost: a
bridge-dependent VST is a **REAPER-only** artifact, no longer a portable VST.
**D1 — Instrument format → DECIDED: native VST3.** JSFX is off the table (retirement
reasoning in §1, in the JSFX box). Native VST3 gives full sampler sophistication, clean
integration, and the VST-host bridge (live `"reasampler"` ext-state, project-awareness,
embedded-UI affordance). The cost — a second codebase in a plugin format — is accepted.
The old "JSFX-first cheap prototype" lean is withdrawn: with D5 locking Windows-only,
JSFX's one advantage (cross-platform-for-free) is worthless here, and its inability to
reach the bridge makes it the wrong tool for the *integrated* product D6 chose.
*My lean, updated but unchanged in direction, for discussion only:* if the goal is
Tier 0–1, **still prototype in JSFX first** — it proves the *sampler value and the
key/velocity seam* with near-zero format commitment, and the file seam you design there is
exactly the portable/export seam a native build would keep anyway. The bridge is an
*integration* advantage, not a *does-the-sampler-work* advantage, so it doesn't change what
the cheapest proof is. Reach for native when either (a) Tier 2+ sophistication is a firm
goal, or (b) **tight live integration with ReaSampler's project state is itself the point**
— that second trigger is new, and it's the bridge's real contribution to this fork.
**D5 — Cross-platform / multiformat → DECIDED: none. Windows-only, VST3-only,
REAPER-only.** The extension is Windows-only; Daniel does not work on other platforms.
This *removes* costs prior drafts carried: no mac/Linux DSP/build/signing matrix, no
multi-format wrapper, no "runs standalone in other hosts" concern, no CLAP-for-portability
argument. REAPER-coupling via the bridge is intended, not a narrowing to regret. This is
the single biggest simplifier — it is *why* bare VST3 is reasonable (§1a): most of what
makes VST3 painful (multi-format, cross-platform) is deleted.
**D2 — Whether to take a JUCE (or any external plugin-SDK) dependency at all.** The
project today is a clean C++ extension with two vendored submodules and a proud
pure-core/shell discipline. A native instrument means a *third* major dependency and a
second build target of a fundamentally different kind. That's a real architectural
weight. JSFX sidesteps it entirely. Flagging it as its own decision because "should we
depend on JUCE" is a bigger standing commitment than "should we build a sampler."
**D6 — One product or two → DECIDED: two products, but tightly integrated.** The
instrument is a *separate artifact* from the extension (the extension stays the pure
capture/organize tool; the instrument is the playback surface — capture and placement, and
now playback, stay distinct acts). But it is **not** a divorced file-only companion: via
the bridge it reads the live `"reasampler"` project ext-state and is project-aware — "two
faces of one tool sharing one state model." The prior draft's open question ("could they
only ever share a file?") is resolved: no, and the shared-live-state integration is the
chosen shape. The retired reasoning for why the loose-companion reading lost: it only ever
looked clean because the prior draft under-weighted the bridge; once the bridge is on the
scale, the integrated reading has the stronger technical affordance, and Daniel took it.
*The bridge's bearing on D2.* Weigh two coupling costs, not one. A native instrument that
uses the bridge takes on **both** a plugin-SDK dependency (JUCE/VST3/CLAP) *and* a
REAPER-API coupling (the same `reaper_plugin_functions.h` surface the extension binds — so
the instrument would vendor and bind against it too, likely resolving pointers via `hostcb`
rather than the extension's startup path). The upside is that the pure sampler core stays
REAPER-free and testable regardless (D3); it's only the *shell* that touches the bridge, so
the coupling is contained where the project already puts REAPER coupling. The downside is
plain: choosing the bridge means choosing a REAPER-only instrument. If Daniel wants a
sampler that also runs in other hosts, the bridge is off the table and D1's native pitch
loses its integration edge — collapsing back toward "native = more DSP power only."
**D3 — Pure-core discipline across the format boundary → CONFIRMED holds (see §1a).** Not
a fork so much as a checked assumption: the sampler's voice engine, envelope math,
key/velocity mapping, and repitch/interpolation live in a pure REAPER-free, DAW-free,
unit-tested core (mirror of `bank_model`/`peaks`/`view_mode_model`); the VST3 wrapper is
the thin shell. Confirmed clean for native, and — importantly — **invariant under
bare-vs-JUCE** (§1a): the shell choice doesn't touch the core. This is the most
ReaSampler-native way to build it and it's assumed, not debated, going forward.
**D3 — Does the pure-core discipline survive the format boundary?** ReaSampler's
identity is *pure REAPER-free testable core + thin shells*. A sampler's **voice engine,
envelope math, key/velocity mapping, and repitch logic are exactly the kind of pure,
testable core** this project excels at — they could live in a REAPER-free, DAW-free,
unit-tested module (mirror of `bank_model`/`peaks`/`view_mode_model`) with the plugin
format (JSFX or VST3 wrapper) as the thin shell around it. **This is the most
ReaSampler-native way to build it** and I'd argue strongly for it regardless of D1: the
sampler DSP core is pure and tested; the format is a shell. The open question is whether
that discipline can hold across a *different plugin format* — with native it's clean C++
so yes; with JSFX the "pure core" would be JSFX code, harder to unit-test in the CTest
harness. That tension is real and feeds back into D1.
### Residual forks — Daniel's to call, in priority order
**D4 — Where the mapping lives (seam fork i/ii/iii from §2).** Bank-owned map,
instrument-owned map, or split (file-intrinsics in the bank, performance-map in the
instrument). *My lean:* **(iii) split** — it's the one that honors ReaSampler's
capture/placement instinct and keeps the bank tool-agnostic. But it's Daniel's call
whether ReaSampler should author instrument definitions at all, or stay purely a
sample library that a *separate* mapping tool arranges.
Four decisions remain live. Ordered by how much they gate the rest.
**D5 — Cross-platform.** The extension is already cross-platform (SWELL). JSFX inherits
that for free. A native plugin re-opens the full cross-platform DSP + UI + build matrix
(Win/mac/Linux, code-signing on mac, etc.) as a *separate* artifact. A cost that lands
entirely on the native path.
**D-A (highest) — Bare Steinberg VST3 SDK vs. JUCE (with the UI-toolkit sub-question).**
This is now *the* central fork, and §1a is the honest assessment feeding it. The
sub-question — **who draws the editor?** — is really the whole fork, because §1a shows the
audio-processing scaffolding is bounded either way.
**D6 — Is this even one product?** The reframe worth surfacing: ReaSampler's thesis is
"a precision *capture/organize* tool; capture and placement are separate acts." A
MIDI-playback instrument is a *different act* — playback. There's a legitimate reading
where the instrument is a **companion product** that shares the bank format, not a
feature *of* ReaSampler — the way a sample library and a sampler that reads it are
related-but-distinct products. That framing might keep ReaSampler sharp (it stays the
capture tool it is) while letting the instrument evolve on its own clock and format.
The alternative reading — it's all one integrated sampler-workstation — is also
coherent.
- *Option A — bare Steinberg SDK + LICE editor.* Take only the VST3 SDK (permissive,
royalty-free, no revenue gate). Use `SingleComponentEffect` for the ~200–400 lines of
once-written audio scaffolding, and draw the editor in **LICE/SWELL — the stack
`bank_panel` already uses** (67 LICE call-sites today). *Pro:* no JUCE dependency, no
AGPL-or-pay posture, house-consistent UI, reuses existing UI muscle, lean CMake
discipline preserved. *Con:* the `IPlugView`↔LICE bridge is real integration work with
edges (window lifecycle, sizing, event routing) — less trodden than dropping in a JUCE
editor, and it should be **spiked before it's promised**.
- *Option B — JUCE.* Take JUCE for its editor framework + parameter management + DSP
utilities. *Pro:* the editor is a solved problem, `AudioProcessorValueTreeState` is
convenient, `juce::dsp` helps at Tier 3. *Con:* a large framework dependency with its own
build system and idioms sitting oddly beside the two-submodule discipline; and the
**license posture** — AGPLv3-or-commercial, free Starter tier below ~$20K/yr revenue (no
splash screen in JUCE 8), Indie ~$3,500 with a ~$200K/yr limit above that. For most of
what JUCE solves (multi-format), **D5 already deleted the need.**
- *Option C — bare SDK + VSTGUI (the SDK's own bundled toolkit).* A middle path: no JUCE,
but a purpose-built VST editor toolkit instead of hand-bridging LICE. Lighter than JUCE,
no extra dependency beyond the SDK. Worth a look if the LICE bridge proves gnarly.
*Re-argued with the bridge on the scale (the prior draft weighed this fork without it).*
The prior draft implicitly assumed the two products could only ever share a **file
format** — which makes "companion product" feel clean and natural, because a shared file
is exactly what independent products share. The bridge changes the terms. A native
instrument can be **tightly, live-integrated with ReaSampler's project state**: same
`"reasampler"` ext-state, same project identity, following the active project — a degree
of coupling that is *more than* "two products that happen to read the same files" and
starts to look like "two faces of one tool sharing one state model." That is a genuine
thumb toward the **one-integrated-product** reading. It's not decisive, and I won't
pretend it is:
- It only applies if the instrument is **native** (JSFX can't reach the bridge, so a
JSFX instrument *is* naturally a file-coupled companion — the format choice and this
fork are entangled: pick JSFX and you've half-answered D6 toward "companion").
- It's an *available* integration, not a *required* one — a native instrument could
still be built to the file seam only, deliberately staying a loose companion for
portability. The bridge widens the option, it doesn't force it.
- The tight-integration reading buys **REAPER-coupling**; the companion reading buys
**portability and independent evolution.** Those are real, opposed goods, and the
bridge doesn't resolve which Daniel wants — it just makes the integrated option
*materially more capable* than the prior draft credited, so the companion reading no
longer wins by default on "well, they can only share a file anyway."
*My honest lean:* **Option A (bare SDK + LICE editor)**, precisely because ReaSampler is
the atypical case where the "take JUCE for the GUI" default is weakest — it already has
working LICE UI competence and a house style. JUCE's headline value (multi-format) is moot
under D5. Fall back to JUCE (Option B) *only if* the `IPlugView`↔LICE spike shows the
bridge is genuinely painful; consider VSTGUI (Option C) as the middle rung before
conceding to a full framework. **Concrete ask: greenlight a small `IPlugView`↔LICE spike
before committing** — it's the one unknown that decides A vs. B, and §1a's LICE-bridge and
Windows-export-name claims are experienced estimates that a spike would convert to fact.
**Net:** the bridge strengthens "one integrated product" from a coherent-but-equal option
into the one with the stronger technical affordance — *if* native-in-REAPER is the path.
It does not pre-decide the fork. This is still the highest-order question and it's Daniel's
to answer, but he should answer it knowing the integrated option is richer than the
file-sharing picture implied — and knowing that choosing JSFX quietly tilts it the other
way.
**D-B — The seam fields: what becomes a bank intrinsic (§2, old D4).** Where the mapping
lives: (i) bank-owned map, (ii) instrument-owned map, or (iii) split — file-*intrinsics*
(root note, loop points) in the bank; *performance-map* (zones, velocity layers,
round-robin, envelopes) in the instrument. *My lean, unchanged:* **(iii) split.** It
mirrors ReaSampler's capture/placement instinct — root note and loop points are *facts
about the captured file* (like sample rate, length, peaks) and belong in the bank;
zones/layers/envelopes are a *performance choice* and belong in the instrument. It keeps
the bank a clean library and makes the bank-index change **small and additive** (root note
+ loop points — the same shape as adding `provenance` in M1). The live-state seam (§2)
means the instrument reads even the performance-map out of shared `"reasampler"` state, so
"who owns which field" is a data-ownership decision, not a transport one. *The one thing to
settle early:* whether root note / loop points get added to `Sample` **now** (so capture
starts populating them and no backfill cliff opens later) or deferred — this is the "design
the seam even if you defer the feature" instinct, and it's cheap to add the fields early.
**D7 — Embedded TCP/MCP UI (new option the bridge surfaces; native/CLAP only).** While
verifying the bridge I found a second REAPER-VST-specific affordance the prior draft never
mentioned: a hosted plugin can **draw its own embedded UI directly in the track/mixer
control panel** (`reaper_plugin_fx_embed.h`: VST2 answers `canDo("hasCockosEmbeddedUI")`
with `0xbeef0000` and draws via `effVendorSpecific/effEditDraw`; VST3 implements
`IReaperUIEmbedInterface`; CLAP exposes `cockos.reaper_embedui`). Concretely: a native
ReaSampler instrument could render a compact keymap/level strip *inline in the TCP/MCP*,
not only in its own plugin window — the same surface REAPER's own JS/embedded FX use. This
is **native/CLAP-only** (JSFX can't do it) and is pure polish, not a Tier-0 need — but it's
a real integration affordance that only exists on the native-in-REAPER path, and it
compounds the D1/D6 tilt: if "ReaSampler's playback surface, deeply woven into REAPER's UI"
is the vision, this is a lever only native reaches. Flagging it for completeness, not
recommending it — it's a Tier-3-ish nicety, deferred by default.
**D-C — Tier scope to commit to first (§3).** Tier 0 ("the bank plays" — one sample,
chromatic, amp envelope, velocity→volume) / Tier 1 ("a keymap" — zoned multisamples with
per-sample root notes) / Tier 2 ("expressive" — velocity layers, round-robin, ADSR, loops)
/ Tier 3 ("polish" — filters, LFOs, choke groups). *My lean:* **scope Tier 0–1 first** as
the "does this belong in ReaSampler's world" proof; hold Tier 2 as the "becomes a tool
people reach for" follow-on; treat Tier 3 as optional-forever. Don't let a Tier-3 feature
list inflate the D-A decision — the editor and DSP needs of Tier 0–1 are modest, which
further weakens the "need JUCE" pull.
**D-D (deferred by default) — Embedded TCP/MCP UI.** A REAPER-hosted VST3 can draw its own
UI *inline in the track/mixer control panel* (`reaper_plugin_fx_embed.h`: VST3 implements
`IReaperUIEmbedInterface`; the same Cockos surface REAPER's own JS/embedded FX use). A
native ReaSampler instrument could render a compact keymap/level strip inline in the
TCP/MCP, not only in its own window — and since this uses the same LICE-class drawing as
Option A above, it composes naturally with the bare-SDK-plus-LICE path. **Pure polish, not
a Tier-0 need — deferred by default.** Flagged only because it's a lever that exists on
this exact (native-in-REAPER, LICE-drawing) path and nowhere else, so if "deeply woven into
REAPER's UI" ever becomes the vision, the groundwork is already the groundwork.
---
## What this doc is asking for
A direction on the two highest-order forks, in order:
With D1/D5/D6 locked, the decision this doc now puts to Daniel is a single ordered set:
1. **D6 — one product or two?** Is the instrument a feature of ReaSampler, or a
companion product sharing the bank format? Everything else sits under this.
2. **D1 — JSFX-first prototype, or straight to native?** Given a target of Tier 0–1 to
start, and the pure-core-as-shell discipline (D3) held either way.
1. **D-A — bare Steinberg SDK + LICE editor, or JUCE?** (My lean: bare SDK + LICE, gated on
a small `IPlugView`↔LICE spike. This is the one that decides the instrument's whole
build shape and dependency posture.)
2. **D-B — split seam, and do root note / loop points get added to `Sample` now?** (My
lean: yes, split; add the intrinsic fields early to avoid a backfill cliff.)
3. **D-C — commit Tier 0–1 first?** (My lean: yes; hold 2, defer 3.)
4. **D-D — embedded TCP/MCP UI:** deferred by default; note only.
Once those two are called, the seam fields (§2, D4) and the tier scope (§3) become
concrete enough to write an actual phase spec. Until then this stays a framing doc with
no PLAN.md footprint — deliberately, so we don't scope an instrument before deciding
whether we're building one.
Once D-A and D-B are called, this becomes concrete enough to write an actual phase spec
(the instrument as its own artifact, with the pure sampler core as a testable module
mirroring `bank_model`). Until then it stays a framing doc with **no PLAN.md footprint** —
deliberately, so nothing is scoped until the build shape (D-A) is chosen.
---
## Sources (for §1a's verified claims)
- Steinberg VST3 SDK — `SingleComponentEffect` class reference (combined processor +
controller; "non-distributable" caveat):
https://steinbergmedia.github.io/vst3_doc/vstsdk/classSteinberg_1_1Vst_1_1SingleComponentEffect.html
- Steinberg VST3 SDK — `IAudioProcessor` class reference (process/setupProcessing/bus
contract):
https://steinbergmedia.github.io/vst3_doc/vstinterfaces/classSteinberg_1_1Vst_1_1IAudioProcessor.html
- VST3 Developer Portal (overview, tutorials, example plugins — AGain / Note Expression
Synth as the skeleton copy-source): https://steinbergmedia.github.io/vst3_dev_portal/
- JUCE 8 EULA (dual AGPLv3 / commercial; tier structure):
https://juce.com/legal/juce-8-licence/
- JUCE forum — revenue limits & JUCE 8 pricing (Starter ~$20K/yr free, Indie ~$200K/yr
limit, ~$3,500): https://forum.juce.com/t/revenue-limits-for-juce-tiers/61058
*Estimate-vs-verified honesty note:* the interface list, `SingleComponentEffect`'s role,
the instrument bus topology, and the JUCE license terms are **verified** from the sources
above. The **line-count band (~200–400)**, the **exact Windows module-export symbol names**
(`InitDll`/`ExitDll`/`GetPluginFactory`), the **factory-macro spellings**
(`BEGIN_FACTORY`/`DEF_CLASS2`), **VSTGUI being bundled in the vendored SDK**, and the
**`IPlugView`↔LICE bridge difficulty** are **experienced estimates** grounded in how the
SDK is shaped — each flagged inline in §1a and each cheap to convert to fact by reading the
vendored SDK headers / running a spike before any of it lands in a plan.