docs: scope MIDI-playback instrument as Phase S (native VST3, second artifact)
Settle forks D-A..D-D: bare Steinberg VST3 SDK + LICE editor; split seam with root-note/loop-points on Sample now; Tier 0-1 committed; embedded TCP/MCP UI scheduled. Phase S added to PLAN.md and CONTEXT.md spec.
This commit is contained in:
+285
@@ -1043,3 +1043,288 @@ No new REAPER *audio* API. New surfaces to verify before use:
|
||||
|
||||
**Build-time residual (not a fork):** the owned-file manifest's exact persistence
|
||||
shape (sibling `"reasampler"` ext-state key vs. folded into the `banks` blob).
|
||||
|
||||
---
|
||||
|
||||
# MIDI-playback instrument — additive phase spec (Phase S — Sampler)
|
||||
|
||||
> **New pillar, its own lettered phase, and — uniquely — its own build artifact.**
|
||||
> Every prior phase (M / D / B / R / V) ships inside the one `reaper_reasampler`
|
||||
> extension binary. Phase S does **not**: a REAPER extension *cannot* be a
|
||||
> MIDI-triggered instrument (it is not a node in any track's signal chain), so the
|
||||
> instrument is a **second, separate binary** — a native **VST3** plugin the user
|
||||
> instantiates on an instrument track — that reads ReaSampler's banks and plays them
|
||||
> MIDI-triggered. Namespaced **`S` (Sampler)** rather than "D" (which would collide
|
||||
> with Design View). The `M`/`D`/`B`/`R`/`V` extension pillars are untouched. Product
|
||||
> framing, the plugin-format reasoning, the bare-VST3-vs-JUCE assessment, and the
|
||||
> settled decision record: `docs/product/midi-playback.md`. Same standing discipline:
|
||||
> **verify every Steinberg VST3 SDK and REAPER/SWELL API name/signature against the
|
||||
> vendored headers before use.**
|
||||
|
||||
## What it is
|
||||
|
||||
A native **VST3 sampler instrument** — a separate product/artifact from the
|
||||
extension — that maps ReaSampler's captured bank samples across a MIDI keyboard and
|
||||
plays them back with a real voice engine (polyphony, velocity, envelopes). The
|
||||
extension stays the sole owner of **capture + organization**; the instrument is the
|
||||
**playback surface**. The two are *tightly integrated but distinct acts*: the
|
||||
extension captures and organizes; the instrument plays. Neither crosses into the
|
||||
other's role — the instrument never captures, the extension never becomes an
|
||||
instrument.
|
||||
|
||||
**Why a VST3 and not the extension (load-bearing, settled — see D1/D5/D6 below).** An
|
||||
instrument track's "read live MIDI, emit audio per-voice, in REAPER's
|
||||
routing/record/render path" contract belongs to VST/VST3/CLAP/JSFX plugins, hosted
|
||||
through an entirely different mechanism than the extension API. The extension SDK's
|
||||
audio-adjacent surfaces (`Audio_RegHardwareHook`, `kbd_OnMidiEvent`, `PlayPreview`,
|
||||
`pcmsrc` subclassing) are each the wrong tool for a live-MIDI instrument — the full
|
||||
reasoning is in `docs/product/midi-playback.md` §1. The instrument is therefore a
|
||||
standard VST3 plugin; this is not an engineering-around-able limitation, it is what
|
||||
the plugin *format* is.
|
||||
|
||||
## The three locked decisions this spec assumes
|
||||
|
||||
Settled by Daniel (2026-07-26); everything below assumes them. Reasoning preserved in
|
||||
`docs/product/midi-playback.md` §4.
|
||||
|
||||
- **D1 — native VST3.** Not JSFX. Full sampler sophistication, clean integration, and
|
||||
access to the REAPER VST-host bridge. JSFX retired (cross-platform-for-free is
|
||||
worthless under D5, and JSFX gets no bridge).
|
||||
- **D5 — Windows-only, VST3-only, REAPER-only.** No cross-platform DSP/build/signing
|
||||
matrix, no multi-format wrapper, no standalone-in-other-hosts concern. REAPER-coupling
|
||||
via the bridge is intended. This is the single biggest simplifier — it deletes most of
|
||||
what makes VST3 painful.
|
||||
- **D6 — two products, tightly integrated.** A separate artifact, but **not** a
|
||||
divorced file-only companion: via the VST-host bridge it reads the live
|
||||
`"reasampler"` project ext-state and is project-aware.
|
||||
|
||||
## The VST-host bridge (the integration mechanism, stated once)
|
||||
|
||||
A VST3 *hosted inside REAPER* can call back into REAPER's own API by resolving
|
||||
function pointers by name over the host callback
|
||||
(`hostcb(&effect, 0xdeadbeef, 0xdeadf00d, 0, "FunctionName", 0.0)` — the same
|
||||
string-keyed API table the extension uses; verified in `video_processor.h` and the
|
||||
`reaper_plugin_functions.h` `GetProjExtState`/`SetProjExtState`/`EnumProjExtState`
|
||||
entries). The plugin can also fetch its **host context** — the track/take/project it
|
||||
was instantiated in (opcode `0xdeadf00e`). **Consequence:** the instrument reads the
|
||||
*same* live `"reasampler"` ext-state that `persist` writes, follows the active
|
||||
project, and needs no "point me at the bank folder" wiring — it asks REAPER which
|
||||
project it is in. This capability exists *because* the plugin is hosted in REAPER; it
|
||||
is the technical affordance D6 leaned on. **Must-verify before build:** confirm the
|
||||
bridge opcodes and the by-name resolution against the vendored
|
||||
`vendor/reaper-sdk/sdk/` headers (`reaper_plugin.h`, `video_processor.h`,
|
||||
`reaper_plugin_functions.h`) — the framing doc's opcode citations are verified from
|
||||
those headers but the exact call marshalling should be confirmed at the spike.
|
||||
|
||||
## The two seams (audio via files, mapping via live state)
|
||||
|
||||
- **File seam (audio, permanent).** The sample **audio** is the on-disk 32-bit-float
|
||||
WAVs — project-relative, travelling with the `.rpp` via the M4 machinery. The
|
||||
instrument resolves those paths **the same way `persist` does** (a shared convention,
|
||||
not a re-implementation). There is no live PCM stream across the bridge, by design.
|
||||
- **Live-state seam (the mapping, via the bridge).** For everything that is *not* raw
|
||||
audio — the bank index, the mapping, which project is active — the instrument reads
|
||||
the live `"reasampler"` ext-state via the bridge. It sees what `persist` last wrote
|
||||
and follows the active project.
|
||||
|
||||
## The seam fields — what becomes a bank intrinsic (D-B, settled 2026-07-26)
|
||||
|
||||
**The split model (option iii) is the settled answer.** It mirrors the
|
||||
capture/placement separation:
|
||||
|
||||
- **Bank intrinsics (facts about the captured file) live on `Sample`.** **Root note**
|
||||
(the MIDI note the sample was recorded at, so it can be repitched across the
|
||||
keyboard — distinct from the existing optional *musical key* field) and **loop
|
||||
points** (sustain-loop start/end for held notes; sample-accurate, zero-crossing-aware)
|
||||
are *facts about the file*, analogous to sample rate, length, and peaks. They are added
|
||||
to `Sample` as an **additive field extension** — the same shape as how `provenance`
|
||||
was added in M1: new optional fields with JSON round-trip, populated at/after capture,
|
||||
defaulting cleanly for pre-existing samples. This keeps the bank a clean,
|
||||
tool-agnostic library (WAVs + facts, readable by anything).
|
||||
- **The performance map (a creative arrangement) lives in the instrument.** **Key
|
||||
zones** (low/high note per sample), **velocity layers**, **round-robin groups**,
|
||||
**amplitude envelopes** (ADSR), and per-sample tuning/gain trim are a *performance
|
||||
choice*, not a fact about a file — they belong to the instrument, not the bank. Under
|
||||
the live-state seam the instrument may still *read* performance-map data out of shared
|
||||
`"reasampler"` state, so "who owns which field" is a data-ownership decision, not a
|
||||
transport one.
|
||||
|
||||
**Why the intrinsic fields are added early (D-B, the backfill-cliff reasoning).** The
|
||||
`Sample` field addition is scheduled as an **early Phase S point** even though the
|
||||
instrument that consumes them lands later. Rationale (the *design-the-seam-even-if-you-
|
||||
defer-the-feature* instinct, same as Fork R-D's owned-file manifest): if the fields are
|
||||
added only when the instrument needs them, every sample captured before then lacks a
|
||||
root note / loop points and must be backfilled by hand. Adding the fields now — so
|
||||
capture starts populating them (or at least defaulting them cleanly) — costs almost
|
||||
nothing and closes the cliff. The field addition touches the **extension** codebase
|
||||
(`bank_model` + capture + persist), is independently shippable, and lands before the
|
||||
instrument build leans on it.
|
||||
|
||||
## Scope tiers (D-C, settled 2026-07-26 — Tier 0–1 committed, Tier 2 held, Tier 3 optional-forever)
|
||||
|
||||
Tiers are minimal → sophisticated; **Tier 0 delivers the core promise** and each tier
|
||||
above is optional depth, not a prerequisite for the one below.
|
||||
|
||||
- **Tier 0 — "the bank plays" (committed).** One sample mapped chromatically across
|
||||
the keyboard from its root note; basic polyphony; a simple amp envelope; velocity →
|
||||
volume. The honest MVP: point a bank sample at a MIDI track and play it repitched.
|
||||
On the native path this is the `SingleComponentEffect` skeleton plus a single-voice
|
||||
core, editor deferrable behind a parameters-only default view.
|
||||
- **Tier 1 — "a keymap" (committed).** Multiple samples zoned across the keyboard (key
|
||||
ranges), each with its own root note — a captured *kit* or a *multisampled instrument*
|
||||
plays correctly. This is where the root-note + key-range seam fields earn their place.
|
||||
One sample per key-region.
|
||||
- **Tier 2 — "expressive" (HELD — noted, not specified).** Velocity layers, round-robin
|
||||
(the anti-machine-gun feature), full ADSR, per-sample tuning/gain trim, sustain loops.
|
||||
Where it becomes a tool people reach for. **Explicitly a follow-on** — its points are
|
||||
not drawn up in this spec; it is recorded as the next depth increment once Tier 0–1
|
||||
proves the instrument belongs in ReaSampler's world.
|
||||
- **Tier 3 — "instrument polish" (optional-forever).** Filters, filter/pitch envelopes,
|
||||
LFOs, per-voice pan, choke groups, a modest FX slot. A direction to leave room for,
|
||||
never a commitment. Do **not** let a Tier-3 feature list inflate the build-shape
|
||||
decisions.
|
||||
|
||||
## The build shape (D-A, settled 2026-07-26 — bare Steinberg VST3 SDK + LICE editor)
|
||||
|
||||
**Settled: bare Steinberg VST3 SDK, no JUCE, with the editor drawn in the same
|
||||
LICE/SWELL stack `bank_panel` already uses.** Reasoning (full assessment in
|
||||
`docs/product/midi-playback.md` §1a and §4 D-A):
|
||||
|
||||
- **The audio-processing scaffolding is bounded.** Using `SingleComponentEffect` (the
|
||||
SDK's combined processor+controller base — sanctioned for a non-distributable,
|
||||
REAPER-only plugin under D5/D6) plus the SDK's factory macros, a silent-but-loading
|
||||
VST3 instrument skeleton is order-of-magnitude a few-hundred lines of
|
||||
adapt-from-example ceremony, written once. The AGain / Note Expression Synth SDK
|
||||
examples are the copy-source. Not a tar pit.
|
||||
- **D5 deletes JUCE's biggest justification.** JUCE exists largely for multi-format /
|
||||
cross-platform, both of which D5 removed. Its one genuine remaining pull is the editor
|
||||
UI — and ReaSampler is the atypical case where even that is weak, because it already
|
||||
has a working, docked, custom-drawn LICE UI (`bank_panel`) and a house style. Drawing
|
||||
the editor in a VST3 `IPlugView` that hosts a LICE surface reuses that muscle, keeps
|
||||
the look house-consistent, and avoids JUCE's AGPL-or-pay license posture (the Steinberg
|
||||
SDK is permissive, no revenue gate).
|
||||
- **The one real edge — the `IPlugView`↔LICE bridge** (window lifecycle, sizing, event
|
||||
routing from the host into the draw/hit-test loop) — is *the same class of work*
|
||||
ReaSampler already did to dock `bank_panel`, not a new competence, but it is less
|
||||
trodden than dropping in a JUCE editor. It is therefore the phase's **opening spike**
|
||||
(below), which also converts §1a's experienced-estimates (Windows module-export
|
||||
symbol names, factory-macro spellings, exact bridge marshalling) into verified fact
|
||||
before the engine build leans on them. VSTGUI (the SDK's bundled toolkit) is the noted
|
||||
fallback rung *only if* the LICE bridge proves gnarlier than the panel work suggests;
|
||||
JUCE is the last resort behind that.
|
||||
|
||||
## The pure core (D3 — the load-bearing split, transplanted)
|
||||
|
||||
**The sampler's voice engine, envelope math, key/velocity mapping, repitch/interpolation,
|
||||
and keymap resolution are a pure, REAPER-free, DAW-free, unit-tested module** — the
|
||||
mirror of `bank_model` / `peaks` / `view_mode_model` / `bank_book`, tested in the CTest
|
||||
harness outside any host. This is the heart of the phase; **test it hard.** The VST3
|
||||
wrapper — the `SingleComponentEffect` subclass, bus setup, the `process` call
|
||||
marshalling MIDI→core and core→audio-buffer, the `IPlugView` LICE editor, and the bridge
|
||||
calls that read `"reasampler"` ext-state — is the **thin shell**, the only part that
|
||||
touches VST3 or REAPER at all. Critically, this split is **invariant under the build-shape
|
||||
choice**: whether the shell is bare-SDK or (hypothetically) JUCE, the pure core is
|
||||
identical, REAPER-free, and tested the same way. The format choice is a shell choice; the
|
||||
core is invariant.
|
||||
|
||||
## Module architecture (preserve the pure/shell split — in the new artifact)
|
||||
|
||||
Pure (no REAPER types, no VST3 types, unit-tested — the mirror of `bank_model`):
|
||||
- **Sampler core** — voice allocation/polyphony, amplitude envelope (ADSR), key→sample
|
||||
and velocity→sample mapping (the keymap), repitch/interpolation from root note, and
|
||||
keymap resolution. REAPER-free *and* VST3-free, unit-tested in CTest against known
|
||||
signals (mirror of how `peaks` asserts an envelope). This is D3's pure core and the
|
||||
heart of the phase.
|
||||
|
||||
Shell (VST3-facing / REAPER-facing, thin):
|
||||
- **VST3 wrapper** — `SingleComponentEffect` subclass: `initialize` (declare an event
|
||||
input bus + an audio output bus, no audio input), `setupProcessing`, `setActive`,
|
||||
`setState`/`getState`, and the hot-path `process` that reads MIDI off the event bus,
|
||||
drives the pure core, and writes the core's per-voice audio to the output bus. Plus the
|
||||
module factory (`GetPluginFactory` + Windows `InitDll`/`ExitDll` — verify exact export
|
||||
names at the spike).
|
||||
- **`IPlugView` LICE editor** — hosts a LICE-drawn surface in the VST3 view seat
|
||||
(window creation/sizing, host→draw/hit-test event routing). Reuses the `bank_panel`
|
||||
LICE/SWELL competence and house style.
|
||||
- **Bridge/state reader** — resolves `GetProjExtState`/`EnumProjExtState` by name over
|
||||
the host callback, fetches the host project context, reads the live `"reasampler"`
|
||||
ext-state (bank index + intrinsic fields + performance map), and resolves WAV audio
|
||||
paths the same project-relative way `persist` does.
|
||||
|
||||
## Precision / invariant implications
|
||||
|
||||
- **The bank is one source; the instrument is another view of it (never a fork).** The
|
||||
instrument is a pure *consumer* of the bank — it does not copy samples, does not own a
|
||||
private sample store, and does not mutate the bank. The bank stays the single
|
||||
authoritative artifact (the one-source-multiple-views instinct). Any instrument path
|
||||
that writes back into the bank or keeps its own sample copies is a bug.
|
||||
- **Capture/placement/playback stay distinct acts.** The instrument reads and plays; it
|
||||
never captures and never inserts into the arrange. The capture load-bearing principle
|
||||
is untouched — Phase S adds a *third* distinct act (playback) without weakening the
|
||||
capture↔placement separation.
|
||||
- **`Sample` field addition is additive and lossless.** Root note + loop points are new
|
||||
optional fields with JSON round-trip, defaulting cleanly for samples captured before
|
||||
the addition — the same additive, backward-compatible shape as `provenance` (M1). No
|
||||
existing `Sample` field changes; no `BankIndex` behavior changes.
|
||||
- **Relative-paths-only survives.** The instrument resolves audio via the M4
|
||||
project-relative machinery; it introduces no absolute paths.
|
||||
|
||||
## Embedded TCP/MCP UI (D-D, settled 2026-07-26 — SCHEDULED as a later Phase S point)
|
||||
|
||||
A REAPER-hosted VST3 can draw its own UI **inline in the track/mixer control panel**
|
||||
via `reaper_plugin_fx_embed.h` (the plugin implements `IReaperUIEmbedInterface`; the
|
||||
same Cockos surface REAPER's own embedded FX use). A ReaSampler instrument can render a
|
||||
compact keymap/level strip inline in the TCP/MCP, not only in its own window. Because
|
||||
this uses the **same LICE-class drawing as the D-A editor path**, it composes naturally
|
||||
with the bare-SDK-plus-LICE build — the groundwork is the groundwork.
|
||||
|
||||
**Settled: scheduled, not deferred.** This is a real, in-phase later point — it lands
|
||||
**after** the main `IPlugView` editor exists (it composes with that LICE path), not a
|
||||
someday-note. It is polish, not a Tier-0 need, so it sequences last in the phase; but it
|
||||
is on the roadmap. **Must-verify before build:** the `IReaperUIEmbedInterface` contract
|
||||
and embed message/lifecycle against `vendor/reaper-sdk/sdk/reaper_plugin_fx_embed.h`.
|
||||
|
||||
## REAPER / Steinberg API surface (verify all signatures)
|
||||
|
||||
- **VST3 SDK (a new vendored dependency — vendor it at the spike).** `FUnknown` and the
|
||||
`IComponent` / `IAudioProcessor` / `IEditController` interface family; the
|
||||
`SingleComponentEffect` / `EditControllerEx1` / `AudioEffect` base classes; the class
|
||||
factory (`GetPluginFactory` + factory macros); `IPlugView` for the editor;
|
||||
`ProcessData` / `ProcessSetup` for the hot path. **Verify** interface members, the
|
||||
base-class overrides, factory-macro spellings, and the Windows module-export symbol
|
||||
names (`InitDll`/`ExitDll`/`GetPluginFactory`) against the vendored SDK at the spike —
|
||||
the framing doc flags several of these as experienced estimates.
|
||||
- **REAPER VST-host bridge.** `hostcb` opcode `0xdeadf00d` (resolve API function by
|
||||
name) and `0xdeadf00e` (host context); the by-name resolution of
|
||||
`GetProjExtState`/`SetProjExtState`/`EnumProjExtState`. **Verify** against
|
||||
`vendor/reaper-sdk/sdk/reaper_plugin.h` + `video_processor.h` +
|
||||
`reaper_plugin_functions.h`.
|
||||
- **Embedded UI (D-D, later point).** `IReaperUIEmbedInterface` and the embed
|
||||
message/lifecycle contract — verify against
|
||||
`vendor/reaper-sdk/sdk/reaper_plugin_fx_embed.h` before use.
|
||||
- **LICE/SWELL editor.** Reuses the `bank_panel` LICE/SWELL drawing surface; verify the
|
||||
`IPlugView`↔LICE window/bitmap bridge at the spike (window creation, sizing, event
|
||||
routing) — the least-trodden edge of the phase.
|
||||
|
||||
## Non-goals / guardrails
|
||||
|
||||
- **The instrument never captures and never inserts into the arrange.** Playback is a
|
||||
read-only act over the bank. Any instrument path that captures, places a timeline item,
|
||||
or writes back into the bank is a bug — reject in review.
|
||||
- **The instrument keeps no private copy of the samples.** It consumes the one
|
||||
authoritative bank; per-instance sample stores are a non-goal (they refork the source
|
||||
the one-source-multiple-views instinct keeps single).
|
||||
- **No cross-platform / multi-format.** Windows-only, VST3-only, REAPER-only (D5). Do
|
||||
not add an AU/AAX/VST2/CLAP wrapper, a mac/Linux build, or a standalone host target.
|
||||
- **The pure core stays REAPER-free *and* VST3-free.** The voice engine / envelope /
|
||||
keymap / repitch module takes no VST3 or REAPER type at its boundary — the shell
|
||||
marshals. Any VST3 or REAPER type leaking into the core is a bug (the D3 split).
|
||||
- **Additive to the extension.** The `Sample` intrinsic-field addition is additive
|
||||
(new optional fields; no existing field or `BankIndex` behavior changes); everything
|
||||
else in Phase S lives in the *second* artifact and does not alter the extension's
|
||||
M/D/B/R/V pillars.
|
||||
- **Do not spec Tier 2/3.** Tier 2 is held (noted, not specified); Tier 3 is
|
||||
optional-forever. Do not let their feature lists drive Tier 0–1's build shape.
|
||||
- **Verify Steinberg SDK, bridge, embed, and LICE-view surfaces** against the vendored
|
||||
headers before use — several §1a claims are experienced estimates until the spike
|
||||
confirms them.
|
||||
|
||||
Reference in New Issue
Block a user