docs(product): fold the REAPER VST-host bridge into MIDI-playback framing

A VST hosted in REAPER can call back into REAPER's own API (name-resolved via audioMaster), so a native ReaSampler instrument reads live project ext-state directly. Re-frames the seam, D6, and the D1/D2 calculus.
2026-07-26 05:49:47 -04:00
parent cce9016279
commit 6659c2dbdf
1 changed files with 176 additions and 33 deletions
@@ -5,10 +5,35 @@ banks. This is a **discussion-shaping doc, not a build plan** — no phase, no P
 points, no settled forks yet. It exists so Daniel and the-boss can react to an honest
 map of the option space before anything is scoped.
-Status: framed by product-designer (2026-07-26). Grounded in the vendored REAPER SDK
+Status: framed by product-designer (2026-07-26), **revised 2026-07-26** to correct a
-headers (`vendor/reaper-sdk/sdk/reaper_plugin.h`, `reaper_plugin_functions.h`) — the
+material omission — the earlier draft missed REAPER's **VST-host bridge**: a VST/VST3
-plugin-format claims below are checked against those headers, not asserted. The
+plugin *hosted inside REAPER* can call back into REAPER's own API from within the
-genuine forks are flagged as **Daniel's to decide**; nothing here pre-decides them.
+plugin, resolving API function pointers by name over the host `audioMaster` callback.
 That single fact changes the integration seam (§2), the "one product or two" question
 (D6), and the JSFX-vs-native calculus (D1/D2). The corrections are made plainly
 in-place, not appended as a footnote; where the prior draft was too pessimistic about
 what a native instrument can reach, it is fixed and the correction is flagged. Grounded
 in the vendored REAPER SDK headers (`vendor/reaper-sdk/sdk/reaper_plugin.h`,
 `reaper_plugin_functions.h`, `video_processor.h`, `reaper_plugin_fx_embed.h`) and
 REAPER's published VST-extensions SDK page (`reaper.fm/sdk/vst/vst_ext.php`) — the
 plugin-format claims below are checked against those, not asserted. The genuine forks
 are flagged as **Daniel's to decide**; nothing here pre-decides them.
 > **The bridge, stated once, up front (the correction).** A hosted VST is not limited to
 > scraping bank WAVs + JSON off disk. REAPER hands the plugin its host callback; calling
 > `hostcb(&effect, 0xdeadbeef, 0xdeadf00d, 0, "FunctionName", 0.0)` resolves *any* REAPER
 > API function by name — the same string-keyed API surface the extension uses (verified:
 > `video_processor.h` line 44 imports `video_CreateVideoProcessor` exactly this way;
 > `GetProjExtState`/`SetProjExtState`/`EnumProjExtState` are ordinary entries in that same
 > string-keyed table, `reaper_plugin_functions.h` lines 8376/8796/9969). The plugin can
 > also fetch its **host context** — the track/take/project it's instantiated in (sibling
 > opcode `0xdeadf00e`, `video_processor.h` line 40; CLAP has `clap_get_reaper_context`,
 > `reaper_plugin.h` line 142). **Consequence:** a native ReaSampler instrument can read
 > the *same* `"reasampler"` project ext-state that `persist` writes — live, project-aware,
 > following the active project — not a file it re-parses off disk. This is a
 > REAPER-VST-specific capability (it exists because the plugin is hosted *in REAPER*), and
 > it is the thumb on the scale the prior draft failed to weigh. The cost of leaning on it
 > is stated honestly in D1/D2: it makes the native plugin **REAPER-coupled.**
 ---
@@ -88,16 +113,25 @@ the single most important thing for Daniel to internalize before scoping anythin
 **Option A — a real VSTi/instrument plugin (JUCE or bare VST3/CLAP SDK) that reads
 ReaSampler's banks.** A separate build artifact: a VST3 (and/or CLAP) sampler plugin
-that the user instantiates on an instrument track. It reads the bank JSON + WAV layout
+that the user instantiates on an instrument track. It maps samples across the keyboard
-ReaSampler writes, maps samples across the keyboard, and plays them MIDI-triggered with
+and plays them MIDI-triggered with a real voice engine. This is the "sophisticated
-a real voice engine. This is the "sophisticated sampler" answer.
+sampler" answer.
 - *Gives you:* everything an instrument is — polyphony, velocity, envelopes, the works,
-  fully integrated into REAPER's routing/render/record path like any VSTi.
+  fully integrated into REAPER's routing/render/record path like any VSTi. **And,
  because it's hosted in REAPER, it reaches ReaSampler's project state directly via the
  VST-host bridge** — it doesn't merely parse bank JSON off disk, it can read the live
  `"reasampler"` ext-state `persist` writes, know its own host project, and follow the
  active project. The prior draft treated Option A as a divorced file-reader; that was
  the omission. A native ReaSampler instrument can be **natively, tightly integrated with
  the extension's project state**, not loosely coupled through a file format alone.
 - *Costs:* a **second codebase in a second plugin format**, almost certainly a new
  dependency (JUCE is the pragmatic choice; bare VST3 SDK is more code, CLAP is leaner
  but younger). It is a real DSP/voice-engine build, not a weekend. Cross-platform DSP,
  its own build/release/signing story, its own UI toolkit. This is a **product-sized
-  commitment**, not a feature.
+  commitment**, not a feature. And leaning on the bridge for state means the instrument
  is **REAPER-coupled** — it stops being a portable VST that runs in any host and becomes
  a REAPER companion. That may be exactly right (see D6), but it is a real narrowing and
  must be a conscious choice, not a side effect.
 **Option B — a JSFX sampler.** JSFX is REAPER's built-in scriptable plugin format
 (text `.jsfx` files, JIT-compiled by REAPER, hostable as an instrument). A JSFX
@@ -112,6 +146,17 @@ instrument *can* receive live MIDI and emit audio on a track. It can load sample
  than in a native plugin. Reading ReaSampler's JSON index from JSFX is friction (JSFX
  is not a general-purpose file parser). Best fit for a **minimal-to-mid** sampler, a
  real ceiling for a **sophisticated** one.
 - *What JSFX can't match, now that the bridge is on the table:* the VST-host bridge is a
  **VST/VST3/CLAP capability** — it is how a *hosted plugin* resolves REAPER API pointers.
  JSFX is not a hosted VST; it has REAPER integration of its own kind (`ext_noinit`,
  `file_*` ops, the shared `gmem[]`, `slider`/parameter plumbing), but it does **not** get
  the same string-keyed REAPER-API surface a native VST does, and in particular has no
  clean, native path to read `persist`'s `"reasampler"` project ext-state the way a native
  VST can. On the JSFX path, the seam is genuinely "read a file ReaSampler wrote next to
  the bank" (hence the sidecar note in §2). This asymmetry is new information: the bridge
  makes **native meaningfully more integratable** than the prior draft assumed — it's not
  just "native is more powerful DSP," it's "native can be *part of ReaSampler's state
  model*, JSFX stays a file-coupled consumer."
 **Option C — the hybrid (recommended framing to explore first).** Keep the extension
 as the sole owner of **capture + organization** (which is its whole existing identity
@@ -130,10 +175,28 @@ and **"how much sampler do we actually want."** Those are §3 and §4.
 ## 2. The integration seam
-The obvious shared artifact is what ReaSampler already produces: **the bank folder
+The prior draft assumed one seam: **the bank folder (project-relative WAVs) + the
-(project-relative WAVs) + the bank/index JSON.** The question is whether that JSON is
+bank/index JSON on disk**, with the playback engine as a file-reader. That's still the
-*sufficient* for a playback engine, or whether playback needs mapping data the index
+seam for *sample audio* (WAVs are files; there's no getting a live PCM stream across the
-doesn't carry today.
+bridge, nor would you want to). But for **everything that isn't the raw audio** — the
 index, the mapping data, which project's bank is active — the bridge opens a **second,
 richer seam** the prior draft missed:
 - **File seam (audio + a portable snapshot).** WAVs on disk; index/mapping as JSON or a
  sidecar. Tool-agnostic, host-agnostic, portable. Any instrument (JSFX or native, in
  REAPER or any other host) can consume it. This is the *only* seam JSFX gets, and the
  *fallback/export* seam for native.
 - **Live-state seam (native-in-REAPER only).** A native VST instance reads the
  `"reasampler"` project ext-state directly via the bridge (`GetProjExtState` /
  `EnumProjExtState`, resolved by name over `hostcb`), and knows its own host project via
  the context callback. It sees what `persist` last wrote, follows the active project, and
  needs no "point me at the right bank folder" wiring — it *asks REAPER* which project it's
  in. This is strictly more than the file seam and it's REAPER-coupled by construction.
 The design question below — "is the JSON sufficient, or does playback need mapping the
 index doesn't carry" — is unchanged. What the bridge changes is *where that mapping can
 live and how the instrument gets it*: for native, the mapping can be **live shared state**
 between extension and instrument, not just a file handed across.
 **What the current index carries** (from `bank_model`'s `Sample`, per CONTEXT.md §Data
 model): id, display name, relative path, source range, channel count, sample rate,
@@ -179,18 +242,29 @@ opinionated part. It also means **the bank index grows only by file-intrinsic fi
 seam ReaSampler already knows how to add (mirror of how `provenance` was added as a
 field in M1 and populated later).
-**A note on format:** if the instrument is native (Option A), it reads the bank JSON
+**A note on format (revised for the bridge):** the friction is asymmetric, and more so
-directly — trivial. If it's JSFX (Option B), reading arbitrary JSON is friction; the
+than the prior draft said.
-seam might need a **simpler sidecar** (a flat `.txt`/key-value map ReaSampler writes
+- *Native (Option A):* doesn't just "read the bank JSON off disk" — it can read the map
-next to the bank, JSFX-parseable) rather than making JSFX parse the index JSON. That's
+  out of **live project ext-state** via the bridge (whichever of i/ii/iii you choose, the
-a concrete cost of the JSFX path and a reason the seam design and the format choice are
+  extension and instrument can share the *same* `"reasampler"` state rather than one
-coupled.
+  writing a file the other re-parses). The `.wav` audio is still files; the mapping need
  not be. Cleanest possible integration.
 - *JSFX (Option B):* no bridge — reading arbitrary JSON is friction, so the seam likely
  needs a **simpler sidecar** (a flat `.txt`/key-value map ReaSampler writes next to the
  bank, JSFX-parseable). On the JSFX path the seam is unavoidably file-based. That's a
  concrete cost of JSFX and a reason the seam design and the format choice stay coupled.
-**Portability caveat:** the bank is project-relative and travels with the `.rpp`
+**Portability caveat (partly dissolved on the native path).** The bank is project-relative
-(M4 machinery). Any instrument consuming it must resolve paths the same way — so the
+and travels with the `.rpp` (M4 machinery). Any *file-seam* consumer must resolve paths
-instrument needs to know the *current* project bank folder. A native plugin instance
+the same way — so a JSFX instrument still needs to be told the current project bank folder
-on a track can be told its folder (saved in plugin state); a JSFX likewise. This is a
+(saved in its own state), a real wiring detail, don't hand-wave it. But a **native**
-solvable wiring detail but a real one — flag it, don't hand-wave it.
+instance sidesteps this: via the context callback it asks REAPER which project it's in,
 then reads that project's bank location straight from ext-state — no "point me at the
 folder" step. What was a shared wiring cost is now a **cost only on the JSFX path.** Note
 the flip side: a native instrument that resolves its bank *only* through the bridge won't
 work outside REAPER at all — so if a portable/exportable instrument is ever a goal, keep
 the file seam as the source of truth and treat the live-state seam as an accelerator, not
 the sole path.
 ---
@@ -245,13 +319,29 @@ None of these are pre-decided here. Each is a genuine fork.
 **D1 — Instrument format: JSFX vs. native VSTi (VST3/CLAP, likely via JUCE).** The
 central fork. JSFX = no new binary, no JUCE, cross-platform free, real ceiling on
-sophistication and awkward bank-JSON reading. Native = full sophistication and clean
+sophistication, awkward bank-JSON reading, **and no VST-host bridge** (file-seam only).
-JSON integration, at the cost of a whole second codebase in a second format with its
+Native = full sophistication, clean integration, **plus the bridge**: live `"reasampler"`
-own build/release/dependency/signing story. *My lean, for discussion only:* if the goal
+ext-state, project-awareness, embedded TCP/MCP UI (see below) — at the cost of a whole
-is Tier 0–1, **prototype in JSFX first** — it proves the seam and the value with near-zero
+second codebase in a second format with its own build/release/dependency/signing story,
-format commitment, and if it hits a ceiling the seam you designed still serves a later
+**and REAPER-coupling if it leans on the bridge.**
-native build. Reach for native when Tier 2+ is a firm goal, not before. But this is
+
-squarely Daniel's call and depends on how sophisticated he actually wants this.
+*What the bridge does to the native column (the honest update).* The prior draft's native
 pitch was "more DSP power + parses JSON cleanly." The bridge adds a category the prior
 draft didn't weigh: native can be **part of ReaSampler's live state model**, not a
 file-coupled consumer. That is a real pull toward native beyond raw DSP — it's the
 difference between "an instrument that reads a ReaSampler export" and "an instrument that
 *is* ReaSampler's playback surface." It doesn't make JSFX wrong; it makes native's ceiling
 higher and its integration tighter than I credited. Weigh it against the coupling cost: a
 bridge-dependent VST is a **REAPER-only** artifact, no longer a portable VST.
 *My lean, updated but unchanged in direction, for discussion only:* if the goal is
 Tier 0–1, **still prototype in JSFX first** — it proves the *sampler value and the
 key/velocity seam* with near-zero format commitment, and the file seam you design there is
 exactly the portable/export seam a native build would keep anyway. The bridge is an
 *integration* advantage, not a *does-the-sampler-work* advantage, so it doesn't change what
 the cheapest proof is. Reach for native when either (a) Tier 2+ sophistication is a firm
 goal, or (b) **tight live integration with ReaSampler's project state is itself the point**
 — that second trigger is new, and it's the bridge's real contribution to this fork.
 **D2 — Whether to take a JUCE (or any external plugin-SDK) dependency at all.** The
 project today is a clean C++ extension with two vendored submodules and a proud
@@ -260,6 +350,17 @@ second build target of a fundamentally different kind. That's a real architectur
 weight. JSFX sidesteps it entirely. Flagging it as its own decision because "should we
 depend on JUCE" is a bigger standing commitment than "should we build a sampler."
 *The bridge's bearing on D2.* Weigh two coupling costs, not one. A native instrument that
 uses the bridge takes on **both** a plugin-SDK dependency (JUCE/VST3/CLAP) *and* a
 REAPER-API coupling (the same `reaper_plugin_functions.h` surface the extension binds — so
 the instrument would vendor and bind against it too, likely resolving pointers via `hostcb`
 rather than the extension's startup path). The upside is that the pure sampler core stays
 REAPER-free and testable regardless (D3); it's only the *shell* that touches the bridge, so
 the coupling is contained where the project already puts REAPER coupling. The downside is
 plain: choosing the bridge means choosing a REAPER-only instrument. If Daniel wants a
 sampler that also runs in other hosts, the bridge is off the table and D1's native pitch
 loses its integration edge — collapsing back toward "native = more DSP power only."
 **D3 — Does the pure-core discipline survive the format boundary?** ReaSampler's
 identity is *pure REAPER-free testable core + thin shells*. A sampler's **voice engine,
 envelope math, key/velocity mapping, and repitch logic are exactly the kind of pure,
@@ -292,8 +393,50 @@ feature *of* ReaSampler — the way a sample library and a sampler that reads it
 related-but-distinct products. That framing might keep ReaSampler sharp (it stays the
 capture tool it is) while letting the instrument evolve on its own clock and format.
 The alternative reading — it's all one integrated sampler-workstation — is also
-coherent. **Which of those two ReaSampler *is* is the highest-order question here, and
+coherent.
-it's Daniel's to answer before format/tier decisions mean much.**
+
 *Re-argued with the bridge on the scale (the prior draft weighed this fork without it).*
 The prior draft implicitly assumed the two products could only ever share a **file
 format** — which makes "companion product" feel clean and natural, because a shared file
 is exactly what independent products share. The bridge changes the terms. A native
 instrument can be **tightly, live-integrated with ReaSampler's project state**: same
 `"reasampler"` ext-state, same project identity, following the active project — a degree
 of coupling that is *more than* "two products that happen to read the same files" and
 starts to look like "two faces of one tool sharing one state model." That is a genuine
 thumb toward the **one-integrated-product** reading. It's not decisive, and I won't
 pretend it is:
  - It only applies if the instrument is **native** (JSFX can't reach the bridge, so a
    JSFX instrument *is* naturally a file-coupled companion — the format choice and this
    fork are entangled: pick JSFX and you've half-answered D6 toward "companion").
  - It's an *available* integration, not a *required* one — a native instrument could
    still be built to the file seam only, deliberately staying a loose companion for
    portability. The bridge widens the option, it doesn't force it.
  - The tight-integration reading buys **REAPER-coupling**; the companion reading buys
    **portability and independent evolution.** Those are real, opposed goods, and the
    bridge doesn't resolve which Daniel wants — it just makes the integrated option
    *materially more capable* than the prior draft credited, so the companion reading no
    longer wins by default on "well, they can only share a file anyway."
 **Net:** the bridge strengthens "one integrated product" from a coherent-but-equal option
 into the one with the stronger technical affordance — *if* native-in-REAPER is the path.
 It does not pre-decide the fork. This is still the highest-order question and it's Daniel's
 to answer, but he should answer it knowing the integrated option is richer than the
 file-sharing picture implied — and knowing that choosing JSFX quietly tilts it the other
 way.
 **D7 — Embedded TCP/MCP UI (new option the bridge surfaces; native/CLAP only).** While
 verifying the bridge I found a second REAPER-VST-specific affordance the prior draft never
 mentioned: a hosted plugin can **draw its own embedded UI directly in the track/mixer
 control panel** (`reaper_plugin_fx_embed.h`: VST2 answers `canDo("hasCockosEmbeddedUI")`
 with `0xbeef0000` and draws via `effVendorSpecific/effEditDraw`; VST3 implements
 `IReaperUIEmbedInterface`; CLAP exposes `cockos.reaper_embedui`). Concretely: a native
 ReaSampler instrument could render a compact keymap/level strip *inline in the TCP/MCP*,
 not only in its own plugin window — the same surface REAPER's own JS/embedded FX use. This
 is **native/CLAP-only** (JSFX can't do it) and is pure polish, not a Tier-0 need — but it's
 a real integration affordance that only exists on the native-in-REAPER path, and it
 compounds the D1/D6 tilt: if "ReaSampler's playback surface, deeply woven into REAPER's UI"
 is the vision, this is a lever only native reaches. Flagging it for completeness, not
 recommending it — it's a Tier-3-ish nicety, deferred by default.
 ---