docs(product): fold the REAPER VST-host bridge into MIDI-playback framing
A VST hosted in REAPER can call back into REAPER's own API (name-resolved via audioMaster), so a native ReaSampler instrument reads live project ext-state directly. Re-frames the seam, D6, and the D1/D2 calculus.
This commit is contained in:
+176
-33
@@ -5,10 +5,35 @@ banks. This is a **discussion-shaping doc, not a build plan** — no phase, no P
|
||||
points, no settled forks yet. It exists so Daniel and the-boss can react to an honest
|
||||
map of the option space before anything is scoped.
|
||||
|
||||
Status: framed by product-designer (2026-07-26). Grounded in the vendored REAPER SDK
|
||||
headers (`vendor/reaper-sdk/sdk/reaper_plugin.h`, `reaper_plugin_functions.h`) — the
|
||||
plugin-format claims below are checked against those headers, not asserted. The
|
||||
genuine forks are flagged as **Daniel's to decide**; nothing here pre-decides them.
|
||||
Status: framed by product-designer (2026-07-26), **revised 2026-07-26** to correct a
|
||||
material omission — the earlier draft missed REAPER's **VST-host bridge**: a VST/VST3
|
||||
plugin *hosted inside REAPER* can call back into REAPER's own API from within the
|
||||
plugin, resolving API function pointers by name over the host `audioMaster` callback.
|
||||
That single fact changes the integration seam (§2), the "one product or two" question
|
||||
(D6), and the JSFX-vs-native calculus (D1/D2). The corrections are made plainly
|
||||
in-place, not appended as a footnote; where the prior draft was too pessimistic about
|
||||
what a native instrument can reach, it is fixed and the correction is flagged. Grounded
|
||||
in the vendored REAPER SDK headers (`vendor/reaper-sdk/sdk/reaper_plugin.h`,
|
||||
`reaper_plugin_functions.h`, `video_processor.h`, `reaper_plugin_fx_embed.h`) and
|
||||
REAPER's published VST-extensions SDK page (`reaper.fm/sdk/vst/vst_ext.php`) — the
|
||||
plugin-format claims below are checked against those, not asserted. The genuine forks
|
||||
are flagged as **Daniel's to decide**; nothing here pre-decides them.
|
||||
|
||||
> **The bridge, stated once, up front (the correction).** A hosted VST is not limited to
|
||||
> scraping bank WAVs + JSON off disk. REAPER hands the plugin its host callback; calling
|
||||
> `hostcb(&effect, 0xdeadbeef, 0xdeadf00d, 0, "FunctionName", 0.0)` resolves *any* REAPER
|
||||
> API function by name — the same string-keyed API surface the extension uses (verified:
|
||||
> `video_processor.h` line 44 imports `video_CreateVideoProcessor` exactly this way;
|
||||
> `GetProjExtState`/`SetProjExtState`/`EnumProjExtState` are ordinary entries in that same
|
||||
> string-keyed table, `reaper_plugin_functions.h` lines 8376/8796/9969). The plugin can
|
||||
> also fetch its **host context** — the track/take/project it's instantiated in (sibling
|
||||
> opcode `0xdeadf00e`, `video_processor.h` line 40; CLAP has `clap_get_reaper_context`,
|
||||
> `reaper_plugin.h` line 142). **Consequence:** a native ReaSampler instrument can read
|
||||
> the *same* `"reasampler"` project ext-state that `persist` writes — live, project-aware,
|
||||
> following the active project — not a file it re-parses off disk. This is a
|
||||
> REAPER-VST-specific capability (it exists because the plugin is hosted *in REAPER*), and
|
||||
> it is the thumb on the scale the prior draft failed to weigh. The cost of leaning on it
|
||||
> is stated honestly in D1/D2: it makes the native plugin **REAPER-coupled.**
|
||||
|
||||
---
|
||||
|
||||
@@ -88,16 +113,25 @@ the single most important thing for Daniel to internalize before scoping anythin
|
||||
|
||||
**Option A — a real VSTi/instrument plugin (JUCE or bare VST3/CLAP SDK) that reads
|
||||
ReaSampler's banks.** A separate build artifact: a VST3 (and/or CLAP) sampler plugin
|
||||
that the user instantiates on an instrument track. It reads the bank JSON + WAV layout
|
||||
ReaSampler writes, maps samples across the keyboard, and plays them MIDI-triggered with
|
||||
a real voice engine. This is the "sophisticated sampler" answer.
|
||||
that the user instantiates on an instrument track. It maps samples across the keyboard
|
||||
and plays them MIDI-triggered with a real voice engine. This is the "sophisticated
|
||||
sampler" answer.
|
||||
- *Gives you:* everything an instrument is — polyphony, velocity, envelopes, the works,
|
||||
fully integrated into REAPER's routing/render/record path like any VSTi.
|
||||
fully integrated into REAPER's routing/render/record path like any VSTi. **And,
|
||||
because it's hosted in REAPER, it reaches ReaSampler's project state directly via the
|
||||
VST-host bridge** — it doesn't merely parse bank JSON off disk, it can read the live
|
||||
`"reasampler"` ext-state `persist` writes, know its own host project, and follow the
|
||||
active project. The prior draft treated Option A as a divorced file-reader; that was
|
||||
the omission. A native ReaSampler instrument can be **natively, tightly integrated with
|
||||
the extension's project state**, not loosely coupled through a file format alone.
|
||||
- *Costs:* a **second codebase in a second plugin format**, almost certainly a new
|
||||
dependency (JUCE is the pragmatic choice; bare VST3 SDK is more code, CLAP is leaner
|
||||
but younger). It is a real DSP/voice-engine build, not a weekend. Cross-platform DSP,
|
||||
its own build/release/signing story, its own UI toolkit. This is a **product-sized
|
||||
commitment**, not a feature.
|
||||
commitment**, not a feature. And leaning on the bridge for state means the instrument
|
||||
is **REAPER-coupled** — it stops being a portable VST that runs in any host and becomes
|
||||
a REAPER companion. That may be exactly right (see D6), but it is a real narrowing and
|
||||
must be a conscious choice, not a side effect.
|
||||
|
||||
**Option B — a JSFX sampler.** JSFX is REAPER's built-in scriptable plugin format
|
||||
(text `.jsfx` files, JIT-compiled by REAPER, hostable as an instrument). A JSFX
|
||||
@@ -112,6 +146,17 @@ instrument *can* receive live MIDI and emit audio on a track. It can load sample
|
||||
than in a native plugin. Reading ReaSampler's JSON index from JSFX is friction (JSFX
|
||||
is not a general-purpose file parser). Best fit for a **minimal-to-mid** sampler, a
|
||||
real ceiling for a **sophisticated** one.
|
||||
- *What JSFX can't match, now that the bridge is on the table:* the VST-host bridge is a
|
||||
**VST/VST3/CLAP capability** — it is how a *hosted plugin* resolves REAPER API pointers.
|
||||
JSFX is not a hosted VST; it has REAPER integration of its own kind (`ext_noinit`,
|
||||
`file_*` ops, the shared `gmem[]`, `slider`/parameter plumbing), but it does **not** get
|
||||
the same string-keyed REAPER-API surface a native VST does, and in particular has no
|
||||
clean, native path to read `persist`'s `"reasampler"` project ext-state the way a native
|
||||
VST can. On the JSFX path, the seam is genuinely "read a file ReaSampler wrote next to
|
||||
the bank" (hence the sidecar note in §2). This asymmetry is new information: the bridge
|
||||
makes **native meaningfully more integratable** than the prior draft assumed — it's not
|
||||
just "native is more powerful DSP," it's "native can be *part of ReaSampler's state
|
||||
model*, JSFX stays a file-coupled consumer."
|
||||
|
||||
**Option C — the hybrid (recommended framing to explore first).** Keep the extension
|
||||
as the sole owner of **capture + organization** (which is its whole existing identity
|
||||
@@ -130,10 +175,28 @@ and **"how much sampler do we actually want."** Those are §3 and §4.
|
||||
|
||||
## 2. The integration seam
|
||||
|
||||
The obvious shared artifact is what ReaSampler already produces: **the bank folder
|
||||
(project-relative WAVs) + the bank/index JSON.** The question is whether that JSON is
|
||||
*sufficient* for a playback engine, or whether playback needs mapping data the index
|
||||
doesn't carry today.
|
||||
The prior draft assumed one seam: **the bank folder (project-relative WAVs) + the
|
||||
bank/index JSON on disk**, with the playback engine as a file-reader. That's still the
|
||||
seam for *sample audio* (WAVs are files; there's no getting a live PCM stream across the
|
||||
bridge, nor would you want to). But for **everything that isn't the raw audio** — the
|
||||
index, the mapping data, which project's bank is active — the bridge opens a **second,
|
||||
richer seam** the prior draft missed:
|
||||
|
||||
- **File seam (audio + a portable snapshot).** WAVs on disk; index/mapping as JSON or a
|
||||
sidecar. Tool-agnostic, host-agnostic, portable. Any instrument (JSFX or native, in
|
||||
REAPER or any other host) can consume it. This is the *only* seam JSFX gets, and the
|
||||
*fallback/export* seam for native.
|
||||
- **Live-state seam (native-in-REAPER only).** A native VST instance reads the
|
||||
`"reasampler"` project ext-state directly via the bridge (`GetProjExtState` /
|
||||
`EnumProjExtState`, resolved by name over `hostcb`), and knows its own host project via
|
||||
the context callback. It sees what `persist` last wrote, follows the active project, and
|
||||
needs no "point me at the right bank folder" wiring — it *asks REAPER* which project it's
|
||||
in. This is strictly more than the file seam and it's REAPER-coupled by construction.
|
||||
|
||||
The design question below — "is the JSON sufficient, or does playback need mapping the
|
||||
index doesn't carry" — is unchanged. What the bridge changes is *where that mapping can
|
||||
live and how the instrument gets it*: for native, the mapping can be **live shared state**
|
||||
between extension and instrument, not just a file handed across.
|
||||
|
||||
**What the current index carries** (from `bank_model`'s `Sample`, per CONTEXT.md §Data
|
||||
model): id, display name, relative path, source range, channel count, sample rate,
|
||||
@@ -179,18 +242,29 @@ opinionated part. It also means **the bank index grows only by file-intrinsic fi
|
||||
seam ReaSampler already knows how to add (mirror of how `provenance` was added as a
|
||||
field in M1 and populated later).
|
||||
|
||||
**A note on format:** if the instrument is native (Option A), it reads the bank JSON
|
||||
directly — trivial. If it's JSFX (Option B), reading arbitrary JSON is friction; the
|
||||
seam might need a **simpler sidecar** (a flat `.txt`/key-value map ReaSampler writes
|
||||
next to the bank, JSFX-parseable) rather than making JSFX parse the index JSON. That's
|
||||
a concrete cost of the JSFX path and a reason the seam design and the format choice are
|
||||
coupled.
|
||||
**A note on format (revised for the bridge):** the friction is asymmetric, and more so
|
||||
than the prior draft said.
|
||||
- *Native (Option A):* doesn't just "read the bank JSON off disk" — it can read the map
|
||||
out of **live project ext-state** via the bridge (whichever of i/ii/iii you choose, the
|
||||
extension and instrument can share the *same* `"reasampler"` state rather than one
|
||||
writing a file the other re-parses). The `.wav` audio is still files; the mapping need
|
||||
not be. Cleanest possible integration.
|
||||
- *JSFX (Option B):* no bridge — reading arbitrary JSON is friction, so the seam likely
|
||||
needs a **simpler sidecar** (a flat `.txt`/key-value map ReaSampler writes next to the
|
||||
bank, JSFX-parseable). On the JSFX path the seam is unavoidably file-based. That's a
|
||||
concrete cost of JSFX and a reason the seam design and the format choice stay coupled.
|
||||
|
||||
**Portability caveat:** the bank is project-relative and travels with the `.rpp`
|
||||
(M4 machinery). Any instrument consuming it must resolve paths the same way — so the
|
||||
instrument needs to know the *current* project bank folder. A native plugin instance
|
||||
on a track can be told its folder (saved in plugin state); a JSFX likewise. This is a
|
||||
solvable wiring detail but a real one — flag it, don't hand-wave it.
|
||||
**Portability caveat (partly dissolved on the native path).** The bank is project-relative
|
||||
and travels with the `.rpp` (M4 machinery). Any *file-seam* consumer must resolve paths
|
||||
the same way — so a JSFX instrument still needs to be told the current project bank folder
|
||||
(saved in its own state), a real wiring detail, don't hand-wave it. But a **native**
|
||||
instance sidesteps this: via the context callback it asks REAPER which project it's in,
|
||||
then reads that project's bank location straight from ext-state — no "point me at the
|
||||
folder" step. What was a shared wiring cost is now a **cost only on the JSFX path.** Note
|
||||
the flip side: a native instrument that resolves its bank *only* through the bridge won't
|
||||
work outside REAPER at all — so if a portable/exportable instrument is ever a goal, keep
|
||||
the file seam as the source of truth and treat the live-state seam as an accelerator, not
|
||||
the sole path.
|
||||
|
||||
---
|
||||
|
||||
@@ -245,13 +319,29 @@ None of these are pre-decided here. Each is a genuine fork.
|
||||
|
||||
**D1 — Instrument format: JSFX vs. native VSTi (VST3/CLAP, likely via JUCE).** The
|
||||
central fork. JSFX = no new binary, no JUCE, cross-platform free, real ceiling on
|
||||
sophistication and awkward bank-JSON reading. Native = full sophistication and clean
|
||||
JSON integration, at the cost of a whole second codebase in a second format with its
|
||||
own build/release/dependency/signing story. *My lean, for discussion only:* if the goal
|
||||
is Tier 0–1, **prototype in JSFX first** — it proves the seam and the value with near-zero
|
||||
format commitment, and if it hits a ceiling the seam you designed still serves a later
|
||||
native build. Reach for native when Tier 2+ is a firm goal, not before. But this is
|
||||
squarely Daniel's call and depends on how sophisticated he actually wants this.
|
||||
sophistication, awkward bank-JSON reading, **and no VST-host bridge** (file-seam only).
|
||||
Native = full sophistication, clean integration, **plus the bridge**: live `"reasampler"`
|
||||
ext-state, project-awareness, embedded TCP/MCP UI (see below) — at the cost of a whole
|
||||
second codebase in a second format with its own build/release/dependency/signing story,
|
||||
**and REAPER-coupling if it leans on the bridge.**
|
||||
|
||||
*What the bridge does to the native column (the honest update).* The prior draft's native
|
||||
pitch was "more DSP power + parses JSON cleanly." The bridge adds a category the prior
|
||||
draft didn't weigh: native can be **part of ReaSampler's live state model**, not a
|
||||
file-coupled consumer. That is a real pull toward native beyond raw DSP — it's the
|
||||
difference between "an instrument that reads a ReaSampler export" and "an instrument that
|
||||
*is* ReaSampler's playback surface." It doesn't make JSFX wrong; it makes native's ceiling
|
||||
higher and its integration tighter than I credited. Weigh it against the coupling cost: a
|
||||
bridge-dependent VST is a **REAPER-only** artifact, no longer a portable VST.
|
||||
|
||||
*My lean, updated but unchanged in direction, for discussion only:* if the goal is
|
||||
Tier 0–1, **still prototype in JSFX first** — it proves the *sampler value and the
|
||||
key/velocity seam* with near-zero format commitment, and the file seam you design there is
|
||||
exactly the portable/export seam a native build would keep anyway. The bridge is an
|
||||
*integration* advantage, not a *does-the-sampler-work* advantage, so it doesn't change what
|
||||
the cheapest proof is. Reach for native when either (a) Tier 2+ sophistication is a firm
|
||||
goal, or (b) **tight live integration with ReaSampler's project state is itself the point**
|
||||
— that second trigger is new, and it's the bridge's real contribution to this fork.
|
||||
|
||||
**D2 — Whether to take a JUCE (or any external plugin-SDK) dependency at all.** The
|
||||
project today is a clean C++ extension with two vendored submodules and a proud
|
||||
@@ -260,6 +350,17 @@ second build target of a fundamentally different kind. That's a real architectur
|
||||
weight. JSFX sidesteps it entirely. Flagging it as its own decision because "should we
|
||||
depend on JUCE" is a bigger standing commitment than "should we build a sampler."
|
||||
|
||||
*The bridge's bearing on D2.* Weigh two coupling costs, not one. A native instrument that
|
||||
uses the bridge takes on **both** a plugin-SDK dependency (JUCE/VST3/CLAP) *and* a
|
||||
REAPER-API coupling (the same `reaper_plugin_functions.h` surface the extension binds — so
|
||||
the instrument would vendor and bind against it too, likely resolving pointers via `hostcb`
|
||||
rather than the extension's startup path). The upside is that the pure sampler core stays
|
||||
REAPER-free and testable regardless (D3); it's only the *shell* that touches the bridge, so
|
||||
the coupling is contained where the project already puts REAPER coupling. The downside is
|
||||
plain: choosing the bridge means choosing a REAPER-only instrument. If Daniel wants a
|
||||
sampler that also runs in other hosts, the bridge is off the table and D1's native pitch
|
||||
loses its integration edge — collapsing back toward "native = more DSP power only."
|
||||
|
||||
**D3 — Does the pure-core discipline survive the format boundary?** ReaSampler's
|
||||
identity is *pure REAPER-free testable core + thin shells*. A sampler's **voice engine,
|
||||
envelope math, key/velocity mapping, and repitch logic are exactly the kind of pure,
|
||||
@@ -292,8 +393,50 @@ feature *of* ReaSampler — the way a sample library and a sampler that reads it
|
||||
related-but-distinct products. That framing might keep ReaSampler sharp (it stays the
|
||||
capture tool it is) while letting the instrument evolve on its own clock and format.
|
||||
The alternative reading — it's all one integrated sampler-workstation — is also
|
||||
coherent. **Which of those two ReaSampler *is* is the highest-order question here, and
|
||||
it's Daniel's to answer before format/tier decisions mean much.**
|
||||
coherent.
|
||||
|
||||
*Re-argued with the bridge on the scale (the prior draft weighed this fork without it).*
|
||||
The prior draft implicitly assumed the two products could only ever share a **file
|
||||
format** — which makes "companion product" feel clean and natural, because a shared file
|
||||
is exactly what independent products share. The bridge changes the terms. A native
|
||||
instrument can be **tightly, live-integrated with ReaSampler's project state**: same
|
||||
`"reasampler"` ext-state, same project identity, following the active project — a degree
|
||||
of coupling that is *more than* "two products that happen to read the same files" and
|
||||
starts to look like "two faces of one tool sharing one state model." That is a genuine
|
||||
thumb toward the **one-integrated-product** reading. It's not decisive, and I won't
|
||||
pretend it is:
|
||||
- It only applies if the instrument is **native** (JSFX can't reach the bridge, so a
|
||||
JSFX instrument *is* naturally a file-coupled companion — the format choice and this
|
||||
fork are entangled: pick JSFX and you've half-answered D6 toward "companion").
|
||||
- It's an *available* integration, not a *required* one — a native instrument could
|
||||
still be built to the file seam only, deliberately staying a loose companion for
|
||||
portability. The bridge widens the option, it doesn't force it.
|
||||
- The tight-integration reading buys **REAPER-coupling**; the companion reading buys
|
||||
**portability and independent evolution.** Those are real, opposed goods, and the
|
||||
bridge doesn't resolve which Daniel wants — it just makes the integrated option
|
||||
*materially more capable* than the prior draft credited, so the companion reading no
|
||||
longer wins by default on "well, they can only share a file anyway."
|
||||
|
||||
**Net:** the bridge strengthens "one integrated product" from a coherent-but-equal option
|
||||
into the one with the stronger technical affordance — *if* native-in-REAPER is the path.
|
||||
It does not pre-decide the fork. This is still the highest-order question and it's Daniel's
|
||||
to answer, but he should answer it knowing the integrated option is richer than the
|
||||
file-sharing picture implied — and knowing that choosing JSFX quietly tilts it the other
|
||||
way.
|
||||
|
||||
**D7 — Embedded TCP/MCP UI (new option the bridge surfaces; native/CLAP only).** While
|
||||
verifying the bridge I found a second REAPER-VST-specific affordance the prior draft never
|
||||
mentioned: a hosted plugin can **draw its own embedded UI directly in the track/mixer
|
||||
control panel** (`reaper_plugin_fx_embed.h`: VST2 answers `canDo("hasCockosEmbeddedUI")`
|
||||
with `0xbeef0000` and draws via `effVendorSpecific/effEditDraw`; VST3 implements
|
||||
`IReaperUIEmbedInterface`; CLAP exposes `cockos.reaper_embedui`). Concretely: a native
|
||||
ReaSampler instrument could render a compact keymap/level strip *inline in the TCP/MCP*,
|
||||
not only in its own plugin window — the same surface REAPER's own JS/embedded FX use. This
|
||||
is **native/CLAP-only** (JSFX can't do it) and is pure polish, not a Tier-0 need — but it's
|
||||
a real integration affordance that only exists on the native-in-REAPER path, and it
|
||||
compounds the D1/D6 tilt: if "ReaSampler's playback surface, deeply woven into REAPER's UI"
|
||||
is the vision, this is a lever only native reaches. Flagging it for completeness, not
|
||||
recommending it — it's a Tier-3-ish nicety, deferred by default.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user