From ea8b97e47bdbaac3b7100d04d45e99e193c985d4 Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Mon, 15 Jun 2026 11:36:46 -0400 Subject: [PATCH] docs: spec WebGL2 Mix visualizer renderer (Phase 10) Replaces the 1-2 FPS Canvas 2D visualizer with a WebGL2 fragment-shader renderer. Four-control row, morphing navy/moss field, in-shader glass. Full spec in product-notes; PLAN.md Phase 10 points at it. --- PLAN.md | 14 + .../mix-visualizer-webgl-renderer.md | 429 ++++++++++++++++++ 2 files changed, 443 insertions(+) create mode 100644 product-notes/mix-visualizer-webgl-renderer.md diff --git a/PLAN.md b/PLAN.md index 3f02d90..cafb492 100644 --- a/PLAN.md +++ b/PLAN.md @@ -178,6 +178,20 @@ Full track decomposition, acceptance criteria, and parallel/dependent analysis: **Dependency shape:** 8.B is the foundation for the CMS tab work (8.A consumes the shared grid; 8.C/8.E layer on once 8.A lands). 8.L follows 8.G and coordinates with 8.E/8.F (same forms). 8.M (legacy-form retirement) follows 8.L and is architectural (route map + addressing decision). On the public side, 8.H (decided H2 — the new release-cardinal archive) gates 8.I. **All Wave 8 tracks are landed** — Phase-9-completion gate (8.A–8.J + 8.L), 8.M, and the post-Phase-9 8.K Mix Visualizer redesign. **Landed tracks:** 8.A, 8.B, 8.C, 8.D, 8.E, 8.F, 8.G, 8.H, 8.I, 8.J, 8.L (2026-06-13); 8.M (2026-06-14); 8.K (2026-06-14). +## Phase 10 — Mix Visualizer WebGL2 Renderer + +The landed Canvas 2D Mix visualizer (8.K) renders at 1–2 FPS and **cannot afford the planned effects** — a staff-engineer analysis found the per-frame killers (full-viewport `shadowBlur`, CSS `backdrop-filter`, per-frame `getBoundingClientRect`) structural to the approach, and the planned effects (bulge, lava-lamp detach, a morphing 2D color field, glass) are all per-pixel/per-frame work — exactly what Canvas 2D is worst at and a fragment shader is best at. + +**Decision (Daniel, explicit): rebuild as a WebGL2 fragment-shader renderer. No Canvas 2D stopgap** — "WebGL as step 1, no pussyfooting." This supersedes 8.K §E's Canvas-2D-default recommendation; the "industry-standard, well-commented, no tricks" discipline carries forward as *textbook WebGL2 with a commented shader*. Target a smooth **60 FPS**. Strictly read-only (no playback-control changes); the duration-derived ~333 samples/sec datum (8.K §F) and the existing Blazor↔JS bridge are both preserved — the datum now lands as a GPU texture rather than a CPU-walked array. + +Adds a **controls row** above the mix details / below the back button: four continuous, session-persistent sliders — **resolution** (relocated 8.K zoom), **bubblyness** (box→liquid bulge), **detach** ("unleash the lava lamp" — blobs pinch off and rise), **color-shift speed** (gradient morph rate). The headline visual is a living 2D **navy↔moss** gradient field (theme tokens from `DeepDrftPalettes`) that varies per-bar *and* shifts along time, never static; plus an in-shader **glass** treatment (specular/Fresnel/frosted/refraction — no CPU backdrop-filter). Persistence mirrors `MixVisualizerZoomState` (widen to a `MixVisualizerControlState` holding all four). + +Full design, renderer architecture, the four effects, acceptance criteria, and phasing: `product-notes/mix-visualizer-webgl-renderer.md`. + +**Sequenced as three waves.** Wave 1 (renderer swap at parity — prove WebGL2 on screen at 60 FPS, bridge intact, no new effects) is the load-bearing prerequisite. Wave 2 (controls row + widened state) and Wave 3 (the four effects in the shader) both follow Wave 1; the four effects within Wave 3 are independently shippable and tunable. **Deferred (Daniel):** control-range guards and motion-speed coupling to bubblyness — he tunes bad ranges by hand once on screen. + +--- + ## Working with this file - **Add items by extending an existing phase first**; only create a new phase when the addition genuinely doesn't fit any of 1–5. Phase numbers are organisational, not sequencing. diff --git a/product-notes/mix-visualizer-webgl-renderer.md b/product-notes/mix-visualizer-webgl-renderer.md new file mode 100644 index 0000000..c558762 --- /dev/null +++ b/product-notes/mix-visualizer-webgl-renderer.md @@ -0,0 +1,429 @@ +# Mix Visualizer — WebGL2 Fragment-Shader Renderer (Design Spec) + +Status: **design-complete, implementation-ready.** Author: product-designer. Date: 2026-06-15 +(interview answers captured 2026-06-15). **No code has been written by this doc.** + +This is the successor to the landed Canvas 2D Mix visualizer (8.K, `COMPLETED.md`). It replaces the +renderer wholesale; it is not a new feature bolted on. A future implementation wave can be dispatched +straight from this spec. + +Cross-references: +- `product-notes/phase-9-mix-visualizer-redesign.md` — the 8.K spec the Canvas 2D renderer was built + from. **The motion model (§A), zoom/Guitar-Hero coupling (§B), read-only contract (§D), and the + datum-resolution analysis (§F, the 333ms-quarter-note-at-180BPM framework) all carry forward + unchanged.** This spec does not re-derive them — it references them and builds the new renderer on + top. +- `DeepDrftPublic/Interop/visualizer/MixVisualizer.ts` — the Canvas 2D module being replaced. +- `DeepDrftPublic.Client/Controls/MixWaveformVisualizer.razor[.cs/.css]` — the Blazor bridge, **largely + preserved** (see §2). +- `DeepDrftPublic.Client/Services/MixVisualizerZoomState.cs` — the session-scoped persistence pattern + the new controls mirror (see §3). +- `DeepDrftPublic.Client/Controls/MixZoomMapping.cs` — the log-space slider↔seconds mapping, reused as-is. +- `DeepDrftShared.Client/Common/DeepDrftPalettes.cs` — the source of truth for the navy/moss theme + tokens (see §4c). + +--- + +## Why a renderer swap, not an iteration + +A staff-engineer analysis concluded the Canvas 2D approach **cannot afford the planned effects**. The +current renderer already strains at 1–2 FPS; the three per-frame killers are full-viewport `shadowBlur`, +the CSS `backdrop-filter: blur()` layer, and per-frame `getBoundingClientRect()`. More decisively: the +effects Daniel wants — bulge, detach, a morphing 2D color field, glass — are **all-dynamic, per-pixel, +per-frame** effects. That is precisely the workload Canvas 2D is worst at and a fragment shader is best +at: in a shader the effects become GLSL evaluated across every pixel in parallel on the GPU, and they +get *cheaper relative to Canvas* as they get fancier. + +**Decision (Daniel, explicit): rebuild as a WebGL2 fragment-shader renderer. No Canvas 2D stopgap.** +"WebGL as step 1, no pussyfooting." This supersedes the 8.K spec's §E recommendation (which defaulted to +Canvas 2D and reserved WebGL only as a fallback) — that recommendation has been overtaken by the effect +roadmap. The 8.K spec's §E "no tricks, well-commented, industry-standard" *intent* still holds; it now +means **textbook WebGL2 with a well-commented shader**, not Canvas. + +--- + +## 1. Goal and scope boundary + +**Goal.** Replace the Canvas 2D Mix visualizer with a WebGL2 fragment-shader renderer that holds a +smooth **60 FPS** while carrying four live, per-pixel visual effects — bulge/bubble shaping, lava-lamp +detachment, a morphing navy↔moss 2D color field, and an in-shader glass treatment — driven by four +persistent continuous-slider controls. The visualizer remains a **read-only, playback-coupled, +windowed, bottom-to-top scrolling** background element on the Mix detail page. + +**In scope.** +- A new WebGL2 fragment-shader rendering module replacing `MixVisualizer.ts` (the rAF loop, scroll/zoom + math, and theme handling move into / alongside the GL pipeline). +- A new **controls row** above the mix details: four continuous sliders (resolution, bubblyness, detach, + color-shift speed), all persistent across page visits. +- The four visual effects, in-shader. +- Preserving the existing Blazor↔JS bridge contract as far as the new inputs allow (§2). + +**Out of scope / unchanged.** +- **No playback-control changes.** The visualizer stays strictly read-only (8.K §D). One-way + `PlaybackPosition` input; no seek, no scrub, no write-back. The dropped `OnSeek` seam stays dropped. +- **No datum format change.** The duration-derived ~333 samples/sec capture (8.K §F, Wave 1, landed) + already feeds the renderer correctly and is the right resolution for the max-zoom anchor. The new + renderer consumes the *same* `WaveformProfileDto` (base64 loudness bytes + duration-from-player). The + only change is that the datum now lands in the GPU as a **texture** rather than a JS array walked on + the CPU (§4b). +- **No change to where the visualizer sits** (full-page background behind `.mix-detail-foreground`), + except that the new controls row sits *above the mix details, below the back button* (§3). +- **Control-range guards** (resolution × bubblyness combinations that look unpleasant) — deferred per + Daniel; he tunes bad ranges by hand once it is on screen. +- **Motion-speed coupling to bubblyness** — deferred. Bubblyness controls *shape/amount only* for now; + scroll speed remains a pure function of zoom (8.K §B), independent of the bubble controls. + +--- + +## 2. Renderer architecture (spec level) + +The implementation choices below the seam are staff-engineer's. This section fixes the *contract* and +the *data-flow shape*, not the GLSL. + +### 2a. Pipeline shape + +A single full-window WebGL2 quad (two triangles covering the canvas), rasterized once per frame, with +**all the visual work in the fragment shader**. Each fragment (pixel) computes: which mix-time it maps +to (from its screen Y, the scroll offset, and the zoom), samples the loudness datum, decides whether it +is inside the ribbon / a bubble / a detached blob, and shades itself with the morphing gradient + glass. +The vertex shader is trivial (pass-through quad); the art is entirely fragment-side. This is the +textbook "shadertoy-style" full-screen-fragment pattern — well-trodden, well-documented, no exotic +tricks. Comment the shader so Daniel can follow it (8.K §E discipline carries forward). + +**This is a deliberate inversion of the Canvas model.** Canvas walked screen rows on the CPU and built a +path; the shader instead asks, per pixel in parallel on the GPU, "am I inside the shape, and what color +am I?" The scroll/zoom math from `MixVisualizer.ts` (time-at-screen-Y, sample-at-time) moves essentially +intact into GLSL — the same equations, evaluated per-fragment instead of per-row. Preserve the +well-commented derivation; it is the part Daniel reads to follow the thing. + +### 2b. How the waveform datum feeds in + +**As a 1-D texture (or a 2-D texture one row tall), not a uniform array.** The datum is up to ~1.8 MB +(~1.8M samples for a 90-min mix at 333/s); uniform arrays are far too small and the wrong tool. A +texture is the standard vehicle: upload the normalized loudness samples once when the datum arrives (the +`setDatum` bridge call), and let the fragment shader sample it with the GPU's built-in linear filtering — +which gives the **smooth, no-stair-stepping interpolation at every zoom** the 8.K spec wanted, for free, +in hardware. Texture coordinate = `mixTime / durationSeconds`; the GPU interpolates between samples. + +Upload happens once per datum (off the animation path), exactly mirroring today's once-per-datum decode. +The per-frame path touches no large buffer — it only updates small uniforms (§2c). This preserves the +existing optimization where a per-tick playback push never re-decodes the datum. + +### 2c. Uniforms — what changes per frame vs. per change + +The fragment shader is parameterized entirely by uniforms. Group them by update cadence: + +- **Per-frame (updated in the rAF loop):** + - `playheadSeconds` — current playback position (drives the scroll). + - `timeSeconds` — a monotonic clock for the always-shifting color field and bubble animation (so the + field morphs and blobs drift even when scroll is slow). This is the one input that makes the field + "never static." +- **Per control change (updated when a slider moves):** + - `visibleSeconds` — the zoom (8.K §B), drives window time-span and apparent scroll speed. + - `bubblyness` — bulge amount (§5b). + - `detach` — lava-lamp detachment amount (§5c). + - `colorShiftSpeed` — gradient morph rate (§5d). (Multiplies `timeSeconds` inside the shader.) +- **Per theme change:** `colorNavy`, `colorMoss` (and any derived stops) — the gradient endpoints, read + from the live palette on a dark-mode toggle (§4c). +- **Per resize:** canvas resolution / aspect. +- **Once per datum:** the datum texture + `durationSeconds` + `samplesPerSecond`. + +### 2d. The Blazor bridge — preserved + +The existing `MixWaveformVisualizer.razor.cs` bridge is **largely reusable** and should be preserved. +What stays: +- Self-fetch of the datum from `ReleaseId` via `IReleaseDataService.GetMixWaveform` (unchanged). +- The cascaded-player coupling: `IStreamingPlayerService.StateChanged` subscription, `IsActivePlayer` + gating on `TrackId`, `CurrentPositionSeconds` / `IsPlaying` / `PlayerDurationSeconds` derivation + (unchanged — this is the playback signal, renderer-agnostic). +- The idempotent `PushDatumAsync` guard (don't re-upload the datum every tick) — even more important now + that the datum is a GPU texture upload. +- The `refreshTheme` push on dark-mode toggle. +- `IAsyncDisposable` teardown of the module handle. +- The module-handle shape (`create(canvas)` → handle with `setDatum` / `setPlayback` / `setZoom` / + `refreshTheme` / `dispose`). **Extend** it with `setBubblyness`, `setDetach`, `setColorShiftSpeed`; + do not redesign it. + +What changes: `create` initializes a WebGL2 context (`canvas.getContext('webgl2')`) instead of `'2d'`, +compiles the shader program, and uploads the datum as a texture. The no-context fallback (return a no-op +handle so the component still renders a plain backdrop) carries forward — now guarding against *no WebGL2* +rather than *no 2D context*. + +### 2e. Animation lives in the rAF/shader loop — NOT in Blazor re-renders + +**Load-bearing constraint.** Today `OnPlayerStateChanged` calls `StateHasChanged()` on every player tick +(~10×/sec) to keep the slider/visibility in sync. That re-render cadence must **not** become the +animation driver — 10 Blazor re-renders/sec is both far below 60 FPS and the wrong layer for animation. +The animation is entirely the GPU rAF loop's job: the shader's `timeSeconds` uniform advances every frame +and the field morphs / blobs drift continuously between player ticks. Blazor pushes *playhead position* +on its tick (cheap uniform write); the *smooth motion between ticks* is the shader interpolating on its +own clock. The rAF loop stays gated on `isPlaying` (cool when paused — 8.K §E), with the same one-shot +redraw-while-idle behavior for zoom/theme/datum/resize changes. + +--- + +## 3. The controls row + +A new horizontal controls section for the visualizer, placed **above the rest of the mix details and +below the "back" button** on the Mix detail page. It holds the four controls in a row. All are +**continuous sliders** (MudSlider, `Step` fine enough to feel continuous — the existing zoom slider uses +`Step="0.001"`, follow that), and all are **persistent across page visits** within a listening session. + +### 3a. The four controls, ranges and defaults + +Daniel asked for sensible default ranges with the explicit note that he will tune them on screen. +Recommendations: + +| # | Control | What it does | Slider range (normalized) | Default | Notes | +|---|---------|--------------|---------------------------|---------|-------| +| 1 | **Resolution** | Visible time-span / zoom (the existing 8.K control, relocated into this row) | 0 → 1, log-mapped to 30 s → 0.333 s | ~10 s window | Reuse `MixZoomMapping` and `MixVisualizerZoomState` exactly as-is. Just move the slider's home into the new row. | +| 2 | **Bubblyness** | Bulge amount of each bar (§5b) | 0 → 1 | ~0.35 | 0 = straight rectangular scrolling bars; 1 = fully rounded liquid silhouettes that bulge from the zero-line outward. Bars stay **attached** at max. | +| 3 | **Detach** ("unleash the lava lamp") | How much bubbles detach from the bar and rise independently (§5c) | 0 → 1 | 0 | 0 = fully attached (whatever bubblyness produced); 1 = blobs separate and float upward freely. Separate axis from bubblyness. | +| 4 | **Color-shift speed** | How fast the gradient field morph cycles (§5d) | 0 → 1, mapped to a slow→quick cycle-rate range | ~0.3 | Map normalized 0→1 onto a cycle period of roughly **60 s (very slow drift) → 4 s (pretty quick)**. Default ~0.3 lands around a ~20 s cycle — alive but not busy. Never fully zero (the field is "always shifting, never static" — §4b); the slow end is "barely perceptible drift," not "frozen." | + +These are starting numbers for feel. The max-zoom 0.333 s anchor for control 1 is fixed (8.K §B); the +other three ranges are Daniel's to tune. + +### 3b. Continuous-slider UX + +All four are bare continuous sliders (no tick marks, no snap), `Size.Small`, `Color.Primary`, each with +an `aria-label`. Lay them in a row with compact labels (icon or short text caption). On mobile the row +may wrap or scroll horizontally — a layout call for staff-engineer, but the controls must remain +present (they are the one thing that affects the visualizer, and none of them is a seek surface — 8.K +§D holds). + +### 3c. Persistence — mirror MixVisualizerZoomState + +The existing `MixVisualizerZoomState` is a DI-**scoped** holder: it survives SPA navigation within one +WASM app instance (open a second mix, the slider keeps its place) and resets to default on a fresh page +load (F5). That is the "persist within session, reset on fresh load" model from 8.K §B, and it is the +pattern to mirror for **all four** controls. + +**Recommendation:** extend the single state object rather than spawning four. Rename/widen +`MixVisualizerZoomState` into a `MixVisualizerControlState` (scoped) holding `VisibleSeconds`, +`Bubblyness`, `Detach`, `ColorShiftSpeed`, each with a `const` default mirrored to the TS tuning anchors. +One injected dependency, one persistence story, four properties. (If Daniel later wants cross-session +persistence — "this mix should remember my lava-lamp settings" — that is a cookie/localStorage upgrade on +this one object, deferred and out of scope now, same as the 8.K note.) + +Keep the C#-side defaults and the TS-side tuning anchors in sync, as the existing +`MixVisualizerZoomState.DefaultVisibleSeconds` / `DEFAULT_VISIBLE_SECONDS` pair already does (comment the +contract on both sides). + +--- + +## 4. The four visual effects + +Described as **intended look + shader-side approach in conceptual terms**. The exact GLSL is +staff-engineer's. Each effect is a continuous function of its control value, so the whole range from +"off" to "maxed" is reachable by dragging one slider. + +### 4a. Waveform geometry (the substrate all effects sit on) + +Traditional **−1..+1 amplitude with 0 at the center line**; peaks extend symmetrically above and below +the zero-line. The waveform **scrolls bottom-to-top** (new audio enters at the bottom, played audio exits +the top — 8.K §A, unchanged). Per-fragment: map screen Y → mix-time (via playhead + zoom), sample the +datum texture for loudness at that time, and the loudness defines the half-width of the symmetric ribbon +about the horizontal center. The "now" line sits at a fixed screen Y (center by default — 8.K §A). This +is the same geometry the Canvas renderer drew; in the shader it becomes a signed-distance test ("is this +pixel inside the ribbon silhouette?"). + +### 4b. The 2D morphing navy↔moss gradient field (the headline visual) + +**Replace the boring grey with a living color field built from the two theme colors — navy blue and moss +green.** Two coupled dimensions: + +1. **Per-bar / along-the-bar (the vertical / amplitude axis).** Color varies along each bar's height — + the zero-line and the peak differ. This gives each bar internal structure: a gradient from center-line + out to the peak. +2. **Along the scroll/time axis (the second dimension).** The mix between navy and moss **shifts along + time** — more green at some moments, more blue at others — and **always shifts, never static**, driven + by the `timeSeconds` clock × `colorShiftSpeed` (§3a control 4). This is what makes it a *field* and not + a fixed gradient. + +**The tension to capture (Daniel's words):** it should read as **a continuous 2D color field** — coherent +across the whole window, neighbors related — *yet each bar should "feel like its own living thing."** +Shader approach to hold both at once: +- A smooth, low-frequency base field (navy↔moss) over (time-axis, amplitude-axis) gives the **field-like + coherence** — sampled with a flowing noise or layered sinusoids in the time dimension so it morphs + continuously. This is the "2D color field" layer. +- A higher-frequency per-bar modulation — keyed off each bar's own index/phase and its instantaneous + loudness — perturbs the field locally so adjacent bars are *related but not identical*, and a loud bar + reads brighter/more-saturated than its quiet neighbor. This is the "own living thing" layer. +- The two compose: coherent field + per-bar liveness = the intended tension. + +Pull the **exact endpoints from the theme tokens** (§4c) so a dark-mode toggle re-themes the field live. + +### 4c. Theme tokens — the concrete colors + +The bespoke palettes live in `DeepDrftShared.Client/Common/DeepDrftPalettes.cs` (the "Charleston in the +Day" / "Lowcountry Summer Nights" identities map onto these). The navy/moss pair: + +- **Navy:** `#17283f` (Primary, light) / `#0D1B2A` (the dark ground) / `#162437` (Surface, dark, elevated). +- **Moss / green:** `#3D7A68` (Secondary light / Primary dark — the interactive green) / `#429d6a` + (Tertiary light, green-interactive) / `#2A5C4F` (PrimaryDarken dark). + +As in the Canvas renderer, the shader cannot resolve `var(--mud-palette-*)` directly — read the computed +CSS custom properties off the canvas element on theme refresh and pass them as `vec3` color uniforms. +**Recommended endpoints:** navy = `--mud-palette-primary` in light / the dark ground in dark; moss = +`--mud-palette-secondary` (light) / `--mud-palette-primary` (dark, where green *is* primary). Staff- +engineer picks the precise var bindings that give the richest navy↔moss spread per palette; the design +intent is "the two theme signature colors, navy and moss, as the two poles of the field." Re-read on +`refreshTheme` so the toggle re-themes live (preserve the existing mechanism). + +### 4d. Bulge / bubble shaping — "bubblyness" (control 2) + +**Intent.** At **minimum**: straight rectangular bars that only scroll bottom-to-top — a clean, +hard-edged waveform. As bubblyness rises: bars **gain space and bulge into rounded, liquid silhouettes +that swell from the center zero-line outward toward the peak** — like a lava-lamp bubble growing. At +**maximum**: fully rounded liquid blobs, but still **attached** to the bar (detachment is a separate +axis — §4e). Bubblyness controls **bulge shape/amount only**, not motion speed (deferred). + +**Shader-side approach.** This is a signed-distance / metaball move. Each bar contributes a field +contribution; `bubblyness` interpolates the per-bar silhouette from a **box SDF** (sharp rectangle, bulge += 0) toward a **rounded capsule / metaball SDF** (bulge = 1) that swells outward from the zero-line. As +bubblyness rises, increase the rounding radius and let neighboring bars' fields blend (metaball +union with a smooth-min) so the silhouette reads as continuous liquid rather than discrete bars. The +"swell from the center outward" is the SDF growing its radius along the amplitude axis with loudness. The +GPU evaluates the SDF per-pixel — this is exactly the per-pixel work that was unaffordable on Canvas and +is cheap in a shader. + +### 4e. Detach — "unleash the lava lamp" (control 3) + +**Intent.** A **separate** control from bubblyness. At 0, bubbles stay attached (whatever shape +bubblyness produced). As detach rises, the bulging bubbles **separate from the bar and rise independently** +— blobs pinch off and float upward on their own, the true lava-lamp experience. At max, freely floating +detached blobs drifting up through the window. + +**Shader-side approach.** Building on the metaball field from §4d: as `detach` rises, displace a fraction +of each bar's bubble mass **upward** by an amount that grows over the bubble's life (the `timeSeconds` +clock gives the rise animation), and **weaken the smooth-min link** between the detaching blob and its +parent bar so the metaball connection thins and pinches off (classic metaball separation — two metaballs +pulling apart show the liquid "neck" thinning then breaking). Detached blobs become independent metaball +centers with their own upward velocity and slight horizontal drift, fading as they near the top. Keep the +count and motion bounded so it stays a hypnotic drift, not a particle storm — detach amount scales *how +much* detaches and *how far it rises*, not an unbounded spawn rate. + +This effect is the clearest argument for the WebGL move: independent floating metaballs with smooth-min +merging, evaluated per-pixel every frame, is the canonical fragment-shader lava-lamp and is essentially +free on the GPU while being impossible at 60 FPS on Canvas 2D. + +### 4f. Glass treatment (delegated to design judgment, in-shader) + +Daniel named refraction/distortion of what's behind, specular highlights/sheen on the bars, frosted +translucency, and a wet glossy surface — and explicitly delegated the aesthetic: "use your terrific +claude design judgment for maximum style." **Hard constraint: achieve glass entirely in the shader. Do +NOT reintroduce a per-frame CPU `backdrop-filter: blur()`** — that was a confirmed perf killer in the +Canvas renderer and is exactly the cost the GPU move exists to eliminate. + +**Recommended concrete, shippable glass treatment (all fragment-shader, all 60-FPS-affordable):** + +1. **Refraction of what's behind.** The visualizer is a full-page background behind the mix-detail + content. True refraction of arbitrary DOM behind a WebGL canvas isn't directly available, so achieve + the *read* of refraction cheaply: (a) where a blob/ribbon's surface curves (high SDF gradient), warp + the gradient-field sampling coordinates by the surface normal — so the color field appears bent + through the glass at the edges, the hallmark of looking through a curved lens; and (b) keep the page + background showing through via translucency (below). This gives "refraction" as an internal distortion + of the field, which reads correctly without needing the actual DOM pixels. +2. **Specular highlights / sheen.** Treat each blob's surface as lit by a fixed virtual light: compute a + surface normal from the SDF gradient and add a sharp specular hotspot (Blinn-Phong-style highlight) + plus a soft broad sheen along the upper edge of each bulge. Highlights drift as blobs move — this is + what sells "wet glossy surface." Cheap: a couple of dot products per fragment. +3. **Frosted translucency.** The ribbon/blobs render at partial alpha over the page (carry forward the + RIBBON_OPACITY-style backdrop intent — it stays a backdrop, not a chart) with a subtle internal + softening (a small in-shader blur of the field, or a frosted noise modulation of alpha) so edges read + soft and lit rather than hard. This is the "frosted glass" read, done in-shader with no CSS filter. +4. **Fresnel edge glow.** Add a Fresnel term (brighter at grazing angles / silhouette edges) so the rims + of blobs catch light — the single most effective "this is glass / lit volume" cue, and one cheap + `pow(1 - dot(n, viewDir), k)` per fragment. + +Net look: **lit, wet, frosted glass blobs of navy-and-moss light drifting upward** — hypnotic, ambient, +unmistakably "lava lamp made of theme-colored glass." The glass is layered *on top of* the color field +and geometry, so all four effects compose into one coherent surface. + +--- + +## 5. Acceptance criteria (observable) + +1. **Frame rate.** The visualizer holds **a smooth 60 FPS** on a mid-range desktop while playing, with + bubblyness, detach, and color-shift-speed all at non-trivial values simultaneously. (The whole point + of the swap — measure it, don't assume it.) Graceful degrade on weaker/mobile devices: drop internal + resolution or effect cost before dropping frames (8.K §E carries forward). +2. **Renderer.** Rendering is WebGL2 fragment-shader; there is no Canvas 2D path and no CSS + `backdrop-filter` on the visualizer. (The old per-frame killers — `shadowBlur`, `backdrop-filter`, + per-frame `getBoundingClientRect`-driven layout — are gone.) +3. **Motion.** A windowed slice scrolls bottom-to-top, coupled to playback; scroll holds when paused; no + scroll when nothing is playing; scrolls in-from-empty at the start and out-to-empty at the end (8.K §A + preserved). The rAF loop is gated on `isPlaying` and burns no frames while paused. +4. **Animation independence.** The color field visibly morphs and detached blobs visibly drift **between** + Blazor player ticks — i.e. motion is smooth at 60 FPS, not stepping at the ~10 Hz `StateHasChanged` + cadence. Animation is driven by the shader clock, not Blazor re-renders. +5. **Controls.** Four continuous sliders sit in a row above the mix details / below the back button. Each + visibly and continuously affects its target as it is dragged: resolution changes the visible + time-span and apparent scroll speed; bubblyness sweeps straight bars → attached liquid bulges; detach + sweeps attached → free-floating rising blobs; color-shift-speed sweeps barely-drifting → briskly + morphing field. +6. **Persistence.** All four control positions survive SPA navigation to another mix within a session and + reset to defaults on a fresh page load (mirrors `MixVisualizerZoomState`). +7. **Gradient.** Bars are filled with a navy↔moss gradient that (a) varies along each bar's height and + (b) shifts along the time axis and never sits fully static; the field reads as coherent across the + window while individual bars read as distinct/alive. Toggling dark mode re-themes the field live with + no reload. +8. **Geometry.** Amplitude is symmetric about a center zero-line (−1..+1), peaks extending both ways. +9. **Glass.** Blobs/ribbon read as lit, frosted, glossy glass — visible specular highlights, soft frosted + edges, edge/Fresnel glow, field distortion at curved surfaces — with no CPU backdrop-filter. +10. **Read-only.** No seek, no scrub, no write-back; the only inputs that affect the visualizer are + playback position (one-way) and the four sliders. No control is a seek surface. +11. **Bridge intact.** The component still self-fetches its datum from `ReleaseId`, couples to the + cascaded player via `TrackId` gating, and tears down its GL resources on dispose without leaking. + +--- + +## 6. Suggested phasing / waves + +The work decomposes naturally into a **parity swap first, effects second** sequence. This de-risks the +hardest architectural step (getting WebGL2 on screen at all, holding 60 FPS, bridge intact) before any +art is layered on, and gives Daniel a working renderer to tune effects against. + +### Wave 1 — Renderer swap at parity (the load-bearing step) +Stand up the WebGL2 pipeline reproducing **today's visual** (the scrolling navy/moss-ish ribbon) but on +the GPU: full-window quad, datum-as-texture, scroll/zoom math ported into the fragment shader, rAF loop +gated on `isPlaying`, bridge contract preserved (`setDatum`/`setPlayback`/`setZoom`/`refreshTheme`/ +`dispose`), no-WebGL2 fallback to a plain backdrop. **Acceptance:** §5 criteria 1 (at parity workload), +2, 3, 4, 10, 11 — a 60-FPS GPU renderer at visual parity with the Canvas version. No new effects yet. +This is where the architecture is proven; everything after is shading. + +### Wave 2 — Controls row + state +Add the four-control row above the mix details, widen `MixVisualizerZoomState` → +`MixVisualizerControlState` (the three new properties + persistence), wire the three new +uniforms (`bubblyness`, `detach`, `colorShiftSpeed`) through the bridge as no-op-until-shader-uses-them +inputs. Resolution relocates here and keeps working. **Acceptance:** §5 criterion 6 and the slider half +of 5 (sliders present, persistent, wired) — even before the shader consumes them visibly. +(Wave 2 can begin once Wave 1's bridge is stable; the controls are inert visually until Wave 3 but the +plumbing is independent.) + +### Wave 3 — The four effects in the shader +Layer the effects onto the proven pipeline, in increasing order of risk: +1. **Gradient field** (§4b) — replace the parity fill with the morphing 2D navy↔moss field. (§5.7) +2. **Bubblyness** (§4d) — box→metaball SDF interpolation. (§5.5 bubblyness) +3. **Detach** (§4e) — metaball pinch-off + rising blobs. (§5.5 detach) +4. **Glass** (§4f) — specular + Fresnel + frosted + field distortion, composed on top. (§5.9) +**Acceptance:** §5 criteria 5 (visual half), 7, 8, 9 — all four effects live and slider-driven at 60 FPS. + +**Dependency shape:** `Wave 1 → (Wave 2 ‖ Wave 3 plumbing) → Wave 3 art`. Wave 1 is the prerequisite for +everything. Wave 2 (controls/state) and Wave 3 (effects) both depend on Wave 1's bridge but the four +effects within Wave 3 are independently shippable and independently tunable — land and tune one slider's +effect at a time. Color-range and bubblyness-range guards stay deferred throughout (Daniel tunes by +hand). + +--- + +## Open items (tuning knobs, none block starting) + +All have recommended defaults inline; Daniel tunes on screen: +- Exact slider ranges for bubblyness / detach / color-shift-speed (§3a). +- The precise palette-var bindings for the navy/moss field endpoints per light/dark (§4c). +- Glass intensity / specular sharpness / Fresnel falloff (§4f) — pure aesthetic tuning. +- Whether the controls row wraps or horizontally scrolls on mobile (§3b) — layout call. +- Whether control state should later persist cross-session (cookie/localStorage) — deferred upgrade to + the one state object (§3c).