ea8b97e47b
Replaces the 1-2 FPS Canvas 2D visualizer with a WebGL2 fragment-shader renderer. Four-control row, morphing navy/moss field, in-shader glass. Full spec in product-notes; PLAN.md Phase 10 points at it.
430 lines
29 KiB
Markdown
430 lines
29 KiB
Markdown
# Mix Visualizer — WebGL2 Fragment-Shader Renderer (Design Spec)
|
||
|
||
Status: **design-complete, implementation-ready.** Author: product-designer. Date: 2026-06-15
|
||
(interview answers captured 2026-06-15). **No code has been written by this doc.**
|
||
|
||
This is the successor to the landed Canvas 2D Mix visualizer (8.K, `COMPLETED.md`). It replaces the
|
||
renderer wholesale; it is not a new feature bolted on. A future implementation wave can be dispatched
|
||
straight from this spec.
|
||
|
||
Cross-references:
|
||
- `product-notes/phase-9-mix-visualizer-redesign.md` — the 8.K spec the Canvas 2D renderer was built
|
||
from. **The motion model (§A), zoom/Guitar-Hero coupling (§B), read-only contract (§D), and the
|
||
datum-resolution analysis (§F, the 333ms-quarter-note-at-180BPM framework) all carry forward
|
||
unchanged.** This spec does not re-derive them — it references them and builds the new renderer on
|
||
top.
|
||
- `DeepDrftPublic/Interop/visualizer/MixVisualizer.ts` — the Canvas 2D module being replaced.
|
||
- `DeepDrftPublic.Client/Controls/MixWaveformVisualizer.razor[.cs/.css]` — the Blazor bridge, **largely
|
||
preserved** (see §2).
|
||
- `DeepDrftPublic.Client/Services/MixVisualizerZoomState.cs` — the session-scoped persistence pattern
|
||
the new controls mirror (see §3).
|
||
- `DeepDrftPublic.Client/Controls/MixZoomMapping.cs` — the log-space slider↔seconds mapping, reused as-is.
|
||
- `DeepDrftShared.Client/Common/DeepDrftPalettes.cs` — the source of truth for the navy/moss theme
|
||
tokens (see §4c).
|
||
|
||
---
|
||
|
||
## Why a renderer swap, not an iteration
|
||
|
||
A staff-engineer analysis concluded the Canvas 2D approach **cannot afford the planned effects**. The
|
||
current renderer already strains at 1–2 FPS; the three per-frame killers are full-viewport `shadowBlur`,
|
||
the CSS `backdrop-filter: blur()` layer, and per-frame `getBoundingClientRect()`. More decisively: the
|
||
effects Daniel wants — bulge, detach, a morphing 2D color field, glass — are **all-dynamic, per-pixel,
|
||
per-frame** effects. That is precisely the workload Canvas 2D is worst at and a fragment shader is best
|
||
at: in a shader the effects become GLSL evaluated across every pixel in parallel on the GPU, and they
|
||
get *cheaper relative to Canvas* as they get fancier.
|
||
|
||
**Decision (Daniel, explicit): rebuild as a WebGL2 fragment-shader renderer. No Canvas 2D stopgap.**
|
||
"WebGL as step 1, no pussyfooting." This supersedes the 8.K spec's §E recommendation (which defaulted to
|
||
Canvas 2D and reserved WebGL only as a fallback) — that recommendation has been overtaken by the effect
|
||
roadmap. The 8.K spec's §E "no tricks, well-commented, industry-standard" *intent* still holds; it now
|
||
means **textbook WebGL2 with a well-commented shader**, not Canvas.
|
||
|
||
---
|
||
|
||
## 1. Goal and scope boundary
|
||
|
||
**Goal.** Replace the Canvas 2D Mix visualizer with a WebGL2 fragment-shader renderer that holds a
|
||
smooth **60 FPS** while carrying four live, per-pixel visual effects — bulge/bubble shaping, lava-lamp
|
||
detachment, a morphing navy↔moss 2D color field, and an in-shader glass treatment — driven by four
|
||
persistent continuous-slider controls. The visualizer remains a **read-only, playback-coupled,
|
||
windowed, bottom-to-top scrolling** background element on the Mix detail page.
|
||
|
||
**In scope.**
|
||
- A new WebGL2 fragment-shader rendering module replacing `MixVisualizer.ts` (the rAF loop, scroll/zoom
|
||
math, and theme handling move into / alongside the GL pipeline).
|
||
- A new **controls row** above the mix details: four continuous sliders (resolution, bubblyness, detach,
|
||
color-shift speed), all persistent across page visits.
|
||
- The four visual effects, in-shader.
|
||
- Preserving the existing Blazor↔JS bridge contract as far as the new inputs allow (§2).
|
||
|
||
**Out of scope / unchanged.**
|
||
- **No playback-control changes.** The visualizer stays strictly read-only (8.K §D). One-way
|
||
`PlaybackPosition` input; no seek, no scrub, no write-back. The dropped `OnSeek` seam stays dropped.
|
||
- **No datum format change.** The duration-derived ~333 samples/sec capture (8.K §F, Wave 1, landed)
|
||
already feeds the renderer correctly and is the right resolution for the max-zoom anchor. The new
|
||
renderer consumes the *same* `WaveformProfileDto` (base64 loudness bytes + duration-from-player). The
|
||
only change is that the datum now lands in the GPU as a **texture** rather than a JS array walked on
|
||
the CPU (§4b).
|
||
- **No change to where the visualizer sits** (full-page background behind `.mix-detail-foreground`),
|
||
except that the new controls row sits *above the mix details, below the back button* (§3).
|
||
- **Control-range guards** (resolution × bubblyness combinations that look unpleasant) — deferred per
|
||
Daniel; he tunes bad ranges by hand once it is on screen.
|
||
- **Motion-speed coupling to bubblyness** — deferred. Bubblyness controls *shape/amount only* for now;
|
||
scroll speed remains a pure function of zoom (8.K §B), independent of the bubble controls.
|
||
|
||
---
|
||
|
||
## 2. Renderer architecture (spec level)
|
||
|
||
The implementation choices below the seam are staff-engineer's. This section fixes the *contract* and
|
||
the *data-flow shape*, not the GLSL.
|
||
|
||
### 2a. Pipeline shape
|
||
|
||
A single full-window WebGL2 quad (two triangles covering the canvas), rasterized once per frame, with
|
||
**all the visual work in the fragment shader**. Each fragment (pixel) computes: which mix-time it maps
|
||
to (from its screen Y, the scroll offset, and the zoom), samples the loudness datum, decides whether it
|
||
is inside the ribbon / a bubble / a detached blob, and shades itself with the morphing gradient + glass.
|
||
The vertex shader is trivial (pass-through quad); the art is entirely fragment-side. This is the
|
||
textbook "shadertoy-style" full-screen-fragment pattern — well-trodden, well-documented, no exotic
|
||
tricks. Comment the shader so Daniel can follow it (8.K §E discipline carries forward).
|
||
|
||
**This is a deliberate inversion of the Canvas model.** Canvas walked screen rows on the CPU and built a
|
||
path; the shader instead asks, per pixel in parallel on the GPU, "am I inside the shape, and what color
|
||
am I?" The scroll/zoom math from `MixVisualizer.ts` (time-at-screen-Y, sample-at-time) moves essentially
|
||
intact into GLSL — the same equations, evaluated per-fragment instead of per-row. Preserve the
|
||
well-commented derivation; it is the part Daniel reads to follow the thing.
|
||
|
||
### 2b. How the waveform datum feeds in
|
||
|
||
**As a 1-D texture (or a 2-D texture one row tall), not a uniform array.** The datum is up to ~1.8 MB
|
||
(~1.8M samples for a 90-min mix at 333/s); uniform arrays are far too small and the wrong tool. A
|
||
texture is the standard vehicle: upload the normalized loudness samples once when the datum arrives (the
|
||
`setDatum` bridge call), and let the fragment shader sample it with the GPU's built-in linear filtering —
|
||
which gives the **smooth, no-stair-stepping interpolation at every zoom** the 8.K spec wanted, for free,
|
||
in hardware. Texture coordinate = `mixTime / durationSeconds`; the GPU interpolates between samples.
|
||
|
||
Upload happens once per datum (off the animation path), exactly mirroring today's once-per-datum decode.
|
||
The per-frame path touches no large buffer — it only updates small uniforms (§2c). This preserves the
|
||
existing optimization where a per-tick playback push never re-decodes the datum.
|
||
|
||
### 2c. Uniforms — what changes per frame vs. per change
|
||
|
||
The fragment shader is parameterized entirely by uniforms. Group them by update cadence:
|
||
|
||
- **Per-frame (updated in the rAF loop):**
|
||
- `playheadSeconds` — current playback position (drives the scroll).
|
||
- `timeSeconds` — a monotonic clock for the always-shifting color field and bubble animation (so the
|
||
field morphs and blobs drift even when scroll is slow). This is the one input that makes the field
|
||
"never static."
|
||
- **Per control change (updated when a slider moves):**
|
||
- `visibleSeconds` — the zoom (8.K §B), drives window time-span and apparent scroll speed.
|
||
- `bubblyness` — bulge amount (§5b).
|
||
- `detach` — lava-lamp detachment amount (§5c).
|
||
- `colorShiftSpeed` — gradient morph rate (§5d). (Multiplies `timeSeconds` inside the shader.)
|
||
- **Per theme change:** `colorNavy`, `colorMoss` (and any derived stops) — the gradient endpoints, read
|
||
from the live palette on a dark-mode toggle (§4c).
|
||
- **Per resize:** canvas resolution / aspect.
|
||
- **Once per datum:** the datum texture + `durationSeconds` + `samplesPerSecond`.
|
||
|
||
### 2d. The Blazor bridge — preserved
|
||
|
||
The existing `MixWaveformVisualizer.razor.cs` bridge is **largely reusable** and should be preserved.
|
||
What stays:
|
||
- Self-fetch of the datum from `ReleaseId` via `IReleaseDataService.GetMixWaveform` (unchanged).
|
||
- The cascaded-player coupling: `IStreamingPlayerService.StateChanged` subscription, `IsActivePlayer`
|
||
gating on `TrackId`, `CurrentPositionSeconds` / `IsPlaying` / `PlayerDurationSeconds` derivation
|
||
(unchanged — this is the playback signal, renderer-agnostic).
|
||
- The idempotent `PushDatumAsync` guard (don't re-upload the datum every tick) — even more important now
|
||
that the datum is a GPU texture upload.
|
||
- The `refreshTheme` push on dark-mode toggle.
|
||
- `IAsyncDisposable` teardown of the module handle.
|
||
- The module-handle shape (`create(canvas)` → handle with `setDatum` / `setPlayback` / `setZoom` /
|
||
`refreshTheme` / `dispose`). **Extend** it with `setBubblyness`, `setDetach`, `setColorShiftSpeed`;
|
||
do not redesign it.
|
||
|
||
What changes: `create` initializes a WebGL2 context (`canvas.getContext('webgl2')`) instead of `'2d'`,
|
||
compiles the shader program, and uploads the datum as a texture. The no-context fallback (return a no-op
|
||
handle so the component still renders a plain backdrop) carries forward — now guarding against *no WebGL2*
|
||
rather than *no 2D context*.
|
||
|
||
### 2e. Animation lives in the rAF/shader loop — NOT in Blazor re-renders
|
||
|
||
**Load-bearing constraint.** Today `OnPlayerStateChanged` calls `StateHasChanged()` on every player tick
|
||
(~10×/sec) to keep the slider/visibility in sync. That re-render cadence must **not** become the
|
||
animation driver — 10 Blazor re-renders/sec is both far below 60 FPS and the wrong layer for animation.
|
||
The animation is entirely the GPU rAF loop's job: the shader's `timeSeconds` uniform advances every frame
|
||
and the field morphs / blobs drift continuously between player ticks. Blazor pushes *playhead position*
|
||
on its tick (cheap uniform write); the *smooth motion between ticks* is the shader interpolating on its
|
||
own clock. The rAF loop stays gated on `isPlaying` (cool when paused — 8.K §E), with the same one-shot
|
||
redraw-while-idle behavior for zoom/theme/datum/resize changes.
|
||
|
||
---
|
||
|
||
## 3. The controls row
|
||
|
||
A new horizontal controls section for the visualizer, placed **above the rest of the mix details and
|
||
below the "back" button** on the Mix detail page. It holds the four controls in a row. All are
|
||
**continuous sliders** (MudSlider, `Step` fine enough to feel continuous — the existing zoom slider uses
|
||
`Step="0.001"`, follow that), and all are **persistent across page visits** within a listening session.
|
||
|
||
### 3a. The four controls, ranges and defaults
|
||
|
||
Daniel asked for sensible default ranges with the explicit note that he will tune them on screen.
|
||
Recommendations:
|
||
|
||
| # | Control | What it does | Slider range (normalized) | Default | Notes |
|
||
|---|---------|--------------|---------------------------|---------|-------|
|
||
| 1 | **Resolution** | Visible time-span / zoom (the existing 8.K control, relocated into this row) | 0 → 1, log-mapped to 30 s → 0.333 s | ~10 s window | Reuse `MixZoomMapping` and `MixVisualizerZoomState` exactly as-is. Just move the slider's home into the new row. |
|
||
| 2 | **Bubblyness** | Bulge amount of each bar (§5b) | 0 → 1 | ~0.35 | 0 = straight rectangular scrolling bars; 1 = fully rounded liquid silhouettes that bulge from the zero-line outward. Bars stay **attached** at max. |
|
||
| 3 | **Detach** ("unleash the lava lamp") | How much bubbles detach from the bar and rise independently (§5c) | 0 → 1 | 0 | 0 = fully attached (whatever bubblyness produced); 1 = blobs separate and float upward freely. Separate axis from bubblyness. |
|
||
| 4 | **Color-shift speed** | How fast the gradient field morph cycles (§5d) | 0 → 1, mapped to a slow→quick cycle-rate range | ~0.3 | Map normalized 0→1 onto a cycle period of roughly **60 s (very slow drift) → 4 s (pretty quick)**. Default ~0.3 lands around a ~20 s cycle — alive but not busy. Never fully zero (the field is "always shifting, never static" — §4b); the slow end is "barely perceptible drift," not "frozen." |
|
||
|
||
These are starting numbers for feel. The max-zoom 0.333 s anchor for control 1 is fixed (8.K §B); the
|
||
other three ranges are Daniel's to tune.
|
||
|
||
### 3b. Continuous-slider UX
|
||
|
||
All four are bare continuous sliders (no tick marks, no snap), `Size.Small`, `Color.Primary`, each with
|
||
an `aria-label`. Lay them in a row with compact labels (icon or short text caption). On mobile the row
|
||
may wrap or scroll horizontally — a layout call for staff-engineer, but the controls must remain
|
||
present (they are the one thing that affects the visualizer, and none of them is a seek surface — 8.K
|
||
§D holds).
|
||
|
||
### 3c. Persistence — mirror MixVisualizerZoomState
|
||
|
||
The existing `MixVisualizerZoomState` is a DI-**scoped** holder: it survives SPA navigation within one
|
||
WASM app instance (open a second mix, the slider keeps its place) and resets to default on a fresh page
|
||
load (F5). That is the "persist within session, reset on fresh load" model from 8.K §B, and it is the
|
||
pattern to mirror for **all four** controls.
|
||
|
||
**Recommendation:** extend the single state object rather than spawning four. Rename/widen
|
||
`MixVisualizerZoomState` into a `MixVisualizerControlState` (scoped) holding `VisibleSeconds`,
|
||
`Bubblyness`, `Detach`, `ColorShiftSpeed`, each with a `const` default mirrored to the TS tuning anchors.
|
||
One injected dependency, one persistence story, four properties. (If Daniel later wants cross-session
|
||
persistence — "this mix should remember my lava-lamp settings" — that is a cookie/localStorage upgrade on
|
||
this one object, deferred and out of scope now, same as the 8.K note.)
|
||
|
||
Keep the C#-side defaults and the TS-side tuning anchors in sync, as the existing
|
||
`MixVisualizerZoomState.DefaultVisibleSeconds` / `DEFAULT_VISIBLE_SECONDS` pair already does (comment the
|
||
contract on both sides).
|
||
|
||
---
|
||
|
||
## 4. The four visual effects
|
||
|
||
Described as **intended look + shader-side approach in conceptual terms**. The exact GLSL is
|
||
staff-engineer's. Each effect is a continuous function of its control value, so the whole range from
|
||
"off" to "maxed" is reachable by dragging one slider.
|
||
|
||
### 4a. Waveform geometry (the substrate all effects sit on)
|
||
|
||
Traditional **−1..+1 amplitude with 0 at the center line**; peaks extend symmetrically above and below
|
||
the zero-line. The waveform **scrolls bottom-to-top** (new audio enters at the bottom, played audio exits
|
||
the top — 8.K §A, unchanged). Per-fragment: map screen Y → mix-time (via playhead + zoom), sample the
|
||
datum texture for loudness at that time, and the loudness defines the half-width of the symmetric ribbon
|
||
about the horizontal center. The "now" line sits at a fixed screen Y (center by default — 8.K §A). This
|
||
is the same geometry the Canvas renderer drew; in the shader it becomes a signed-distance test ("is this
|
||
pixel inside the ribbon silhouette?").
|
||
|
||
### 4b. The 2D morphing navy↔moss gradient field (the headline visual)
|
||
|
||
**Replace the boring grey with a living color field built from the two theme colors — navy blue and moss
|
||
green.** Two coupled dimensions:
|
||
|
||
1. **Per-bar / along-the-bar (the vertical / amplitude axis).** Color varies along each bar's height —
|
||
the zero-line and the peak differ. This gives each bar internal structure: a gradient from center-line
|
||
out to the peak.
|
||
2. **Along the scroll/time axis (the second dimension).** The mix between navy and moss **shifts along
|
||
time** — more green at some moments, more blue at others — and **always shifts, never static**, driven
|
||
by the `timeSeconds` clock × `colorShiftSpeed` (§3a control 4). This is what makes it a *field* and not
|
||
a fixed gradient.
|
||
|
||
**The tension to capture (Daniel's words):** it should read as **a continuous 2D color field** — coherent
|
||
across the whole window, neighbors related — *yet each bar should "feel like its own living thing."**
|
||
Shader approach to hold both at once:
|
||
- A smooth, low-frequency base field (navy↔moss) over (time-axis, amplitude-axis) gives the **field-like
|
||
coherence** — sampled with a flowing noise or layered sinusoids in the time dimension so it morphs
|
||
continuously. This is the "2D color field" layer.
|
||
- A higher-frequency per-bar modulation — keyed off each bar's own index/phase and its instantaneous
|
||
loudness — perturbs the field locally so adjacent bars are *related but not identical*, and a loud bar
|
||
reads brighter/more-saturated than its quiet neighbor. This is the "own living thing" layer.
|
||
- The two compose: coherent field + per-bar liveness = the intended tension.
|
||
|
||
Pull the **exact endpoints from the theme tokens** (§4c) so a dark-mode toggle re-themes the field live.
|
||
|
||
### 4c. Theme tokens — the concrete colors
|
||
|
||
The bespoke palettes live in `DeepDrftShared.Client/Common/DeepDrftPalettes.cs` (the "Charleston in the
|
||
Day" / "Lowcountry Summer Nights" identities map onto these). The navy/moss pair:
|
||
|
||
- **Navy:** `#17283f` (Primary, light) / `#0D1B2A` (the dark ground) / `#162437` (Surface, dark, elevated).
|
||
- **Moss / green:** `#3D7A68` (Secondary light / Primary dark — the interactive green) / `#429d6a`
|
||
(Tertiary light, green-interactive) / `#2A5C4F` (PrimaryDarken dark).
|
||
|
||
As in the Canvas renderer, the shader cannot resolve `var(--mud-palette-*)` directly — read the computed
|
||
CSS custom properties off the canvas element on theme refresh and pass them as `vec3` color uniforms.
|
||
**Recommended endpoints:** navy = `--mud-palette-primary` in light / the dark ground in dark; moss =
|
||
`--mud-palette-secondary` (light) / `--mud-palette-primary` (dark, where green *is* primary). Staff-
|
||
engineer picks the precise var bindings that give the richest navy↔moss spread per palette; the design
|
||
intent is "the two theme signature colors, navy and moss, as the two poles of the field." Re-read on
|
||
`refreshTheme` so the toggle re-themes live (preserve the existing mechanism).
|
||
|
||
### 4d. Bulge / bubble shaping — "bubblyness" (control 2)
|
||
|
||
**Intent.** At **minimum**: straight rectangular bars that only scroll bottom-to-top — a clean,
|
||
hard-edged waveform. As bubblyness rises: bars **gain space and bulge into rounded, liquid silhouettes
|
||
that swell from the center zero-line outward toward the peak** — like a lava-lamp bubble growing. At
|
||
**maximum**: fully rounded liquid blobs, but still **attached** to the bar (detachment is a separate
|
||
axis — §4e). Bubblyness controls **bulge shape/amount only**, not motion speed (deferred).
|
||
|
||
**Shader-side approach.** This is a signed-distance / metaball move. Each bar contributes a field
|
||
contribution; `bubblyness` interpolates the per-bar silhouette from a **box SDF** (sharp rectangle, bulge
|
||
= 0) toward a **rounded capsule / metaball SDF** (bulge = 1) that swells outward from the zero-line. As
|
||
bubblyness rises, increase the rounding radius and let neighboring bars' fields blend (metaball
|
||
union with a smooth-min) so the silhouette reads as continuous liquid rather than discrete bars. The
|
||
"swell from the center outward" is the SDF growing its radius along the amplitude axis with loudness. The
|
||
GPU evaluates the SDF per-pixel — this is exactly the per-pixel work that was unaffordable on Canvas and
|
||
is cheap in a shader.
|
||
|
||
### 4e. Detach — "unleash the lava lamp" (control 3)
|
||
|
||
**Intent.** A **separate** control from bubblyness. At 0, bubbles stay attached (whatever shape
|
||
bubblyness produced). As detach rises, the bulging bubbles **separate from the bar and rise independently**
|
||
— blobs pinch off and float upward on their own, the true lava-lamp experience. At max, freely floating
|
||
detached blobs drifting up through the window.
|
||
|
||
**Shader-side approach.** Building on the metaball field from §4d: as `detach` rises, displace a fraction
|
||
of each bar's bubble mass **upward** by an amount that grows over the bubble's life (the `timeSeconds`
|
||
clock gives the rise animation), and **weaken the smooth-min link** between the detaching blob and its
|
||
parent bar so the metaball connection thins and pinches off (classic metaball separation — two metaballs
|
||
pulling apart show the liquid "neck" thinning then breaking). Detached blobs become independent metaball
|
||
centers with their own upward velocity and slight horizontal drift, fading as they near the top. Keep the
|
||
count and motion bounded so it stays a hypnotic drift, not a particle storm — detach amount scales *how
|
||
much* detaches and *how far it rises*, not an unbounded spawn rate.
|
||
|
||
This effect is the clearest argument for the WebGL move: independent floating metaballs with smooth-min
|
||
merging, evaluated per-pixel every frame, is the canonical fragment-shader lava-lamp and is essentially
|
||
free on the GPU while being impossible at 60 FPS on Canvas 2D.
|
||
|
||
### 4f. Glass treatment (delegated to design judgment, in-shader)
|
||
|
||
Daniel named refraction/distortion of what's behind, specular highlights/sheen on the bars, frosted
|
||
translucency, and a wet glossy surface — and explicitly delegated the aesthetic: "use your terrific
|
||
claude design judgment for maximum style." **Hard constraint: achieve glass entirely in the shader. Do
|
||
NOT reintroduce a per-frame CPU `backdrop-filter: blur()`** — that was a confirmed perf killer in the
|
||
Canvas renderer and is exactly the cost the GPU move exists to eliminate.
|
||
|
||
**Recommended concrete, shippable glass treatment (all fragment-shader, all 60-FPS-affordable):**
|
||
|
||
1. **Refraction of what's behind.** The visualizer is a full-page background behind the mix-detail
|
||
content. True refraction of arbitrary DOM behind a WebGL canvas isn't directly available, so achieve
|
||
the *read* of refraction cheaply: (a) where a blob/ribbon's surface curves (high SDF gradient), warp
|
||
the gradient-field sampling coordinates by the surface normal — so the color field appears bent
|
||
through the glass at the edges, the hallmark of looking through a curved lens; and (b) keep the page
|
||
background showing through via translucency (below). This gives "refraction" as an internal distortion
|
||
of the field, which reads correctly without needing the actual DOM pixels.
|
||
2. **Specular highlights / sheen.** Treat each blob's surface as lit by a fixed virtual light: compute a
|
||
surface normal from the SDF gradient and add a sharp specular hotspot (Blinn-Phong-style highlight)
|
||
plus a soft broad sheen along the upper edge of each bulge. Highlights drift as blobs move — this is
|
||
what sells "wet glossy surface." Cheap: a couple of dot products per fragment.
|
||
3. **Frosted translucency.** The ribbon/blobs render at partial alpha over the page (carry forward the
|
||
RIBBON_OPACITY-style backdrop intent — it stays a backdrop, not a chart) with a subtle internal
|
||
softening (a small in-shader blur of the field, or a frosted noise modulation of alpha) so edges read
|
||
soft and lit rather than hard. This is the "frosted glass" read, done in-shader with no CSS filter.
|
||
4. **Fresnel edge glow.** Add a Fresnel term (brighter at grazing angles / silhouette edges) so the rims
|
||
of blobs catch light — the single most effective "this is glass / lit volume" cue, and one cheap
|
||
`pow(1 - dot(n, viewDir), k)` per fragment.
|
||
|
||
Net look: **lit, wet, frosted glass blobs of navy-and-moss light drifting upward** — hypnotic, ambient,
|
||
unmistakably "lava lamp made of theme-colored glass." The glass is layered *on top of* the color field
|
||
and geometry, so all four effects compose into one coherent surface.
|
||
|
||
---
|
||
|
||
## 5. Acceptance criteria (observable)
|
||
|
||
1. **Frame rate.** The visualizer holds **a smooth 60 FPS** on a mid-range desktop while playing, with
|
||
bubblyness, detach, and color-shift-speed all at non-trivial values simultaneously. (The whole point
|
||
of the swap — measure it, don't assume it.) Graceful degrade on weaker/mobile devices: drop internal
|
||
resolution or effect cost before dropping frames (8.K §E carries forward).
|
||
2. **Renderer.** Rendering is WebGL2 fragment-shader; there is no Canvas 2D path and no CSS
|
||
`backdrop-filter` on the visualizer. (The old per-frame killers — `shadowBlur`, `backdrop-filter`,
|
||
per-frame `getBoundingClientRect`-driven layout — are gone.)
|
||
3. **Motion.** A windowed slice scrolls bottom-to-top, coupled to playback; scroll holds when paused; no
|
||
scroll when nothing is playing; scrolls in-from-empty at the start and out-to-empty at the end (8.K §A
|
||
preserved). The rAF loop is gated on `isPlaying` and burns no frames while paused.
|
||
4. **Animation independence.** The color field visibly morphs and detached blobs visibly drift **between**
|
||
Blazor player ticks — i.e. motion is smooth at 60 FPS, not stepping at the ~10 Hz `StateHasChanged`
|
||
cadence. Animation is driven by the shader clock, not Blazor re-renders.
|
||
5. **Controls.** Four continuous sliders sit in a row above the mix details / below the back button. Each
|
||
visibly and continuously affects its target as it is dragged: resolution changes the visible
|
||
time-span and apparent scroll speed; bubblyness sweeps straight bars → attached liquid bulges; detach
|
||
sweeps attached → free-floating rising blobs; color-shift-speed sweeps barely-drifting → briskly
|
||
morphing field.
|
||
6. **Persistence.** All four control positions survive SPA navigation to another mix within a session and
|
||
reset to defaults on a fresh page load (mirrors `MixVisualizerZoomState`).
|
||
7. **Gradient.** Bars are filled with a navy↔moss gradient that (a) varies along each bar's height and
|
||
(b) shifts along the time axis and never sits fully static; the field reads as coherent across the
|
||
window while individual bars read as distinct/alive. Toggling dark mode re-themes the field live with
|
||
no reload.
|
||
8. **Geometry.** Amplitude is symmetric about a center zero-line (−1..+1), peaks extending both ways.
|
||
9. **Glass.** Blobs/ribbon read as lit, frosted, glossy glass — visible specular highlights, soft frosted
|
||
edges, edge/Fresnel glow, field distortion at curved surfaces — with no CPU backdrop-filter.
|
||
10. **Read-only.** No seek, no scrub, no write-back; the only inputs that affect the visualizer are
|
||
playback position (one-way) and the four sliders. No control is a seek surface.
|
||
11. **Bridge intact.** The component still self-fetches its datum from `ReleaseId`, couples to the
|
||
cascaded player via `TrackId` gating, and tears down its GL resources on dispose without leaking.
|
||
|
||
---
|
||
|
||
## 6. Suggested phasing / waves
|
||
|
||
The work decomposes naturally into a **parity swap first, effects second** sequence. This de-risks the
|
||
hardest architectural step (getting WebGL2 on screen at all, holding 60 FPS, bridge intact) before any
|
||
art is layered on, and gives Daniel a working renderer to tune effects against.
|
||
|
||
### Wave 1 — Renderer swap at parity (the load-bearing step)
|
||
Stand up the WebGL2 pipeline reproducing **today's visual** (the scrolling navy/moss-ish ribbon) but on
|
||
the GPU: full-window quad, datum-as-texture, scroll/zoom math ported into the fragment shader, rAF loop
|
||
gated on `isPlaying`, bridge contract preserved (`setDatum`/`setPlayback`/`setZoom`/`refreshTheme`/
|
||
`dispose`), no-WebGL2 fallback to a plain backdrop. **Acceptance:** §5 criteria 1 (at parity workload),
|
||
2, 3, 4, 10, 11 — a 60-FPS GPU renderer at visual parity with the Canvas version. No new effects yet.
|
||
This is where the architecture is proven; everything after is shading.
|
||
|
||
### Wave 2 — Controls row + state
|
||
Add the four-control row above the mix details, widen `MixVisualizerZoomState` →
|
||
`MixVisualizerControlState` (the three new properties + persistence), wire the three new
|
||
uniforms (`bubblyness`, `detach`, `colorShiftSpeed`) through the bridge as no-op-until-shader-uses-them
|
||
inputs. Resolution relocates here and keeps working. **Acceptance:** §5 criterion 6 and the slider half
|
||
of 5 (sliders present, persistent, wired) — even before the shader consumes them visibly.
|
||
(Wave 2 can begin once Wave 1's bridge is stable; the controls are inert visually until Wave 3 but the
|
||
plumbing is independent.)
|
||
|
||
### Wave 3 — The four effects in the shader
|
||
Layer the effects onto the proven pipeline, in increasing order of risk:
|
||
1. **Gradient field** (§4b) — replace the parity fill with the morphing 2D navy↔moss field. (§5.7)
|
||
2. **Bubblyness** (§4d) — box→metaball SDF interpolation. (§5.5 bubblyness)
|
||
3. **Detach** (§4e) — metaball pinch-off + rising blobs. (§5.5 detach)
|
||
4. **Glass** (§4f) — specular + Fresnel + frosted + field distortion, composed on top. (§5.9)
|
||
**Acceptance:** §5 criteria 5 (visual half), 7, 8, 9 — all four effects live and slider-driven at 60 FPS.
|
||
|
||
**Dependency shape:** `Wave 1 → (Wave 2 ‖ Wave 3 plumbing) → Wave 3 art`. Wave 1 is the prerequisite for
|
||
everything. Wave 2 (controls/state) and Wave 3 (effects) both depend on Wave 1's bridge but the four
|
||
effects within Wave 3 are independently shippable and independently tunable — land and tune one slider's
|
||
effect at a time. Color-range and bubblyness-range guards stay deferred throughout (Daniel tunes by
|
||
hand).
|
||
|
||
---
|
||
|
||
## Open items (tuning knobs, none block starting)
|
||
|
||
All have recommended defaults inline; Daniel tunes on screen:
|
||
- Exact slider ranges for bubblyness / detach / color-shift-speed (§3a).
|
||
- The precise palette-var bindings for the navy/moss field endpoints per light/dark (§4c).
|
||
- Glass intensity / specular sharpness / Fresnel falloff (§4f) — pure aesthetic tuning.
|
||
- Whether the controls row wraps or horizontally scrolls on mobile (§3b) — layout call.
|
||
- Whether control state should later persist cross-session (cookie/localStorage) — deferred upgrade to
|
||
the one state object (§3c).
|