docs: spec WebGL2 Mix visualizer renderer (Phase 10)
Replaces the 1-2 FPS Canvas 2D visualizer with a WebGL2 fragment-shader renderer. Four-control row, morphing navy/moss field, in-shader glass. Full spec in product-notes; PLAN.md Phase 10 points at it.
This commit is contained in:
daniel-c-harvey
@@ -0,0 +1,429 @@
|
||||
# Mix Visualizer — WebGL2 Fragment-Shader Renderer (Design Spec)
|
||||
|
||||
Status: **design-complete, implementation-ready.** Author: product-designer. Date: 2026-06-15
|
||||
(interview answers captured 2026-06-15). **No code has been written by this doc.**
|
||||
|
||||
This is the successor to the landed Canvas 2D Mix visualizer (8.K, `COMPLETED.md`). It replaces the
|
||||
renderer wholesale; it is not a new feature bolted on. A future implementation wave can be dispatched
|
||||
straight from this spec.
|
||||
|
||||
Cross-references:
|
||||
- `product-notes/phase-9-mix-visualizer-redesign.md` — the 8.K spec the Canvas 2D renderer was built
|
||||
from. **The motion model (§A), zoom/Guitar-Hero coupling (§B), read-only contract (§D), and the
|
||||
datum-resolution analysis (§F, the 333ms-quarter-note-at-180BPM framework) all carry forward
|
||||
unchanged.** This spec does not re-derive them — it references them and builds the new renderer on
|
||||
top.
|
||||
- `DeepDrftPublic/Interop/visualizer/MixVisualizer.ts` — the Canvas 2D module being replaced.
|
||||
- `DeepDrftPublic.Client/Controls/MixWaveformVisualizer.razor[.cs/.css]` — the Blazor bridge, **largely
|
||||
preserved** (see §2).
|
||||
- `DeepDrftPublic.Client/Services/MixVisualizerZoomState.cs` — the session-scoped persistence pattern
|
||||
the new controls mirror (see §3).
|
||||
- `DeepDrftPublic.Client/Controls/MixZoomMapping.cs` — the log-space slider↔seconds mapping, reused as-is.
|
||||
- `DeepDrftShared.Client/Common/DeepDrftPalettes.cs` — the source of truth for the navy/moss theme
|
||||
tokens (see §4c).
|
||||
|
||||
---
|
||||
|
||||
## Why a renderer swap, not an iteration
|
||||
|
||||
A staff-engineer analysis concluded the Canvas 2D approach **cannot afford the planned effects**. The
|
||||
current renderer already strains at 1–2 FPS; the three per-frame killers are full-viewport `shadowBlur`,
|
||||
the CSS `backdrop-filter: blur()` layer, and per-frame `getBoundingClientRect()`. More decisively: the
|
||||
effects Daniel wants — bulge, detach, a morphing 2D color field, glass — are **all-dynamic, per-pixel,
|
||||
per-frame** effects. That is precisely the workload Canvas 2D is worst at and a fragment shader is best
|
||||
at: in a shader the effects become GLSL evaluated across every pixel in parallel on the GPU, and they
|
||||
get *cheaper relative to Canvas* as they get fancier.
|
||||
|
||||
**Decision (Daniel, explicit): rebuild as a WebGL2 fragment-shader renderer. No Canvas 2D stopgap.**
|
||||
"WebGL as step 1, no pussyfooting." This supersedes the 8.K spec's §E recommendation (which defaulted to
|
||||
Canvas 2D and reserved WebGL only as a fallback) — that recommendation has been overtaken by the effect
|
||||
roadmap. The 8.K spec's §E "no tricks, well-commented, industry-standard" *intent* still holds; it now
|
||||
means **textbook WebGL2 with a well-commented shader**, not Canvas.
|
||||
|
||||
---
|
||||
|
||||
## 1. Goal and scope boundary
|
||||
|
||||
**Goal.** Replace the Canvas 2D Mix visualizer with a WebGL2 fragment-shader renderer that holds a
|
||||
smooth **60 FPS** while carrying four live, per-pixel visual effects — bulge/bubble shaping, lava-lamp
|
||||
detachment, a morphing navy↔moss 2D color field, and an in-shader glass treatment — driven by four
|
||||
persistent continuous-slider controls. The visualizer remains a **read-only, playback-coupled,
|
||||
windowed, bottom-to-top scrolling** background element on the Mix detail page.
|
||||
|
||||
**In scope.**
|
||||
- A new WebGL2 fragment-shader rendering module replacing `MixVisualizer.ts` (the rAF loop, scroll/zoom
|
||||
math, and theme handling move into / alongside the GL pipeline).
|
||||
- A new **controls row** above the mix details: four continuous sliders (resolution, bubblyness, detach,
|
||||
color-shift speed), all persistent across page visits.
|
||||
- The four visual effects, in-shader.
|
||||
- Preserving the existing Blazor↔JS bridge contract as far as the new inputs allow (§2).
|
||||
|
||||
**Out of scope / unchanged.**
|
||||
- **No playback-control changes.** The visualizer stays strictly read-only (8.K §D). One-way
|
||||
`PlaybackPosition` input; no seek, no scrub, no write-back. The dropped `OnSeek` seam stays dropped.
|
||||
- **No datum format change.** The duration-derived ~333 samples/sec capture (8.K §F, Wave 1, landed)
|
||||
already feeds the renderer correctly and is the right resolution for the max-zoom anchor. The new
|
||||
renderer consumes the *same* `WaveformProfileDto` (base64 loudness bytes + duration-from-player). The
|
||||
only change is that the datum now lands in the GPU as a **texture** rather than a JS array walked on
|
||||
the CPU (§4b).
|
||||
- **No change to where the visualizer sits** (full-page background behind `.mix-detail-foreground`),
|
||||
except that the new controls row sits *above the mix details, below the back button* (§3).
|
||||
- **Control-range guards** (resolution × bubblyness combinations that look unpleasant) — deferred per
|
||||
Daniel; he tunes bad ranges by hand once it is on screen.
|
||||
- **Motion-speed coupling to bubblyness** — deferred. Bubblyness controls *shape/amount only* for now;
|
||||
scroll speed remains a pure function of zoom (8.K §B), independent of the bubble controls.
|
||||
|
||||
---
|
||||
|
||||
## 2. Renderer architecture (spec level)
|
||||
|
||||
The implementation choices below the seam are staff-engineer's. This section fixes the *contract* and
|
||||
the *data-flow shape*, not the GLSL.
|
||||
|
||||
### 2a. Pipeline shape
|
||||
|
||||
A single full-window WebGL2 quad (two triangles covering the canvas), rasterized once per frame, with
|
||||
**all the visual work in the fragment shader**. Each fragment (pixel) computes: which mix-time it maps
|
||||
to (from its screen Y, the scroll offset, and the zoom), samples the loudness datum, decides whether it
|
||||
is inside the ribbon / a bubble / a detached blob, and shades itself with the morphing gradient + glass.
|
||||
The vertex shader is trivial (pass-through quad); the art is entirely fragment-side. This is the
|
||||
textbook "shadertoy-style" full-screen-fragment pattern — well-trodden, well-documented, no exotic
|
||||
tricks. Comment the shader so Daniel can follow it (8.K §E discipline carries forward).
|
||||
|
||||
**This is a deliberate inversion of the Canvas model.** Canvas walked screen rows on the CPU and built a
|
||||
path; the shader instead asks, per pixel in parallel on the GPU, "am I inside the shape, and what color
|
||||
am I?" The scroll/zoom math from `MixVisualizer.ts` (time-at-screen-Y, sample-at-time) moves essentially
|
||||
intact into GLSL — the same equations, evaluated per-fragment instead of per-row. Preserve the
|
||||
well-commented derivation; it is the part Daniel reads to follow the thing.
|
||||
|
||||
### 2b. How the waveform datum feeds in
|
||||
|
||||
**As a 1-D texture (or a 2-D texture one row tall), not a uniform array.** The datum is up to ~1.8 MB
|
||||
(~1.8M samples for a 90-min mix at 333/s); uniform arrays are far too small and the wrong tool. A
|
||||
texture is the standard vehicle: upload the normalized loudness samples once when the datum arrives (the
|
||||
`setDatum` bridge call), and let the fragment shader sample it with the GPU's built-in linear filtering —
|
||||
which gives the **smooth, no-stair-stepping interpolation at every zoom** the 8.K spec wanted, for free,
|
||||
in hardware. Texture coordinate = `mixTime / durationSeconds`; the GPU interpolates between samples.
|
||||
|
||||
Upload happens once per datum (off the animation path), exactly mirroring today's once-per-datum decode.
|
||||
The per-frame path touches no large buffer — it only updates small uniforms (§2c). This preserves the
|
||||
existing optimization where a per-tick playback push never re-decodes the datum.
|
||||
|
||||
### 2c. Uniforms — what changes per frame vs. per change
|
||||
|
||||
The fragment shader is parameterized entirely by uniforms. Group them by update cadence:
|
||||
|
||||
- **Per-frame (updated in the rAF loop):**
|
||||
- `playheadSeconds` — current playback position (drives the scroll).
|
||||
- `timeSeconds` — a monotonic clock for the always-shifting color field and bubble animation (so the
|
||||
field morphs and blobs drift even when scroll is slow). This is the one input that makes the field
|
||||
"never static."
|
||||
- **Per control change (updated when a slider moves):**
|
||||
- `visibleSeconds` — the zoom (8.K §B), drives window time-span and apparent scroll speed.
|
||||
- `bubblyness` — bulge amount (§5b).
|
||||
- `detach` — lava-lamp detachment amount (§5c).
|
||||
- `colorShiftSpeed` — gradient morph rate (§5d). (Multiplies `timeSeconds` inside the shader.)
|
||||
- **Per theme change:** `colorNavy`, `colorMoss` (and any derived stops) — the gradient endpoints, read
|
||||
from the live palette on a dark-mode toggle (§4c).
|
||||
- **Per resize:** canvas resolution / aspect.
|
||||
- **Once per datum:** the datum texture + `durationSeconds` + `samplesPerSecond`.
|
||||
|
||||
### 2d. The Blazor bridge — preserved
|
||||
|
||||
The existing `MixWaveformVisualizer.razor.cs` bridge is **largely reusable** and should be preserved.
|
||||
What stays:
|
||||
- Self-fetch of the datum from `ReleaseId` via `IReleaseDataService.GetMixWaveform` (unchanged).
|
||||
- The cascaded-player coupling: `IStreamingPlayerService.StateChanged` subscription, `IsActivePlayer`
|
||||
gating on `TrackId`, `CurrentPositionSeconds` / `IsPlaying` / `PlayerDurationSeconds` derivation
|
||||
(unchanged — this is the playback signal, renderer-agnostic).
|
||||
- The idempotent `PushDatumAsync` guard (don't re-upload the datum every tick) — even more important now
|
||||
that the datum is a GPU texture upload.
|
||||
- The `refreshTheme` push on dark-mode toggle.
|
||||
- `IAsyncDisposable` teardown of the module handle.
|
||||
- The module-handle shape (`create(canvas)` → handle with `setDatum` / `setPlayback` / `setZoom` /
|
||||
`refreshTheme` / `dispose`). **Extend** it with `setBubblyness`, `setDetach`, `setColorShiftSpeed`;
|
||||
do not redesign it.
|
||||
|
||||
What changes: `create` initializes a WebGL2 context (`canvas.getContext('webgl2')`) instead of `'2d'`,
|
||||
compiles the shader program, and uploads the datum as a texture. The no-context fallback (return a no-op
|
||||
handle so the component still renders a plain backdrop) carries forward — now guarding against *no WebGL2*
|
||||
rather than *no 2D context*.
|
||||
|
||||
### 2e. Animation lives in the rAF/shader loop — NOT in Blazor re-renders
|
||||
|
||||
**Load-bearing constraint.** Today `OnPlayerStateChanged` calls `StateHasChanged()` on every player tick
|
||||
(~10×/sec) to keep the slider/visibility in sync. That re-render cadence must **not** become the
|
||||
animation driver — 10 Blazor re-renders/sec is both far below 60 FPS and the wrong layer for animation.
|
||||
The animation is entirely the GPU rAF loop's job: the shader's `timeSeconds` uniform advances every frame
|
||||
and the field morphs / blobs drift continuously between player ticks. Blazor pushes *playhead position*
|
||||
on its tick (cheap uniform write); the *smooth motion between ticks* is the shader interpolating on its
|
||||
own clock. The rAF loop stays gated on `isPlaying` (cool when paused — 8.K §E), with the same one-shot
|
||||
redraw-while-idle behavior for zoom/theme/datum/resize changes.
|
||||
|
||||
---
|
||||
|
||||
## 3. The controls row
|
||||
|
||||
A new horizontal controls section for the visualizer, placed **above the rest of the mix details and
|
||||
below the "back" button** on the Mix detail page. It holds the four controls in a row. All are
|
||||
**continuous sliders** (MudSlider, `Step` fine enough to feel continuous — the existing zoom slider uses
|
||||
`Step="0.001"`, follow that), and all are **persistent across page visits** within a listening session.
|
||||
|
||||
### 3a. The four controls, ranges and defaults
|
||||
|
||||
Daniel asked for sensible default ranges with the explicit note that he will tune them on screen.
|
||||
Recommendations:
|
||||
|
||||
| # | Control | What it does | Slider range (normalized) | Default | Notes |
|
||||
|---|---------|--------------|---------------------------|---------|-------|
|
||||
| 1 | **Resolution** | Visible time-span / zoom (the existing 8.K control, relocated into this row) | 0 → 1, log-mapped to 30 s → 0.333 s | ~10 s window | Reuse `MixZoomMapping` and `MixVisualizerZoomState` exactly as-is. Just move the slider's home into the new row. |
|
||||
| 2 | **Bubblyness** | Bulge amount of each bar (§5b) | 0 → 1 | ~0.35 | 0 = straight rectangular scrolling bars; 1 = fully rounded liquid silhouettes that bulge from the zero-line outward. Bars stay **attached** at max. |
|
||||
| 3 | **Detach** ("unleash the lava lamp") | How much bubbles detach from the bar and rise independently (§5c) | 0 → 1 | 0 | 0 = fully attached (whatever bubblyness produced); 1 = blobs separate and float upward freely. Separate axis from bubblyness. |
|
||||
| 4 | **Color-shift speed** | How fast the gradient field morph cycles (§5d) | 0 → 1, mapped to a slow→quick cycle-rate range | ~0.3 | Map normalized 0→1 onto a cycle period of roughly **60 s (very slow drift) → 4 s (pretty quick)**. Default ~0.3 lands around a ~20 s cycle — alive but not busy. Never fully zero (the field is "always shifting, never static" — §4b); the slow end is "barely perceptible drift," not "frozen." |
|
||||
|
||||
These are starting numbers for feel. The max-zoom 0.333 s anchor for control 1 is fixed (8.K §B); the
|
||||
other three ranges are Daniel's to tune.
|
||||
|
||||
### 3b. Continuous-slider UX
|
||||
|
||||
All four are bare continuous sliders (no tick marks, no snap), `Size.Small`, `Color.Primary`, each with
|
||||
an `aria-label`. Lay them in a row with compact labels (icon or short text caption). On mobile the row
|
||||
may wrap or scroll horizontally — a layout call for staff-engineer, but the controls must remain
|
||||
present (they are the one thing that affects the visualizer, and none of them is a seek surface — 8.K
|
||||
§D holds).
|
||||
|
||||
### 3c. Persistence — mirror MixVisualizerZoomState
|
||||
|
||||
The existing `MixVisualizerZoomState` is a DI-**scoped** holder: it survives SPA navigation within one
|
||||
WASM app instance (open a second mix, the slider keeps its place) and resets to default on a fresh page
|
||||
load (F5). That is the "persist within session, reset on fresh load" model from 8.K §B, and it is the
|
||||
pattern to mirror for **all four** controls.
|
||||
|
||||
**Recommendation:** extend the single state object rather than spawning four. Rename/widen
|
||||
`MixVisualizerZoomState` into a `MixVisualizerControlState` (scoped) holding `VisibleSeconds`,
|
||||
`Bubblyness`, `Detach`, `ColorShiftSpeed`, each with a `const` default mirrored to the TS tuning anchors.
|
||||
One injected dependency, one persistence story, four properties. (If Daniel later wants cross-session
|
||||
persistence — "this mix should remember my lava-lamp settings" — that is a cookie/localStorage upgrade on
|
||||
this one object, deferred and out of scope now, same as the 8.K note.)
|
||||
|
||||
Keep the C#-side defaults and the TS-side tuning anchors in sync, as the existing
|
||||
`MixVisualizerZoomState.DefaultVisibleSeconds` / `DEFAULT_VISIBLE_SECONDS` pair already does (comment the
|
||||
contract on both sides).
|
||||
|
||||
---
|
||||
|
||||
## 4. The four visual effects
|
||||
|
||||
Described as **intended look + shader-side approach in conceptual terms**. The exact GLSL is
|
||||
staff-engineer's. Each effect is a continuous function of its control value, so the whole range from
|
||||
"off" to "maxed" is reachable by dragging one slider.
|
||||
|
||||
### 4a. Waveform geometry (the substrate all effects sit on)
|
||||
|
||||
Traditional **−1..+1 amplitude with 0 at the center line**; peaks extend symmetrically above and below
|
||||
the zero-line. The waveform **scrolls bottom-to-top** (new audio enters at the bottom, played audio exits
|
||||
the top — 8.K §A, unchanged). Per-fragment: map screen Y → mix-time (via playhead + zoom), sample the
|
||||
datum texture for loudness at that time, and the loudness defines the half-width of the symmetric ribbon
|
||||
about the horizontal center. The "now" line sits at a fixed screen Y (center by default — 8.K §A). This
|
||||
is the same geometry the Canvas renderer drew; in the shader it becomes a signed-distance test ("is this
|
||||
pixel inside the ribbon silhouette?").
|
||||
|
||||
### 4b. The 2D morphing navy↔moss gradient field (the headline visual)
|
||||
|
||||
**Replace the boring grey with a living color field built from the two theme colors — navy blue and moss
|
||||
green.** Two coupled dimensions:
|
||||
|
||||
1. **Per-bar / along-the-bar (the vertical / amplitude axis).** Color varies along each bar's height —
|
||||
the zero-line and the peak differ. This gives each bar internal structure: a gradient from center-line
|
||||
out to the peak.
|
||||
2. **Along the scroll/time axis (the second dimension).** The mix between navy and moss **shifts along
|
||||
time** — more green at some moments, more blue at others — and **always shifts, never static**, driven
|
||||
by the `timeSeconds` clock × `colorShiftSpeed` (§3a control 4). This is what makes it a *field* and not
|
||||
a fixed gradient.
|
||||
|
||||
**The tension to capture (Daniel's words):** it should read as **a continuous 2D color field** — coherent
|
||||
across the whole window, neighbors related — *yet each bar should "feel like its own living thing."**
|
||||
Shader approach to hold both at once:
|
||||
- A smooth, low-frequency base field (navy↔moss) over (time-axis, amplitude-axis) gives the **field-like
|
||||
coherence** — sampled with a flowing noise or layered sinusoids in the time dimension so it morphs
|
||||
continuously. This is the "2D color field" layer.
|
||||
- A higher-frequency per-bar modulation — keyed off each bar's own index/phase and its instantaneous
|
||||
loudness — perturbs the field locally so adjacent bars are *related but not identical*, and a loud bar
|
||||
reads brighter/more-saturated than its quiet neighbor. This is the "own living thing" layer.
|
||||
- The two compose: coherent field + per-bar liveness = the intended tension.
|
||||
|
||||
Pull the **exact endpoints from the theme tokens** (§4c) so a dark-mode toggle re-themes the field live.
|
||||
|
||||
### 4c. Theme tokens — the concrete colors
|
||||
|
||||
The bespoke palettes live in `DeepDrftShared.Client/Common/DeepDrftPalettes.cs` (the "Charleston in the
|
||||
Day" / "Lowcountry Summer Nights" identities map onto these). The navy/moss pair:
|
||||
|
||||
- **Navy:** `#17283f` (Primary, light) / `#0D1B2A` (the dark ground) / `#162437` (Surface, dark, elevated).
|
||||
- **Moss / green:** `#3D7A68` (Secondary light / Primary dark — the interactive green) / `#429d6a`
|
||||
(Tertiary light, green-interactive) / `#2A5C4F` (PrimaryDarken dark).
|
||||
|
||||
As in the Canvas renderer, the shader cannot resolve `var(--mud-palette-*)` directly — read the computed
|
||||
CSS custom properties off the canvas element on theme refresh and pass them as `vec3` color uniforms.
|
||||
**Recommended endpoints:** navy = `--mud-palette-primary` in light / the dark ground in dark; moss =
|
||||
`--mud-palette-secondary` (light) / `--mud-palette-primary` (dark, where green *is* primary). Staff-
|
||||
engineer picks the precise var bindings that give the richest navy↔moss spread per palette; the design
|
||||
intent is "the two theme signature colors, navy and moss, as the two poles of the field." Re-read on
|
||||
`refreshTheme` so the toggle re-themes live (preserve the existing mechanism).
|
||||
|
||||
### 4d. Bulge / bubble shaping — "bubblyness" (control 2)
|
||||
|
||||
**Intent.** At **minimum**: straight rectangular bars that only scroll bottom-to-top — a clean,
|
||||
hard-edged waveform. As bubblyness rises: bars **gain space and bulge into rounded, liquid silhouettes
|
||||
that swell from the center zero-line outward toward the peak** — like a lava-lamp bubble growing. At
|
||||
**maximum**: fully rounded liquid blobs, but still **attached** to the bar (detachment is a separate
|
||||
axis — §4e). Bubblyness controls **bulge shape/amount only**, not motion speed (deferred).
|
||||
|
||||
**Shader-side approach.** This is a signed-distance / metaball move. Each bar contributes a field
|
||||
contribution; `bubblyness` interpolates the per-bar silhouette from a **box SDF** (sharp rectangle, bulge
|
||||
= 0) toward a **rounded capsule / metaball SDF** (bulge = 1) that swells outward from the zero-line. As
|
||||
bubblyness rises, increase the rounding radius and let neighboring bars' fields blend (metaball
|
||||
union with a smooth-min) so the silhouette reads as continuous liquid rather than discrete bars. The
|
||||
"swell from the center outward" is the SDF growing its radius along the amplitude axis with loudness. The
|
||||
GPU evaluates the SDF per-pixel — this is exactly the per-pixel work that was unaffordable on Canvas and
|
||||
is cheap in a shader.
|
||||
|
||||
### 4e. Detach — "unleash the lava lamp" (control 3)
|
||||
|
||||
**Intent.** A **separate** control from bubblyness. At 0, bubbles stay attached (whatever shape
|
||||
bubblyness produced). As detach rises, the bulging bubbles **separate from the bar and rise independently**
|
||||
— blobs pinch off and float upward on their own, the true lava-lamp experience. At max, freely floating
|
||||
detached blobs drifting up through the window.
|
||||
|
||||
**Shader-side approach.** Building on the metaball field from §4d: as `detach` rises, displace a fraction
|
||||
of each bar's bubble mass **upward** by an amount that grows over the bubble's life (the `timeSeconds`
|
||||
clock gives the rise animation), and **weaken the smooth-min link** between the detaching blob and its
|
||||
parent bar so the metaball connection thins and pinches off (classic metaball separation — two metaballs
|
||||
pulling apart show the liquid "neck" thinning then breaking). Detached blobs become independent metaball
|
||||
centers with their own upward velocity and slight horizontal drift, fading as they near the top. Keep the
|
||||
count and motion bounded so it stays a hypnotic drift, not a particle storm — detach amount scales *how
|
||||
much* detaches and *how far it rises*, not an unbounded spawn rate.
|
||||
|
||||
This effect is the clearest argument for the WebGL move: independent floating metaballs with smooth-min
|
||||
merging, evaluated per-pixel every frame, is the canonical fragment-shader lava-lamp and is essentially
|
||||
free on the GPU while being impossible at 60 FPS on Canvas 2D.
|
||||
|
||||
### 4f. Glass treatment (delegated to design judgment, in-shader)
|
||||
|
||||
Daniel named refraction/distortion of what's behind, specular highlights/sheen on the bars, frosted
|
||||
translucency, and a wet glossy surface — and explicitly delegated the aesthetic: "use your terrific
|
||||
claude design judgment for maximum style." **Hard constraint: achieve glass entirely in the shader. Do
|
||||
NOT reintroduce a per-frame CPU `backdrop-filter: blur()`** — that was a confirmed perf killer in the
|
||||
Canvas renderer and is exactly the cost the GPU move exists to eliminate.
|
||||
|
||||
**Recommended concrete, shippable glass treatment (all fragment-shader, all 60-FPS-affordable):**
|
||||
|
||||
1. **Refraction of what's behind.** The visualizer is a full-page background behind the mix-detail
|
||||
content. True refraction of arbitrary DOM behind a WebGL canvas isn't directly available, so achieve
|
||||
the *read* of refraction cheaply: (a) where a blob/ribbon's surface curves (high SDF gradient), warp
|
||||
the gradient-field sampling coordinates by the surface normal — so the color field appears bent
|
||||
through the glass at the edges, the hallmark of looking through a curved lens; and (b) keep the page
|
||||
background showing through via translucency (below). This gives "refraction" as an internal distortion
|
||||
of the field, which reads correctly without needing the actual DOM pixels.
|
||||
2. **Specular highlights / sheen.** Treat each blob's surface as lit by a fixed virtual light: compute a
|
||||
surface normal from the SDF gradient and add a sharp specular hotspot (Blinn-Phong-style highlight)
|
||||
plus a soft broad sheen along the upper edge of each bulge. Highlights drift as blobs move — this is
|
||||
what sells "wet glossy surface." Cheap: a couple of dot products per fragment.
|
||||
3. **Frosted translucency.** The ribbon/blobs render at partial alpha over the page (carry forward the
|
||||
RIBBON_OPACITY-style backdrop intent — it stays a backdrop, not a chart) with a subtle internal
|
||||
softening (a small in-shader blur of the field, or a frosted noise modulation of alpha) so edges read
|
||||
soft and lit rather than hard. This is the "frosted glass" read, done in-shader with no CSS filter.
|
||||
4. **Fresnel edge glow.** Add a Fresnel term (brighter at grazing angles / silhouette edges) so the rims
|
||||
of blobs catch light — the single most effective "this is glass / lit volume" cue, and one cheap
|
||||
`pow(1 - dot(n, viewDir), k)` per fragment.
|
||||
|
||||
Net look: **lit, wet, frosted glass blobs of navy-and-moss light drifting upward** — hypnotic, ambient,
|
||||
unmistakably "lava lamp made of theme-colored glass." The glass is layered *on top of* the color field
|
||||
and geometry, so all four effects compose into one coherent surface.
|
||||
|
||||
---
|
||||
|
||||
## 5. Acceptance criteria (observable)
|
||||
|
||||
1. **Frame rate.** The visualizer holds **a smooth 60 FPS** on a mid-range desktop while playing, with
|
||||
bubblyness, detach, and color-shift-speed all at non-trivial values simultaneously. (The whole point
|
||||
of the swap — measure it, don't assume it.) Graceful degrade on weaker/mobile devices: drop internal
|
||||
resolution or effect cost before dropping frames (8.K §E carries forward).
|
||||
2. **Renderer.** Rendering is WebGL2 fragment-shader; there is no Canvas 2D path and no CSS
|
||||
`backdrop-filter` on the visualizer. (The old per-frame killers — `shadowBlur`, `backdrop-filter`,
|
||||
per-frame `getBoundingClientRect`-driven layout — are gone.)
|
||||
3. **Motion.** A windowed slice scrolls bottom-to-top, coupled to playback; scroll holds when paused; no
|
||||
scroll when nothing is playing; scrolls in-from-empty at the start and out-to-empty at the end (8.K §A
|
||||
preserved). The rAF loop is gated on `isPlaying` and burns no frames while paused.
|
||||
4. **Animation independence.** The color field visibly morphs and detached blobs visibly drift **between**
|
||||
Blazor player ticks — i.e. motion is smooth at 60 FPS, not stepping at the ~10 Hz `StateHasChanged`
|
||||
cadence. Animation is driven by the shader clock, not Blazor re-renders.
|
||||
5. **Controls.** Four continuous sliders sit in a row above the mix details / below the back button. Each
|
||||
visibly and continuously affects its target as it is dragged: resolution changes the visible
|
||||
time-span and apparent scroll speed; bubblyness sweeps straight bars → attached liquid bulges; detach
|
||||
sweeps attached → free-floating rising blobs; color-shift-speed sweeps barely-drifting → briskly
|
||||
morphing field.
|
||||
6. **Persistence.** All four control positions survive SPA navigation to another mix within a session and
|
||||
reset to defaults on a fresh page load (mirrors `MixVisualizerZoomState`).
|
||||
7. **Gradient.** Bars are filled with a navy↔moss gradient that (a) varies along each bar's height and
|
||||
(b) shifts along the time axis and never sits fully static; the field reads as coherent across the
|
||||
window while individual bars read as distinct/alive. Toggling dark mode re-themes the field live with
|
||||
no reload.
|
||||
8. **Geometry.** Amplitude is symmetric about a center zero-line (−1..+1), peaks extending both ways.
|
||||
9. **Glass.** Blobs/ribbon read as lit, frosted, glossy glass — visible specular highlights, soft frosted
|
||||
edges, edge/Fresnel glow, field distortion at curved surfaces — with no CPU backdrop-filter.
|
||||
10. **Read-only.** No seek, no scrub, no write-back; the only inputs that affect the visualizer are
|
||||
playback position (one-way) and the four sliders. No control is a seek surface.
|
||||
11. **Bridge intact.** The component still self-fetches its datum from `ReleaseId`, couples to the
|
||||
cascaded player via `TrackId` gating, and tears down its GL resources on dispose without leaking.
|
||||
|
||||
---
|
||||
|
||||
## 6. Suggested phasing / waves
|
||||
|
||||
The work decomposes naturally into a **parity swap first, effects second** sequence. This de-risks the
|
||||
hardest architectural step (getting WebGL2 on screen at all, holding 60 FPS, bridge intact) before any
|
||||
art is layered on, and gives Daniel a working renderer to tune effects against.
|
||||
|
||||
### Wave 1 — Renderer swap at parity (the load-bearing step)
|
||||
Stand up the WebGL2 pipeline reproducing **today's visual** (the scrolling navy/moss-ish ribbon) but on
|
||||
the GPU: full-window quad, datum-as-texture, scroll/zoom math ported into the fragment shader, rAF loop
|
||||
gated on `isPlaying`, bridge contract preserved (`setDatum`/`setPlayback`/`setZoom`/`refreshTheme`/
|
||||
`dispose`), no-WebGL2 fallback to a plain backdrop. **Acceptance:** §5 criteria 1 (at parity workload),
|
||||
2, 3, 4, 10, 11 — a 60-FPS GPU renderer at visual parity with the Canvas version. No new effects yet.
|
||||
This is where the architecture is proven; everything after is shading.
|
||||
|
||||
### Wave 2 — Controls row + state
|
||||
Add the four-control row above the mix details, widen `MixVisualizerZoomState` →
|
||||
`MixVisualizerControlState` (the three new properties + persistence), wire the three new
|
||||
uniforms (`bubblyness`, `detach`, `colorShiftSpeed`) through the bridge as no-op-until-shader-uses-them
|
||||
inputs. Resolution relocates here and keeps working. **Acceptance:** §5 criterion 6 and the slider half
|
||||
of 5 (sliders present, persistent, wired) — even before the shader consumes them visibly.
|
||||
(Wave 2 can begin once Wave 1's bridge is stable; the controls are inert visually until Wave 3 but the
|
||||
plumbing is independent.)
|
||||
|
||||
### Wave 3 — The four effects in the shader
|
||||
Layer the effects onto the proven pipeline, in increasing order of risk:
|
||||
1. **Gradient field** (§4b) — replace the parity fill with the morphing 2D navy↔moss field. (§5.7)
|
||||
2. **Bubblyness** (§4d) — box→metaball SDF interpolation. (§5.5 bubblyness)
|
||||
3. **Detach** (§4e) — metaball pinch-off + rising blobs. (§5.5 detach)
|
||||
4. **Glass** (§4f) — specular + Fresnel + frosted + field distortion, composed on top. (§5.9)
|
||||
**Acceptance:** §5 criteria 5 (visual half), 7, 8, 9 — all four effects live and slider-driven at 60 FPS.
|
||||
|
||||
**Dependency shape:** `Wave 1 → (Wave 2 ‖ Wave 3 plumbing) → Wave 3 art`. Wave 1 is the prerequisite for
|
||||
everything. Wave 2 (controls/state) and Wave 3 (effects) both depend on Wave 1's bridge but the four
|
||||
effects within Wave 3 are independently shippable and independently tunable — land and tune one slider's
|
||||
effect at a time. Color-range and bubblyness-range guards stay deferred throughout (Daniel tunes by
|
||||
hand).
|
||||
|
||||
---
|
||||
|
||||
## Open items (tuning knobs, none block starting)
|
||||
|
||||
All have recommended defaults inline; Daniel tunes on screen:
|
||||
- Exact slider ranges for bubblyness / detach / color-shift-speed (§3a).
|
||||
- The precise palette-var bindings for the navy/moss field endpoints per light/dark (§4c).
|
||||
- Glass intensity / specular sharpness / Fresnel falloff (§4f) — pure aesthetic tuning.
|
||||
- Whether the controls row wraps or horizontally scrolls on mobile (§3b) — layout call.
|
||||
- Whether control state should later persist cross-session (cookie/localStorage) — deferred upgrade to
|
||||
the one state object (§3c).
|
||||
Reference in New Issue
Block a user