docs(plan): Phase 18 OQ resolutions + VBR-safe accurate Opus seek model

This commit is contained in:
daniel-c-harvey
2026-06-23 05:26:58 -04:00
parent 6af6677a12
commit e3a4364b8c
3 changed files with 543 additions and 183 deletions
+82 -47
View File
@@ -468,23 +468,44 @@ decoder-side `AudioPlayer.createFormatDecoder` is a **wired** strategy registry
step that *derives* an Opus 320 artifact per track (nothing derives Opus today), and (2) a **per-format
delivery selection** so one track serves as either WAV or Opus on request.
**Architectural spine — a derived artifact + a delivery param + one new decoder; three new leaf
implementations, zero changes to existing format code (the strong OCP signal).** Transcode is a new
processor sibling in `DeepDrftContent`, invoked post-store alongside `WaveformProfileService`,
**failure-tolerant and off the hot path** (background/queued — a 1 GB WAV transcode must not block the
upload response) — mirroring the landed waveform-datum pattern (derive at ingest, regenerate via a CMS
bulk action + ApiKey endpoint). The Opus bytes are a **derived artifact** stored like the high-res
waveform datum (recommend a dedicated `track-opus` vault, the `track-waveforms` precedent; final call
staff-engineer's). Delivery adds a **`?format=opus|lossless` param** (mirroring the existing `offset`
param threading through `TrackProxyController`) resolved server-side to the right artifact + content-type,
with a **lossless fallback** when no Opus artifact exists (additive, never 404/silence). The player gains
one `OpusFormatDecoder` (`IFormatDecoder`): Ogg-page-aligned segmenting (`OggS` scan — the FLAC
frame-sync analogue), `OpusHead` setup-bytes carry (the FLAC `streamInfoBytes` analogue), and an
**approximate** page-interpolation `calculateByteOffset` (Opus is VBR/paged — this is exactly the Phase
21 C5 case). **Browser constraint flagged:** Ogg-Opus `decodeAudioData` is Safari-18.4+ only (Chrome/FF
long-standing), so the Opus default must be **capability-gated** — fall back to the universal lossless
**Open questions RESOLVED (Daniel, 2026-06-23).** OQ1 selection UX → **global, via a new public-site
Settings menu** (not a bare app-bar control); OQ2 default → **Opus by default, capability-gated** (defer
network-awareness); OQ3 remembered → **persisted via the dark-mode seam** (cookie → prerender-read →
`PersistentComponentState` → client cookie service); OQ4 → **always-on Opus + Backfill-Opus**; OQ5 →
**Ogg Opus**; OQ6 transcode model → **background job after the file is available, with a visible
Post-Processing phase on the CMS upload meter.** One new tuning OQ (OQ7: seek-index granularity — recommend
~12 s buckets) is non-blocking.
**Architectural spine — a derived artifact set + a delivery param + one new decoder + a precomputed
accurate seek index; leaf implementations only, zero changes to existing format code (the strong OCP
signal).** Transcode is a new processor sibling in `DeepDrftContent`, invoked post-store alongside
`WaveformProfileService` **as a background job** (a 1 GB WAV transcode must not block the upload; the source
is stored and the track plays lossless *first*, then Opus is derived)mirroring the landed waveform-datum
pattern (derive at ingest, regenerate via a CMS bulk action + ApiKey endpoint). The Opus bytes are a
**derived artifact** stored like the high-res waveform datum (recommend a dedicated `track-opus` vault, the
`track-waveforms` precedent; final call staff-engineer's). Delivery adds a **`?format=opus|lossless` param**
(mirroring the existing `offset` param threading through `TrackProxyController`) resolved server-side to the
right artifact + content-type, with a **lossless fallback** when no Opus artifact exists (additive, never
404/silence). The player gains one `OpusFormatDecoder` (`IFormatDecoder`): Ogg-page-aligned segmenting
(`OggS` scan — the FLAC frame-sync analogue) and `OpusHead`/`OpusTags` setup-bytes carry (the FLAC
`streamInfoBytes` analogue). **Browser constraint flagged:** Ogg-Opus `decodeAudioData` is Safari-18.4+ only
(Chrome/FF long-standing), so the Opus default is **capability-gated** — fall back to the universal lossless
path on browsers that can't decode it.
**VBR-safe ACCURATE seeking (Daniel, 2026-06-23 — supersedes the earlier "approximate" hand-wave).** Raw
byte-offset seek and rough page interpolation are inadequate for VBR Opus — there is no linear time↔byte
relationship. The fix is an **accurate transfer function built at transcode time** (the one moment the
whole encoded stream is walked): a precomputed **seek index** mapping Ogg-page `granulepos` (48 kHz sample
counts → time) → exact byte offset (recommend ~12 s buckets snapped to page starts; ~58 KB for a 1-hour
mix). The decode **setup header** (`OpusHead`/`OpusTags`, needed to decode any mid-stream slice) is made
available too. Recommended concrete design: **one sidecar artifact per track = `[setup header][seek
index]`, built at transcode, stored beside the Opus bytes, fetched once on track load**, parsed into
`OpusSeekData`. Client seek flow: `calculateByteOffset(t)` binary-searches the index for the exact page
offset → `Range: bytes=X-` fetch (landed Phase 4 primitive, unchanged) → prepend the cached setup header →
decode → fine re-sync to `t` within the bucket. **The listener lands at the correct time, not
approximately** (AC9), **without** the full PCM in memory — so it composes with Phase 21 windowed refill,
which calls the **same** index resolver. The earlier "approximate page-interpolation" language is rejected.
**Constraints/invariants:** keep the bespoke graph (no MSE); preprocessing is **additive** (WAV path
untouched, byte-for-byte; a track with no Opus artifact still plays losslessly); reuse the landed
`Range`/offset seek path; no format branches leak outside the new decoder + one selection arm + the
@@ -492,38 +513,49 @@ transcode/delivery seam; transcode failure must not block ingest; format selecti
decision resolving one `EntryKey` to one of two artifacts (one source, two views — **not** a second
`TrackEntity` row, which would fracture share/queue/play-count/release identity).
Sequenced as five waves. `18.1 → 18.2 → {18.3, 18.4} → 18.5`. **18.1 (ingest transcode + derived
artifact) is the cold-start prerequisite** — nothing downstream has bytes to serve or decode until an
Opus artifact exists.
Sequenced as six waves. `18.1 → 18.2 → {18.3, 18.4} → 18.5`, with `18.6` (Settings menu) able to run in
parallel (it needs only 18.3's format mechanism before its toggle is live). **18.1 (ingest transcode +
seek-index + setup-header derived artifacts) is the cold-start prerequisite** — nothing downstream has
bytes to serve, decode, or seek against until those artifacts exist.
- **18.1 — Ingest transcode: derive + store the Opus artifact (cold-start; load-bearing).** New
- **18.1 — Ingest transcode + seek-index + setup-header (cold-start; load-bearing).** New
`OpusTranscodeService`/processor in `DeepDrftContent`, invoked post-store from
`UnifiedTrackService.UploadAsync` alongside `WaveformProfileService`; produces Ogg Opus fullband 320;
stores it as a derived artifact (recommend a `track-opus` vault). Failure-tolerant; off the hot path
(background/queued). **Independent of the delivery/decoder waves — can begin immediately.**
- **18.2 — Storage + lookup contract.** The derived-artifact key/vault convention + server-side "given
`EntryKey` + format, return the right `AudioBinary` + content-type," including the lossless fallback.
**Depends on 18.1.**
- **18.3 — Delivery: `?format=opus|lossless` param + proxy threading.** On the `DeepDrftAPI` stream
endpoint (resolves via 18.2), forwarded through `TrackProxyController` (mirror `offset`), `Range`
serving the chosen artifact; player sends it via `TrackMediaClient`. **Depends on 18.2; parallel-ok
with 18.4.**
- **18.4 — `OpusFormatDecoder` in the player stack.** New `IFormatDecoder` (Ogg-page segmenting,
`OpusHead` carry, approximate page-interpolation `calculateByteOffset` with an `OpusSeekData`
accelerator) + one arm in `createFormatDecoder` on `audio/ogg`/`audio/opus`; capability detection for
the lossless fallback. **Depends on 18.2; parallel-ok with 18.3.**
- **18.5 — Backfill + selection UX + end-to-end validation.** "Backfill Opus" CMS bulk action (third
sibling to Generate-Profiles / Backfill-High-res) + replace-audio Opus regeneration; the listener
selection control (recommend a global persisted quality toggle); the AC1AC8 acceptance pass including
the Phase-21 handshake (Opus is windowable by the same machinery). **Depends on 18.118.4.**
`UnifiedTrackService.UploadAsync` alongside `WaveformProfileService` **as a background job** (OQ6);
produces Ogg Opus fullband 320; **walks the encoded stream once to build the granule→byte seek index and
extract the `OpusHead`/`OpusTags` setup header**; stores the Opus bytes **and** the combined seek/setup
**sidecar** as derived artifacts (recommend a `track-opus` vault). Failure-tolerant. **Independent of the
delivery/decoder waves — can begin immediately.**
- **18.2 — Storage + lookup contract.** The derived-artifact key/vault convention (Opus bytes + sidecar) +
server-side "given `EntryKey` + format, return the right `AudioBinary` + content-type (+ the sidecar),"
including the lossless fallback. **Depends on 18.1.**
- **18.3 — Delivery: `?format=opus|lossless` param + sidecar serving + proxy threading.** On the
`DeepDrftAPI` stream endpoint (resolves via 18.2), forwarded through `TrackProxyController` (mirror
`offset`), `Range` serving the chosen artifact; **plus serving the seek/setup sidecar**; player sends the
format param via `TrackMediaClient`. **Depends on 18.2; parallel-ok with 18.4.**
- **18.4 — `OpusFormatDecoder` + index-based seek resolver in the player stack.** New `IFormatDecoder`
(Ogg-page segmenting via `OggS` scan, `OpusHead`/`OpusTags` setup carry from the cached sidecar,
**`calculateByteOffset` that binary-searches the precomputed seek index** — NOT interpolation — with an
`OpusSeekData` accelerator holding the parsed index + setup bytes, and the one-time sidecar fetch+parse on
track load) + one arm in `createFormatDecoder` on `audio/ogg`/`audio/opus`; capability detection for the
lossless fallback. **Depends on 18.2; parallel-ok with 18.3.**
- **18.5 — Backfill + replace-audio + end-to-end validation (incl. seek accuracy).** "Backfill Opus" CMS
bulk action (third sibling to Generate-Profiles / Backfill-High-res), rebuilding Opus bytes + sidecar for
existing tracks; replace-audio Opus + sidecar regeneration; the AC1AC10 acceptance pass **including AC9
(an Opus seek lands at the correct time, not approximately)** and the Phase-21 handshake (Opus windowable
via the index resolver + sidecar setup header). **Depends on 18.118.4.**
- **18.6 — Public Settings menu + quality toggle (the listener selection UX).** New public-site
Settings-menu shell (app-bar trigger + MudBlazor menu + a settings-item abstraction + a
`PublicSiteSettings`/`ListenerSettings` object + the dark-mode-pattern persistence seam: `streamQuality`
cookie, a `DeepDrftPublic` prerender-read service, `PersistentComponentState` bridge, client cookie
service); the **quality toggle is its first occupant** (Low-data/Lossless, Opus default, capability-gated)
+ the CMS upload meter's **Post-Processing phase** (OQ6). Built design-for-adaptability so dark mode can
plug in later without restructuring (not migrated now). **Depends on 18.3** for the toggle; the menu shell
can be built ahead. *Splittable* (shell, then toggle) if Daniel wants the shell proven first.
**Dependency shape:** `18.1 → 18.2 → {18.3 ∥ 18.4} → 18.5`; 18.1 is the only cold-start wave.
**Phase-level: 18 precedes Phase 21.** **Open questions for Daniel (spec §6):** selection UX (recommend a
single global quality toggle); default policy (recommend Opus-by-default, capability-gated; defer
network-awareness); whether the choice is remembered + scope (recommend persisted cookie/`localStorage`,
the dark-mode precedent); per-upload Opus opt-out vs. always-on (recommend always-on); Ogg-vs-CAF/WebM
container (recommend Ogg Opus as directed); transcode execution model (background/queued — a track is
lossless-only briefly until its Opus finishes; confirm acceptable). None block 18.1.
**Dependency shape:** `18.1 → 18.2 → {18.3 ∥ 18.4} → 18.5`; `18.6 ∥` (needs 18.3 for the live toggle);
18.1 is the only cold-start wave. **Phase-level: 18 precedes Phase 21** (windowed refill consumes the Phase
18 seek-index resolver). **OQ1OQ6 RESOLVED (above); OQ7 (seek-index granularity, recommend ~12 s buckets)
is a non-blocking tuning steer.** None block 18.1.
---
@@ -539,9 +571,12 @@ endpoint, no schema change.
derived Ogg Opus 320 low-data path, Phase 18) is a prerequisite that comes first; windowing must work
across **both** delivery formats. Phase 21's C5 invariant already anticipated this ("must not foreclose
MP3/FLAC"); **Opus is now the concrete VBR/paged driver** — windowing an Opus stream uses the decoder's
*approximate* byte↔time mapping (`OpusFormatDecoder.calculateByteOffset`, Ogg-page interpolation), not
the exact CBR-WAV `byteRate` math. Build the window machinery format-agnostically so it inherits Opus
for free.
**accurate index-based** byte↔time mapping (`OpusFormatDecoder.calculateByteOffset`, a binary search in the
Phase 18 precomputed seek index — *not* the exact CBR-WAV `byteRate` math, and *not* approximate page
interpolation: VBR-safe and exact, per the Phase 18 seek-model resolution 2026-06-23). The windowed refill
controller calls the **same** index resolver an explicit seek does, and a window opening away from byte 0
still decodes via the Phase 18 sidecar setup header. Build the window machinery format-agnostically so it
inherits Opus for free.
The network path already streams in adaptive 1664 KB chunks. The accumulation is on the **decode
side**: `PlaybackScheduler` holds an `AudioBuffer[]` it **never evicts** ("Supports pause/resume/seek by