docs: record Phase 18 (Opus low-data streaming) in COMPLETED; stage PLAN for Phase 21

This commit is contained in:
daniel-c-harvey
2026-06-23 21:48:39 -04:00
parent 8902ce4d63
commit bbcf8be677
2 changed files with 31 additions and 115 deletions
+3 -115
View File
@@ -443,119 +443,9 @@ not the same work; this phase does not satisfy or depend on that one.
---
## Phase 18 — Opus Low-Data Streaming (dual-format lossless + Opus delivery)
## Phase 18 — Opus Low-Data Streaming (Completed)
The concrete realization of the long-deferred **"Non-WAV formats"** intent (`CONTEXT.md §5`). Daniel's
direction (2026-06-23): **two delivery formats per track — the existing lossless WAV path, and a new
low-data Ogg Opus (fullband, 320 kbps) path — so the listener gets a choice, with Opus the
bandwidth-friendly default-candidate.** Lossless streaming becomes *optional*, not the only path. The
bespoke Web Audio decode→schedule graph is **retained by deliberate choice** — Opus feeds the same
`IFormatDecoder` seam, not an HTML `<media>` element or MSE (the decision shared with Phase 21 OQ5).
**Sequenced BEFORE Phase 21** — windowing must work across both formats. Surfaces: ingest/preprocessing
in `DeepDrftContent` (`AudioProcessor`/router/`WaveformProfileService`) + `DeepDrftAPI`
(`UnifiedTrackService.UploadAsync`, replace-audio); delivery/decode in `DeepDrftAPI` (stream endpoint +
`Range`) + `DeepDrftPublic` proxy + `DeepDrftPublic.Client` player stack + `DeepDrftPublic/Interop/audio`
TS decoders. Full design, the three directions with SOLID/road-not-taken rationale, the storage and
delivery options, the Opus decoder + seek math, acceptance criteria, open questions, and wave
decomposition: `product-notes/phase-18-opus-low-data-streaming.md`.
**Much further along than the backlog line implies (verified 2026-06-23).** The multi-format *substrate*
already exists on both sides: the producer-side `AudioProcessorRouter` routes `.wav`/`.mp3`/`.flac` and
`TrackContentService.AddTrackAsync` is format-agnostic (it **stores originals**, no transcode); the
decoder-side `AudioPlayer.createFormatDecoder` is a **wired** strategy registry dispatching on
`Content-Type` (WAV/MP3/FLAC decoders all present — correcting the Phase 21 spec's stale
"implemented-not-wired" note). **The actual gap is Daniel's specific ask:** (1) a **transcode-at-ingest**
step that *derives* an Opus 320 artifact per track (nothing derives Opus today), and (2) a **per-format
delivery selection** so one track serves as either WAV or Opus on request.
**Open questions RESOLVED (Daniel, 2026-06-23).** OQ1 selection UX → **global, via a new public-site
Settings menu** (not a bare app-bar control); OQ2 default → **Opus by default, capability-gated** (defer
network-awareness); OQ3 remembered → **persisted via the dark-mode seam** (cookie → prerender-read →
`PersistentComponentState` → client cookie service); OQ4 → **always-on Opus + Backfill-Opus**; OQ5 →
**Ogg Opus**; OQ6 transcode model → **background job after the file is available, with a visible
Post-Processing phase on the CMS upload meter.** OQ7 (seek-index granularity) → **0.5 s (half-second)
buckets** (~115 KB index for a 1-hour mix).
**Architectural spine — a derived artifact set + a delivery param + one new decoder + a precomputed
accurate seek index; leaf implementations only, zero changes to existing format code (the strong OCP
signal).** Transcode is a new processor sibling in `DeepDrftContent`, invoked post-store alongside
`WaveformProfileService` **as a background job** (a 1 GB WAV transcode must not block the upload; the source
is stored and the track plays lossless *first*, then Opus is derived) — mirroring the landed waveform-datum
pattern (derive at ingest, regenerate via a CMS bulk action + ApiKey endpoint). The Opus bytes are a
**derived artifact** stored like the high-res waveform datum (recommend a dedicated `track-opus` vault, the
`track-waveforms` precedent; final call staff-engineer's). Delivery adds a **`?format=opus|lossless` param**
(mirroring the existing `offset` param threading through `TrackProxyController`) resolved server-side to the
right artifact + content-type, with a **lossless fallback** when no Opus artifact exists (additive, never
404/silence). The player gains one `OpusFormatDecoder` (`IFormatDecoder`): Ogg-page-aligned segmenting
(`OggS` scan — the FLAC frame-sync analogue) and `OpusHead`/`OpusTags` setup-bytes carry (the FLAC
`streamInfoBytes` analogue). **Browser constraint flagged:** Ogg-Opus `decodeAudioData` is Safari-18.4+ only
(Chrome/FF long-standing), so the Opus default is **capability-gated** — fall back to the universal lossless
path on browsers that can't decode it.
**VBR-safe ACCURATE seeking (Daniel, 2026-06-23 — supersedes the earlier "approximate" hand-wave).** Raw
byte-offset seek and rough page interpolation are inadequate for VBR Opus — there is no linear time↔byte
relationship. The fix is an **accurate transfer function built at transcode time** (the one moment the
whole encoded stream is walked): a precomputed **seek index** mapping Ogg-page `granulepos` (48 kHz sample
counts → time) → exact byte offset (**0.5 s buckets** snapped to page starts — OQ7; ~7,200 entries ×
16 bytes ≈ ~115 KB for a 1-hour mix). The decode **setup header** (`OpusHead`/`OpusTags`, needed to decode any mid-stream slice) is made
available too. Recommended concrete design: **one sidecar artifact per track = `[setup header][seek
index]`, built at transcode, stored beside the Opus bytes, fetched once on track load**, parsed into
`OpusSeekData`. Client seek flow: `calculateByteOffset(t)` binary-searches the index for the exact page
offset → `Range: bytes=X-` fetch (landed Phase 4 primitive, unchanged) → prepend the cached setup header →
decode → fine re-sync to `t` within the bucket. **The listener lands at the correct time, not
approximately** (AC9), **without** the full PCM in memory — so it composes with Phase 21 windowed refill,
which calls the **same** index resolver. The earlier "approximate page-interpolation" language is rejected.
**Constraints/invariants:** keep the bespoke graph (no MSE); preprocessing is **additive** (WAV path
untouched, byte-for-byte; a track with no Opus artifact still plays losslessly); reuse the landed
`Range`/offset seek path; no format branches leak outside the new decoder + one selection arm + the
transcode/delivery seam; transcode failure must not block ingest; format selection is a delivery-time
decision resolving one `EntryKey` to one of two artifacts (one source, two views — **not** a second
`TrackEntity` row, which would fracture share/queue/play-count/release identity).
Sequenced as six waves. `18.1 → 18.2 → {18.3, 18.4} → 18.5`, with `18.6` (Settings menu) able to run in
parallel (it needs only 18.3's format mechanism before its toggle is live). **18.1 (ingest transcode +
seek-index + setup-header derived artifacts) is the cold-start prerequisite** — nothing downstream has
bytes to serve, decode, or seek against until those artifacts exist.
- **18.1 — Ingest transcode + seek-index + setup-header (cold-start; load-bearing).** New
`OpusTranscodeService`/processor in `DeepDrftContent`, invoked post-store from
`UnifiedTrackService.UploadAsync` alongside `WaveformProfileService` **as a background job** (OQ6);
produces Ogg Opus fullband 320; **walks the encoded stream once to build the granule→byte seek index and
extract the `OpusHead`/`OpusTags` setup header**; stores the Opus bytes **and** the combined seek/setup
**sidecar** as derived artifacts (recommend a `track-opus` vault). Failure-tolerant. **Independent of the
delivery/decoder waves — can begin immediately.**
- **18.2 — Storage + lookup contract.** The derived-artifact key/vault convention (Opus bytes + sidecar) +
server-side "given `EntryKey` + format, return the right `AudioBinary` + content-type (+ the sidecar),"
including the lossless fallback. **Depends on 18.1.**
- **18.3 — Delivery: `?format=opus|lossless` param + sidecar serving + proxy threading.** On the
`DeepDrftAPI` stream endpoint (resolves via 18.2), forwarded through `TrackProxyController` (mirror
`offset`), `Range` serving the chosen artifact; **plus serving the seek/setup sidecar**; player sends the
format param via `TrackMediaClient`. **Depends on 18.2; parallel-ok with 18.4.**
- **18.4 — `OpusFormatDecoder` + index-based seek resolver in the player stack.** New `IFormatDecoder`
(Ogg-page segmenting via `OggS` scan, `OpusHead`/`OpusTags` setup carry from the cached sidecar,
**`calculateByteOffset` that binary-searches the precomputed seek index** — NOT interpolation — with an
`OpusSeekData` accelerator holding the parsed index + setup bytes, and the one-time sidecar fetch+parse on
track load) + one arm in `createFormatDecoder` on `audio/ogg`/`audio/opus`; capability detection for the
lossless fallback. **Depends on 18.2; parallel-ok with 18.3.**
- **18.5 — Backfill + replace-audio + end-to-end validation (incl. seek accuracy).** "Backfill Opus" CMS
bulk action (third sibling to Generate-Profiles / Backfill-High-res), rebuilding Opus bytes + sidecar for
existing tracks; replace-audio Opus + sidecar regeneration; the AC1AC10 acceptance pass **including AC9
(an Opus seek lands at the correct time, not approximately)** and the Phase-21 handshake (Opus windowable
via the index resolver + sidecar setup header). **Depends on 18.118.4.**
- **18.6 — Public Settings menu + quality toggle (the listener selection UX).** New public-site
Settings-menu shell (app-bar trigger + MudBlazor menu + a settings-item abstraction + a
`PublicSiteSettings`/`ListenerSettings` object + the dark-mode-pattern persistence seam: `streamQuality`
cookie, a `DeepDrftPublic` prerender-read service, `PersistentComponentState` bridge, client cookie
service); the **quality toggle is its first occupant** (Low-data/Lossless, Opus default, capability-gated)
+ the CMS upload meter's **Post-Processing phase** (OQ6). Built design-for-adaptability so dark mode can
plug in later without restructuring (not migrated now). **Depends on 18.3** for the toggle; the menu shell
can be built ahead. *Splittable* (shell, then toggle) if Daniel wants the shell proven first.
**Dependency shape:** `18.1 → 18.2 → {18.3 ∥ 18.4} → 18.5`; `18.6 ∥` (needs 18.3 for the live toggle);
18.1 is the only cold-start wave. **Phase-level: 18 precedes Phase 21** (windowed refill consumes the Phase
18 seek-index resolver). **OQ1OQ7 RESOLVED (above); OQ7 (seek-index granularity) = 0.5 s buckets.** None
block 18.1.
See `COMPLETED.md` for Phase 18 — dual-format lossless + Opus delivery, including the as-built WebCodecs decoder divergence — which landed on `streaming-overhaul` (2026-06-23).
---
@@ -567,9 +457,7 @@ plays without the whole decoded PCM accumulating in the browser. **Public listen
(`DeepDrftPublic.Client` player stack + `DeepDrftPublic` TypeScript audio interop); no CMS, no API
endpoint, no schema change.
**Sequenced AFTER Phase 18 (Opus Low-Data Streaming) — Daniel, 2026-06-23.** Format support (the
derived Ogg Opus 320 low-data path, Phase 18) is a prerequisite that comes first; windowing must work
across **both** delivery formats. Phase 21's C5 invariant already anticipated this ("must not foreclose
**Phase 18 (Opus Low-Data Streaming) has landed — Phase 21 is the next pickup.** The derived Ogg Opus 320 low-data path (Phase 18, `COMPLETED.md`) is the prerequisite; windowing must work across **both** delivery formats. Phase 21's C5 invariant already anticipated this ("must not foreclose
MP3/FLAC"); **Opus is now the concrete VBR/paged driver** — windowing an Opus stream uses the decoder's
**accurate index-based** byte↔time mapping (`OpusFormatDecoder.calculateByteOffset`, a binary search in the
Phase 18 precomputed seek index — *not* the exact CBR-WAV `byteRate` math, and *not* approximate page