docs: record Phase 18 (Opus low-data streaming) in COMPLETED; stage PLAN for Phase 21
This commit is contained in:
@@ -443,119 +443,9 @@ not the same work; this phase does not satisfy or depend on that one.
|
||||
|
||||
---
|
||||
|
||||
## Phase 18 — Opus Low-Data Streaming (dual-format lossless + Opus delivery)
|
||||
## Phase 18 — Opus Low-Data Streaming (Completed)
|
||||
|
||||
The concrete realization of the long-deferred **"Non-WAV formats"** intent (`CONTEXT.md §5`). Daniel's
|
||||
direction (2026-06-23): **two delivery formats per track — the existing lossless WAV path, and a new
|
||||
low-data Ogg Opus (fullband, 320 kbps) path — so the listener gets a choice, with Opus the
|
||||
bandwidth-friendly default-candidate.** Lossless streaming becomes *optional*, not the only path. The
|
||||
bespoke Web Audio decode→schedule graph is **retained by deliberate choice** — Opus feeds the same
|
||||
`IFormatDecoder` seam, not an HTML `<media>` element or MSE (the decision shared with Phase 21 OQ5).
|
||||
**Sequenced BEFORE Phase 21** — windowing must work across both formats. Surfaces: ingest/preprocessing
|
||||
in `DeepDrftContent` (`AudioProcessor`/router/`WaveformProfileService`) + `DeepDrftAPI`
|
||||
(`UnifiedTrackService.UploadAsync`, replace-audio); delivery/decode in `DeepDrftAPI` (stream endpoint +
|
||||
`Range`) + `DeepDrftPublic` proxy + `DeepDrftPublic.Client` player stack + `DeepDrftPublic/Interop/audio`
|
||||
TS decoders. Full design, the three directions with SOLID/road-not-taken rationale, the storage and
|
||||
delivery options, the Opus decoder + seek math, acceptance criteria, open questions, and wave
|
||||
decomposition: `product-notes/phase-18-opus-low-data-streaming.md`.
|
||||
|
||||
**Much further along than the backlog line implies (verified 2026-06-23).** The multi-format *substrate*
|
||||
already exists on both sides: the producer-side `AudioProcessorRouter` routes `.wav`/`.mp3`/`.flac` and
|
||||
`TrackContentService.AddTrackAsync` is format-agnostic (it **stores originals**, no transcode); the
|
||||
decoder-side `AudioPlayer.createFormatDecoder` is a **wired** strategy registry dispatching on
|
||||
`Content-Type` (WAV/MP3/FLAC decoders all present — correcting the Phase 21 spec's stale
|
||||
"implemented-not-wired" note). **The actual gap is Daniel's specific ask:** (1) a **transcode-at-ingest**
|
||||
step that *derives* an Opus 320 artifact per track (nothing derives Opus today), and (2) a **per-format
|
||||
delivery selection** so one track serves as either WAV or Opus on request.
|
||||
|
||||
**Open questions RESOLVED (Daniel, 2026-06-23).** OQ1 selection UX → **global, via a new public-site
|
||||
Settings menu** (not a bare app-bar control); OQ2 default → **Opus by default, capability-gated** (defer
|
||||
network-awareness); OQ3 remembered → **persisted via the dark-mode seam** (cookie → prerender-read →
|
||||
`PersistentComponentState` → client cookie service); OQ4 → **always-on Opus + Backfill-Opus**; OQ5 →
|
||||
**Ogg Opus**; OQ6 transcode model → **background job after the file is available, with a visible
|
||||
Post-Processing phase on the CMS upload meter.** OQ7 (seek-index granularity) → **0.5 s (half-second)
|
||||
buckets** (~115 KB index for a 1-hour mix).
|
||||
|
||||
**Architectural spine — a derived artifact set + a delivery param + one new decoder + a precomputed
|
||||
accurate seek index; leaf implementations only, zero changes to existing format code (the strong OCP
|
||||
signal).** Transcode is a new processor sibling in `DeepDrftContent`, invoked post-store alongside
|
||||
`WaveformProfileService` **as a background job** (a 1 GB WAV transcode must not block the upload; the source
|
||||
is stored and the track plays lossless *first*, then Opus is derived) — mirroring the landed waveform-datum
|
||||
pattern (derive at ingest, regenerate via a CMS bulk action + ApiKey endpoint). The Opus bytes are a
|
||||
**derived artifact** stored like the high-res waveform datum (recommend a dedicated `track-opus` vault, the
|
||||
`track-waveforms` precedent; final call staff-engineer's). Delivery adds a **`?format=opus|lossless` param**
|
||||
(mirroring the existing `offset` param threading through `TrackProxyController`) resolved server-side to the
|
||||
right artifact + content-type, with a **lossless fallback** when no Opus artifact exists (additive, never
|
||||
404/silence). The player gains one `OpusFormatDecoder` (`IFormatDecoder`): Ogg-page-aligned segmenting
|
||||
(`OggS` scan — the FLAC frame-sync analogue) and `OpusHead`/`OpusTags` setup-bytes carry (the FLAC
|
||||
`streamInfoBytes` analogue). **Browser constraint flagged:** Ogg-Opus `decodeAudioData` is Safari-18.4+ only
|
||||
(Chrome/FF long-standing), so the Opus default is **capability-gated** — fall back to the universal lossless
|
||||
path on browsers that can't decode it.
|
||||
|
||||
**VBR-safe ACCURATE seeking (Daniel, 2026-06-23 — supersedes the earlier "approximate" hand-wave).** Raw
|
||||
byte-offset seek and rough page interpolation are inadequate for VBR Opus — there is no linear time↔byte
|
||||
relationship. The fix is an **accurate transfer function built at transcode time** (the one moment the
|
||||
whole encoded stream is walked): a precomputed **seek index** mapping Ogg-page `granulepos` (48 kHz sample
|
||||
counts → time) → exact byte offset (**0.5 s buckets** snapped to page starts — OQ7; ~7,200 entries ×
|
||||
16 bytes ≈ ~115 KB for a 1-hour mix). The decode **setup header** (`OpusHead`/`OpusTags`, needed to decode any mid-stream slice) is made
|
||||
available too. Recommended concrete design: **one sidecar artifact per track = `[setup header][seek
|
||||
index]`, built at transcode, stored beside the Opus bytes, fetched once on track load**, parsed into
|
||||
`OpusSeekData`. Client seek flow: `calculateByteOffset(t)` binary-searches the index for the exact page
|
||||
offset → `Range: bytes=X-` fetch (landed Phase 4 primitive, unchanged) → prepend the cached setup header →
|
||||
decode → fine re-sync to `t` within the bucket. **The listener lands at the correct time, not
|
||||
approximately** (AC9), **without** the full PCM in memory — so it composes with Phase 21 windowed refill,
|
||||
which calls the **same** index resolver. The earlier "approximate page-interpolation" language is rejected.
|
||||
|
||||
**Constraints/invariants:** keep the bespoke graph (no MSE); preprocessing is **additive** (WAV path
|
||||
untouched, byte-for-byte; a track with no Opus artifact still plays losslessly); reuse the landed
|
||||
`Range`/offset seek path; no format branches leak outside the new decoder + one selection arm + the
|
||||
transcode/delivery seam; transcode failure must not block ingest; format selection is a delivery-time
|
||||
decision resolving one `EntryKey` to one of two artifacts (one source, two views — **not** a second
|
||||
`TrackEntity` row, which would fracture share/queue/play-count/release identity).
|
||||
|
||||
Sequenced as six waves. `18.1 → 18.2 → {18.3, 18.4} → 18.5`, with `18.6` (Settings menu) able to run in
|
||||
parallel (it needs only 18.3's format mechanism before its toggle is live). **18.1 (ingest transcode +
|
||||
seek-index + setup-header derived artifacts) is the cold-start prerequisite** — nothing downstream has
|
||||
bytes to serve, decode, or seek against until those artifacts exist.
|
||||
|
||||
- **18.1 — Ingest transcode + seek-index + setup-header (cold-start; load-bearing).** New
|
||||
`OpusTranscodeService`/processor in `DeepDrftContent`, invoked post-store from
|
||||
`UnifiedTrackService.UploadAsync` alongside `WaveformProfileService` **as a background job** (OQ6);
|
||||
produces Ogg Opus fullband 320; **walks the encoded stream once to build the granule→byte seek index and
|
||||
extract the `OpusHead`/`OpusTags` setup header**; stores the Opus bytes **and** the combined seek/setup
|
||||
**sidecar** as derived artifacts (recommend a `track-opus` vault). Failure-tolerant. **Independent of the
|
||||
delivery/decoder waves — can begin immediately.**
|
||||
- **18.2 — Storage + lookup contract.** The derived-artifact key/vault convention (Opus bytes + sidecar) +
|
||||
server-side "given `EntryKey` + format, return the right `AudioBinary` + content-type (+ the sidecar),"
|
||||
including the lossless fallback. **Depends on 18.1.**
|
||||
- **18.3 — Delivery: `?format=opus|lossless` param + sidecar serving + proxy threading.** On the
|
||||
`DeepDrftAPI` stream endpoint (resolves via 18.2), forwarded through `TrackProxyController` (mirror
|
||||
`offset`), `Range` serving the chosen artifact; **plus serving the seek/setup sidecar**; player sends the
|
||||
format param via `TrackMediaClient`. **Depends on 18.2; parallel-ok with 18.4.**
|
||||
- **18.4 — `OpusFormatDecoder` + index-based seek resolver in the player stack.** New `IFormatDecoder`
|
||||
(Ogg-page segmenting via `OggS` scan, `OpusHead`/`OpusTags` setup carry from the cached sidecar,
|
||||
**`calculateByteOffset` that binary-searches the precomputed seek index** — NOT interpolation — with an
|
||||
`OpusSeekData` accelerator holding the parsed index + setup bytes, and the one-time sidecar fetch+parse on
|
||||
track load) + one arm in `createFormatDecoder` on `audio/ogg`/`audio/opus`; capability detection for the
|
||||
lossless fallback. **Depends on 18.2; parallel-ok with 18.3.**
|
||||
- **18.5 — Backfill + replace-audio + end-to-end validation (incl. seek accuracy).** "Backfill Opus" CMS
|
||||
bulk action (third sibling to Generate-Profiles / Backfill-High-res), rebuilding Opus bytes + sidecar for
|
||||
existing tracks; replace-audio Opus + sidecar regeneration; the AC1–AC10 acceptance pass **including AC9
|
||||
(an Opus seek lands at the correct time, not approximately)** and the Phase-21 handshake (Opus windowable
|
||||
via the index resolver + sidecar setup header). **Depends on 18.1–18.4.**
|
||||
- **18.6 — Public Settings menu + quality toggle (the listener selection UX).** New public-site
|
||||
Settings-menu shell (app-bar trigger + MudBlazor menu + a settings-item abstraction + a
|
||||
`PublicSiteSettings`/`ListenerSettings` object + the dark-mode-pattern persistence seam: `streamQuality`
|
||||
cookie, a `DeepDrftPublic` prerender-read service, `PersistentComponentState` bridge, client cookie
|
||||
service); the **quality toggle is its first occupant** (Low-data/Lossless, Opus default, capability-gated)
|
||||
+ the CMS upload meter's **Post-Processing phase** (OQ6). Built design-for-adaptability so dark mode can
|
||||
plug in later without restructuring (not migrated now). **Depends on 18.3** for the toggle; the menu shell
|
||||
can be built ahead. *Splittable* (shell, then toggle) if Daniel wants the shell proven first.
|
||||
|
||||
**Dependency shape:** `18.1 → 18.2 → {18.3 ∥ 18.4} → 18.5`; `18.6 ∥` (needs 18.3 for the live toggle);
|
||||
18.1 is the only cold-start wave. **Phase-level: 18 precedes Phase 21** (windowed refill consumes the Phase
|
||||
18 seek-index resolver). **OQ1–OQ7 RESOLVED (above); OQ7 (seek-index granularity) = 0.5 s buckets.** None
|
||||
block 18.1.
|
||||
See `COMPLETED.md` for Phase 18 — dual-format lossless + Opus delivery, including the as-built WebCodecs decoder divergence — which landed on `streaming-overhaul` (2026-06-23).
|
||||
|
||||
---
|
||||
|
||||
@@ -567,9 +457,7 @@ plays without the whole decoded PCM accumulating in the browser. **Public listen
|
||||
(`DeepDrftPublic.Client` player stack + `DeepDrftPublic` TypeScript audio interop); no CMS, no API
|
||||
endpoint, no schema change.
|
||||
|
||||
**Sequenced AFTER Phase 18 (Opus Low-Data Streaming) — Daniel, 2026-06-23.** Format support (the
|
||||
derived Ogg Opus 320 low-data path, Phase 18) is a prerequisite that comes first; windowing must work
|
||||
across **both** delivery formats. Phase 21's C5 invariant already anticipated this ("must not foreclose
|
||||
**Phase 18 (Opus Low-Data Streaming) has landed — Phase 21 is the next pickup.** The derived Ogg Opus 320 low-data path (Phase 18, `COMPLETED.md`) is the prerequisite; windowing must work across **both** delivery formats. Phase 21's C5 invariant already anticipated this ("must not foreclose
|
||||
MP3/FLAC"); **Opus is now the concrete VBR/paged driver** — windowing an Opus stream uses the decoder's
|
||||
**accurate index-based** byte↔time mapping (`OpusFormatDecoder.calculateByteOffset`, a binary search in the
|
||||
Phase 18 precomputed seek index — *not* the exact CBR-WAV `byteRate` math, and *not* approximate page
|
||||
|
||||
Reference in New Issue
Block a user