docs: record streaming-stabilization arc (Opus + HW-accel-off fixes)

Document the back-pressure water marks, genuine end-of-playback gating, rebuffer hysteresis, 48kHz Opus pre-align, decodePressure auto-throttle, and HW-accel detection / lava default-off, plus the off-main-thread-decode fallback note.
This commit is contained in:
daniel-c-harvey
2026-06-26 10:59:48 -04:00
parent d7a373cdb0
commit 634eb611eb
3 changed files with 29 additions and 1 deletions
+16
View File
@@ -40,6 +40,22 @@ Newest entries at the top. Group by phase/wave header (mirroring `PLAN.md` / `CM
Together with the existing decoded-side 96 MB ceiling (Phase 21.2) and the segmented fetch (Phase 21 Direction B), these complete the three-layer memory bound: raw queue, decoded queue, and network.
### Phase 21 — Post-landing: streaming stabilization (landed 2026-06-26)
**Landed:** 2026-06-26 on `streaming-overhaul`. Resolves playback instability that surfaced specifically with Opus + hardware acceleration off: the Opus underrun/false-end cycle, visualizer starvation of WebCodecs decode, and an OOM path from mid-decode `AudioContext` teardown. All changes are TypeScript-only (`DeepDrftPublic/Interop/audio/` + `DeepDrftPublic/Interop/visualizer/`) plus the C# service that calls them; no API, no schema, no CMS change.
1. **Back-pressure water marks widened to 60s high / 30s low.** `PlaybackScheduler.ts` `DEFAULT_FORWARD_HIGH_WATER_SECONDS` / `DEFAULT_FORWARD_LOW_WATER_SECONDS` doubled from 30/15. The 96 MB decoded-byte ceiling is unchanged and remains the hard OOM bound; production pauses on `lookahead ≥ high OR bytes > cap`, whichever first. The wider time window absorbs per-packet WebCodecs decode jitter (Opus 48 kHz stereo ≈ 0.37 MB/s → 60 s ≈ 23 MB — well within the cap) without false back-pressure on the Opus decode ramp.
2. **End-of-playback gated on genuine completion (`streamComplete` + `underrun_`).** `PlaybackScheduler` now distinguishes a drained-but-still-streaming queue (startup/underrun gap) from a truly finished track (stream complete AND queue drained). `streamComplete` (set by `setStreamComplete`, called from `AudioPlayer.markStreamComplete`) is the discriminator. When the queue drains while `streamComplete` is false the scheduler parks in `underrun_` state (anchors frozen, `isActive_` false) and resumes from `scheduleNewBuffers` when buffers arrive. `finishPlayback()` is the single genuine-end path, reached only when both conditions hold simultaneously. Previously a drained Opus queue during the WebCodecs startup ramp fired `onPlaybackEnded`, nulling Duration and advancing the queue.
3. **Rebuffer hysteresis — 1s decoded-lead gate.** `hasMinimumPlaybackLead()` gates Opus playback START and underrun RESUME (`DEFAULT_MIN_PLAYBACK_LEAD_SECONDS = 1.0`), preventing the decode-ramp thrash (resume on a 20 ms Opus packet → drain immediately → re-park → repeat). `streamComplete` overrides the gate so a short tail plays out rather than parking forever. WAV keeps its `hasMinimumBuffers(6)` buffer-count start gate unchanged — large synchronous WAV segments rarely underrun and that gate's character must not change. **Complete-without-start force-start:** `StreamingAudioPlayerService.TryStartPlaybackAsync()` is called after `MarkStreamCompleteAsync` when `_streamingPlaybackStarted` is still false, preventing an ultra-short Opus track (total audio below the 1s lead) from sitting loaded-but-not-playing.
4. **Opus `AudioContext` pre-aligned to 48 kHz.** `AudioPlayer.initializeStreaming` calls `contextManager.recreateWithSampleRate(OPUS_SAMPLE_RATE)` when the content type resolves as Opus, before any bytes flow. The context is already at 48 kHz when the decoder's own lazy `recreateWithSampleRate` check runs, so it short-circuits (no-op). Eliminates the mid-decode context teardown-and-rebuild that previously double-decoded the stream and OOM'd the tab under HW accel off.
5. **`decodePressure` singleton + `WaveformVisualizer` auto-throttle (~15 fps under pressure).** `DeepDrftPublic/Interop/audio/decodePressure.ts` exports a process-wide `DecodePressureSignal` singleton shared between the audio pipeline (producer: `OpusStreamDecoder` yield-cap events and `PlaybackScheduler` underrun-parking events) and the visualizer (consumer). Engages only on sustained stress (≥ 5 `report()` calls within a 2500 ms window) and holds for a 1000 ms minimum dwell before releasing, suppressing lone startup-ramp blips. `WaveformVisualizer.ts` checks `decodePressure.isUnderPressure()` each rAF frame; when engaged it skips frames so at most one renders per `PRESSURE_THROTTLE_FRAME_MS` (`1000/15` ms, ~15 fps), cutting main-thread WebGL software-render + physics cost so WebCodecs decode recovers. A no-op under HW accel — healthy decode never calls `report()` and `isUnderPressure()` stays false indefinitely.
6. **Hardware-acceleration detection + `ApplyCapabilityDefault`.** `DeepDrftPublic/Interop/visualizer/hwAccel.ts` probes `WEBGL_debug_renderer_info.UNMASKED_RENDERER_WEBGL` for known software-renderer signatures (SwiftShader, llvmpipe, softpipe, Microsoft Basic Render Driver, Mesa Offscreen, "software"). Absence of the debug extension is treated as accelerated — the probe disables lava only on positive evidence of software rendering, not on uncertainty. `WaveformVisualizerControlState.ApplyCapabilityDefault(bool hardwareAccelerated)` is called once per session by the visualizer bridge on first interactive render: when `hardwareAccelerated` is false it sets `LavaEnabled = false` (the expensive main-thread subsystem) while leaving `WaveformEnabled` on, coerces Theater Mode, and raises `Changed` once so all observers see the default in a single cycle. Guarded by `_capabilityDefaultApplied` so it runs at most once per session and never overrides an explicit in-session user toggle. With HW accel present the call is a no-op.
---
## Phase 18 — Opus Low-Data Streaming (landed 2026-06-23)