docs: record streaming raw-queue and per-chunk memory bounds
Document the three-layer memory bound (raw queue, decoded queue, network) in the streaming seam after the HW-accel-off OOM fix landed on streaming-overhaul.
This commit is contained in:
@@ -66,11 +66,16 @@ If step 1 succeeds and step 2 fails, audio is orphaned in the vault (no rollback
|
|||||||
|
|
||||||
The player is not fetch-then-play:
|
The player is not fetch-then-play:
|
||||||
|
|
||||||
1. Client calls `GET api/track/{id}` on DeepDrftContent and receives WAV bytes as a stream (`HttpCompletionOption.ResponseHeadersRead`).
|
1. Client calls `GET api/track/{id}` on DeepDrftContent and receives the first bounded segment (`Range: bytes=0-{SegmentSizeBytes-1}`, 4 MB) via `HttpCompletionOption.ResponseHeadersRead`.
|
||||||
2. `StreamingAudioPlayerService` reads in adaptive 16–64 KB chunks, pushes each via `AudioInteropService.processStreamingChunk`.
|
2. `StreamingAudioPlayerService` reads in adaptive 16–64 KB chunks within each segment, pushes each via `AudioInteropService.processStreamingChunk`.
|
||||||
3. TypeScript `StreamDecoder` parses WAV header, decodes chunks to `AudioBuffer`s. `PlaybackScheduler` schedules them on a Web Audio graph.
|
3. TypeScript `StreamDecoder` parses the format header, decodes chunks to `AudioBuffer`s. `PlaybackScheduler` schedules them on a Web Audio graph.
|
||||||
4. Playback starts as soon as a min buffer is queued; UI duration from parsed header (not waiting for full file).
|
4. Playback starts as soon as a min buffer is queued; UI duration from parsed header (not waiting for full file).
|
||||||
5. **Seek beyond buffer**: if seek target is past what's decoded, client issues `GET api/track/{id}` with `Range: bytes={byteOffset}-`. Server streams raw bytes from that file-absolute offset with a `206 Partial Content` response. Player retains the parsed WAV header and feeds the raw PCM continuation into the existing decode pipeline.
|
5. **Seek beyond buffer**: if seek target is past what's decoded, `StreamingAudioPlayerService` issues a new bounded segment starting at the seek byte offset. Server responds 206; player retains the parsed header and feeds the raw continuation into the existing decode pipeline.
|
||||||
|
|
||||||
|
**Memory bounding (three complementary layers, all required):**
|
||||||
|
- **Raw-queue bound (`StreamDecoder`):** `releaseConsumedChunks()` front-compacts `rawChunks` after each aligned segment is decoded, using a `discardedBytes` absolute cursor so all offsets remain absolute even as the array's front moves. Without this, a long WAV (e.g. a 92-min mix ≈ 970 MB raw) accumulates its entire decoded-from body in `rawChunks` regardless of the decode-side bounds below.
|
||||||
|
- **Decoded-queue bound (`PlaybackScheduler`):** `evictPlayedBuffers()` discards already-played `AudioBuffer`s, capping the scheduler's forward fill to a 96 MB ceiling (Phase 21.2). Decoded PCM is larger than source (Web Audio uses 32-bit float — a 16-bit stereo WAV roughly doubles; Opus decodes to the same float footprint).
|
||||||
|
- **Network bound (segmented fetch, Phase 21 Direction B):** The forward stream is fetched as sequential bounded `bytes=cursor-{cursor+4MB-1}` Range requests via `RunSegmentedStreamAsync`; the next segment is fetched only after `DrainBackpressureAsync` confirms the scheduler is below low-water. Because each segment is fully consumed before the next is issued, the browser holds at most ~one segment of raw bytes. A per-chunk drain inside the segment loop (gated on `_streamingPlaybackStarted` so it cannot deadlock first-audio) additionally prevents high-density codecs (e.g. Opus, where a 4 MB segment is ~100 s of audio) from decoding the whole segment eagerly ahead of the playhead before the inter-segment gate runs.
|
||||||
|
|
||||||
Keep this seam clean — it is the most architecturally load-bearing part of the playback path.
|
Keep this seam clean — it is the most architecturally load-bearing part of the playback path.
|
||||||
|
|
||||||
|
|||||||
@@ -30,6 +30,16 @@ Newest entries at the top. Group by phase/wave header (mirroring `PLAN.md` / `CM
|
|||||||
|
|
||||||
- **Design memo:** `product-notes/phase-21-windowed-streaming-buffer.md` (note: the spec recommended Direction A; the as-built pivot to Direction B is annotated at the top of that doc and in §3.2/§3.3).
|
- **Design memo:** `product-notes/phase-21-windowed-streaming-buffer.md` (note: the spec recommended Direction A; the as-built pivot to Direction B is annotated at the top of that doc and in §3.2/§3.3).
|
||||||
|
|
||||||
|
### Phase 21 — Post-landing: raw-queue and per-chunk bounds (landed 2026-06-24)
|
||||||
|
|
||||||
|
**Landed:** 2026-06-24 on `streaming-overhaul`. Fixes a Chrome tab crash (main-process OOM with hardware acceleration OFF) on long-track playback. Two unbounded memory queues remained after Phase 21 landed; this patch adds complementary bounds on both.
|
||||||
|
|
||||||
|
1. **`StreamingAudioPlayerService` — per-chunk drain inside segments.** Phase 21 Direction B's inter-segment back-pressure gate is matched to WAV byte density (~24 s per 4 MB segment), but not to Opus: at 320 kbps a 4 MB segment is ~100 s of decodable audio. The inner chunk loop had no fill check, so the entire segment decoded eagerly ahead of the playhead — piling decoded f32 PCM into main-process RAM before the inter-segment gate ever ran. Fix: an extracted `DrainBackpressureAsync` helper (shared with the inter-segment gate) is called per chunk once the scheduler is over high-water, gated on `_streamingPlaybackStarted` so it cannot deadlock first-audio (the playhead doesn't advance until playback starts, so draining to low-water would block indefinitely before the first buffer plays). Also added load-generation diagnostic logging (`_loadGeneration`) to confirm single-load-per-play-action in-browser.
|
||||||
|
|
||||||
|
2. **`StreamDecoder.ts` — `releaseConsumedChunks()` front-compaction of `rawChunks`.** `rawChunks` retained the entire decoded-from stream body because consumed chunks were never released — a 92-min WAV mix (≈ 970 MB raw) accumulated all of it in the browser. Phase 21.2's 96 MB decoded-side ceiling does not reach this raw queue. Fix: `releaseConsumedChunks()` walks `rawChunks` after each aligned segment is decoded and splices off all fully-consumed chunks (any chunk whose end ≤ `processedBytes`), advancing `discardedBytes` (an absolute cursor into the logical stream) so `extractAlignedData` can continue walking from the correct position even though `rawChunks[0]` no longer begins at byte 0. This is the raw-side analogue of `PlaybackScheduler.evictPlayedBuffers` (the decoded side).
|
||||||
|
|
||||||
|
Together with the existing decoded-side 96 MB ceiling (Phase 21.2) and the segmented fetch (Phase 21 Direction B), these complete the three-layer memory bound: raw queue, decoded queue, and network.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Phase 18 — Opus Low-Data Streaming (landed 2026-06-23)
|
## Phase 18 — Opus Low-Data Streaming (landed 2026-06-23)
|
||||||
|
|||||||
Reference in New Issue
Block a user