daniel/deepdrft
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 13 Wiki Activity

docs: record streaming raw-queue and per-chunk memory bounds

Browse Source 
Document the three-layer memory bound (raw queue, decoded queue, network) in the streaming seam after the HW-accel-off OOM fix landed on streaming-overhaul.
This commit is contained in:
daniel-c-harvey
2026-06-24 22:09:01 -04:00
parent 8206c0bdaf
commit 7adc35dd5d
2 changed files with 19 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+9 -4
View File
@@ -66,11 +66,16 @@ If step 1 succeeds and step 2 fails, audio is orphaned in the vault (no rollback
The player is not fetch-then-play:
1. Client calls `GET api/track/{id}` on DeepDrftContent and receives WAV bytes as a stream (`HttpCompletionOption.ResponseHeadersRead`).
2. `StreamingAudioPlayerService` reads in adaptive 16–64 KB chunks, pushes each via `AudioInteropService.processStreamingChunk`.
3. TypeScript `StreamDecoder` parses WAV header, decodes chunks to `AudioBuffer`s. `PlaybackScheduler` schedules them on a Web Audio graph.
1. Client calls `GET api/track/{id}` on DeepDrftContent and receives the first bounded segment (`Range: bytes=0-{SegmentSizeBytes-1}`, 4 MB) via `HttpCompletionOption.ResponseHeadersRead`.
2. `StreamingAudioPlayerService` reads in adaptive 16–64 KB chunks within each segment, pushes each via `AudioInteropService.processStreamingChunk`.
3. TypeScript `StreamDecoder` parses the format header, decodes chunks to `AudioBuffer`s. `PlaybackScheduler` schedules them on a Web Audio graph.
4. Playback starts as soon as a min buffer is queued; UI duration from parsed header (not waiting for full file).
5. **Seek beyond buffer**: if seek target is past what's decoded, client issues `GET api/track/{id}` with `Range: bytes={byteOffset}-`. Server streams raw bytes from that file-absolute offset with a `206 Partial Content` response. Player retains the parsed WAV header and feeds the raw PCM continuation into the existing decode pipeline.
5. **Seek beyond buffer**: if seek target is past what's decoded, `StreamingAudioPlayerService` issues a new bounded segment starting at the seek byte offset. Server responds 206; player retains the parsed header and feeds the raw continuation into the existing decode pipeline.
**Memory bounding (three complementary layers, all required):**
- **Raw-queue bound (`StreamDecoder`):** `releaseConsumedChunks()` front-compacts `rawChunks` after each aligned segment is decoded, using a `discardedBytes` absolute cursor so all offsets remain absolute even as the array's front moves. Without this, a long WAV (e.g. a 92-min mix ≈ 970 MB raw) accumulates its entire decoded-from body in `rawChunks` regardless of the decode-side bounds below.
- **Decoded-queue bound (`PlaybackScheduler`):** `evictPlayedBuffers()` discards already-played `AudioBuffer`s, capping the scheduler's forward fill to a 96 MB ceiling (Phase 21.2). Decoded PCM is larger than source (Web Audio uses 32-bit float — a 16-bit stereo WAV roughly doubles; Opus decodes to the same float footprint).
- **Network bound (segmented fetch, Phase 21 Direction B):** The forward stream is fetched as sequential bounded `bytes=cursor-{cursor+4MB-1}` Range requests via `RunSegmentedStreamAsync`; the next segment is fetched only after `DrainBackpressureAsync` confirms the scheduler is below low-water. Because each segment is fully consumed before the next is issued, the browser holds at most ~one segment of raw bytes. A per-chunk drain inside the segment loop (gated on `_streamingPlaybackStarted` so it cannot deadlock first-audio) additionally prevents high-density codecs (e.g. Opus, where a 4 MB segment is ~100 s of audio) from decoding the whole segment eagerly ahead of the playhead before the inter-segment gate runs.
Keep this seam clean — it is the most architecturally load-bearing part of the playback path.