Compare commits
87 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| aa4fae1faf | |||
| 8d1272e36f | |||
| 6a043b622e | |||
| 2af0d8650b | |||
| ca44979b08 | |||
| bf5b314aed | |||
| afa862a67b | |||
| d72263aea1 | |||
| 1e17ffc380 | |||
| a98cef1ba7 | |||
| 4351ae04be | |||
| c1f2cafd8b | |||
| 634eb611eb | |||
| d7a373cdb0 | |||
| 020a945843 | |||
| 1aef30f67d | |||
| 0e8b85bbcb | |||
| 374f09150f | |||
| 9bfa921703 | |||
| 4fe2d564d9 | |||
| 5d9f410cd8 | |||
| 76f7f389a3 | |||
| 61e185a2f7 | |||
| 9347f11ff0 | |||
| f0d1463619 | |||
| aa0b64329f | |||
| 4ab430d232 | |||
| beec36a382 | |||
| 79bbbd4956 | |||
| 48e58c266d | |||
| 67422e922d | |||
| 3aed5c129f | |||
| e98e616997 | |||
| 8fa37f995b | |||
| 0800167511 | |||
| 8a6acd5f5f | |||
| be9de8d77c | |||
| d686fe48ce | |||
| 7adc35dd5d | |||
| 8206c0bdaf | |||
| aeec582957 | |||
| 036ee1f78e | |||
| c1e6930c70 | |||
| cc9d20184d | |||
| e7762e35e8 | |||
| 11faf8888f | |||
| adbd376d42 | |||
| cb899a2913 | |||
| def297e7d9 | |||
| 369cb86437 | |||
| c7629c15a4 | |||
| 9c95a5f23e | |||
| b93881cd66 | |||
| af4cb186f3 | |||
| 121983b19d | |||
| 29e8747c69 | |||
| 518479e7ae | |||
| a2becf45d6 | |||
| 07f29a8216 | |||
| ed606d94c7 | |||
| ccf7d3dbe3 | |||
| bbcf8be677 | |||
| 8902ce4d63 | |||
| eb58ae4a72 | |||
| d80b777e9f | |||
| 5a75da1769 | |||
| 7f3fb74126 | |||
| d0118997b6 | |||
| 5b78efaad4 | |||
| 81d4b42b72 | |||
| 77c6c42c94 | |||
| ab75bbf6c1 | |||
| 59f48bb8cb | |||
| c63c7ca033 | |||
| e5366bc4ec | |||
| 2bde4908d7 | |||
| dce5530890 | |||
| 8afcd3784f | |||
| 261289c1b8 | |||
| 740d01a67f | |||
| e807ddb91b | |||
| 19793ba1c3 | |||
| e845dc3496 | |||
| ba064cc136 | |||
| b3dadbb572 | |||
| 6add30a4ff | |||
| 33d6f34d8a |
@@ -42,7 +42,7 @@ Server-side (SSR): Both clients point directly at DeepDrftAPI (server-to-server,
|
||||
- Root contains typed **MediaVaults** (Media, Image, Audio)
|
||||
- Each vault has a JSON `index` file listing entries + per-entry metadata
|
||||
- Entries are user-supplied strings sanitized to `[a-zA-Z0-9-]` + file extension
|
||||
- Binary hierarchy: `FileBinary` → `MediaBinary` (+ Extension/MIME) → `AudioBinary` (+ Duration/Bitrate) | `ImageBinary` (+ AspectRatio)
|
||||
- Binary hierarchy (model shapes): `FileBinary` → `MediaBinary` (+ Extension/MIME) → `AudioBinary` (+ Duration/Bitrate) | `ImageBinary` (+ AspectRatio). **Non-delivery read path** (`LoadResourceAsync<AudioBinary>`) returns a full-buffer `AudioBinary` — still used for non-delivery operations (e.g., duration backfill). **Audio delivery path** streams via `GetEntryStreamAsync` (Opus artifact, `track-opus` vault) or `OpenAudioMediaStreamAsync` (lossless source, `tracks` vault) — a seekable, disk-backed `Stream` per request, never a whole-file `byte[]` (read-side OOM fix, parallel to the store-side). The **write/store path** is streaming: audio processors return a `ProcessedAudio` plan (metadata + streamed `WriteToAsync` callback); `RegisterResourceStreamingAsync` / `MediaVault.AddEntryStreamingAsync` write bytes to a temp file then `File.Move` atomic-rename into place — the full `AudioBinary` buffer is never materialized on this path.
|
||||
- **Error-handling philosophy**: public operations swallow exceptions and return `null`/`false` — callers must check return values, not catch.
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -57,20 +57,39 @@ The split between host projects (`DeepDrftPublic`, `DeepDrftManager`, `DeepDrftC
|
||||
|
||||
`TrackEntity` holds *only* metadata. The link to binary content is `EntryKey` (string) — the entry id inside the `tracks` vault in FileDatabase. Dual-database add flow:
|
||||
|
||||
1. `DeepDrftContent.TrackService.AddTrackFromWavAsync` processes WAV, generates entry GUID, stores audio in vault, returns unpersisted `TrackEntity`.
|
||||
1. `TrackContentService.AddTrackAsync` routes the audio file by extension (`AudioProcessorRouter`), produces a `ProcessedAudio` plan (bounded-header metadata + streamed `WriteToAsync` callback — no whole-file `AudioBinary` buffer), and stores it in the vault via `FileDatabase.RegisterResourceStreamingAsync` / `MediaVault.AddEntryStreamingAsync` (atomic temp→rename on the Linux host). Returns an unpersisted `TrackEntity` with `DurationSeconds` populated from the header parse. Wave 1 OOM fix.
|
||||
2. `DeepDrftAPI.Services.UnifiedTrackService.UploadAsync` persists the entity to SQL via `DeepDrftData.TrackManager` and returns the persisted entity with `Id`.
|
||||
|
||||
If step 1 succeeds and step 2 fails, audio is orphaned in the vault (no rollback today).
|
||||
|
||||
The Opus transcode derived-artifact path (`OpusTranscodeService.TranscodeAndStoreAsync`) also streams its entire pipeline: extension and duration are read from the vault index (no body load); the source bytes are opened via `TrackContentService.OpenAudioMediaStreamAsync` and bounded-copied to a staging file; the encoded Ogg output is walked from a `FileStream` via `OggOpusParser.WalkAsync(Stream)` (bounded one-page-at-a-time buffer; byte-identical to the retained whole-buffer oracle `Walk(ReadOnlySpan<byte>)`); and stored via `RegisterResourceStreamingAsync`. The sidecar (a few KB, inherently bounded) retains the whole-buffer write. Completes the store-path OOM-fix arc.
|
||||
|
||||
### Streaming-first audio playback
|
||||
|
||||
The player is not fetch-then-play:
|
||||
|
||||
1. Client calls `GET api/track/{id}` on DeepDrftContent and receives WAV bytes as a stream (`HttpCompletionOption.ResponseHeadersRead`).
|
||||
2. `StreamingAudioPlayerService` reads in adaptive 16–64 KB chunks, pushes each via `AudioInteropService.processStreamingChunk`.
|
||||
3. TypeScript `StreamDecoder` parses WAV header, decodes chunks to `AudioBuffer`s. `PlaybackScheduler` schedules them on a Web Audio graph.
|
||||
1. Client calls `GET api/track/{id}` on DeepDrftContent and receives the first bounded segment (`Range: bytes=0-{SegmentSizeBytes-1}`, 4 MB) via `HttpCompletionOption.ResponseHeadersRead`.
|
||||
2. `StreamingAudioPlayerService` reads in adaptive 16–64 KB chunks within each segment, pushes each via `AudioInteropService.processStreamingChunk`.
|
||||
3. TypeScript `StreamDecoder` parses the format header, decodes chunks to `AudioBuffer`s. `PlaybackScheduler` schedules them on a Web Audio graph.
|
||||
4. Playback starts as soon as a min buffer is queued; UI duration from parsed header (not waiting for full file).
|
||||
5. **Seek beyond buffer**: if seek target is past what's decoded, client issues `GET api/track/{id}` with `Range: bytes={byteOffset}-`. Server streams raw bytes from that file-absolute offset with a `206 Partial Content` response. Player retains the parsed WAV header and feeds the raw PCM continuation into the existing decode pipeline.
|
||||
5. **Seek beyond buffer**: if seek target is past what's decoded, `StreamingAudioPlayerService` issues a new bounded segment starting at the seek byte offset. Server responds 206; player retains the parsed header and feeds the raw continuation into the existing decode pipeline.
|
||||
|
||||
**Memory bounding (three complementary layers, all required):**
|
||||
- **Raw-queue bound (`StreamDecoder`):** `releaseConsumedChunks()` front-compacts `rawChunks` after each aligned segment is decoded, using a `discardedBytes` absolute cursor so all offsets remain absolute even as the array's front moves. Without this, a long WAV (e.g. a 92-min mix ≈ 970 MB raw) accumulates its entire decoded-from body in `rawChunks` regardless of the decode-side bounds below.
|
||||
- **Decoded-queue bound (`PlaybackScheduler`):** `evictPlayedBuffers()` discards already-played `AudioBuffer`s, capping the scheduler's forward fill to a 96 MB ceiling (Phase 21.2). Decoded PCM is larger than source (Web Audio uses 32-bit float — a 16-bit stereo WAV roughly doubles; Opus decodes to the same float footprint).
|
||||
- **Network bound (segmented fetch, Phase 21 Direction B):** The forward stream is fetched as sequential bounded `bytes=cursor-{cursor+4MB-1}` Range requests via `RunSegmentedStreamAsync`; the next segment is fetched only after `DrainBackpressureAsync` confirms the scheduler is below low-water. Because each segment is fully consumed before the next is issued, the browser holds at most ~one segment of raw bytes. A per-chunk drain inside the segment loop (gated on `_streamingPlaybackStarted` so it cannot deadlock first-audio) additionally prevents high-density codecs (e.g. Opus, where a 4 MB segment is ~100 s of audio) from decoding the whole segment eagerly ahead of the playhead before the inter-segment gate runs.
|
||||
|
||||
**Playback stability invariants (streaming-stabilization arc):**
|
||||
- **Back-pressure water marks:** forward fill 60s high / 30s low (`PlaybackScheduler.DEFAULT_FORWARD_HIGH_WATER_SECONDS` / `...LOW_WATER_SECONDS`). Production pauses on `lookahead ≥ high OR decoded bytes > 96 MB ceiling`, whichever first. The time window is a jitter cushion for Opus's async decode ramp; the byte cap is the hard OOM guarantee and is unchanged.
|
||||
- **Genuine end-of-playback:** `PlaybackScheduler` uses a `streamComplete` flag (set by `setStreamComplete`) combined with an `underrun_` park/resume state to distinguish a drained-but-still-streaming queue (startup/underrun gap → park, resume on refill) from a truly finished track (stream complete AND queue drained → `finishPlayback()`, the single genuine-end path). Prevents the false `onPlaybackEnded` that previously fired when the Opus WebCodecs queue momentarily drained during the decode ramp.
|
||||
- **Rebuffer hysteresis:** Opus playback start and underrun-resume are gated on `hasMinimumPlaybackLead()` (1s decoded lead, `DEFAULT_MIN_PLAYBACK_LEAD_SECONDS`); WAV keeps `hasMinimumBuffers(6)`. `streamComplete` overrides the gate so a short tail still plays out. `StreamingAudioPlayerService.TryStartPlaybackAsync()` force-starts after `MarkStreamCompleteAsync` when the threshold was never crossed (ultra-short track protection).
|
||||
- **Opus `AudioContext` pre-aligned to 48 kHz** in `AudioPlayer.initializeStreaming` before any bytes flow, eliminating the mid-decode context teardown that previously OOM'd the tab under software rendering.
|
||||
|
||||
**Visualizer / decode contention (decodePressure + hwAccel):**
|
||||
- `DeepDrftPublic/Interop/audio/decodePressure.ts` exports a shared `DecodePressureSignal` singleton. The audio pipeline (`OpusStreamDecoder` yield-cap events, `PlaybackScheduler` underrun-parking events) calls `report()` on sustained lag; the visualizer calls `isUnderPressure()` each rAF frame. The signal engages only after ≥ 5 reports within 2500 ms with a 1s minimum hold (hysteresis); a lone startup-ramp blip never engages.
|
||||
- Under pressure, `WaveformVisualizer.ts` throttles its rAF loop to ~15 fps (`PRESSURE_THROTTLE_FRAME_MS = 1000/15`), cutting main-thread WebGL software-render + physics cost so WebCodecs decode recovers. A no-op under HW accel.
|
||||
- `DeepDrftPublic/Interop/visualizer/hwAccel.ts` probes `WEBGL_debug_renderer_info.UNMASKED_RENDERER_WEBGL` for software-renderer signatures (SwiftShader, llvmpipe, softpipe, Microsoft Basic Render, etc.). Absence of the debug extension is treated as accelerated — lava is disabled only on positive evidence. On first interactive render, `WaveformVisualizerControlState.ApplyCapabilityDefault(bool)` applies a one-time scoped default: `LavaEnabled = false` (the expensive subsystem) when no HW accel is detected, `WaveformEnabled` stays on. Guarded by `_capabilityDefaultApplied`; never overrides an explicit user toggle.
|
||||
- **Known limitation / deferred escalation:** HW-accel-off Opus playback is made usable by defaulting lava off and throttling under pressure. If starvation recurs (e.g. waveform-only path also proves too costly, or a software renderer slips the probe), the documented next step is moving Opus WebCodecs decode off the main thread (Web Worker / AudioWorklet) so it stops competing with main-thread rendering. Not a current implementation — a recorded fallback.
|
||||
|
||||
Keep this seam clean — it is the most architecturally load-bearing part of the playback path.
|
||||
|
||||
|
||||
+80
-1
@@ -47,6 +47,83 @@ Newest entries at the top. Group by phase/wave header (mirroring `PLAN.md` / `CM
|
||||
|
||||
---
|
||||
|
||||
## Phase 21 — Windowed Streaming Buffer (landed 2026-06-24)
|
||||
|
||||
**Landed:** 2026-06-24 on `streaming-overhaul`.
|
||||
|
||||
- **What:** Bounded client memory for long audio streams. Playing a 1 GB+ DJ Mix (the Phase 9 `Mix` medium — a single long track) no longer accumulates the full decoded PCM in the browser; instead the player holds only a sliding forward window and discards what has already played. Four waves, all landed. Public listener site only (`DeepDrftPublic.Client` player stack + `DeepDrftPublic` TypeScript audio interop); no CMS, no API endpoint, no schema change.
|
||||
|
||||
- **Why:** The `PlaybackScheduler` held an `AudioBuffer[]` it never evicted — both decode paths (`StreamDecoder`/`IFormatDecoder` for WAV/MP3/FLAC, and `OggDemuxer` → `OpusStreamDecoder` for Opus) pushed into it without limit. Decoded PCM is larger than the source (Web Audio is 32-bit float per sample/channel — a 16-bit stereo WAV roughly doubles once decoded; a low-data Opus mix decodes to the same float footprint regardless of how few compressed bytes arrived), so a 1 GB WAV or Opus mix could accumulate ~2 GB of retained float in the browser. The Opus path also had a second upstream accumulation locus: the WebCodecs `AudioDecoder` work queue and `decodedQueue: AudioData[]`.
|
||||
|
||||
- **Shape (by wave):**
|
||||
- **21.1 — Partial eviction in `PlaybackScheduler` (cold-start; load-bearing; shared by both paths).** Drop already-played buffers while keeping position/index/time-anchor bookkeeping exact against a buffer array that no longer begins at absolute time 0. Written once, serves both decode paths (they `addBuffer` identically). This is the hardest correctness work in the phase. No refill yet.
|
||||
- **21.2 — Back-pressure (one fill signal, two throttle sites).** Bound the unplayed region by stopping production above a high-water mark and resuming below low-water, driven by `PlaybackScheduler.evaluateProductionPause()` — a single shared scheduler-fill signal (OQ6/OQ7 resolved: shared controller, per-path hook). **21.2a** — `StreamingAudioPlayerService` stops fetching the next segment above high-water, polling `evaluateProductionPause()` at 100 ms cadence until the fill drains below low-water (serves both paths). **21.2b** — the Opus demux/decode feed is additionally stopped when the same signal is set, so the WebCodecs decode queue and `decodedQueue` do not balloon behind a paused segment loop (Opus only; no WAV analogue). Together with 21.1 this bounds both the played and unplayed sides on both formats.
|
||||
- **21.3 — Seek-back-past-window refill + clean failure recovery.** When a backward seek lands earlier than the retained tail, the existing seek-beyond-buffer path is reused pointed at the earlier offset (whichever resolver the active path ships: `IFormatDecoder`/`StreamDecoder` for WAV; the live `resolveOpusByteOffset` + `OpusStreamDecoder.reinitializeForRangeContinuation` for Opus). Minimal AC6 refill-failure handling added: `RecoverFromFailedRefill` clears the scheduler and surfaces a user-visible error rather than leaving the player wedged.
|
||||
- **21.4 — Validation + Direction A→B pivot (the critical wave — see as-built divergence below).** Network-memory bounding confirmed in Daniel's browser run: the segmented approach delivers ~4 MB segments pacing with playback, with the browser holding ~one segment of raw bytes rather than the full artifact.
|
||||
|
||||
- **As-built divergence — Direction A→B pivot (important).** The spec recommended **Direction A** (sliding window on one open-ended forward stream, relying on pausing `ReadAsync` / the segment loop to backpressure the socket) with **Direction B** (discrete bounded `Range: bytes=start-end` segments, each fetched only when the scheduler drains below low-water) held as the documented fallback "if Direction A's back-pressure proves leaky in practice."
|
||||
|
||||
**21.4 browser validation proved it leaky.** On Blazor WASM, the browser `fetch` API buffers the entire HTTP response body regardless of how slowly the application reads it: pausing reads bounded the *decode* but not the *network download*, so the entire ~970 MB body accumulated in browser memory even though the application only decoded a window of it. Direction A's core assumption — that pausing `ReadAsync` closes the TCP flow-control window before the browser caches the whole body — does not hold in the WASM `fetch` runtime.
|
||||
|
||||
**We pivoted to Direction B.** The forward stream now issues sequential bounded `bytes=cursor-(cursor+SegmentSizeBytes-1)` Range requests (`SegmentSizeBytes = 4 MB`), each fetched only after the scheduler drains below low-water — via `RunSegmentedStreamAsync` in `StreamingAudioPlayerService`. Because each 4 MB request is fully consumed before the next is issued, the browser holds at most ~one segment of raw bytes regardless of file size. The decode-side windowing (21.1/21.2) pairs with it: the segment loop's segment-gate replaces the raw `ReadAsync` pause as the production throttle (21.2a), and the Opus decode-ahead throttle (21.2b) hooks the same `evaluateProductionPause()` signal. Seek and refill converge on the same segmented loop. Direction B is recorded as the shipped approach; Direction A is recorded as tried-in-validation and found insufficient for the WASM `fetch` runtime.
|
||||
|
||||
- **What Phase 21 does NOT include:** the full AC matrix re-run (Opus seek-storm, visualizer, rapid-seek concurrency under the new segmented loop) beyond the network-memory bounding confirmed in 21.4. Those acceptance criteria remain the validation baseline for any follow-on work touching this seam.
|
||||
|
||||
- **Design memo:** `product-notes/phase-21-windowed-streaming-buffer.md` (note: the spec recommended Direction A; the as-built pivot to Direction B is annotated at the top of that doc and in §3.2/§3.3).
|
||||
|
||||
### Phase 21 — Post-landing: raw-queue and per-chunk bounds (landed 2026-06-24)
|
||||
|
||||
**Landed:** 2026-06-24 on `streaming-overhaul`. Fixes a Chrome tab crash (main-process OOM with hardware acceleration OFF) on long-track playback. Two unbounded memory queues remained after Phase 21 landed; this patch adds complementary bounds on both.
|
||||
|
||||
1. **`StreamingAudioPlayerService` — per-chunk drain inside segments.** Phase 21 Direction B's inter-segment back-pressure gate is matched to WAV byte density (~24 s per 4 MB segment), but not to Opus: at 320 kbps a 4 MB segment is ~100 s of decodable audio. The inner chunk loop had no fill check, so the entire segment decoded eagerly ahead of the playhead — piling decoded f32 PCM into main-process RAM before the inter-segment gate ever ran. Fix: an extracted `DrainBackpressureAsync` helper (shared with the inter-segment gate) is called per chunk once the scheduler is over high-water, gated on `_streamingPlaybackStarted` so it cannot deadlock first-audio (the playhead doesn't advance until playback starts, so draining to low-water would block indefinitely before the first buffer plays). Also added load-generation diagnostic logging (`_loadGeneration`) to confirm single-load-per-play-action in-browser.
|
||||
|
||||
2. **`StreamDecoder.ts` — `releaseConsumedChunks()` front-compaction of `rawChunks`.** `rawChunks` retained the entire decoded-from stream body because consumed chunks were never released — a 92-min WAV mix (≈ 970 MB raw) accumulated all of it in the browser. Phase 21.2's 96 MB decoded-side ceiling does not reach this raw queue. Fix: `releaseConsumedChunks()` walks `rawChunks` after each aligned segment is decoded and splices off all fully-consumed chunks (any chunk whose end ≤ `processedBytes`), advancing `discardedBytes` (an absolute cursor into the logical stream) so `extractAlignedData` can continue walking from the correct position even though `rawChunks[0]` no longer begins at byte 0. This is the raw-side analogue of `PlaybackScheduler.evictPlayedBuffers` (the decoded side).
|
||||
|
||||
Together with the existing decoded-side 96 MB ceiling (Phase 21.2) and the segmented fetch (Phase 21 Direction B), these complete the three-layer memory bound: raw queue, decoded queue, and network.
|
||||
|
||||
### Phase 21 — Post-landing: streaming stabilization (landed 2026-06-26)
|
||||
|
||||
**Landed:** 2026-06-26 on `streaming-overhaul`. Resolves playback instability that surfaced specifically with Opus + hardware acceleration off: the Opus underrun/false-end cycle, visualizer starvation of WebCodecs decode, and an OOM path from mid-decode `AudioContext` teardown. All changes are TypeScript-only (`DeepDrftPublic/Interop/audio/` + `DeepDrftPublic/Interop/visualizer/`) plus the C# service that calls them; no API, no schema, no CMS change.
|
||||
|
||||
1. **Back-pressure water marks widened to 60s high / 30s low.** `PlaybackScheduler.ts` `DEFAULT_FORWARD_HIGH_WATER_SECONDS` / `DEFAULT_FORWARD_LOW_WATER_SECONDS` doubled from 30/15. The 96 MB decoded-byte ceiling is unchanged and remains the hard OOM bound; production pauses on `lookahead ≥ high OR bytes > cap`, whichever first. The wider time window absorbs per-packet WebCodecs decode jitter (Opus 48 kHz stereo ≈ 0.37 MB/s → 60 s ≈ 23 MB — well within the cap) without false back-pressure on the Opus decode ramp.
|
||||
|
||||
2. **End-of-playback gated on genuine completion (`streamComplete` + `underrun_`).** `PlaybackScheduler` now distinguishes a drained-but-still-streaming queue (startup/underrun gap) from a truly finished track (stream complete AND queue drained). `streamComplete` (set by `setStreamComplete`, called from `AudioPlayer.markStreamComplete`) is the discriminator. When the queue drains while `streamComplete` is false the scheduler parks in `underrun_` state (anchors frozen, `isActive_` false) and resumes from `scheduleNewBuffers` when buffers arrive. `finishPlayback()` is the single genuine-end path, reached only when both conditions hold simultaneously. Previously a drained Opus queue during the WebCodecs startup ramp fired `onPlaybackEnded`, nulling Duration and advancing the queue.
|
||||
|
||||
3. **Rebuffer hysteresis — 1s decoded-lead gate.** `hasMinimumPlaybackLead()` gates Opus playback START and underrun RESUME (`DEFAULT_MIN_PLAYBACK_LEAD_SECONDS = 1.0`), preventing the decode-ramp thrash (resume on a 20 ms Opus packet → drain immediately → re-park → repeat). `streamComplete` overrides the gate so a short tail plays out rather than parking forever. WAV keeps its `hasMinimumBuffers(6)` buffer-count start gate unchanged — large synchronous WAV segments rarely underrun and that gate's character must not change. **Complete-without-start force-start:** `StreamingAudioPlayerService.TryStartPlaybackAsync()` is called after `MarkStreamCompleteAsync` when `_streamingPlaybackStarted` is still false, preventing an ultra-short Opus track (total audio below the 1s lead) from sitting loaded-but-not-playing.
|
||||
|
||||
4. **Opus `AudioContext` pre-aligned to 48 kHz.** `AudioPlayer.initializeStreaming` calls `contextManager.recreateWithSampleRate(OPUS_SAMPLE_RATE)` when the content type resolves as Opus, before any bytes flow. The context is already at 48 kHz when the decoder's own lazy `recreateWithSampleRate` check runs, so it short-circuits (no-op). Eliminates the mid-decode context teardown-and-rebuild that previously double-decoded the stream and OOM'd the tab under HW accel off.
|
||||
|
||||
5. **`decodePressure` singleton + `WaveformVisualizer` auto-throttle (~15 fps under pressure).** `DeepDrftPublic/Interop/audio/decodePressure.ts` exports a process-wide `DecodePressureSignal` singleton shared between the audio pipeline (producer: `OpusStreamDecoder` yield-cap events and `PlaybackScheduler` underrun-parking events) and the visualizer (consumer). Engages only on sustained stress (≥ 5 `report()` calls within a 2500 ms window) and holds for a 1000 ms minimum dwell before releasing, suppressing lone startup-ramp blips. `WaveformVisualizer.ts` checks `decodePressure.isUnderPressure()` each rAF frame; when engaged it skips frames so at most one renders per `PRESSURE_THROTTLE_FRAME_MS` (`1000/15` ms, ~15 fps), cutting main-thread WebGL software-render + physics cost so WebCodecs decode recovers. A no-op under HW accel — healthy decode never calls `report()` and `isUnderPressure()` stays false indefinitely.
|
||||
|
||||
6. **Hardware-acceleration detection + `ApplyCapabilityDefault`.** `DeepDrftPublic/Interop/visualizer/hwAccel.ts` probes `WEBGL_debug_renderer_info.UNMASKED_RENDERER_WEBGL` for known software-renderer signatures (SwiftShader, llvmpipe, softpipe, Microsoft Basic Render Driver, Mesa Offscreen, "software"). Absence of the debug extension is treated as accelerated — the probe disables lava only on positive evidence of software rendering, not on uncertainty. `WaveformVisualizerControlState.ApplyCapabilityDefault(bool hardwareAccelerated)` is called once per session by the visualizer bridge on first interactive render: when `hardwareAccelerated` is false it sets `LavaEnabled = false` (the expensive main-thread subsystem) while leaving `WaveformEnabled` on, coerces Theater Mode, and raises `Changed` once so all observers see the default in a single cycle. Guarded by `_capabilityDefaultApplied` so it runs at most once per session and never overrides an explicit in-session user toggle. With HW accel present the call is a no-op.
|
||||
|
||||
---
|
||||
|
||||
## Phase 18 — Opus Low-Data Streaming (landed 2026-06-23)
|
||||
|
||||
_Note: Two distinct efforts share the "Phase 18" label — phase numbers are organisational, not sequential. See also **Phase 18 — Theme / Dark-Mode Remediation (landed 2026-06-19)** below._
|
||||
|
||||
**Landed:** 2026-06-23 on `streaming-overhaul`.
|
||||
|
||||
- **What:** Dual-format audio delivery — the existing lossless WAV path plus a new low-data Ogg Opus (fullband, 320 kbps) path — giving listeners a quality choice. Opus is the default (capability-gated). Six waves, all landed.
|
||||
|
||||
- **Why:** Lossless WAV works but imposes high bandwidth cost, particularly for long DJ Mixes. Opus at 320 kbps is indistinguishable for typical listening and an order of magnitude smaller, making the site usable on metered or constrained connections. The bespoke Web Audio decode→schedule graph is retained by deliberate choice; Opus feeds the same `IFormatDecoder` seam rather than an HTML `<media>` element or MSE.
|
||||
|
||||
- **Shape (by wave):**
|
||||
- **18.1 — Ingest transcode + seek-index sidecar.** New `OpusTranscodeService`/processor in `DeepDrftContent`, invoked post-store from `UnifiedTrackService.UploadAsync` as a background job (failure-tolerant; the track plays lossless first). Produces the Ogg Opus 320 artifact and a combined seek/setup **sidecar** (0.5 s-bucketed granule→byte index + `OpusHead`/`OpusTags` setup header), both stored in a new `track-opus` vault. **ffmpeg/libopus** added to the deploy bootstrap as a runtime prereq on the API host.
|
||||
- **18.2 — Storage + lookup contract.** Server-side "given `EntryKey` + format, return the right artifact + content-type, with lossless fallback when no Opus artifact exists."
|
||||
- **18.3 — Delivery: `?format=opus|lossless` + sidecar serving + proxy threading.** `?format=` param on the stream endpoint (Range preserved); sidecar-serving endpoint; param forwarded through `TrackProxyController`; `TrackMediaClient` sends the format param.
|
||||
- **18.4 — Client Opus decoder + index-based seek + capability detection.** New `OpusFormatDecoder` (`IFormatDecoder`): Ogg-page demux, setup-header carry from the cached sidecar, `calculateByteOffset` binary-searching the precomputed seek index (VBR-safe accurate seek — not interpolation). Capability gate falls back to the lossless path on browsers without Opus support. Testing-phase fixes also landed: a valid ffmpeg-generated capability-probe sample and correct duration reporting (fixed the seekbar + visualizer in Opus mode).
|
||||
- **18.5 — End-to-end integration.** Player format selection (Opus-by-default, capability-gated); sidecar fetch/inject on track load; **Backfill-Opus** CMS bulk action (third sibling to Generate-Profiles / Backfill-High-res); replace-audio regenerates Opus + sidecar.
|
||||
- **18.6 — Settings menu + quality toggle + CMS status.** New public-site **Settings menu** (app-bar trigger + settings-item abstraction + `ListenerSettings`, persisted via the dark-mode-pattern seam: `streamQuality` cookie → prerender-read → `PersistentComponentState` → client cookie service); streaming-quality toggle (Low-data/Lossless) as its first occupant. CMS **Opus-status badge** on the track grid. CMS upload meter **Post-Processing** phase visualizing background Opus transcode.
|
||||
|
||||
- **As-built divergence — the Opus decoder (important).** The originally-spec'd "segment-and-decode-each-page via `decodeAudioData`" model was implemented and then **replaced**: browser testing showed it glitches at every segment boundary and broke seeking (Opus has pre-skip + inter-frame state, so independent per-segment decode is architecturally wrong). The landed decoder is a **WebCodecs `AudioDecoder` streaming pipeline** (Ogg demux → packets → stateful continuous decode), with index-driven seek and a WebCodecs-based capability gate. Daniel's call at the decision point — offered a decode-whole stopgap (Option A) vs. the proper WebCodecs streaming decoder (Option B), he chose B: "Let's go for B, why do you like half-solutions so much" (the same no-stopgap posture as the Mix visualizer WebGL2 decision, where he said "no stopgap — WebGL as step 1, no pussyfooting"). The segmented approach is recorded as tried-and-replaced.
|
||||
|
||||
- **What Phase 18 does NOT include:** windowed / bounded-memory streaming. Phase 18 still fetches the whole (smaller) Opus artifact; bounding client memory to a sliding forward window is **Phase 21's job**.
|
||||
|
||||
- **Design memo:** `product-notes/phase-18-opus-low-data-streaming.md`.
|
||||
---
|
||||
|
||||
## Phase 20 — Theater Mode (landed 2026-06-20)
|
||||
|
||||
**Landed:** 2026-06-20 on dev. Pending: final manual browser/GPU smoke-test on dev.
|
||||
@@ -76,6 +153,8 @@ Newest entries at the top. Group by phase/wave header (mirroring `PLAN.md` / `CM
|
||||
|
||||
## Phase 18 — Theme / Dark-Mode Remediation (landed 2026-06-19)
|
||||
|
||||
_Note: Two distinct efforts share the "Phase 18" label — phase numbers are organisational, not sequential. See also **Phase 18 — Opus Low-Data Streaming (landed 2026-06-23)** above._
|
||||
|
||||
**Landed:** 2026-06-19 on dev (Wave 1 + Wave 2 + Wave 3).
|
||||
|
||||
- **What:** A DRY token pass resolving six theming symptoms (five in dark mode, one in light) that all traced to three root causes: neutral page surfaces bound to constant brand tokens, the play chip bound to a constant light-grey, and no theme-aware popover-surface token. Resolved as one coherent pass via a shared token layer rather than per-component patches.
|
||||
@@ -245,7 +324,7 @@ Newest entries at the top. Group by phase/wave header (mirroring `PLAN.md` / `CM
|
||||
|
||||
- **Shape:**
|
||||
- **Client — `IAnonIdProvider` / `AnonIdProvider`** (`DeepDrftPublic.Client/Services/IAnonIdProvider.cs`, `AnonIdProvider.cs`): `IAnonIdProvider` exposes `string? Current` (synchronous cached read, safe on the unload path) and `ValueTask EnsureLoadedAsync()` (warms the cache from `localStorage` via JS interop — idempotent, best-effort, never throws). `AnonIdProvider` is the production implementation over the `window.DeepDrftAnonId.get` interop call. Degrades to null when `localStorage` is unavailable (private mode / blocked / partitioned iframe) — missing id is the accepted graceful path; over-counting is the direction of error (§3). Scoped (per-session cache); the token itself outlives the session in `localStorage`.
|
||||
- **Client — TypeScript interop** (`DeepDrftPublic/Interop/telemetry/anonid.ts`): mints and reads the `localStorage` GUID. Exposes `window.DeepDrftAnonId.get`. Returns null without throwing when storage is unavailable.
|
||||
- **Client — TypeScript interop** (`DeepDrftPublic/Interop/session/anonid.ts`): mints and reads the `localStorage` GUID. Exposes `window.DeepDrftAnonId.get`. Returns null without throwing when storage is unavailable.
|
||||
- **Client — `BeaconPlayEventSink`** (`DeepDrftPublic.Client/Services/BeaconPlayEventSink.cs`): now injects `IAnonIdProvider`; reads `_anonId.Current` synchronously at emit time and sets `PlayEventDto.AnonId`. Null id produces an anonId-less payload (the field is omitted from the wire JSON entirely via `WhenWritingNull` — the API treats absent and null identically).
|
||||
- **Client — `ShareTracker`** (`DeepDrftPublic.Client/Services/ShareTracker.cs`): now injects `IAnonIdProvider`; reads `_anonId.Current` at share time and sets `ShareEventDto.AnonId`. Same null-omit posture as the play sink.
|
||||
- **API — `EventController`** (`DeepDrftAPI/Controllers/EventController.cs`): `TryNormalizeAnonId` helper on both `POST api/event/play` and `POST api/event/share` — whitespace-only / empty / null collapses to null (valid anonId-less event); a token longer than 64 chars is rejected with `400 Bad Request` rather than truncated (truncation would collide distinct listeners onto one prefix); valid tokens are trimmed and passed through.
|
||||
|
||||
+19
-10
@@ -11,7 +11,7 @@ Dual-database authority for tracks (SQL metadata + FileDatabase binary), release
|
||||
## What lives here now (only)
|
||||
|
||||
- `Program.cs`, `Startup.cs`: HTTP host config, DI wiring, middleware setup, port binding. AuthBlocks startup: `AddAuthBlocks`, `UseAuthBlocksStartupAsync`, `MapAuthBlocks`, authentication/authorization middleware.
|
||||
- `Services/UnifiedTrackService.cs`: Host-internal orchestrator. Coordinates vault write + SQL persist for upload (`UploadAsync`), and SQL delete + vault remove for delete (`DeleteAsync`).
|
||||
- `Services/UnifiedTrackService.cs`: Host-internal orchestrator. Coordinates streaming vault write + SQL persist for upload (`UploadAsync`), and SQL delete + vault remove for delete (`DeleteAsync`). The upload/replace hot path streams audio into the vault via the `ProcessedAudio` plan (Wave 1 OOM fix) and then computes both waveform datums in a single bounded streaming pass via `TryStoreWaveformDatumsAsync` (Wave 2 OOM fix) — neither path buffers the whole audio file in a managed `byte[]`.
|
||||
- `Services/UnifiedReleaseService.cs`: Host-internal orchestrator. Coordinates release mutations (mix waveform compute + store, session hero-image upload + link).
|
||||
- `Controllers/TrackController.cs`: Track endpoints (see below).
|
||||
- `Controllers/ReleaseController.cs`: Release endpoints (see below).
|
||||
@@ -32,12 +32,12 @@ Dual-database authority for tracks (SQL metadata + FileDatabase binary), release
|
||||
|
||||
### GET api/track/{trackId} (unauthenticated)
|
||||
|
||||
Returns the WAV bytes from the `tracks` vault with HTTP Range support.
|
||||
Streams the track's audio bytes from disk with HTTP Range support. An optional `?format=opus` query parameter selects between the derived Opus artifact and the lossless source.
|
||||
|
||||
- **Route parameter `trackId`** (string): the entry id inside the `tracks` vault (i.e. `TrackEntity.EntryKey`).
|
||||
- **Range header** (optional): HTTP Range header for byte-range requests (e.g., `Range: bytes=1000-`). Server responds with `206 Partial Content` and streams from the requested offset.
|
||||
- Streams the file directly from disk with `enableRangeProcessing: true`, supporting both full-file and partial-range requests without synthesizing WAV headers or buffering.
|
||||
- Returns 200 for full-file requests, 206 for Range requests, 404 if track not found, 500 if vault operations fail (with error swallowing — the vault returns `null`).
|
||||
- **Query parameter `format`** (optional): `opus` requests the derived Ogg Opus artifact from the `track-opus` vault when present, falling back to lossless when it is not (C2 — never 404 if any audio exists); omitted or any other value delivers the lossless source in its stored format (WAV/MP3/FLAC). Both arms stream from a seekable disk `FileStream` via `File(stream, contentType, enableRangeProcessing: true)` — no whole-file `byte[]`; `Content-Type` reflects what was actually served (e.g., `audio/ogg` on an Opus hit, the source's real MIME on a lossless request or C2 fallback).
|
||||
- **Range header** (optional): HTTP Range header for byte-range requests (e.g., `Range: bytes=1000-`). Server responds with `206 Partial Content` and streams from the requested offset. Honoured on both arms (seekable `FileStream` in both cases).
|
||||
- Returns 200 for full-file requests, 206 for Range requests, 404 if no audio artifact exists for the track, 500 if vault operations fail.
|
||||
|
||||
### GET api/track/albums (unauthenticated)
|
||||
|
||||
@@ -74,7 +74,7 @@ Admin backfill: computes and stores a waveform profile for an existing track fro
|
||||
|
||||
- **Header `ApiKey`**: required. Validated by `ApiKeyAuthenticationMiddleware`.
|
||||
- **Route parameter `trackId`** (string): the entry id (TrackEntity.EntryKey).
|
||||
- Fetches audio from vault, decodes it, computes a loudness profile, and stores the profile in the `waveform-profiles` vault.
|
||||
- Streams the vault audio in bounded ≤80 KB chunks (no whole-file load) via `WaveformProfileService.ComputeAndStoreProfileStreamingAsync`. Tri-state result: `null` = no vault audio → 404; `false` = audio present but not WAV-decodable or vault write failed → 500; `true` = stored → 200.
|
||||
- Returns 200 on success. Returns 404 if no audio is stored under that key. Returns 500 if WAV decoding or vault write fails.
|
||||
|
||||
### GET api/track/{trackId}/waveform/high-res (unauthenticated)
|
||||
@@ -91,7 +91,7 @@ Server-side trigger: compute and store the per-track high-res datum for any trac
|
||||
|
||||
- **Header `ApiKey`**: required. Validated by `ApiKeyAuthenticationMiddleware`.
|
||||
- **Route parameter `trackId`** (string): the entry id (TrackEntity.EntryKey).
|
||||
- Calls `WaveformProfileService.ComputeAndStoreHighResAsync` via `UnifiedTrackService`.
|
||||
- Reads the duration from vault index metadata (no audio body load), then streams the vault audio in bounded chunks via `WaveformProfileService.ComputeAndStoreHighResStreamingAsync`. Tri-state result: `null` = no vault audio → 404; `false` = audio present but not WAV-decodable or vault write failed → 500; `true` = stored → 200.
|
||||
- Returns 200 on success. Returns 404 if no audio stored under that key. Returns 500 on compute/storage failure.
|
||||
|
||||
### GET api/track/meta/by-key/{entryKey} (unauthenticated)
|
||||
@@ -110,6 +110,15 @@ Admin backfill view: returns every track with flags indicating whether each wave
|
||||
- **Response**: `List<WaveformStatusDto>` with `TrackId`, `EntryKey`, `TrackName`, `HasProfile` (bool — 512-bucket player-bar seeker profile in `waveform-profiles` vault), and `HasHighRes` (bool — duration-derived high-res visualizer datum in `track-waveforms` vault).
|
||||
- Returns 200 on success. Returns 500 on query error.
|
||||
|
||||
### GET api/track/opus-status ([ApiKeyAuthorize])
|
||||
|
||||
Admin Post-Processing view: returns every track with a flag indicating whether it has a complete Opus artifact (both audio AND seek/setup sidecar present in the `track-opus` vault). Used by the CMS to show the Backfill-Opus badge and to poll per-track Post-Processing status after an upload. Mirrors the shape and auth posture of `GET api/track/waveform-status`.
|
||||
|
||||
- **Header `ApiKey`**: required. Validated by `ApiKeyAuthenticationMiddleware`.
|
||||
- **Response**: `List<OpusStatusDto>` with `TrackId`, `EntryKey`, `TrackName`, and `HasOpus` (bool — true only when both the Opus audio entry and the seek/setup sidecar entry are present in `track-opus`; a half-derived track counts as incomplete).
|
||||
- `HasOpus` is resolved via `TrackFormatResolver.HasOpusAsync` — an **index-only** existence check (`MediaVault.HasIndexEntry` for both entries; no file-body load). The endpoint loops over the whole catalogue, so a body load per track would stream the full library sequentially; the index lookup costs zero disk reads per track.
|
||||
- Returns 200 on success. Returns 500 on query error.
|
||||
|
||||
### POST api/track/duration/backfill ([ApiKeyAuthorize])
|
||||
|
||||
Admin backfill: for every track whose `DurationSeconds` SQL column is still null, reads the `AudioBinary.Duration` from the vault and writes it to SQL. Idempotent — a re-run only touches still-null rows; rows that already have a value are skipped.
|
||||
@@ -170,7 +179,7 @@ Soft-delete a release row. Used by the albums browser to remove an orphaned rele
|
||||
- `releaseId` (long?, optional): the SQL release ID to attach this track to. Omit (null) on the first row of a submit — this is the **CREATE path**, which mints a new release and blocks a pre-existing (title, artist) with 409. Set to the release id returned by row 1 for rows 2..N of a within-batch multi-track Cut — this is the **ATTACH path**, which skips the (title, artist) pre-existing check and attaches directly to the already-created release after validating the id matches the natural key. The upload form is create-only; appending to a pre-existing release must go through the edit tools.
|
||||
- The upload stream is copied to a staging file under the **upload staging directory** (resolved from `Upload:StagingPath`, defaulting to a `staging` subdirectory under the FileDatabase vault path — on the data disk, **never** `Path.GetTempPath()`) with the appropriate extension (`.wav`, `.mp3`, or `.flac`). The audio processor reads from disk and requires the correct extension for format detection. The staging file is always deleted in a `finally` block — success or failure. The framework's own multipart file-section buffer is relocated off the system temp mount too: `Startup.ConfigureDomainServices` sets the `ASPNETCORE_TEMP` env var to the same staging directory, so neither on-disk copy of a large body lands on `/tmp` (a small RAM-backed tmpfs on the Linux host).
|
||||
- `[RequestSizeLimit(~1.86 GB / 2_000_000_000)]` + `[RequestFormLimits(MultipartBodyLengthLimit = 2_000_000_000)]` lift the per-request ceiling above the framework default (~28 MB) so production-sized files are accepted. The body is streamed to the staging file, not buffered in memory.
|
||||
- `UnifiedTrackService.UploadAsync` orchestrates: release resolution (CREATE or ATTACH, see above) → `TrackContentService.AddTrackAsync` (format-agnostic vault write via router) → `TrackManager` (SQL persist with `createdByUserId`). Release resolution runs the cardinality guard on both paths and, on the CREATE path, calls `ITrackService.FindOrCreateRelease` (returns `(ReleaseDto Release, bool WasCreated)`); if `WasCreated` is false, a concurrent upload won the race and the request is rejected as a duplicate rather than silently attaching.
|
||||
- `UnifiedTrackService.UploadAsync` orchestrates: release resolution (CREATE or ATTACH, see above) → `TrackContentService.AddTrackAsync` (format-agnostic streaming vault write via router — audio is streamed to the vault via the `ProcessedAudio` plan, no whole-file buffer — Wave 1 OOM fix) → `TrackManager` (SQL persist with `createdByUserId`) → `TryStoreWaveformDatumsAsync` (best-effort: reads duration from vault index metadata, then computes both waveform datums from a single bounded streaming pass over the stored audio — Wave 2 OOM fix). Release resolution runs the cardinality guard on both paths and, on the CREATE path, calls `ITrackService.FindOrCreateRelease` (returns `(ReleaseDto Release, bool WasCreated)`); if `WasCreated` is false, a concurrent upload won the race and the request is rejected as a duplicate rather than silently attaching.
|
||||
- Returns 200 with the **persisted** `TrackDto` JSON (Id populated) on success. Returns 400 for missing/invalid form fields or unsupported audio format. Returns 409 for two distinct domain conditions: a pre-existing (title, artist) duplicate on the CREATE path (`DUPLICATE_RELEASE:` marker → 409 Conflict), or a track-number conflict within the release (`CARDINALITY_VIOLATION:` marker → 409 Conflict). Returns 500 if processing fails.
|
||||
|
||||
### DELETE api/track/{id:long} ([ApiKeyAuthorize])
|
||||
@@ -190,7 +199,7 @@ Soft-delete a release row. Used by the albums browser to remove an orphaned rele
|
||||
- **Route parameter `id`** (long): the SQL track ID.
|
||||
- **Form field `audioFile`** (`IFormFile`, required): the replacement audio bytes. File name must end in `.wav`, `.mp3`, or `.flac`.
|
||||
- `[RequestSizeLimit(~1.86 GB / 2_000_000_000)]` + `[RequestFormLimits(MultipartBodyLengthLimit = 2_000_000_000)]` mirror the upload ceiling. The body is streamed to a staging file under the upload staging directory (the same off-`/tmp` data-disk location as the upload path; correct extension preserved for the audio processor), always deleted in a `finally` block.
|
||||
- Calls `UnifiedTrackService.ReplaceAudioAsync`, which: looks up SQL row by id → calls `TrackContentService.ReplaceTrackAudioAsync(entryKey, tempFilePath)` (registers new audio under the existing `EntryKey`; removes the stale backing file only on a cross-format swap, after the new write succeeds) → regenerates both waveform datums (best-effort; a datum failure is logged and swallowed) → writes the new audio's duration to `DurationSeconds` via `ITrackService.SetDuration` (unconditional overwrite; a failure is surfaced, not swallowed, to prevent derived aggregates like `MixRuntimeSeconds` from silently going stale).
|
||||
- Calls `UnifiedTrackService.ReplaceAudioAsync`, which: looks up SQL row by id → calls `TrackContentService.ReplaceTrackAudioAsync(entryKey, tempFilePath)` (streams new audio into the vault under the existing `EntryKey` via the `ProcessedAudio` plan, no whole-file buffer — Wave 1 OOM fix; removes the stale backing file only on a cross-format swap, after the new write succeeds) → regenerates both waveform datums from a single bounded streaming pass over the freshly stored audio via `TryStoreWaveformDatumsAsync` (best-effort; a datum failure is logged and swallowed — Wave 2 OOM fix) → writes the new audio's duration to `DurationSeconds` via `ITrackService.SetDuration` (unconditional overwrite; a failure is surfaced, not swallowed, to prevent derived aggregates like `MixRuntimeSeconds` from silently going stale).
|
||||
- Returns 200 on success. Returns 400 if the file is missing or the format is unsupported. Returns 404 if the track id is not found. Returns 500 if vault processing fails.
|
||||
|
||||
### GET api/track/page (unauthenticated)
|
||||
@@ -288,7 +297,7 @@ Legacy endpoint: formerly served the high-res waveform datum for a Mix release f
|
||||
|
||||
### POST api/release/{id:long}/mix/waveform ([ApiKeyAuthorize])
|
||||
|
||||
Server-side trigger: fetch the Mix's track audio from the vault, compute the duration-derived high-res datum, store it in the `track-waveforms` vault under the track's `EntryKey`, and link it via `MixMetadata.WaveformEntryKey`. Delegates to `WaveformProfileService.ComputeAndStoreHighResAsync` — the same shared seam used by the upload path and the generalized CMS generate action. No request body.
|
||||
Server-side trigger: fetch the Mix's track audio from the vault, compute the duration-derived high-res datum, store it in the `track-waveforms` vault under the track's `EntryKey`, and link it via `MixMetadata.WaveformEntryKey`. Delegates to `WaveformProfileService.ComputeAndStoreHighResStreamingAsync` (streams the vault audio in bounded chunks, no whole-file buffer) — the same shared seam used by the upload path and the generalized CMS generate action. No request body.
|
||||
|
||||
- **Header `ApiKey`**: required. Validated by `ApiKeyAuthenticationMiddleware`.
|
||||
- **Route parameter `id`** (long): the SQL release ID.
|
||||
|
||||
@@ -112,10 +112,10 @@ public class ReleaseController : ControllerBase
|
||||
}
|
||||
|
||||
// POST api/release/{id}/mix/waveform ([ApiKeyAuthorize], no body)
|
||||
// Server-side trigger: fetch the Mix's track audio from the vault, compute a duration-derived high-res
|
||||
// waveform via ComputeAndStoreHighResAsync, store it in the track-waveforms vault, and set
|
||||
// MixMetadata.WaveformEntryKey. 404 when the release is missing or has no stored audio; 500 on
|
||||
// compute/storage failure. Declared before "{id:long}".
|
||||
// Server-side trigger: stream the Mix's track audio from the vault, compute a duration-derived
|
||||
// high-res waveform, store it in the track-waveforms vault, and set MixMetadata.WaveformEntryKey.
|
||||
// 404 when the release is missing or has no stored audio; 500 on compute/storage failure. Declared
|
||||
// before "{id:long}".
|
||||
[ApiKeyAuthorize]
|
||||
[HttpPost("{id:long}/mix/waveform")]
|
||||
public async Task<ActionResult> GenerateMixWaveform(long id, CancellationToken ct = default)
|
||||
|
||||
@@ -5,6 +5,7 @@ using DeepDrftContent.Constants;
|
||||
using DeepDrftContent.FileDatabase.Services;
|
||||
using DeepDrftContent.FileDatabase.Models;
|
||||
using DeepDrftContent.Processors;
|
||||
using DeepDrftContent.Processors.Opus;
|
||||
using DeepDrftData;
|
||||
using DeepDrftModels.DTOs;
|
||||
using DeepDrftModels.Enums;
|
||||
@@ -20,6 +21,7 @@ public class TrackController : ControllerBase
|
||||
private readonly UnifiedTrackService _unifiedService;
|
||||
private readonly ITrackService _sqlTrackService;
|
||||
private readonly WaveformProfileService _waveformProfileService;
|
||||
private readonly TrackFormatResolver _formatResolver;
|
||||
private readonly UploadStagingDirectory _stagingDirectory;
|
||||
private readonly ILogger<TrackController> _logger;
|
||||
|
||||
@@ -35,6 +37,7 @@ public class TrackController : ControllerBase
|
||||
UnifiedTrackService unifiedService,
|
||||
ITrackService sqlTrackService,
|
||||
WaveformProfileService waveformProfileService,
|
||||
TrackFormatResolver formatResolver,
|
||||
UploadStagingDirectory stagingDirectory,
|
||||
ILogger<TrackController> logger)
|
||||
{
|
||||
@@ -43,6 +46,7 @@ public class TrackController : ControllerBase
|
||||
_unifiedService = unifiedService;
|
||||
_sqlTrackService = sqlTrackService;
|
||||
_waveformProfileService = waveformProfileService;
|
||||
_formatResolver = formatResolver;
|
||||
_stagingDirectory = stagingDirectory;
|
||||
_logger = logger;
|
||||
}
|
||||
@@ -245,6 +249,40 @@ public class TrackController : ControllerBase
|
||||
return Ok(status);
|
||||
}
|
||||
|
||||
// GET api/track/opus-status ([ApiKeyAuthorize])
|
||||
// Admin Post-Processing view (18.6): returns every track with a flag for whether it carries a COMPLETE
|
||||
// Opus artifact — both the Opus audio AND the seek/setup sidecar present (TrackFormatResolver.HasOpusAsync,
|
||||
// the same completeness rule the 18.5 Backfill-Opus pass enqueues against; a half-derived track counts as
|
||||
// missing). Mirrors GET waveform-status exactly: same ApiKey auth, same unpaged whole-catalogue shape, same
|
||||
// literal-route placement before "{trackId}". The CMS reads it to show the Backfill-Opus "missing N" badge
|
||||
// and to poll per-track Post-Processing status after an upload.
|
||||
[ApiKeyAuthorize]
|
||||
[HttpGet("opus-status")]
|
||||
public async Task<ActionResult> GetOpusStatus()
|
||||
{
|
||||
var tracks = await _sqlTrackService.GetAll();
|
||||
if (!tracks.Success || tracks.Value is null)
|
||||
{
|
||||
var error = tracks.Messages.FirstOrDefault()?.Message ?? "Unknown error";
|
||||
_logger.LogError("GetOpusStatus failed to load tracks: {Error}", error);
|
||||
return StatusCode(500, "Failed to load tracks");
|
||||
}
|
||||
|
||||
var status = new List<OpusStatusDto>(tracks.Value.Count);
|
||||
foreach (var track in tracks.Value)
|
||||
{
|
||||
status.Add(new OpusStatusDto
|
||||
{
|
||||
TrackId = track.Id,
|
||||
EntryKey = track.EntryKey,
|
||||
TrackName = track.TrackName,
|
||||
HasOpus = await _formatResolver.HasOpusAsync(track.EntryKey),
|
||||
});
|
||||
}
|
||||
|
||||
return Ok(status);
|
||||
}
|
||||
|
||||
// POST api/track/duration/backfill ([ApiKeyAuthorize], no body)
|
||||
// One-time admin backfill: for every track whose SQL duration is still null, read the duration from
|
||||
// the vault audio and write it to SQL. Mirrors the waveform backfill posture. Idempotent — a re-run
|
||||
@@ -265,6 +303,27 @@ public class TrackController : ControllerBase
|
||||
return Ok(new { updated = result.Value.Updated, skipped = result.Value.Skipped });
|
||||
}
|
||||
|
||||
// POST api/track/opus/backfill ([ApiKeyAuthorize], no body)
|
||||
// Backfill-Opus (18.5, OQ4): enqueue a background Opus derive for every track lacking a complete Opus
|
||||
// artifact (audio + sidecar). Mirrors the duration-backfill posture — enqueue-only and non-blocking, the
|
||||
// transcodes run on the shared serial worker. Idempotent: a re-run only schedules tracks still missing
|
||||
// Opus. Returns { enqueued, skipped }. Declared in the literal-route block (before "{trackId}") so the
|
||||
// "opus/backfill" segment is never treated as a trackId; distinct shape from "{trackId}/opus" (per-track).
|
||||
[ApiKeyAuthorize]
|
||||
[HttpPost("opus/backfill")]
|
||||
public async Task<ActionResult> BackfillOpus(CancellationToken cancellationToken)
|
||||
{
|
||||
var result = await _unifiedService.BackfillOpusAsync(cancellationToken);
|
||||
if (!result.Success)
|
||||
{
|
||||
var error = result.Messages.FirstOrDefault()?.Message ?? "Unknown error";
|
||||
_logger.LogError("BackfillOpus failed: {Error}", error);
|
||||
return StatusCode(500, error);
|
||||
}
|
||||
|
||||
return Ok(new { enqueued = result.Value.Enqueued, skipped = result.Value.Skipped });
|
||||
}
|
||||
|
||||
// POST api/track/upload: raw audio in (multipart/form-data) + metadata → persisted TrackDto out.
|
||||
// Accepts .wav, .mp3, and .flac. Used by the CMS upload flow on DeepDrftManager; that host
|
||||
// proxies the upload here so it never touches the vault disk path or SQL directly.
|
||||
@@ -642,10 +701,28 @@ public class TrackController : ControllerBase
|
||||
|
||||
// --- Parameterized routes ---
|
||||
|
||||
// GET api/track/{trackId}?format=opus|lossless (unauthenticated)
|
||||
// Streams the track's audio bytes with HTTP Range support. The optional `format` selector (Phase 18.3)
|
||||
// picks the delivery rendering: absent or unrecognized ⇒ Lossless (byte-identical to pre-Phase-18 —
|
||||
// the existing zero-copy disk-stream path, untouched); `opus` ⇒ the derived Ogg Opus 320 artifact
|
||||
// when present, falling back to lossless when it is not (C2 — never 404/silence). The Opus path streams
|
||||
// the resolved artifact from a seekable disk FileStream via File(..., enableRangeProcessing: true) —
|
||||
// no whole-file byte[] — so Range: bytes=X- still yields 206 (load-bearing for streaming + seek),
|
||||
// matching the lossless disk-stream's range behavior.
|
||||
[HttpGet("{trackId}")]
|
||||
public async Task<ActionResult> GetTrack(string trackId)
|
||||
public async Task<ActionResult> GetTrack(string trackId, [FromQuery] string? format = null)
|
||||
{
|
||||
_logger.LogInformation("GetTrack called with trackId: {TrackId}", trackId);
|
||||
_logger.LogInformation("GetTrack called with trackId: {TrackId}, format: {Format}", trackId, format);
|
||||
|
||||
// Only `opus` diverges from today's behavior; everything else (null, "lossless", garbage) takes the
|
||||
// unchanged lossless disk-stream path below, preserving the large-file zero-copy streaming. Routing
|
||||
// lossless through the resolver would force the whole source (up to ~1 GB) into memory per request —
|
||||
// a regression the resolver's in-memory byte[] result is fine for Opus (small) but not for lossless.
|
||||
if (Enum.TryParse<AudioFormat>(format, ignoreCase: true, out var requestedFormat)
|
||||
&& requestedFormat == AudioFormat.Opus)
|
||||
{
|
||||
return await GetTrackOpusAsync(trackId);
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
@@ -664,7 +741,7 @@ public class TrackController : ControllerBase
|
||||
}
|
||||
|
||||
// Resolve MIME and log before handing the stream to File().
|
||||
// If anything here throws, the finally block disposes the wrapper
|
||||
// If anything here throws, the catch block disposes the wrapper
|
||||
// (and its inner FileStream) so neither leaks. On the success path
|
||||
// File() takes ownership of the inner stream; ASP.NET Core disposes
|
||||
// it after the response body is sent. The wrapper is a thin struct
|
||||
@@ -678,16 +755,15 @@ public class TrackController : ControllerBase
|
||||
streamMimeType = MimeTypeExtensions.GetMimeType(mediaStream.Extension);
|
||||
streamLength = mediaStream.Stream.Length;
|
||||
innerStream = mediaStream.Stream;
|
||||
_logger.LogInformation(
|
||||
"Streaming track from disk: {TrackId}, Size: {Size} bytes",
|
||||
trackId, streamLength);
|
||||
}
|
||||
catch
|
||||
{
|
||||
await mediaStream.DisposeAsync();
|
||||
throw;
|
||||
}
|
||||
|
||||
_logger.LogInformation(
|
||||
"Streaming track from disk: {TrackId}, Size: {Size} bytes",
|
||||
trackId, streamLength);
|
||||
// enableRangeProcessing: true — seek is served by HTTP Range requests.
|
||||
// The FileStream is seekable, so ASP.NET Core honours an incoming
|
||||
// Range header by slicing the file and responding 206 Partial Content.
|
||||
@@ -700,6 +776,76 @@ public class TrackController : ControllerBase
|
||||
}
|
||||
}
|
||||
|
||||
// The ?format=opus arm of GetTrack. Resolves the Opus artifact (or the lossless fallback when none
|
||||
// exists, C2) via TrackFormatResolver and streams the resolved bytes from a seekable disk FileStream —
|
||||
// never a whole-file byte[] (a ~220 MB Opus / ~970 MB lossless managed allocation per request was the
|
||||
// read-path OOM defect this closes). enableRangeProcessing:true is load-bearing: the seekable FileStream
|
||||
// lets ASP.NET honour Range: bytes=X- with a 206 slice (seek + Phase 21 windowing). The resolver reports
|
||||
// the *actually-served* format via ResolvedAudio, so the content-type matches the bytes (audio/ogg on a
|
||||
// hit, the source MIME on a fallback) and the eventual client decoder dispatches correctly.
|
||||
//
|
||||
// Disposal mirrors the lossless GetTrack path exactly: File() takes ownership of the stream on success
|
||||
// and disposes it after the response; the inner try disposes ResolvedAudio (and its FileStream) on any
|
||||
// pre-handoff throw so a handle never leaks.
|
||||
private async Task<ActionResult> GetTrackOpusAsync(string trackId)
|
||||
{
|
||||
try
|
||||
{
|
||||
var resolved = await _formatResolver.ResolveAsync(trackId, AudioFormat.Opus);
|
||||
if (resolved is null)
|
||||
{
|
||||
_logger.LogWarning("Track not found for Opus request: {TrackId}", trackId);
|
||||
return NotFound();
|
||||
}
|
||||
|
||||
string contentType;
|
||||
long streamLength;
|
||||
Stream innerStream;
|
||||
try
|
||||
{
|
||||
contentType = resolved.ContentType;
|
||||
// Length from the seekable FileStream — a metadata read, not a body load.
|
||||
streamLength = resolved.Stream.Length;
|
||||
innerStream = resolved.Stream;
|
||||
_logger.LogInformation(
|
||||
"Streaming track {TrackId} as {Format} ({Size} bytes, {ContentType})",
|
||||
trackId, resolved.ResolvedFormat, streamLength, contentType);
|
||||
}
|
||||
catch
|
||||
{
|
||||
await resolved.DisposeAsync();
|
||||
throw;
|
||||
}
|
||||
|
||||
return File(innerStream, contentType, enableRangeProcessing: true);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Error retrieving track as Opus: {TrackId}", trackId);
|
||||
return StatusCode(500, "Internal server error");
|
||||
}
|
||||
}
|
||||
|
||||
// GET api/track/{trackId}/opus/seekdata (unauthenticated)
|
||||
// Returns the Opus setup-header + granule→byte seek-index sidecar bytes (Phase 18.3). The client
|
||||
// fetches this once on track load and parses it into OpusSeekData (18.4) before issuing any Opus seek.
|
||||
// Raw octet-stream — the bytes are the OpusSidecar blob exactly as 18.1 stored them. 404 when no sidecar
|
||||
// is stored (no Opus artifact yet, or an older derive predating the sidecar); the client then degrades
|
||||
// to lossless, mirroring the C2 posture of the audio path. Same public auth posture as the audio stream.
|
||||
// The "opus/seekdata" literal suffix keeps this distinct from the audio and waveform routes.
|
||||
[HttpGet("{trackId}/opus/seekdata")]
|
||||
public async Task<ActionResult> GetOpusSeekData(string trackId)
|
||||
{
|
||||
var sidecar = await _formatResolver.GetOpusSidecarAsync(trackId);
|
||||
if (sidecar is null)
|
||||
{
|
||||
_logger.LogInformation("No Opus sidecar for track: {TrackId}", trackId);
|
||||
return NotFound();
|
||||
}
|
||||
|
||||
return File(sidecar, "application/octet-stream");
|
||||
}
|
||||
|
||||
// GET api/track/{trackId}/waveform (unauthenticated)
|
||||
// Returns the stored waveform loudness profile for a track, base64-encoded. Public listener
|
||||
// data, same auth posture as GET api/track/{trackId} streaming. 404 when no profile is stored
|
||||
@@ -756,15 +902,18 @@ public class TrackController : ControllerBase
|
||||
[HttpPost("{trackId}/waveform")]
|
||||
public async Task<ActionResult> GenerateWaveform(string trackId)
|
||||
{
|
||||
var audio = await _trackContentService.GetAudioBinaryAsync(trackId);
|
||||
if (audio is null)
|
||||
// Streaming compute (Wave 2): the WAV is read from the vault in bounded chunks, never buffered
|
||||
// whole. Tri-state: null = no vault audio (404), false = present but uncomputable / write failed
|
||||
// (500), true = stored.
|
||||
var stored = await _waveformProfileService.ComputeAndStoreProfileStreamingAsync(
|
||||
_ => _trackContentService.OpenAudioStreamAsync(trackId), trackId, HttpContext.RequestAborted);
|
||||
if (stored is null)
|
||||
{
|
||||
_logger.LogWarning("GenerateWaveform: no audio in vault for {TrackId}", trackId);
|
||||
return NotFound();
|
||||
}
|
||||
|
||||
var stored = await _waveformProfileService.ComputeAndStoreAsync(audio.Buffer, trackId);
|
||||
if (!stored)
|
||||
if (stored is false)
|
||||
{
|
||||
_logger.LogError("GenerateWaveform: profile computation/storage failed for {TrackId}", trackId);
|
||||
return StatusCode(500, "Failed to generate waveform profile.");
|
||||
@@ -784,16 +933,27 @@ public class TrackController : ControllerBase
|
||||
[HttpPost("{trackId}/waveform/high-res")]
|
||||
public async Task<ActionResult> GenerateHighResWaveform(string trackId)
|
||||
{
|
||||
var audio = await _trackContentService.GetAudioBinaryAsync(trackId);
|
||||
if (audio is null)
|
||||
// The high-res bucket count is duration-derived. Read the duration from the vault index metadata
|
||||
// (no body load); its absence means the track has no vault audio → 404.
|
||||
var duration = await _trackContentService.GetAudioDurationAsync(trackId);
|
||||
if (duration is null)
|
||||
{
|
||||
_logger.LogWarning("GenerateHighResWaveform: no audio in vault for {TrackId}", trackId);
|
||||
return NotFound();
|
||||
}
|
||||
|
||||
var stored = await _waveformProfileService.ComputeAndStoreHighResAsync(
|
||||
audio.Buffer, trackId, audio.Duration);
|
||||
if (!stored)
|
||||
// Streaming compute (Wave 2): bounded read of the vault WAV. Tri-state mapping as in
|
||||
// GenerateWaveform — null (entry vanished between the metadata read and the compute) → 404.
|
||||
var stored = await _waveformProfileService.ComputeAndStoreHighResStreamingAsync(
|
||||
_ => _trackContentService.OpenAudioStreamAsync(trackId), trackId, duration.Value,
|
||||
HttpContext.RequestAborted);
|
||||
if (stored is null)
|
||||
{
|
||||
_logger.LogWarning("GenerateHighResWaveform: no audio in vault for {TrackId}", trackId);
|
||||
return NotFound();
|
||||
}
|
||||
|
||||
if (stored is false)
|
||||
{
|
||||
_logger.LogError("GenerateHighResWaveform: computation/storage failed for {TrackId}", trackId);
|
||||
return StatusCode(500, "Failed to generate high-res waveform datum.");
|
||||
@@ -802,6 +962,32 @@ public class TrackController : ControllerBase
|
||||
return Ok();
|
||||
}
|
||||
|
||||
// POST api/track/{trackId}/opus ([ApiKeyAuthorize])
|
||||
// Per-track Opus (re)derive trigger (18.5): schedule a single track's background transcode. Enqueue-only
|
||||
// and non-blocking — the transcode runs on the shared serial worker; this returns as soon as it is
|
||||
// scheduled. Re-runnable: overwrites any prior artifact in place. trackId is the EntryKey. 404 when the
|
||||
// track id is unknown. The "opus" literal suffix keeps this distinct from the audio/waveform routes and
|
||||
// from the parameterized PUT "{trackId}". Returns 202 Accepted — the work is queued, not done inline.
|
||||
[ApiKeyAuthorize]
|
||||
[HttpPost("{trackId}/opus")]
|
||||
public async Task<ActionResult> GenerateOpus(string trackId, CancellationToken cancellationToken)
|
||||
{
|
||||
var result = await _unifiedService.EnqueueOpusAsync(trackId, cancellationToken);
|
||||
if (result.Success)
|
||||
{
|
||||
return Accepted();
|
||||
}
|
||||
|
||||
var error = result.Messages.FirstOrDefault()?.Message ?? "Unknown error";
|
||||
if (string.Equals(error, UnifiedTrackService.TrackNotFoundMessage, StringComparison.Ordinal))
|
||||
{
|
||||
return NotFound();
|
||||
}
|
||||
|
||||
_logger.LogError("GenerateOpus failed for {TrackId}: {Error}", trackId, error);
|
||||
return StatusCode(500, error);
|
||||
}
|
||||
|
||||
[ApiKeyAuthorize]
|
||||
[HttpPut("{trackId}")]
|
||||
public async Task<ActionResult> PutTrack(string trackId, [FromBody] AudioBinaryDto track)
|
||||
|
||||
@@ -4,6 +4,7 @@ using DeepDrftAPI;
|
||||
using DeepDrftAPI.Middleware;
|
||||
using DeepDrftAPI.Models;
|
||||
using DeepDrftAPI.Services;
|
||||
using DeepDrftAPI.Services.Opus;
|
||||
using DeepDrftData;
|
||||
using DeepDrftData.Data;
|
||||
using DeepDrftData.Repositories;
|
||||
@@ -66,6 +67,13 @@ builder.Services
|
||||
.AddScoped<ITrackService>(sp => sp.GetRequiredService<TrackManager>());
|
||||
builder.Services.AddScoped<UnifiedTrackService>();
|
||||
|
||||
// Background Opus transcode (Phase 18.1, OQ6). One singleton is both the enqueue seam
|
||||
// (IOpusTranscodeQueue, injected into the scoped UnifiedTrackService) and the hosted drain loop
|
||||
// (IHostedService). It resolves OpusTranscodeService — a singleton — so no scope is captured.
|
||||
builder.Services.AddSingleton<OpusTranscodeBackgroundService>();
|
||||
builder.Services.AddSingleton<IOpusTranscodeQueue>(sp => sp.GetRequiredService<OpusTranscodeBackgroundService>());
|
||||
builder.Services.AddHostedService(sp => sp.GetRequiredService<OpusTranscodeBackgroundService>());
|
||||
|
||||
// Phase 16 anonymous telemetry — append-only event logs + incremental play-counter rollup (all SQL).
|
||||
// EventManager is the IEventService boundary; EventRepository owns the EF writes and the
|
||||
// release-resolution + counter-bump transaction.
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
namespace DeepDrftAPI.Services.Opus;
|
||||
|
||||
/// <summary>
|
||||
/// The enqueue seam for the background Opus transcode (OQ6 / §3.1a). <see cref="UnifiedTrackService"/>
|
||||
/// depends only on this thin interface — not on the worker — so adding the background derive to the
|
||||
/// upload/replace paths costs one small dependency, not the whole transcode graph. Enqueuing is
|
||||
/// non-blocking and best-effort: a freshly uploaded track is already persisted and playable losslessly
|
||||
/// before anything is enqueued, and the transcode runs off the request thread.
|
||||
/// </summary>
|
||||
public interface IOpusTranscodeQueue
|
||||
{
|
||||
/// <summary>
|
||||
/// Schedules a background Opus derive for the track identified by <paramref name="entryKey"/>. Returns
|
||||
/// immediately. A dropped or failed enqueue must not affect the caller — the track remains
|
||||
/// lossless-only and eligible for backfill.
|
||||
/// </summary>
|
||||
void Enqueue(string entryKey);
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
using System.Threading.Channels;
|
||||
using DeepDrftContent.Processors.Opus;
|
||||
|
||||
namespace DeepDrftAPI.Services.Opus;
|
||||
|
||||
/// <summary>
|
||||
/// The background worker behind <see cref="IOpusTranscodeQueue"/> (OQ6 / §3.1a). An unbounded in-process
|
||||
/// channel buffers EntryKeys enqueued by the upload and replace-audio paths; a single hosted loop drains
|
||||
/// them one at a time and runs <see cref="OpusTranscodeService.TranscodeAndStoreAsync"/> for each. Serial
|
||||
/// by design — a transcode is CPU-heavy (§3.1), so running them concurrently would starve request
|
||||
/// handling; one-at-a-time keeps the derive strictly off the hot path without saturating the host.
|
||||
///
|
||||
/// This worker IS the queue (implements <see cref="IOpusTranscodeQueue"/>) so enqueue and drain share one
|
||||
/// channel with no extra indirection. It is registered as a singleton and surfaced under both the
|
||||
/// interface and <see cref="IHostedService"/>.
|
||||
/// </summary>
|
||||
public sealed class OpusTranscodeBackgroundService : BackgroundService, IOpusTranscodeQueue
|
||||
{
|
||||
private readonly Channel<string> _channel =
|
||||
Channel.CreateUnbounded<string>(new UnboundedChannelOptions { SingleReader = true });
|
||||
|
||||
private readonly OpusTranscodeService _transcodeService;
|
||||
private readonly ILogger<OpusTranscodeBackgroundService> _logger;
|
||||
|
||||
public OpusTranscodeBackgroundService(
|
||||
OpusTranscodeService transcodeService,
|
||||
ILogger<OpusTranscodeBackgroundService> logger)
|
||||
{
|
||||
_transcodeService = transcodeService;
|
||||
_logger = logger;
|
||||
}
|
||||
|
||||
public void Enqueue(string entryKey)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(entryKey))
|
||||
return;
|
||||
|
||||
if (!_channel.Writer.TryWrite(entryKey))
|
||||
{
|
||||
// Unbounded writer only rejects after Complete(), i.e. during shutdown. The track stays
|
||||
// lossless-only and is eligible for backfill, so a dropped enqueue is non-fatal — log it.
|
||||
_logger.LogWarning("Opus transcode: could not enqueue {EntryKey} (queue closed).", entryKey);
|
||||
}
|
||||
}
|
||||
|
||||
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
|
||||
{
|
||||
await foreach (var entryKey in _channel.Reader.ReadAllAsync(stoppingToken))
|
||||
{
|
||||
try
|
||||
{
|
||||
await _transcodeService.TranscodeAndStoreAsync(entryKey, stoppingToken);
|
||||
}
|
||||
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
|
||||
{
|
||||
break; // host shutting down
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
// TranscodeAndStoreAsync already swallows expected failures; this guards the loop against
|
||||
// anything unexpected so one bad track never kills the worker.
|
||||
_logger.LogError(ex, "Opus transcode: unhandled failure draining {EntryKey}; worker continues.", entryKey);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public override Task StopAsync(CancellationToken cancellationToken)
|
||||
{
|
||||
_channel.Writer.TryComplete();
|
||||
return base.StopAsync(cancellationToken);
|
||||
}
|
||||
}
|
||||
@@ -143,8 +143,9 @@ public class UnifiedReleaseService
|
||||
return Result.CreateFailResult(MixHasNoTrackMessage);
|
||||
}
|
||||
|
||||
var audio = await _trackContentService.GetAudioBinaryAsync(entryKey);
|
||||
if (audio is null)
|
||||
// Duration from the vault index metadata (no body load); its absence means no vault audio.
|
||||
var duration = await _trackContentService.GetAudioDurationAsync(entryKey);
|
||||
if (duration is null)
|
||||
{
|
||||
_logger.LogWarning("TriggerMixWaveform: no audio in vault for {EntryKey} (release {ReleaseId})", entryKey, releaseId);
|
||||
return Result.CreateFailResult(MixTrackNoAudioMessage);
|
||||
@@ -152,10 +153,18 @@ public class UnifiedReleaseService
|
||||
|
||||
// Duration-derived, constant-time-resolution capture (≈333 samples/sec) so long mixes are not
|
||||
// under-sampled by a fixed bucket count — see WaveformResolution / spec §F. Same per-track
|
||||
// high-res datum every track now carries (phase-12 §5).
|
||||
var computed = await _waveformProfileService.ComputeAndStoreHighResAsync(
|
||||
audio.Buffer, entryKey, audio.Duration);
|
||||
if (!computed)
|
||||
// high-res datum every track now carries (phase-12 §5). Streamed from the vault in bounded
|
||||
// chunks (Wave 2): a ~GB mix is never buffered whole. Tri-state — null = entry vanished after
|
||||
// the metadata read; false = uncomputable / write failed.
|
||||
var computed = await _waveformProfileService.ComputeAndStoreHighResStreamingAsync(
|
||||
_ => _trackContentService.OpenAudioStreamAsync(entryKey), entryKey, duration.Value, ct);
|
||||
if (computed is null)
|
||||
{
|
||||
_logger.LogWarning("TriggerMixWaveform: no audio in vault for {EntryKey} (release {ReleaseId})", entryKey, releaseId);
|
||||
return Result.CreateFailResult(MixTrackNoAudioMessage);
|
||||
}
|
||||
|
||||
if (computed is false)
|
||||
{
|
||||
_logger.LogError("TriggerMixWaveform: waveform computation/storage failed for {EntryKey}", entryKey);
|
||||
return Result.CreateFailResult("Failed to compute the Mix waveform.");
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
using DeepDrftAPI.Services.Opus;
|
||||
using DeepDrftContent;
|
||||
using DeepDrftContent.Constants;
|
||||
using DeepDrftContent.Processors;
|
||||
using DeepDrftContent.Processors.Opus;
|
||||
using DeepDrftData;
|
||||
using DeepDrftModels.DTOs;
|
||||
using DeepDrftModels.Enums;
|
||||
@@ -39,6 +41,8 @@ public class UnifiedTrackService
|
||||
private readonly ITrackService _sqlTrackService;
|
||||
private readonly FileDb _fileDatabase;
|
||||
private readonly WaveformProfileService _waveformProfileService;
|
||||
private readonly IOpusTranscodeQueue _opusTranscodeQueue;
|
||||
private readonly TrackFormatResolver _formatResolver;
|
||||
private readonly ILogger<UnifiedTrackService> _logger;
|
||||
|
||||
public UnifiedTrackService(
|
||||
@@ -46,12 +50,16 @@ public class UnifiedTrackService
|
||||
ITrackService sqlTrackService,
|
||||
FileDb fileDatabase,
|
||||
WaveformProfileService waveformProfileService,
|
||||
IOpusTranscodeQueue opusTranscodeQueue,
|
||||
TrackFormatResolver formatResolver,
|
||||
ILogger<UnifiedTrackService> logger)
|
||||
{
|
||||
_contentTrackContentService = contentTrackContentService;
|
||||
_sqlTrackService = sqlTrackService;
|
||||
_fileDatabase = fileDatabase;
|
||||
_waveformProfileService = waveformProfileService;
|
||||
_opusTranscodeQueue = opusTranscodeQueue;
|
||||
_formatResolver = formatResolver;
|
||||
_logger = logger;
|
||||
}
|
||||
|
||||
@@ -138,7 +146,7 @@ public class UnifiedTrackService
|
||||
}
|
||||
|
||||
var unpersisted = await _contentTrackContentService.AddTrackAsync(
|
||||
tempFilePath, trackName, artist, album, genre, releaseDate, originalFileName: originalFileName);
|
||||
tempFilePath, trackName, artist, album, genre, releaseDate, originalFileName: originalFileName, cancellationToken: ct);
|
||||
|
||||
if (unpersisted is null)
|
||||
{
|
||||
@@ -219,6 +227,11 @@ public class UnifiedTrackService
|
||||
// frontend, so a failure here is logged and swallowed — never fails the upload.
|
||||
await TryStoreWaveformDatumsAsync(unpersisted.EntryKey, ct);
|
||||
|
||||
// Schedule the low-data Opus derive (OQ6 / §3.1a): the track is persisted and lossless-playable
|
||||
// NOW; the transcode + seek-index build run on a background worker. Non-blocking and best-effort
|
||||
// — the upload response never waits on it, and a transcode failure leaves the track lossless-only.
|
||||
_opusTranscodeQueue.Enqueue(unpersisted.EntryKey);
|
||||
|
||||
return saveResult;
|
||||
}
|
||||
|
||||
@@ -269,31 +282,25 @@ public class UnifiedTrackService
|
||||
|
||||
var entryKey = lookup.Value.EntryKey;
|
||||
|
||||
var newAudio = await _contentTrackContentService.ReplaceTrackAudioAsync(entryKey, tempFilePath);
|
||||
if (newAudio is null)
|
||||
var newDuration = await _contentTrackContentService.ReplaceTrackAudioAsync(entryKey, tempFilePath, ct);
|
||||
if (newDuration is null)
|
||||
{
|
||||
_logger.LogWarning("ReplaceAudioAsync: content swap returned null for track {TrackId} ({EntryKey})", trackId, entryKey);
|
||||
return Result.CreateFailResult("Failed to process and store the replacement audio.");
|
||||
}
|
||||
|
||||
// The old waveform no longer matches the new bytes. Regenerate both datums in place; keyed
|
||||
// by the same EntryKey, the re-run overwrites the stale data (proven re-runnable). The
|
||||
// freshly stored buffer is the authoritative source — no re-read of the vault needed.
|
||||
try
|
||||
{
|
||||
await _waveformProfileService.ComputeAndStoreAsync(newAudio.Buffer, entryKey);
|
||||
await _waveformProfileService.ComputeAndStoreHighResAsync(newAudio.Buffer, entryKey, newAudio.Duration);
|
||||
}
|
||||
catch (Exception ex) when (ex is not OperationCanceledException)
|
||||
{
|
||||
_logger.LogError(ex, "ReplaceAudioAsync: waveform regen failed for {EntryKey}; replace unaffected.", entryKey);
|
||||
}
|
||||
// The old waveform no longer matches the new bytes. Regenerate both datums in place, keyed by
|
||||
// the same EntryKey (the re-run overwrites the stale data). The store path no longer hands back
|
||||
// a buffer, so the waveform compute re-reads the freshly stored audio from the vault — the same
|
||||
// path the upload uses. That re-read is now a bounded streaming pass (Wave 2); neither the store
|
||||
// nor the compute holds the whole file. Best-effort throughout: a datum failure never fails the replace.
|
||||
await TryStoreWaveformDatumsAsync(entryKey, ct);
|
||||
|
||||
// Write the new duration to SQL. The vault bytes are already swapped, so this is the
|
||||
// authoritative metadata update for the replace. A failure here is surfaced (unlike the
|
||||
// best-effort waveform regen above) because a stale DurationSeconds silently corrupts
|
||||
// derived aggregates (e.g. MixRuntimeSeconds on the home stats endpoint).
|
||||
var durationWrite = await _sqlTrackService.SetDuration(trackId, newAudio.Duration, ct);
|
||||
var durationWrite = await _sqlTrackService.SetDuration(trackId, newDuration.Value, ct);
|
||||
if (!durationWrite.Success)
|
||||
{
|
||||
var error = durationWrite.Messages.FirstOrDefault()?.Message ?? "Unknown error";
|
||||
@@ -303,20 +310,26 @@ public class UnifiedTrackService
|
||||
return Result.CreateFailResult("Audio replaced but duration metadata could not be updated.");
|
||||
}
|
||||
|
||||
// The stale Opus artifact (if any) no longer matches the new source. Schedule a background
|
||||
// regenerate — the transcode service overwrites the prior artifacts in place keyed by the same
|
||||
// EntryKey. Best-effort, off the request thread, mirrors the waveform regen above.
|
||||
_opusTranscodeQueue.Enqueue(entryKey);
|
||||
|
||||
return Result.CreatePassResult();
|
||||
}
|
||||
|
||||
// Compute and store both waveform datums for a freshly uploaded track: the fixed 512-bucket profile
|
||||
// the player-bar seeker consumes, and the duration-derived high-res datum the lava visualizer
|
||||
// consumes (phase-12 §5 — every track now carries one, computed at upload). Both source the same
|
||||
// audio: read it back from the vault once (the authoritative parsed duration + the stored buffer)
|
||||
// rather than re-reading and re-parsing the temp file. Best-effort throughout — never fails upload.
|
||||
// consumes (phase-12 §5 — every track now carries one, computed at upload). Both are reduced in a
|
||||
// SINGLE streaming pass over the vault audio (Wave 2): the duration comes from the vault index
|
||||
// metadata (no body load) and the PCM is streamed in bounded chunks through two accumulators, so a
|
||||
// ~GB mix never lands its whole body in a managed byte[]. Best-effort throughout — never fails upload.
|
||||
private async Task TryStoreWaveformDatumsAsync(string entryKey, CancellationToken ct)
|
||||
{
|
||||
try
|
||||
{
|
||||
var audio = await _contentTrackContentService.GetAudioBinaryAsync(entryKey);
|
||||
if (audio is null)
|
||||
var duration = await _contentTrackContentService.GetAudioDurationAsync(entryKey);
|
||||
if (duration is null)
|
||||
{
|
||||
_logger.LogWarning(
|
||||
"Waveform datum step: no audio in vault for {EntryKey} immediately after store; skipping.",
|
||||
@@ -324,8 +337,8 @@ public class UnifiedTrackService
|
||||
return;
|
||||
}
|
||||
|
||||
await _waveformProfileService.ComputeAndStoreAsync(audio.Buffer, entryKey);
|
||||
await _waveformProfileService.ComputeAndStoreHighResAsync(audio.Buffer, entryKey, audio.Duration);
|
||||
await _waveformProfileService.ComputeAndStoreAllStreamingAsync(
|
||||
_ => _contentTrackContentService.OpenAudioStreamAsync(entryKey), entryKey, duration.Value, ct);
|
||||
}
|
||||
catch (Exception ex) when (ex is not OperationCanceledException)
|
||||
{
|
||||
@@ -356,8 +369,11 @@ public class UnifiedTrackService
|
||||
{
|
||||
ct.ThrowIfCancellationRequested();
|
||||
|
||||
var audio = await _contentTrackContentService.GetAudioBinaryAsync(track.EntryKey);
|
||||
if (audio is null)
|
||||
// Read the duration from the vault index metadata (no audio body load) — the same value the
|
||||
// processor wrote at upload. Bounds this admin path too (Wave 2): a backfill over a catalogue
|
||||
// of long mixes no longer pulls each whole file into memory just to read its runtime.
|
||||
var duration = await _contentTrackContentService.GetAudioDurationAsync(track.EntryKey);
|
||||
if (duration is null)
|
||||
{
|
||||
_logger.LogWarning("BackfillDurationsAsync: no vault audio for {EntryKey} (track {Id}); skipping.",
|
||||
track.EntryKey, track.Id);
|
||||
@@ -365,7 +381,7 @@ public class UnifiedTrackService
|
||||
continue;
|
||||
}
|
||||
|
||||
var write = await _sqlTrackService.UpdateDuration(track.Id, audio.Duration, ct);
|
||||
var write = await _sqlTrackService.UpdateDuration(track.Id, duration.Value, ct);
|
||||
if (!write.Success)
|
||||
{
|
||||
var error = write.Messages.FirstOrDefault()?.Message ?? "Unknown error";
|
||||
@@ -381,6 +397,69 @@ public class UnifiedTrackService
|
||||
return ResultContainer<(int, int)>.CreatePassResult((updated, skipped));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Backfill-Opus (18.5, OQ4): enqueue a background Opus derive for every non-deleted track that lacks a
|
||||
/// complete Opus artifact (missing audio OR missing sidecar — a half-derived track is treated as missing
|
||||
/// and re-derived). Mirrors the duration-backfill posture: enumerate SQL rows, check each against the
|
||||
/// <c>track-opus</c> vault, schedule the misses. Enqueue-only and non-blocking — the actual transcodes run
|
||||
/// on the shared background worker, serially (the same queue the upload/replace paths feed), so this
|
||||
/// returns as soon as the misses are scheduled rather than waiting on CPU-heavy transcodes. Idempotent:
|
||||
/// a re-run only enqueues tracks still missing Opus, and already-queued/in-flight derives simply overwrite
|
||||
/// in place. Returns (enqueued, skipped) — skipped = tracks that already have a complete Opus artifact.
|
||||
/// </summary>
|
||||
public async Task<ResultContainer<(int Enqueued, int Skipped)>> BackfillOpusAsync(CancellationToken ct)
|
||||
{
|
||||
var all = await _sqlTrackService.GetAll();
|
||||
if (!all.Success || all.Value is null)
|
||||
{
|
||||
var error = all.Messages.FirstOrDefault()?.Message ?? "Unknown error";
|
||||
_logger.LogError("BackfillOpusAsync: failed to load tracks: {Error}", error);
|
||||
return ResultContainer<(int, int)>.CreateFailResult($"Could not load tracks: {error}");
|
||||
}
|
||||
|
||||
var enqueued = 0;
|
||||
var skipped = 0;
|
||||
foreach (var track in all.Value)
|
||||
{
|
||||
ct.ThrowIfCancellationRequested();
|
||||
|
||||
if (await _formatResolver.HasOpusAsync(track.EntryKey))
|
||||
{
|
||||
skipped++;
|
||||
continue;
|
||||
}
|
||||
|
||||
_opusTranscodeQueue.Enqueue(track.EntryKey);
|
||||
enqueued++;
|
||||
}
|
||||
|
||||
_logger.LogInformation("BackfillOpusAsync complete: {Enqueued} enqueued, {Skipped} already had Opus.",
|
||||
enqueued, skipped);
|
||||
return ResultContainer<(int, int)>.CreatePassResult((enqueued, skipped));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Per-track Opus (re)derive trigger (18.5): schedule a background transcode for one track. Returns false
|
||||
/// only when the track id is unknown; the enqueue itself is non-blocking and best-effort, like the bulk
|
||||
/// backfill. Re-runnable — overwrites any prior artifact in place.
|
||||
/// </summary>
|
||||
public async Task<Result> EnqueueOpusAsync(string entryKey, CancellationToken ct)
|
||||
{
|
||||
var lookup = await _sqlTrackService.GetByEntryKey(entryKey);
|
||||
if (!lookup.Success)
|
||||
{
|
||||
var error = lookup.Messages.FirstOrDefault()?.Message ?? "Unknown error";
|
||||
_logger.LogError("EnqueueOpusAsync: lookup failed for {EntryKey}: {Error}", entryKey, error);
|
||||
return Result.CreateFailResult("Failed to load track.");
|
||||
}
|
||||
|
||||
if (lookup.Value is null)
|
||||
return Result.CreateFailResult(TrackNotFoundMessage);
|
||||
|
||||
_opusTranscodeQueue.Enqueue(entryKey);
|
||||
return Result.CreatePassResult();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Delete a track's SQL row, then its vault entry. SQL is the source of truth: a SQL delete
|
||||
/// failure fails the operation (and leaves the vault untouched), but a subsequent vault delete
|
||||
|
||||
@@ -4,6 +4,7 @@ using DeepDrftContent.Constants;
|
||||
using DeepDrftContent.FileDatabase.Models;
|
||||
using DeepDrftContent.FileDatabase.Services;
|
||||
using DeepDrftContent.Processors;
|
||||
using DeepDrftContent.Processors.Opus;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using NetBlocks.Utilities.Environment;
|
||||
|
||||
@@ -44,6 +45,7 @@ namespace DeepDrftAPI
|
||||
InitializeTrackVault(db).GetAwaiter().GetResult();
|
||||
InitializeImageVault(db).GetAwaiter().GetResult();
|
||||
InitializeTrackWaveformsVault(db).GetAwaiter().GetResult();
|
||||
InitializeTrackOpusVault(db).GetAwaiter().GetResult();
|
||||
return db;
|
||||
});
|
||||
|
||||
@@ -65,6 +67,21 @@ namespace DeepDrftAPI
|
||||
Environment.SetEnvironmentVariable("ASPNETCORE_TEMP", stagingPath);
|
||||
builder.Services.AddSingleton(new UploadStagingDirectory(stagingPath));
|
||||
|
||||
// Opus low-data transcode (Phase 18.1). The domain service lives in DeepDrftContent; the host
|
||||
// owns only the engine config and the background worker. Bitrate/ffmpeg-path come from the
|
||||
// OpusTranscode config section; StagingPath is forced to the same data-disk staging directory
|
||||
// the upload path uses so large transcode temp files never land on the /tmp tmpfs.
|
||||
builder.Services.Configure<OpusTranscodeOptions>(
|
||||
builder.Configuration.GetSection(nameof(OpusTranscodeOptions)));
|
||||
builder.Services.PostConfigure<OpusTranscodeOptions>(o => o.StagingPath = stagingPath);
|
||||
builder.Services.AddSingleton<FfmpegOpusEncoder>();
|
||||
builder.Services.AddSingleton<OpusTranscodeService>();
|
||||
|
||||
// Opus delivery format resolution + sidecar lookup (Phase 18.2). The seam 18.3 calls behind
|
||||
// the ?format= stream param and the sidecar path. Stateless over the FileDatabase + content
|
||||
// service singletons; the lossless branch reuses the existing read path unchanged (C2).
|
||||
builder.Services.AddSingleton<TrackFormatResolver>();
|
||||
|
||||
return Task.CompletedTask;
|
||||
}
|
||||
|
||||
@@ -107,5 +124,15 @@ namespace DeepDrftAPI
|
||||
await fileDatabase.CreateVaultAsync(VaultConstants.TrackWaveforms, MediaVaultType.Media);
|
||||
}
|
||||
}
|
||||
|
||||
// Ensure the track-opus vault exists (Phase 18.1). Holds the derived low-data Ogg Opus artifacts
|
||||
// — the Opus audio bytes and the setup-header + seek-index sidecar — keyed by the track's EntryKey.
|
||||
private static async Task InitializeTrackOpusVault(FileDatabase fileDatabase)
|
||||
{
|
||||
if (!fileDatabase.HasVault(VaultConstants.TrackOpus))
|
||||
{
|
||||
await fileDatabase.CreateVaultAsync(VaultConstants.TrackOpus, MediaVaultType.Audio);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,9 +1,7 @@
|
||||
{
|
||||
"Logging": {
|
||||
"LogLevel": {
|
||||
"Default": "Information",
|
||||
"Microsoft.AspNetCore": "Warning",
|
||||
"DeepDrftContent.Controllers.TrackController": "Information"
|
||||
"Default": "Information"
|
||||
}
|
||||
},
|
||||
"AllowedHosts": "*",
|
||||
|
||||
+55
-16
@@ -18,7 +18,9 @@ DeepDrftContent.Services/
|
||||
│ ├── Services/ # FileDatabase, MediaVault, IndexSystem, IndexWatcher
|
||||
│ └── Utils/ # StructuralMap, StructuralSet, FileUtils
|
||||
├── Processors/
|
||||
│ └── AudioProcessor.cs # WAV file parsing, metadata extraction
|
||||
│ ├── AudioProcessor.cs # WAV file parsing, metadata extraction, streaming PCM header parse
|
||||
│ ├── AudioStoreStream.cs # Bounded streaming copy/prefix helpers used by the processors
|
||||
│ └── ProcessedAudio.cs # Store-path plan: metadata + streamed WriteToAsync callback
|
||||
├── Constants/
|
||||
│ └── VaultConstants.cs # Vault name definitions
|
||||
├── TrackService.cs # Content-side orchestrator
|
||||
@@ -47,6 +49,8 @@ FileBinary (base: byte buffer)
|
||||
|
||||
Each has a matching `*Dto` variant for base64 JSON transport (e.g., `AudioBinaryDto` with buffer encoded as base64).
|
||||
|
||||
**Read/load path**: vault reads (`LoadResourceAsync<AudioBinary>`) still return a full-buffer `AudioBinary`. **Write/store path** is streaming: audio processors return a `ProcessedAudio` plan (metadata + `WriteToAsync` callback); `RegisterResourceStreamingAsync` / `MediaVault.AddEntryStreamingAsync` write bytes to a temp file, then `File.Move` atomic-rename into place. The full `AudioBinary` buffer is never materialized on the store path.
|
||||
|
||||
### Index lifecycle
|
||||
|
||||
- **DirectoryIndex**: Root index file (at `{rootPath}/index`). Tracks which vaults exist.
|
||||
@@ -70,19 +74,33 @@ public async Task<bool> RegisterResourceAsync(string vaultId, string entryId, Fi
|
||||
try { /* store and update index */ }
|
||||
catch { return false; } // Swallow, return false
|
||||
}
|
||||
|
||||
// Streaming counterpart (Wave 1 OOM fix) — same bool contract:
|
||||
public async Task<bool> RegisterResourceStreamingAsync(
|
||||
string vaultId, string entryId, MetaData metaData,
|
||||
Func<Stream, CancellationToken, Task> writeContent, CancellationToken ct)
|
||||
{
|
||||
try { /* stream to temp → atomic rename → update index */ }
|
||||
catch { return false; } // Swallow (logs non-cancel exceptions), return false
|
||||
}
|
||||
```
|
||||
|
||||
`MediaVault.AddEntryStreamingAsync` (called internally) writes bytes to a temp file in the vault directory, then `File.Move(temp, final, overwrite: true)` (atomic POSIX rename on the Linux prod host), then updates the index. A cancel or I/O fault before the rename leaves any prior backing file intact; the temp file is cleaned up best-effort.
|
||||
|
||||
**Callers must check return values.** Do not change this without a deliberate design pass — it's embedded in all FileDatabase tests and client code.
|
||||
|
||||
## Audio processors
|
||||
|
||||
Multi-format support via router pattern. All processors live in `DeepDrftContent/Processors/`:
|
||||
|
||||
- `AudioProcessor.ProcessWavFileAsync(filePath)`: WAV-specific processor. Validates RIFF/WAVE structure and format code. Accepts standard PCM (audioFormat=1) and WAVE_FORMAT_EXTENSIBLE (audioFormat=0xFFFE) when the SubFormat GUID indicates PCM. Normalizes EXTENSIBLE-PCM uploads to standard 44-byte PCM WAV before storing. Parses fmt and data chunks; extracts duration and bitrate. Returns `AudioBinary` with metadata. On parse failure, logs warning and returns defaults (180s / 1411 kbps / 44.1 kHz / 16-bit stereo). Accepts standard PCM (audioFormat=1), WAVE_FORMAT_EXTENSIBLE with PCM SubFormat (0x0001), IEEE Float SubFormat (0x0003), and Padded 24-in-32 containers; normalizes Float and padded inputs to standard 24-bit PCM before storage.
|
||||
- `Mp3AudioProcessor.ProcessMp3FileAsync(filePath)`: MP3 processor. Skips ID3v2 tag, finds first valid MPEG frame sync, decodes frame header (bitrate, sample rate, channels). Reads Xing/Info header for VBR total-frame count (accurate duration); VBRI header as fallback; CBR estimate from file size otherwise. Returns `AudioBinary` with original bytes and `.mp3` extension. On parse failure, falls back to defaults (180s / 320 kbps).
|
||||
- `FlacAudioProcessor.ProcessFlacFileAsync(filePath)`: FLAC processor. Validates `fLaC` magic, reads STREAMINFO metadata block (20-bit sample rate, 3-bit channels, 5-bit bits-per-sample, 36-bit total samples — all bit-packed). Computes duration from `totalSamples / sampleRate`; average bitrate from file size. Returns `AudioBinary` with original bytes and `.flac` extension. On parse failure, falls back to defaults (180s / 1411 kbps).
|
||||
- `AudioProcessorRouter.ProcessAudioFileAsync(filePath)`: Routes by extension — `.wav` → `AudioProcessor`, `.mp3` → `Mp3AudioProcessor`, `.flac` → `FlacAudioProcessor`. Throws `ArgumentException` for unsupported extensions.
|
||||
- `WaveformProfileService.ComputeAndStoreHighResAsync(entryKey)`: The shared compute seam for the duration-derived high-res waveform datum (~333 samples/sec). Medium-neutral — computes for any track by `EntryKey`, stores in the `track-waveforms` vault. Called by the upload path (`UnifiedTrackService.UploadAsync` for every new track), the CMS per-row generate action, and the Mix release trigger (now a legacy delegate). Phase 12 generalization of the former Mix-only compute.
|
||||
- `AudioProcessor.ProcessWavFileAsync(filePath)`: WAV-specific processor. Validates RIFF/WAVE structure and format code. Accepts standard PCM (audioFormat=1) and WAVE_FORMAT_EXTENSIBLE (audioFormat=0xFFFE) when the SubFormat GUID indicates PCM. Normalizes EXTENSIBLE-PCM uploads to standard 44-byte PCM WAV before storing. Parses fmt and data chunks; extracts duration and bitrate. **Returns `ProcessedAudio`** (metadata + streamed `WriteToAsync` callback — no whole-file buffer). Header window grows in 64 KB steps capped at 8 MB; the audio body is never read during processing. Standard PCM WAV is stored verbatim (passthrough via `AudioStoreStream.CopyFileAsync`); EXTENSIBLE-PCM / IEEE-float / padded-container WAVs stream their normalization to standard 24-bit PCM. On parse failure, falls back to defaults (180s / 1411 kbps). Also exposes `TryExtractPcm(ReadOnlySpan<byte>)` for the whole-buffer waveform parity oracle and `TryReadPcmStreamInfoAsync(stream, totalLength)` for the streaming waveform compute path (bounded header parse from a stream; returns `WavPcmStreamInfo?` with `DataStart`/`DataLength`/format fields).
|
||||
- `Mp3AudioProcessor.ProcessMp3FileAsync(filePath)`: MP3 processor. Reads a bounded ≤8 MB prefix for header parsing; skips ID3v2 tag, finds first valid MPEG frame sync, decodes frame header (bitrate, sample rate, channels). Reads Xing/Info header for VBR total-frame count (accurate duration); VBRI header as fallback; CBR estimate from file size otherwise. **Returns `ProcessedAudio`** (passthrough plan — MP3 stored unmodified). On parse failure, falls back to defaults (180s / 320 kbps).
|
||||
- `FlacAudioProcessor.ProcessFlacFileAsync(filePath)`: FLAC processor. Reads a bounded ≤64 KB prefix. Validates `fLaC` magic, reads STREAMINFO metadata block (20-bit sample rate, 3-bit channels, 5-bit bits-per-sample, 36-bit total samples — all bit-packed). Computes duration from `totalSamples / sampleRate`; average bitrate from file size. **Returns `ProcessedAudio`** (passthrough plan — FLAC stored unmodified). On parse failure, falls back to defaults (180s / 1411 kbps).
|
||||
- `AudioProcessorRouter.ProcessAudioFileAsync(filePath)`: Routes by extension — `.wav` → `AudioProcessor`, `.mp3` → `Mp3AudioProcessor`, `.flac` → `FlacAudioProcessor`. **Returns `ProcessedAudio?`**. Throws `ArgumentException` for unsupported extensions.
|
||||
- `ProcessedAudio`: Store-path plan returned by all processors. Carries `Extension`, `Duration`, `Bitrate`, `Size`, and a `WriteToAsync(destination, ct)` callback that streams the canonical vault bytes to the destination without materializing the whole file. `ProcessedAudio.Passthrough(sourcePath, ...)` builds a passthrough plan via `AudioStoreStream.CopyFileAsync`.
|
||||
- `AudioStoreStream`: Internal bounded-buffer streaming helpers. `CopyFileAsync(sourcePath, destination, ct)` does a bounded 80 KB disk-to-disk copy; `ReadPrefixAsync(path, cap, ct)` reads at most `cap` bytes from the start of a file (used by processors for header parsing without loading the body).
|
||||
- `ILoudnessAlgorithm` / `ILoudnessAccumulator`: The loudness strategy interface now exposes both `Compute(ReadOnlySpan<byte>, ...)` (whole-buffer, retained as the parity oracle for tests — no production callers) and `CreateAccumulator(pcmByteLength, ...)` → `ILoudnessAccumulator` (streaming: `Add(ReadOnlySpan<byte>)` / `Finish()` → `double[]`). `RmsLoudnessAlgorithm` implements both; `Compute` is defined in terms of the accumulator, so streaming and whole-buffer outputs are byte-identical.
|
||||
- `WaveformProfileService`: Streaming loudness compute + vault store. The whole-buffer `ComputeAndStoreAsync` / `ComputeAndStoreHighResAsync` methods are **retained but have no production callers** — they exist as the byte-identity parity oracle for tests; do not delete them. The production paths are: `ComputeAndStoreProfileStreamingAsync` (512-bucket seeker profile, tri-state `bool?`), `ComputeAndStoreHighResStreamingAsync` (duration-derived high-res datum, tri-state `bool?`), and `ComputeAndStoreAllStreamingAsync` (both datums in a SINGLE streaming pass, used by the upload/replace hot path, tri-state `bool?`). Tri-state: `null` = no backing audio stream; `false` = audio present but not WAV-decodable or vault write failed; `true` = stored. Streaming reads the WAV in bounded ≤80 KB chunks through one accumulator per datum; peak memory is O(bucket arrays + read buffer), independent of file size.
|
||||
- `WaveformResolution`: Enum / constants controlling bucket density for the high-res compute. Renamed from `MixWaveformResolution` in Phase 12.
|
||||
|
||||
Vault stores original bytes with correct extension and MIME type (inferred from file extension or content-type header at upload time).
|
||||
@@ -101,20 +119,41 @@ The primary entry point is `TrackContentService.AddTrackAsync(filePath, mimeType
|
||||
|
||||
## Content-side TrackService (orchestrator)
|
||||
|
||||
### AddTrackFromWavAsync(filePath)
|
||||
### AddTrackAsync(audioFilePath, ...)
|
||||
|
||||
1. Reads a WAV file from disk.
|
||||
2. Calls `AudioProcessor.ProcessWavFileAsync` → `AudioBinary`.
|
||||
3. Generates a GUID entry key (via `Guid.NewGuid().ToString()`).
|
||||
4. Ensures the `tracks` vault exists (creates if missing).
|
||||
5. Calls `FileDatabase.RegisterResourceAsync("tracks", entryKey, audioBinary)`.
|
||||
6. Returns a populated `TrackEntity` (with `Id = 0` since it's not yet in SQL).
|
||||
The primary upload entry point. Format-agnostic — routes by extension via `AudioProcessorRouter`.
|
||||
|
||||
**Note**: The caller (CLI or web) is responsible for then saving this entity to SQL via `DeepDrftWeb.Services.TrackService.Create`. If the vault write succeeds and SQL write fails, audio is orphaned (no compensating rollback).
|
||||
1. Calls `AudioProcessorRouter.ProcessAudioFileAsync(filePath)` → `ProcessedAudio` plan (no whole-file buffer — Wave 1 OOM fix).
|
||||
2. Generates a GUID entry key (via `Guid.NewGuid().ToString()`).
|
||||
3. Ensures the `tracks` vault exists (creates if missing).
|
||||
4. Builds `MetaData` from the plan's header-extracted fields and calls `FileDatabase.RegisterResourceStreamingAsync("tracks", entryKey, metaData, processed.WriteToAsync)` — bytes are streamed from the staging file to the vault via a bounded copy; `MediaVault.AddEntryStreamingAsync` writes to a temp file then atomic-renames into place.
|
||||
5. Returns a populated `TrackEntity` (with `Id = 0` since it's not yet in SQL, and `DurationSeconds` populated from the header parse).
|
||||
|
||||
If the vault write succeeds and SQL write fails, audio is orphaned (no compensating rollback).
|
||||
|
||||
### AddTrackFromWavAsync(filePath, ...)
|
||||
|
||||
Backward-compatible shim — delegates to `AddTrackAsync`. The router accepts WAV alongside MP3 and FLAC, so this carries no WAV-specific logic of its own.
|
||||
|
||||
### ReplaceTrackAudioAsync(entryKey, audioFilePath)
|
||||
|
||||
Swaps the vault bytes for an existing track in place, under the same `entryKey`. Captures the old extension from the vault index metadata (not by loading the file) so a cross-format swap can clean up the stale backing file post-success. Streams the new audio via `ProcessedAudio` + `RegisterResourceStreamingAsync` (no whole-file buffer — Wave 1 OOM fix). Returns the new audio's duration on success, `null` on failure (original audio left intact).
|
||||
|
||||
### GetAudioBinaryAsync(entryKey)
|
||||
|
||||
Reads audio from the `tracks` vault via `FileDatabase.LoadResourceAsync<AudioBinary>("tracks", entryKey)`. Returns `null` if not found or read fails.
|
||||
Reads audio from the `tracks` vault via `FileDatabase.LoadResourceAsync<AudioBinary>("tracks", entryKey)`. Returns a full-buffer `AudioBinary` or `null` if not found. This is a full-buffer convenience read — **not** on the audio-delivery hot path. The delivery path now uses `TrackFormatResolver` (Opus: `MediaVault.GetEntryStreamAsync`; lossless: `OpenAudioMediaStreamAsync`). `GetAudioBinaryAsync` is retained as a read-back oracle used by tests (`AudioStoreStreamingTests`, `TrackReplaceAudioTests`, `TrackFormatDeliveryTests`).
|
||||
|
||||
### OpenAudioStreamAsync(entryKey)
|
||||
|
||||
Returns a read-only, seekable `Stream` over a track's vault audio without buffering the whole file. Used by the streaming waveform compute path (`WaveformProfileService`). Caller owns and must dispose the stream. Returns `null` if the entry has no backing file.
|
||||
|
||||
### OpenAudioMediaStreamAsync(entryKey)
|
||||
|
||||
Returns a `MediaStream?` — a read-only, non-buffering stream over a track's vault audio together with its stored extension, or `null` if the entry has no backing file. Same non-buffering contract as `OpenAudioStreamAsync`, but exposes `.Extension` alongside `.Stream` so the caller can name a staging file by the original format (the Opus transcode stages the source with the correct extension so ffmpeg can detect the format). The caller owns the returned `MediaStream` and must dispose it. Follows the FileDatabase swallow-and-return-null contract.
|
||||
|
||||
### GetAudioDurationAsync(entryKey)
|
||||
|
||||
Reads the stored audio duration from the vault index metadata only — no audio body load. Used by the streaming waveform compute to derive the duration-based bucket count, and by `UnifiedTrackService.BackfillDurationsAsync`. Returns `null` if the entry is unknown or carries no audio metadata.
|
||||
|
||||
### InitializeTracksVaultAsync()
|
||||
|
||||
@@ -122,7 +161,7 @@ Safety call to ensure the `tracks` vault exists (creates if missing). Called on
|
||||
|
||||
## Vault constants
|
||||
|
||||
`VaultConstants.Tracks = "tracks"`, `VaultConstants.Images = "images"`, and `VaultConstants.TrackWaveforms = "track-waveforms"` — the vault names in production use. `TrackWaveforms` holds the per-track high-res waveform datum keyed by `TrackEntity.EntryKey` (Phase 12; renamed from the former `mix-waveforms`, which was Mix-only). New vault names go here when adding new vault types.
|
||||
`VaultConstants.Tracks = "tracks"`, `VaultConstants.Images = "images"`, `VaultConstants.TrackWaveforms = "track-waveforms"`, and `VaultConstants.TrackOpus = "track-opus"` — the vault names in production use. `TrackWaveforms` holds the per-track high-res waveform datum keyed by `TrackEntity.EntryKey` (Phase 12; renamed from the former `mix-waveforms`, which was Mix-only). `TrackOpus` holds two entries per track: the derived Opus audio (keyed by `entryKey`, extension `.opus`) and the combined setup-header + seek-index sidecar (keyed by `{entryKey}-sidecar`, extension `.opusidx`). New vault names go here when adding new vault types.
|
||||
|
||||
## Service registration
|
||||
|
||||
|
||||
@@ -28,4 +28,13 @@ public static class VaultConstants
|
||||
/// The datum resolution is duration-derived (≈333 samples/sec, see <c>WaveformResolution</c>).
|
||||
/// </summary>
|
||||
public const string TrackWaveforms = "track-waveforms";
|
||||
|
||||
/// <summary>
|
||||
/// Vault name for the derived low-data Ogg Opus artifacts, keyed by the track's EntryKey (Phase 18,
|
||||
/// S2). Holds two entries per track: the Opus audio bytes (<c>.opus</c>) and the combined setup-header
|
||||
/// + granule→byte seek-index sidecar (<c>.opusidx</c>). Both are best-effort derived artifacts —
|
||||
/// regenerable, and a track without them still plays losslessly. Distinct from the source <c>tracks</c>
|
||||
/// vault so the source means exactly one thing (mirrors the <c>track-waveforms</c> precedent).
|
||||
/// </summary>
|
||||
public const string TrackOpus = "track-opus";
|
||||
}
|
||||
@@ -206,6 +206,7 @@ public static class MimeTypeExtensions
|
||||
{ ".flac", "audio/flac" },
|
||||
{ ".aac", "audio/aac" },
|
||||
{ ".ogg", "audio/ogg" },
|
||||
{ ".opus", "audio/ogg" },
|
||||
{ ".m4a", "audio/mp4" }
|
||||
};
|
||||
|
||||
|
||||
@@ -178,6 +178,46 @@ public class FileDatabase : DirectoryIndexDirectory, IDisposable
|
||||
return false;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Registers a resource by streaming its bytes into the vault, without materializing the whole
|
||||
/// file in a managed <c>byte[]</c> (the store-path OOM fix). The caller supplies the index
|
||||
/// <paramref name="metaData"/> and a <paramref name="writeContent"/> callback that emits bytes to
|
||||
/// the backing stream. Swallows exceptions and returns false, matching
|
||||
/// <see cref="RegisterResourceAsync"/>'s contract — callers check the bool.
|
||||
/// </summary>
|
||||
public async Task<bool> RegisterResourceStreamingAsync(
|
||||
string vaultId,
|
||||
string entryId,
|
||||
MetaData metaData,
|
||||
Func<Stream, CancellationToken, Task> writeContent,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
try
|
||||
{
|
||||
var directoryVault = _vaults.Get(vaultId);
|
||||
if (directoryVault != null)
|
||||
{
|
||||
var written = await directoryVault.AddEntryStreamingAsync(entryId, metaData, writeContent, cancellationToken);
|
||||
_logger.LogInformation(
|
||||
"Streamed {Bytes} bytes into vault {VaultId} entry {EntryId} (no whole-file buffer).",
|
||||
written, vaultId, entryId);
|
||||
return true;
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
// Swallow and return false, matching RegisterResourceAsync. Log at error for real failures
|
||||
// only — a normal client cancel (OperationCanceledException) is not an error condition and
|
||||
// would spam the error log on every client disconnect during a large upload or replace.
|
||||
if (ex is not OperationCanceledException)
|
||||
{
|
||||
_logger.LogError(ex, "RegisterResourceStreamingAsync failed for vault {VaultId} entry {EntryId}", vaultId, entryId);
|
||||
}
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Removes a resource from a specific vault. Returns null if the vault does not exist,
|
||||
/// false if the entry was not found, true if the entry was removed. Distinguishing
|
||||
|
||||
@@ -56,6 +56,63 @@ public abstract class MediaVault : VaultIndexDirectory
|
||||
await FileUtils.PutFileAsync(mediaPath, buffer);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Streams an entry's bytes into the vault without ever materializing the whole file in memory.
|
||||
/// The metadata is supplied by the caller (there is no in-memory <see cref="FileBinary"/> to infer
|
||||
/// it from) — the store path (upload / replace-audio) sources its bytes from a staging file, not a
|
||||
/// buffer. Returns the number of bytes written, for the caller to log.
|
||||
///
|
||||
/// Write ordering (atomic-replace guarantee): bytes are streamed to a temp file in the same vault
|
||||
/// directory, the temp file is renamed over the final backing-file path (POSIX <c>rename(2)</c> —
|
||||
/// atomic on the Linux prod host), and the index is updated only after the rename succeeds.
|
||||
/// This ordering ensures: (a) the index never advertises a not-yet-present file; (b) a client
|
||||
/// disconnect or I/O fault during the write leaves any prior backing file intact and the index
|
||||
/// unchanged; (c) the temp file is cleaned up best-effort on any failure before re-throwing so the
|
||||
/// vault directory stays tidy. The caller treats a thrown exception as a failed register.
|
||||
/// </summary>
|
||||
public async Task<long> AddEntryStreamingAsync(
|
||||
string entryId,
|
||||
MetaData metaData,
|
||||
Func<Stream, CancellationToken, Task> writeContent,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
var finalPath = GetMediaPathFromEntryKey(entryId, metaData.Extension);
|
||||
var tempPath = Path.Combine(RootPath, Path.GetRandomFileName() + ".tmp");
|
||||
|
||||
try
|
||||
{
|
||||
long bytesWritten;
|
||||
await using (var tempStream = new FileStream(
|
||||
tempPath, FileMode.CreateNew, FileAccess.Write, FileShare.None,
|
||||
bufferSize: 81920, useAsync: true))
|
||||
{
|
||||
await writeContent(tempStream, cancellationToken);
|
||||
await tempStream.FlushAsync(cancellationToken);
|
||||
bytesWritten = tempStream.Length;
|
||||
}
|
||||
|
||||
// Rename into place — atomic on the Linux prod host (POSIX rename(2)); overwrites any
|
||||
// existing same-extension backing file safely on the replace path.
|
||||
File.Move(tempPath, finalPath, overwrite: true);
|
||||
|
||||
// Update the index only after the file is durably in place. A crash between Move and
|
||||
// AddToIndexAsync leaves an unreferenced file on disk (a harmless orphan recoverable
|
||||
// by a vault scan); a crash or cancel during the temp write leaves the original backing
|
||||
// file and the index both unchanged.
|
||||
await AddToIndexAsync(entryId, metaData);
|
||||
|
||||
return bytesWritten;
|
||||
}
|
||||
catch
|
||||
{
|
||||
// Best-effort temp-file cleanup. After a successful rename tempPath is gone and the
|
||||
// delete is a no-op. After a write failure or cancel tempPath holds partial bytes that
|
||||
// must be removed so the vault directory stays tidy.
|
||||
try { if (File.Exists(tempPath)) File.Delete(tempPath); } catch { /* best-effort */ }
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Retrieves an entry from the vault (MediaVaultType inferred from T)
|
||||
/// </summary>
|
||||
|
||||
@@ -7,12 +7,22 @@ namespace DeepDrftContent.Processors;
|
||||
/// </summary>
|
||||
public class AudioProcessor
|
||||
{
|
||||
// Header parsing never needs the audio body. Read the file in 64 KB steps until the data-chunk
|
||||
// header is locatable, capping the window so a pathological file with an enormous pre-data header
|
||||
// cannot drive an unbounded allocation — such a file simply falls through to default metadata and
|
||||
// passthrough storage, the same outcome as any unparseable WAV.
|
||||
private const int HeaderWindowStep = 64 * 1024;
|
||||
private const int HeaderWindowCap = 8 * 1024 * 1024;
|
||||
|
||||
/// <summary>
|
||||
/// Processes a WAV file and creates an AudioBinary object
|
||||
/// Processes a WAV file into a <see cref="ProcessedAudio"/> store plan: extracts metadata from a
|
||||
/// bounded header window (never the whole file) and returns a streamed writer for the canonical
|
||||
/// vault bytes. Standard PCM is stored verbatim (passthrough copy); EXTENSIBLE-PCM / IEEE-float /
|
||||
/// padded-container WAVs are normalized to a plain 44-byte standard-PCM WAV, written progressively
|
||||
/// so the vault only ever holds a format the streaming pipeline already handles.
|
||||
/// </summary>
|
||||
/// <param name="filePath">Path to the WAV file</param>
|
||||
/// <returns>AudioBinary object with metadata</returns>
|
||||
public async Task<AudioBinary?> ProcessWavFileAsync(string filePath)
|
||||
public async Task<ProcessedAudio?> ProcessWavFileAsync(string filePath, CancellationToken cancellationToken = default)
|
||||
{
|
||||
if (!File.Exists(filePath))
|
||||
{
|
||||
@@ -26,30 +36,197 @@ public class AudioProcessor
|
||||
|
||||
try
|
||||
{
|
||||
var buffer = await File.ReadAllBytesAsync(filePath);
|
||||
var wavInfo = ExtractWavMetadata(buffer);
|
||||
var fileLength = new FileInfo(filePath).Length;
|
||||
var window = await ReadWavHeaderWindowAsync(filePath, cancellationToken);
|
||||
var wavInfo = ExtractWavMetadata(window);
|
||||
|
||||
// EXTENSIBLE-PCM is byte-compatible with standard PCM but carries a 40+ byte fmt chunk
|
||||
// the streaming pipeline never expects. Normalize to a plain 44-byte PCM WAV at storage
|
||||
// time so the vault only ever holds standard PCM and the client decode path stays unchanged.
|
||||
var storedBuffer = wavInfo.IsExtensible ? NormalizeToStandardPcm(buffer, wavInfo) : buffer;
|
||||
if (!wavInfo.IsExtensible)
|
||||
{
|
||||
// Standard PCM (or the default-fallback path, which reports IsExtensible = false):
|
||||
// the source bytes are already a format the pipeline handles, so store them verbatim.
|
||||
return ProcessedAudio.Passthrough(filePath, ".wav", wavInfo.Duration, wavInfo.Bitrate, fileLength);
|
||||
}
|
||||
|
||||
var parameters = new AudioBinaryParams(
|
||||
Buffer: storedBuffer,
|
||||
Size: storedBuffer.Length,
|
||||
Extension: ".wav",
|
||||
Duration: wavInfo.Duration,
|
||||
Bitrate: wavInfo.Bitrate
|
||||
);
|
||||
// EXTENSIBLE → streamed normalization. The output data size is derivable from the source
|
||||
// data size alone (no body read needed): verbatim keeps it, float drops 1 byte per sample
|
||||
// (4→3), padded keeps only the valid bytes per container sample.
|
||||
var dataStart = (long)wavInfo.DataChunkPos + 8;
|
||||
var available = fileLength - dataStart;
|
||||
var srcDataSize = Math.Min((long)wavInfo.DataSize, available);
|
||||
|
||||
return new AudioBinary(parameters);
|
||||
NormalizeMode mode;
|
||||
int outBitsPerSample;
|
||||
long outDataSize;
|
||||
int containerBytes = 0;
|
||||
int validBytes = 0;
|
||||
if (wavInfo.IsFloat)
|
||||
{
|
||||
mode = NormalizeMode.Float;
|
||||
outBitsPerSample = 24;
|
||||
outDataSize = (srcDataSize / 4) * 3;
|
||||
}
|
||||
else if (wavInfo.IsPaddedContainer)
|
||||
{
|
||||
mode = NormalizeMode.Padded;
|
||||
outBitsPerSample = wavInfo.BitsPerSample;
|
||||
containerBytes = wavInfo.ContainerBitsPerSample / 8;
|
||||
validBytes = wavInfo.BitsPerSample / 8;
|
||||
outDataSize = (srcDataSize / containerBytes) * validBytes;
|
||||
}
|
||||
else
|
||||
{
|
||||
mode = NormalizeMode.Verbatim;
|
||||
outBitsPerSample = wavInfo.BitsPerSample;
|
||||
outDataSize = srcDataSize;
|
||||
}
|
||||
|
||||
var channels = wavInfo.Channels;
|
||||
var sampleRate = wavInfo.SampleRate;
|
||||
|
||||
return new ProcessedAudio(
|
||||
".wav", wavInfo.Duration, wavInfo.Bitrate, 44 + outDataSize,
|
||||
(destination, ct) => WriteNormalizedWavAsync(
|
||||
filePath, dataStart, srcDataSize, channels, sampleRate, outBitsPerSample,
|
||||
outDataSize, mode, containerBytes, validBytes, destination, ct));
|
||||
}
|
||||
catch (Exception ex)
|
||||
catch (Exception ex) when (ex is not OperationCanceledException)
|
||||
{
|
||||
throw new InvalidOperationException($"Failed to process WAV file: {ex.Message}", ex);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Reads only enough of the file to contain the fmt chunk and the data chunk's 8-byte header, so
|
||||
/// metadata parsing never loads the (potentially ~GB) audio body. Grows the window in 64 KB steps
|
||||
/// until the data chunk is locatable or EOF/<see cref="HeaderWindowCap"/> is hit.
|
||||
/// </summary>
|
||||
private static async Task<byte[]> ReadWavHeaderWindowAsync(string filePath, CancellationToken ct)
|
||||
{
|
||||
await using var fs = new FileStream(
|
||||
filePath, FileMode.Open, FileAccess.Read, FileShare.Read,
|
||||
bufferSize: HeaderWindowStep, useAsync: true);
|
||||
|
||||
using var ms = new MemoryStream();
|
||||
var buffer = new byte[HeaderWindowStep];
|
||||
while (ms.Length < HeaderWindowCap)
|
||||
{
|
||||
var read = await fs.ReadAsync(buffer, ct);
|
||||
if (read == 0)
|
||||
break;
|
||||
ms.Write(buffer, 0, read);
|
||||
|
||||
// FindChunk returns -1 on a partial window (the data chunk isn't reachable yet), so keep
|
||||
// reading until it is found or the cap/EOF is hit. On normal files the data chunk header
|
||||
// sits within the first 64 KB, so this loop runs exactly once.
|
||||
var soFar = ms.ToArray();
|
||||
if (FindChunk(soFar, "data") >= 0)
|
||||
return soFar;
|
||||
}
|
||||
|
||||
return ms.ToArray();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Writes a normalized standard-PCM WAV to <paramref name="destination"/>: the 44-byte header
|
||||
/// followed by the data region streamed from the source in bounded, sample-aligned chunks. No
|
||||
/// whole-file buffer is ever held — peak memory is O(chunk), independent of duration.
|
||||
/// </summary>
|
||||
private async Task WriteNormalizedWavAsync(
|
||||
string sourcePath, long dataStart, long srcDataSize,
|
||||
int channels, int sampleRate, int outBitsPerSample, long outDataSize,
|
||||
NormalizeMode mode, int containerBytes, int validBytes,
|
||||
Stream destination, CancellationToken ct)
|
||||
{
|
||||
var header = BuildStandardPcmHeader(channels, sampleRate, outBitsPerSample, outDataSize);
|
||||
await destination.WriteAsync(header, ct);
|
||||
|
||||
await using var src = new FileStream(
|
||||
sourcePath, FileMode.Open, FileAccess.Read, FileShare.Read,
|
||||
bufferSize: 81920, useAsync: true);
|
||||
src.Seek(dataStart, SeekOrigin.Begin);
|
||||
|
||||
switch (mode)
|
||||
{
|
||||
case NormalizeMode.Verbatim:
|
||||
await CopyBoundedAsync(src, destination, srcDataSize, ct);
|
||||
break;
|
||||
case NormalizeMode.Float:
|
||||
// Each 4-byte float sample becomes 3 bytes of 24-bit PCM.
|
||||
await TransformBoundedAsync(src, destination, srcDataSize, unit: 4,
|
||||
transform: (buf, len) => ConvertFloatTo24BitPcm(buf, 0, len), ct);
|
||||
break;
|
||||
case NormalizeMode.Padded:
|
||||
await TransformBoundedAsync(src, destination, srcDataSize, unit: containerBytes,
|
||||
transform: (buf, len) => RepackPaddedContainer(buf, 0, len, containerBytes * 8, validBytes * 8), ct);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Bounded copy of exactly <paramref name="totalBytes"/> from src to dest.</summary>
|
||||
private static async Task CopyBoundedAsync(Stream src, Stream dest, long totalBytes, CancellationToken ct)
|
||||
{
|
||||
var buffer = new byte[81920];
|
||||
var remaining = totalBytes;
|
||||
while (remaining > 0)
|
||||
{
|
||||
var want = (int)Math.Min(buffer.Length, remaining);
|
||||
var read = await src.ReadAsync(buffer.AsMemory(0, want), ct);
|
||||
if (read == 0)
|
||||
break;
|
||||
await dest.WriteAsync(buffer.AsMemory(0, read), ct);
|
||||
remaining -= read;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Streams <paramref name="totalBytes"/> of source data through <paramref name="transform"/> in
|
||||
/// sample-aligned chunks, writing each transformed chunk to <paramref name="dest"/>. The read
|
||||
/// buffer is a multiple of <paramref name="unit"/>; leftover bytes that do not complete a sample
|
||||
/// are carried into the next read, and a final partial sample is dropped (matching the
|
||||
/// whole-buffer transforms' integer-division behavior).
|
||||
/// </summary>
|
||||
private static async Task TransformBoundedAsync(
|
||||
Stream src, Stream dest, long totalBytes, int unit,
|
||||
Func<byte[], int, byte[]> transform, CancellationToken ct)
|
||||
{
|
||||
var bufLen = Math.Max(unit, (81920 / unit) * unit);
|
||||
var buffer = new byte[bufLen];
|
||||
var remaining = totalBytes;
|
||||
var carried = 0;
|
||||
while (remaining > 0)
|
||||
{
|
||||
var want = (int)Math.Min(bufLen - carried, remaining);
|
||||
if (want == 0)
|
||||
break;
|
||||
var read = await src.ReadAsync(buffer.AsMemory(carried, want), ct);
|
||||
if (read == 0)
|
||||
break;
|
||||
remaining -= read;
|
||||
|
||||
var filled = carried + read;
|
||||
var whole = (filled / unit) * unit;
|
||||
if (whole > 0)
|
||||
{
|
||||
var output = transform(buffer, whole);
|
||||
await dest.WriteAsync(output, ct);
|
||||
}
|
||||
|
||||
carried = filled - whole;
|
||||
if (carried > 0)
|
||||
Array.Copy(buffer, whole, buffer, 0, carried);
|
||||
}
|
||||
}
|
||||
|
||||
private enum NormalizeMode
|
||||
{
|
||||
/// <summary>Sample bytes already standard PCM (EXTENSIBLE-PCM, depth == container width).</summary>
|
||||
Verbatim,
|
||||
/// <summary>IEEE float samples converted to 24-bit PCM.</summary>
|
||||
Float,
|
||||
/// <summary>Padded container (e.g. 24-in-32) re-packed to the valid depth.</summary>
|
||||
Padded
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Extracts the raw PCM data region and format parameters from a WAV buffer, reusing the
|
||||
/// same chunk-walk and validation as metadata extraction. Returns null if the buffer is not
|
||||
@@ -111,6 +288,105 @@ public class AudioProcessor
|
||||
return new PcmData(pcm, metadata.Channels, metadata.SampleRate, metadata.BitsPerSample);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Reads only the WAV header region from <paramref name="stream"/> (a bounded window, never the
|
||||
/// audio body) and returns where the PCM data region begins, how long it is, and the format
|
||||
/// parameters needed to decode it — the streaming counterpart of <see cref="TryExtractPcm"/>. The
|
||||
/// data length is clamped against <paramref name="totalFileLength"/> (the true backing-file size),
|
||||
/// so the caller streams exactly the present PCM. Returns null for the same inputs
|
||||
/// <see cref="TryExtractPcm"/> rejects — non-WAV bytes (mp3/flac), float, and padded-container
|
||||
/// EXTENSIBLE — so the caller treats null as "no profile computable" and continues gracefully.
|
||||
///
|
||||
/// <paramref name="stream"/> must be positioned at the start; on return its position is past the
|
||||
/// header window (the caller seeks to <c>DataStart</c> before streaming the body). No whole-file
|
||||
/// buffer is allocated — peak memory is the bounded header window.
|
||||
/// </summary>
|
||||
public async Task<WavPcmStreamInfo?> TryReadPcmStreamInfoAsync(
|
||||
Stream stream, long totalFileLength, CancellationToken cancellationToken = default)
|
||||
{
|
||||
var window = await ReadWavHeaderWindowAsync(stream, cancellationToken);
|
||||
if (window is null)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
var validation = ValidateWavStructure(window);
|
||||
if (!validation.IsValid || validation.IsFloat)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
WavMetadata metadata;
|
||||
try
|
||||
{
|
||||
metadata = ParseWavMetadata(window, validation);
|
||||
ValidateAudioParameters(metadata);
|
||||
if (metadata.IsPaddedContainer)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
}
|
||||
catch
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
long dataStart = validation.DataChunkPos + 8;
|
||||
if (dataStart > totalFileLength)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
var available = totalFileLength - dataStart;
|
||||
var dataLength = Math.Min((long)metadata.DataSize, available);
|
||||
if (dataLength <= 0)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
return new WavPcmStreamInfo(
|
||||
dataStart, dataLength, metadata.Channels, metadata.SampleRate, metadata.BitsPerSample);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Reads enough of <paramref name="stream"/> to contain the fmt chunk and the data chunk's 8-byte
|
||||
/// header, growing in 64 KB steps until the data chunk is locatable or EOF / the
|
||||
/// <see cref="HeaderWindowCap"/> is reached. Bails after the first read when the bytes are not a
|
||||
/// RIFF/WAVE container, so a non-WAV stream (mp3/flac) costs one read, not the full cap. Returns
|
||||
/// null only when nothing could be read.
|
||||
/// </summary>
|
||||
private static async Task<byte[]?> ReadWavHeaderWindowAsync(Stream stream, CancellationToken ct)
|
||||
{
|
||||
using var ms = new MemoryStream();
|
||||
var buffer = new byte[HeaderWindowStep];
|
||||
while (ms.Length < HeaderWindowCap)
|
||||
{
|
||||
var read = await stream.ReadAsync(buffer, ct);
|
||||
if (read == 0)
|
||||
break;
|
||||
ms.Write(buffer, 0, read);
|
||||
|
||||
var soFar = ms.ToArray();
|
||||
|
||||
// Early-out for non-WAV input: once at least the 12-byte RIFF/WAVE preamble is in hand,
|
||||
// a missing signature means this will never be a WAV — stop rather than read to the cap.
|
||||
if (soFar.Length >= 12 && !HasRiffWaveSignature(soFar))
|
||||
return soFar;
|
||||
|
||||
// FindChunk returns -1 until the data chunk header is fully in the window; on a normal
|
||||
// file it sits within the first 64 KB so this loop runs exactly once.
|
||||
if (FindChunk(soFar, "data") >= 0)
|
||||
return soFar;
|
||||
}
|
||||
|
||||
return ms.Length > 0 ? ms.ToArray() : null;
|
||||
}
|
||||
|
||||
private static bool HasRiffWaveSignature(byte[] buffer) =>
|
||||
buffer.Length >= 12
|
||||
&& System.Text.Encoding.ASCII.GetString(buffer, 0, 4) == "RIFF"
|
||||
&& System.Text.Encoding.ASCII.GetString(buffer, 8, 4) == "WAVE";
|
||||
|
||||
/// <summary>
|
||||
/// Extracts metadata from WAV file buffer with comprehensive validation
|
||||
/// </summary>
|
||||
@@ -317,50 +593,17 @@ public class AudioProcessor
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Rebuilds an EXTENSIBLE WAV as a canonical 44-byte-header standard PCM WAV (audioFormat = 1)
|
||||
/// so the vault only ever holds a format the streaming pipeline already handles. Three source
|
||||
/// shapes are normalized:
|
||||
/// <list type="bullet">
|
||||
/// <item>EXTENSIBLE-PCM (depth == container): sample bytes are byte-identical to standard PCM and
|
||||
/// copied verbatim; only the header is replaced.</item>
|
||||
/// <item>IEEE float: 32-bit float samples are converted to 24-bit signed integer PCM.</item>
|
||||
/// <item>Padded container (e.g. 24-in-32): the padding/sign-extension bytes are stripped, keeping
|
||||
/// the lowest valid bytes per sample.</item>
|
||||
/// </list>
|
||||
/// The output header always reports the valid bit depth (<see cref="WavMetadata.BitsPerSample"/>).
|
||||
/// Builds the canonical 44-byte standard-PCM WAV header (audioFormat = 1) for a normalized stream.
|
||||
/// The body is written separately so no whole-file buffer is allocated; this only emits the header
|
||||
/// the streaming pipeline expects, reporting the valid (post-normalization) bit depth.
|
||||
/// </summary>
|
||||
private byte[] NormalizeToStandardPcm(byte[] buffer, WavMetadata metadata)
|
||||
private static byte[] BuildStandardPcmHeader(int channels, int sampleRate, int outBitsPerSample, long dataSize)
|
||||
{
|
||||
// Clamp the declared data size to what is actually present; some encoders overshoot.
|
||||
var dataStart = metadata.DataChunkPos + 8;
|
||||
var available = buffer.Length - dataStart;
|
||||
var srcDataSize = Math.Min(metadata.DataSize, available);
|
||||
|
||||
byte[] dataBytes;
|
||||
int outBitsPerSample;
|
||||
if (metadata.IsFloat)
|
||||
{
|
||||
dataBytes = ConvertFloatTo24BitPcm(buffer, dataStart, srcDataSize);
|
||||
outBitsPerSample = 24;
|
||||
}
|
||||
else if (metadata.IsPaddedContainer)
|
||||
{
|
||||
dataBytes = RepackPaddedContainer(buffer, dataStart, srcDataSize, metadata.ContainerBitsPerSample, metadata.BitsPerSample);
|
||||
outBitsPerSample = metadata.BitsPerSample;
|
||||
}
|
||||
else
|
||||
{
|
||||
dataBytes = new byte[srcDataSize];
|
||||
Array.Copy(buffer, dataStart, dataBytes, 0, srcDataSize);
|
||||
outBitsPerSample = metadata.BitsPerSample;
|
||||
}
|
||||
|
||||
var dataSize = dataBytes.Length;
|
||||
const int headerSize = 44;
|
||||
var result = new byte[headerSize + dataSize];
|
||||
var result = new byte[headerSize];
|
||||
|
||||
var blockAlign = (ushort)(metadata.Channels * (outBitsPerSample / 8));
|
||||
var byteRate = (uint)(metadata.SampleRate * blockAlign);
|
||||
var blockAlign = (ushort)(channels * (outBitsPerSample / 8));
|
||||
var byteRate = (uint)(sampleRate * blockAlign);
|
||||
|
||||
// RIFF header
|
||||
System.Text.Encoding.ASCII.GetBytes("RIFF").CopyTo(result, 0);
|
||||
@@ -371,8 +614,8 @@ public class AudioProcessor
|
||||
System.Text.Encoding.ASCII.GetBytes("fmt ").CopyTo(result, 12);
|
||||
BitConverter.GetBytes((uint)16).CopyTo(result, 16);
|
||||
BitConverter.GetBytes((ushort)1).CopyTo(result, 20); // audioFormat = PCM
|
||||
BitConverter.GetBytes((ushort)metadata.Channels).CopyTo(result, 22);
|
||||
BitConverter.GetBytes((uint)metadata.SampleRate).CopyTo(result, 24);
|
||||
BitConverter.GetBytes((ushort)channels).CopyTo(result, 22);
|
||||
BitConverter.GetBytes((uint)sampleRate).CopyTo(result, 24);
|
||||
BitConverter.GetBytes(byteRate).CopyTo(result, 28);
|
||||
BitConverter.GetBytes(blockAlign).CopyTo(result, 32);
|
||||
BitConverter.GetBytes((ushort)outBitsPerSample).CopyTo(result, 34);
|
||||
@@ -381,8 +624,6 @@ public class AudioProcessor
|
||||
System.Text.Encoding.ASCII.GetBytes("data").CopyTo(result, 36);
|
||||
BitConverter.GetBytes((uint)dataSize).CopyTo(result, 40);
|
||||
|
||||
Array.Copy(dataBytes, 0, result, headerSize, dataSize);
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -459,7 +700,7 @@ public class AudioProcessor
|
||||
/// <summary>
|
||||
/// Finds a chunk in the WAV file buffer with proper alignment handling
|
||||
/// </summary>
|
||||
private int FindChunk(byte[] buffer, string chunkId)
|
||||
private static int FindChunk(byte[] buffer, string chunkId)
|
||||
{
|
||||
var chunkBytes = System.Text.Encoding.ASCII.GetBytes(chunkId);
|
||||
int offset = 12; // Start after RIFF header
|
||||
@@ -556,4 +797,21 @@ public readonly record struct PcmData(
|
||||
ReadOnlyMemory<byte> Pcm,
|
||||
int Channels,
|
||||
int SampleRate,
|
||||
int BitsPerSample);
|
||||
|
||||
/// <summary>
|
||||
/// Where a WAV's PCM data region lives and how to decode it, without the bytes themselves — the
|
||||
/// streaming counterpart of <see cref="PcmData"/>. The caller seeks to <see cref="DataStart"/> and
|
||||
/// streams exactly <see cref="DataLength"/> bytes through a loudness accumulator.
|
||||
/// </summary>
|
||||
/// <param name="DataStart">Absolute byte offset of the first PCM sample (past the data chunk header).</param>
|
||||
/// <param name="DataLength">PCM region length in bytes, clamped to what the backing file actually holds.</param>
|
||||
/// <param name="Channels">Number of interleaved channels.</param>
|
||||
/// <param name="SampleRate">Samples per second.</param>
|
||||
/// <param name="BitsPerSample">Bit depth per sample (8, 16, 24, or 32).</param>
|
||||
public readonly record struct WavPcmStreamInfo(
|
||||
long DataStart,
|
||||
long DataLength,
|
||||
int Channels,
|
||||
int SampleRate,
|
||||
int BitsPerSample);
|
||||
@@ -24,18 +24,18 @@ public class AudioProcessorRouter
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Processes <paramref name="filePath"/> with the processor matching its extension, returning an
|
||||
/// <see cref="AudioBinary"/> carrying the stored bytes and extracted metadata. Throws
|
||||
/// <see cref="ArgumentException"/> for unsupported extensions.
|
||||
/// Processes <paramref name="filePath"/> with the processor matching its extension, returning a
|
||||
/// <see cref="ProcessedAudio"/> store plan (extracted metadata plus a streamed writer for the
|
||||
/// canonical vault bytes). Throws <see cref="ArgumentException"/> for unsupported extensions.
|
||||
/// </summary>
|
||||
public async Task<AudioBinary?> ProcessAudioFileAsync(string filePath)
|
||||
public async Task<ProcessedAudio?> ProcessAudioFileAsync(string filePath, CancellationToken cancellationToken = default)
|
||||
{
|
||||
var ext = Path.GetExtension(filePath).ToLowerInvariant();
|
||||
return ext switch
|
||||
{
|
||||
".wav" => await _wavProcessor.ProcessWavFileAsync(filePath),
|
||||
".mp3" => await _mp3Processor.ProcessMp3FileAsync(filePath),
|
||||
".flac" => await _flacProcessor.ProcessFlacFileAsync(filePath),
|
||||
".wav" => await _wavProcessor.ProcessWavFileAsync(filePath, cancellationToken),
|
||||
".mp3" => await _mp3Processor.ProcessMp3FileAsync(filePath, cancellationToken),
|
||||
".flac" => await _flacProcessor.ProcessFlacFileAsync(filePath, cancellationToken),
|
||||
_ => throw new ArgumentException($"Unsupported audio format: {ext}", nameof(filePath)),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
namespace DeepDrftContent.Processors;
|
||||
|
||||
/// <summary>
|
||||
/// Bounded-buffer streaming primitives shared by the audio processors on the store path. None of
|
||||
/// these hold the whole file in memory: copies move a fixed window at a time, and the header read
|
||||
/// caps its allocation regardless of file size.
|
||||
/// </summary>
|
||||
internal static class AudioStoreStream
|
||||
{
|
||||
private const int CopyBufferSize = 81920; // 80 KB — matches the controller staging copy.
|
||||
|
||||
/// <summary>
|
||||
/// Bounded disk-to-disk copy of <paramref name="sourcePath"/> into <paramref name="destination"/>.
|
||||
/// Used for passthrough formats whose stored bytes equal the source bytes. Hand-rolled rather than
|
||||
/// <see cref="Stream.CopyToAsync(Stream)"/> because <c>FileStream</c>'s override writes in 128 KB
|
||||
/// blocks; this keeps every write at or below <see cref="CopyBufferSize"/>, so peak managed memory
|
||||
/// is provably O(buffer), never O(filesize).
|
||||
/// </summary>
|
||||
public static async Task CopyFileAsync(string sourcePath, Stream destination, CancellationToken ct)
|
||||
{
|
||||
await using var src = new FileStream(
|
||||
sourcePath, FileMode.Open, FileAccess.Read, FileShare.Read,
|
||||
bufferSize: CopyBufferSize, useAsync: true);
|
||||
|
||||
var buffer = new byte[CopyBufferSize];
|
||||
int read;
|
||||
while ((read = await src.ReadAsync(buffer, ct)) > 0)
|
||||
{
|
||||
await destination.WriteAsync(buffer.AsMemory(0, read), ct);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Reads at most <paramref name="cap"/> bytes from the start of <paramref name="path"/> — enough
|
||||
/// for header/metadata parsing without loading the (potentially ~GB) body. Bounds the allocation
|
||||
/// at <c>min(cap, fileLength)</c>. Size-based metadata (e.g. average bitrate) must use the true
|
||||
/// file length, supplied separately, not the prefix length.
|
||||
/// </summary>
|
||||
public static async Task<byte[]> ReadPrefixAsync(string path, long cap, CancellationToken ct)
|
||||
{
|
||||
await using var fs = new FileStream(
|
||||
path, FileMode.Open, FileAccess.Read, FileShare.Read,
|
||||
bufferSize: CopyBufferSize, useAsync: true);
|
||||
|
||||
var length = (int)Math.Min(cap, fs.Length);
|
||||
var buffer = new byte[length];
|
||||
var total = 0;
|
||||
while (total < length)
|
||||
{
|
||||
var read = await fs.ReadAsync(buffer.AsMemory(total, length - total), ct);
|
||||
if (read == 0)
|
||||
break;
|
||||
total += read;
|
||||
}
|
||||
|
||||
return total == length ? buffer : buffer[..total];
|
||||
}
|
||||
}
|
||||
@@ -12,7 +12,11 @@ public class FlacAudioProcessor
|
||||
private const double FallbackDuration = 180.0;
|
||||
private const int FallbackBitrate = 1411;
|
||||
|
||||
public async Task<AudioBinary?> ProcessFlacFileAsync(string filePath)
|
||||
// STREAMINFO is mandatory and always the first metadata block, immediately after the 4-byte magic
|
||||
// (data at offset 8, 34 bytes). A small prefix read covers it without loading the body.
|
||||
private const long HeaderCap = 64 * 1024;
|
||||
|
||||
public async Task<ProcessedAudio?> ProcessFlacFileAsync(string filePath, CancellationToken cancellationToken = default)
|
||||
{
|
||||
if (!File.Exists(filePath))
|
||||
{
|
||||
@@ -24,25 +28,21 @@ public class FlacAudioProcessor
|
||||
throw new ArgumentException("File must be a FLAC file", nameof(filePath));
|
||||
}
|
||||
|
||||
var buffer = await File.ReadAllBytesAsync(filePath);
|
||||
var meta = ExtractFlacMetadata(buffer);
|
||||
var fileLength = new FileInfo(filePath).Length;
|
||||
var window = await AudioStoreStream.ReadPrefixAsync(filePath, HeaderCap, cancellationToken);
|
||||
var meta = ExtractFlacMetadata(window, fileLength);
|
||||
|
||||
var parameters = new AudioBinaryParams(
|
||||
Buffer: buffer,
|
||||
Size: buffer.Length,
|
||||
Extension: ".flac",
|
||||
Duration: meta.Duration,
|
||||
Bitrate: meta.Bitrate);
|
||||
|
||||
return new AudioBinary(parameters);
|
||||
// FLAC is stored unmodified — passthrough the original bytes via a streamed disk-to-disk copy.
|
||||
return ProcessedAudio.Passthrough(filePath, ".flac", meta.Duration, meta.Bitrate, fileLength);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Validates the <c>fLaC</c> magic and the leading STREAMINFO block, then computes duration from
|
||||
/// total-samples / sample-rate and average bitrate from file size. On any parse failure, logs a
|
||||
/// warning and returns synthetic defaults — never throws.
|
||||
/// warning and returns synthetic defaults — never throws. <paramref name="fileLength"/> is the true
|
||||
/// file size (the header window may be shorter), used for the average-bitrate computation.
|
||||
/// </summary>
|
||||
private static FlacMetadata ExtractFlacMetadata(byte[] buffer)
|
||||
private static FlacMetadata ExtractFlacMetadata(byte[] buffer, long fileLength)
|
||||
{
|
||||
try
|
||||
{
|
||||
@@ -84,7 +84,7 @@ public class FlacAudioProcessor
|
||||
|
||||
var duration = (double)totalSamples / sampleRate;
|
||||
var bitrate = duration > 0
|
||||
? (int)(buffer.LongLength * 8L / (duration * 1000))
|
||||
? (int)(fileLength * 8L / (duration * 1000))
|
||||
: FallbackBitrate;
|
||||
|
||||
return new FlacMetadata { Duration = duration, Bitrate = bitrate };
|
||||
|
||||
@@ -20,4 +20,46 @@ public interface ILoudnessAlgorithm
|
||||
/// is 1. All zeros when the signal is silent (peak is 0) or no samples are present.
|
||||
/// </returns>
|
||||
double[] Compute(ReadOnlySpan<byte> pcmData, int channels, int sampleRate, int bitsPerSample, int bucketCount);
|
||||
|
||||
/// <summary>
|
||||
/// Creates a stateful accumulator that reduces the same loudness profile from PCM fed in bounded
|
||||
/// chunks rather than from one contiguous buffer. The streaming waveform path uses this so a long
|
||||
/// track's PCM is never materialized whole in a managed <c>byte[]</c>. The accumulator's output is
|
||||
/// byte-identical to <see cref="Compute"/> for the same total PCM, because <see cref="Compute"/> is
|
||||
/// itself defined in terms of one — the single source of truth for the loudness reduction.
|
||||
/// </summary>
|
||||
/// <param name="pcmByteLength">
|
||||
/// Total length of the PCM data region in bytes. Required up front because the bucket each frame
|
||||
/// lands in is derived from the frame's position relative to the total frame count.
|
||||
/// </param>
|
||||
/// <param name="channels">Number of interleaved channels; averaged to mono per frame.</param>
|
||||
/// <param name="sampleRate">Samples per second (used for the envelope-smoothing time base).</param>
|
||||
/// <param name="bitsPerSample">Bit depth (8 unsigned, 16/24/32 signed) used to decode samples.</param>
|
||||
/// <param name="bucketCount">Number of equal time slices to reduce the signal to.</param>
|
||||
ILoudnessAccumulator CreateAccumulator(
|
||||
long pcmByteLength, int channels, int sampleRate, int bitsPerSample, int bucketCount);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Stateful, single-pass reducer for one loudness profile. Frames are fed via <see cref="Add"/> in
|
||||
/// arbitrary (non-frame-aligned) chunks — a partial frame straddling a chunk boundary is carried
|
||||
/// internally — and <see cref="Finish"/> emits the peak-normalized <c>double[bucketCount]</c>. Not
|
||||
/// thread-safe; feed one stream sequentially. Reusable across the same stream's chunks only, not
|
||||
/// across streams.
|
||||
/// </summary>
|
||||
public interface ILoudnessAccumulator
|
||||
{
|
||||
/// <summary>
|
||||
/// Feeds the next run of PCM bytes (interleaved, little-endian). Need not be frame-aligned; bytes
|
||||
/// that do not complete a frame are retained until the next call. Bytes past the total frame count
|
||||
/// declared at construction are ignored, matching the whole-buffer path's trailing-partial-frame drop.
|
||||
/// </summary>
|
||||
void Add(ReadOnlySpan<byte> pcmChunk);
|
||||
|
||||
/// <summary>
|
||||
/// Finalizes and returns the peak-normalized loudness profile (<c>double[bucketCount]</c>, each in
|
||||
/// [0, 1]). All zeros for silence or a degenerate (no-frame) input. Call once, after the last
|
||||
/// <see cref="Add"/>.
|
||||
/// </summary>
|
||||
double[] Finish();
|
||||
}
|
||||
|
||||
@@ -25,7 +25,13 @@ public class Mp3AudioProcessor
|
||||
private const double FallbackDuration = 180.0;
|
||||
private const int FallbackBitrate = 320;
|
||||
|
||||
public async Task<AudioBinary?> ProcessMp3FileAsync(string filePath)
|
||||
// Metadata lives in the leading ID3v2 tag plus the first MPEG frame. Cap the header read so a
|
||||
// large MP3 is not pulled into memory whole just to read it; a tag larger than this (very large
|
||||
// embedded art) simply falls back to the CBR/default estimate, never an OOM. The body is stored
|
||||
// by streaming the original file, not from this window.
|
||||
private const long HeaderCap = 8 * 1024 * 1024;
|
||||
|
||||
public async Task<ProcessedAudio?> ProcessMp3FileAsync(string filePath, CancellationToken cancellationToken = default)
|
||||
{
|
||||
if (!File.Exists(filePath))
|
||||
{
|
||||
@@ -37,24 +43,21 @@ public class Mp3AudioProcessor
|
||||
throw new ArgumentException("File must be an MP3 file", nameof(filePath));
|
||||
}
|
||||
|
||||
var buffer = await File.ReadAllBytesAsync(filePath);
|
||||
var meta = ExtractMp3Metadata(buffer);
|
||||
var fileLength = new FileInfo(filePath).Length;
|
||||
var window = await AudioStoreStream.ReadPrefixAsync(filePath, HeaderCap, cancellationToken);
|
||||
var meta = ExtractMp3Metadata(window, fileLength);
|
||||
|
||||
var parameters = new AudioBinaryParams(
|
||||
Buffer: buffer,
|
||||
Size: buffer.Length,
|
||||
Extension: ".mp3",
|
||||
Duration: meta.Duration,
|
||||
Bitrate: meta.Bitrate);
|
||||
|
||||
return new AudioBinary(parameters);
|
||||
// MP3 is stored unmodified — passthrough the original bytes via a streamed disk-to-disk copy.
|
||||
return ProcessedAudio.Passthrough(filePath, ".mp3", meta.Duration, meta.Bitrate, fileLength);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Parses the first valid MPEG frame (after any ID3v2 tag) and any Xing/VBRI tag inside it.
|
||||
/// On any parse failure, logs a warning and returns synthetic defaults — never throws.
|
||||
/// <paramref name="fileLength"/> is the true file size (the header window may be shorter), used
|
||||
/// for the CBR duration estimate.
|
||||
/// </summary>
|
||||
private static Mp3Metadata ExtractMp3Metadata(byte[] buffer)
|
||||
private static Mp3Metadata ExtractMp3Metadata(byte[] buffer, long fileLength)
|
||||
{
|
||||
try
|
||||
{
|
||||
@@ -65,7 +68,7 @@ public class Mp3AudioProcessor
|
||||
}
|
||||
|
||||
var header = DecodeFrameHeader(buffer, frameStart);
|
||||
var duration = ComputeDuration(buffer, frameStart, header);
|
||||
var duration = ComputeDuration(buffer, frameStart, header, fileLength);
|
||||
|
||||
return new Mp3Metadata { Duration = duration, Bitrate = header.BitrateKbps };
|
||||
}
|
||||
@@ -202,7 +205,7 @@ public class Mp3AudioProcessor
|
||||
/// Computes duration from a Xing/Info or VBRI tag (accurate for VBR) when present; otherwise
|
||||
/// falls back to the CBR estimate fileSize / (bitrate_kbps * 125). Guards divide-by-zero.
|
||||
/// </summary>
|
||||
private static double ComputeDuration(byte[] buffer, int frameStart, FrameHeader header)
|
||||
private static double ComputeDuration(byte[] buffer, int frameStart, FrameHeader header, long fileLength)
|
||||
{
|
||||
var xingFrames = ReadXingFrameCount(buffer, frameStart, header);
|
||||
if (xingFrames > 0 && header.SampleRate > 0)
|
||||
@@ -216,10 +219,10 @@ public class Mp3AudioProcessor
|
||||
return (double)vbriFrames * header.SamplesPerFrame / header.SampleRate;
|
||||
}
|
||||
|
||||
// CBR fallback: bitrate_kbps * 1000 / 8 bytes per second = bitrate_kbps * 125.
|
||||
// Exclude the ID3v2 tag bytes (everything before frameStart) from the estimate.
|
||||
// CBR fallback: bitrate_kbps * 1000 / 8 bytes per second = bitrate_kbps * 125. Uses the true
|
||||
// file length (not the bounded header window), excluding the ID3v2 tag bytes before frameStart.
|
||||
var bytesPerSecond = header.BitrateKbps * 125;
|
||||
return bytesPerSecond > 0 ? (double)(buffer.Length - frameStart) / bytesPerSecond : FallbackDuration;
|
||||
return bytesPerSecond > 0 ? (double)(fileLength - frameStart) / bytesPerSecond : FallbackDuration;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
|
||||
@@ -0,0 +1,140 @@
|
||||
using System.Diagnostics;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
|
||||
namespace DeepDrftContent.Processors.Opus;
|
||||
|
||||
/// <summary>
|
||||
/// Encodes a source audio file (any format the source vault holds — WAV/MP3/FLAC) to Ogg Opus fullband
|
||||
/// 320 kbps by shelling out to FFmpeg (libopus). FFmpeg is chosen over a managed encoder because it
|
||||
/// muxes a correct Ogg container with accurate granule positions across every input format — the page
|
||||
/// structure the seek-index walk depends on — which a raw libopus binding does not provide. The external
|
||||
/// <c>ffmpeg</c> binary is therefore a host runtime prerequisite (flagged in the wave handoff).
|
||||
/// </summary>
|
||||
public sealed class FfmpegOpusEncoder
|
||||
{
|
||||
private readonly OpusTranscodeOptions _options;
|
||||
private readonly ILogger<FfmpegOpusEncoder> _logger;
|
||||
|
||||
public FfmpegOpusEncoder(IOptions<OpusTranscodeOptions> options, ILogger<FfmpegOpusEncoder> logger)
|
||||
{
|
||||
_options = options.Value;
|
||||
_logger = logger;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Transcodes <paramref name="sourcePath"/> to an Ogg Opus file at <paramref name="destinationPath"/>.
|
||||
/// Returns true on a clean exit with a non-empty output. Returns false (logged) on a non-zero exit,
|
||||
/// a timeout, a missing ffmpeg binary, or any process failure — a transcode failure must never throw
|
||||
/// to the caller (C6); the background worker treats false as "leave the track lossless-only".
|
||||
/// </summary>
|
||||
public async Task<bool> EncodeAsync(string sourcePath, string destinationPath, CancellationToken ct)
|
||||
{
|
||||
var ffmpeg = string.IsNullOrWhiteSpace(_options.FfmpegPath) ? "ffmpeg" : _options.FfmpegPath;
|
||||
|
||||
// -vn drops any cover-art video stream; -map a:0 takes the first audio stream; -ar 48000 forces
|
||||
// fullband (Opus internally resamples to 48 kHz anyway, but stating it keeps granulepos math
|
||||
// unambiguous); libopus VBR at the target bitrate; -f ogg for an explicit Ogg container; -y
|
||||
// overwrites the (pre-created, empty) destination temp file.
|
||||
var args = new[]
|
||||
{
|
||||
"-hide_banner", "-nostdin", "-loglevel", "error",
|
||||
"-i", sourcePath,
|
||||
"-vn", "-map", "a:0",
|
||||
"-c:a", "libopus", "-b:a", $"{_options.BitrateKbps}k",
|
||||
"-ar", "48000",
|
||||
"-f", "ogg",
|
||||
"-y", destinationPath,
|
||||
};
|
||||
|
||||
var psi = new ProcessStartInfo(ffmpeg)
|
||||
{
|
||||
RedirectStandardError = true,
|
||||
RedirectStandardOutput = true,
|
||||
UseShellExecute = false,
|
||||
CreateNoWindow = true,
|
||||
};
|
||||
foreach (var arg in args)
|
||||
psi.ArgumentList.Add(arg);
|
||||
|
||||
using var process = new Process { StartInfo = psi };
|
||||
|
||||
try
|
||||
{
|
||||
if (!process.Start())
|
||||
{
|
||||
_logger.LogError("Opus transcode: ffmpeg failed to start ({Ffmpeg}).", ffmpeg);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
// Most commonly a missing binary (Win32Exception "file not found"). This is the ops
|
||||
// prerequisite failing — log loudly so it is unmistakable in the deploy logs.
|
||||
_logger.LogError(ex,
|
||||
"Opus transcode: could not launch ffmpeg ({Ffmpeg}). Is the ffmpeg binary installed on the host?",
|
||||
ffmpeg);
|
||||
return false;
|
||||
}
|
||||
|
||||
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(ct);
|
||||
timeout.CancelAfter(TimeSpan.FromSeconds(_options.TimeoutSeconds));
|
||||
|
||||
// Drain stderr concurrently — ffmpeg can block writing diagnostics if the pipe is not read.
|
||||
var stderrTask = process.StandardError.ReadToEndAsync(timeout.Token);
|
||||
|
||||
try
|
||||
{
|
||||
await process.WaitForExitAsync(timeout.Token);
|
||||
}
|
||||
catch (OperationCanceledException) when (ct.IsCancellationRequested)
|
||||
{
|
||||
TryKill(process);
|
||||
await SafeStderr(stderrTask); // observe to avoid unobserved-task warnings
|
||||
throw; // genuine shutdown cancellation — let it propagate
|
||||
}
|
||||
catch (OperationCanceledException)
|
||||
{
|
||||
TryKill(process);
|
||||
await SafeStderr(stderrTask); // observe to avoid unobserved-task warnings
|
||||
_logger.LogError("Opus transcode: ffmpeg exceeded the {Timeout}s timeout for {Source}.",
|
||||
_options.TimeoutSeconds, sourcePath);
|
||||
return false;
|
||||
}
|
||||
|
||||
var stderr = await SafeStderr(stderrTask);
|
||||
if (process.ExitCode != 0)
|
||||
{
|
||||
_logger.LogError("Opus transcode: ffmpeg exited {Code} for {Source}. stderr: {Stderr}",
|
||||
process.ExitCode, sourcePath, stderr);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!File.Exists(destinationPath) || new FileInfo(destinationPath).Length == 0)
|
||||
{
|
||||
_logger.LogError("Opus transcode: ffmpeg exited 0 but produced no output for {Source}.", sourcePath);
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
private void TryKill(Process process)
|
||||
{
|
||||
try
|
||||
{
|
||||
if (!process.HasExited)
|
||||
process.Kill(entireProcessTree: true);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogWarning(ex, "Opus transcode: failed to kill timed-out ffmpeg process.");
|
||||
}
|
||||
}
|
||||
|
||||
private static async Task<string> SafeStderr(Task<string> stderrTask)
|
||||
{
|
||||
try { return await stderrTask; }
|
||||
catch { return "<stderr unavailable>"; }
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
namespace DeepDrftContent.Processors.Opus;
|
||||
|
||||
/// <summary>
|
||||
/// Wire-format constants for the Ogg-Opus derived artifacts. Centralised so the seek-index codec,
|
||||
/// the page walker, and the tests agree on one set of magic numbers.
|
||||
/// </summary>
|
||||
public static class OggOpusConstants
|
||||
{
|
||||
/// <summary>Opus granule positions are always sample counts at 48 kHz, regardless of input rate.</summary>
|
||||
public const double OpusSampleRate = 48000.0;
|
||||
|
||||
/// <summary>One seek-index entry per this many seconds of audio (OQ7 — 0.5 s buckets).</summary>
|
||||
public const double SeekBucketSeconds = 0.5;
|
||||
|
||||
/// <summary>The Ogg page capture pattern "OggS" — every page starts with these four bytes.</summary>
|
||||
public static ReadOnlySpan<byte> CapturePattern => "OggS"u8;
|
||||
|
||||
/// <summary>Magic signature opening an OpusHead identification header packet.</summary>
|
||||
public static ReadOnlySpan<byte> OpusHeadSignature => "OpusHead"u8;
|
||||
|
||||
/// <summary>Magic signature opening an OpusTags comment header packet.</summary>
|
||||
public static ReadOnlySpan<byte> OpusTagsSignature => "OpusTags"u8;
|
||||
|
||||
/// <summary>
|
||||
/// Fixed size of an Ogg page header before the segment table: capture(4) + version(1) +
|
||||
/// header-type(1) + granulepos(8) + serial(4) + sequence(4) + checksum(4) + page-segments(1).
|
||||
/// </summary>
|
||||
public const int OggPageHeaderSize = 27;
|
||||
|
||||
/// <summary>Byte offset of the 64-bit granule position within an Ogg page header.</summary>
|
||||
public const int GranulePositionOffset = 6;
|
||||
|
||||
/// <summary>Byte offset of the page-segment count (the segment-table length) within the header.</summary>
|
||||
public const int PageSegmentCountOffset = 26;
|
||||
|
||||
/// <summary>Sentinel granule position for a page that ends mid-packet (no usable timestamp).</summary>
|
||||
public const ulong NoGranulePosition = 0xFFFFFFFFFFFFFFFFUL;
|
||||
|
||||
/// <summary>
|
||||
/// Minimum byte length of an <c>OpusHead</c> packet payload to safely read <c>pre_skip</c>.
|
||||
/// RFC 7845 §5.1: "OpusHead"(8) + version(1) + channels(1) + pre_skip(2) = 12 bytes minimum.
|
||||
/// </summary>
|
||||
public const int OpusHeadMinSize = 12;
|
||||
|
||||
/// <summary>
|
||||
/// Byte offset of <c>pre_skip</c> within the full <c>OpusHead</c> packet payload (including the
|
||||
/// magic). RFC 7845 §5.1: "OpusHead"(8) + version(1) + channels(1) = 10 bytes before pre_skip.
|
||||
/// </summary>
|
||||
public const int OpusHeadPreSkipOffset = 10;
|
||||
|
||||
/// <summary>
|
||||
/// Header size of the serialized seek-index blob:
|
||||
/// totalBytes(8) + duration(8) + count(4) + preSkip(2) + reserved(2) = 24 bytes.
|
||||
/// </summary>
|
||||
public const int SeekIndexHeaderSize = 24;
|
||||
|
||||
/// <summary>Size of one serialized seek point: granulepos(8) + byteOffset(8).</summary>
|
||||
public const int SeekPointSize = 16;
|
||||
|
||||
/// <summary>Vault-resource extension for the Opus audio bytes.</summary>
|
||||
public const string OpusExtension = ".opus";
|
||||
|
||||
/// <summary>Vault-resource extension for the combined setup-header + seek-index sidecar.</summary>
|
||||
public const string SidecarExtension = ".opusidx";
|
||||
}
|
||||
@@ -0,0 +1,310 @@
|
||||
using System.Buffers.Binary;
|
||||
|
||||
namespace DeepDrftContent.Processors.Opus;
|
||||
|
||||
/// <summary>
|
||||
/// The result of walking an encoded Ogg Opus stream once: the captured setup header (the leading
|
||||
/// <c>OpusHead</c> + <c>OpusTags</c> pages, verbatim) and the bucketed granule→byte seek index. This
|
||||
/// is everything the sidecar artifact carries (§3.4a) — built at transcode time so delivery never
|
||||
/// re-walks the stream.
|
||||
/// </summary>
|
||||
/// <param name="SetupHeaderBytes">The leading setup pages (OpusHead + OpusTags), exactly as they
|
||||
/// appear at the start of the stream, ready to prepend to any mid-stream page run before decode.</param>
|
||||
/// <param name="SeekIndex">The accurate, 0.5 s-bucketed granule→byte transfer function.</param>
|
||||
public sealed record OggOpusWalk(byte[] SetupHeaderBytes, OggOpusSeekIndex SeekIndex);
|
||||
|
||||
/// <summary>
|
||||
/// Pure Ogg-Opus stream walker. Reads the page structure directly (the <c>OggS</c> capture pattern and
|
||||
/// the 27-byte page header) to (1) capture the setup-header pages and (2) record, for every audio page,
|
||||
/// its end granule position and exact byte offset — bucketed to 0.5 s with each bucket boundary snapped
|
||||
/// to the nearest enclosing page start. No external dependency: the encoder (FFmpeg) produces the bytes;
|
||||
/// this turns them into the seek artifact deterministically, so it is unit-testable without a codec.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Two entry points share one <see cref="WalkState"/> page-processing core, so they produce byte-identical
|
||||
/// output by construction (the project's parity-oracle convention, mirroring
|
||||
/// <c>RmsLoudnessAlgorithm.Compute</c> over its accumulator):
|
||||
/// <list type="bullet">
|
||||
/// <item><see cref="Walk(ReadOnlySpan{byte})"/> — the whole-buffer overload, retained as the byte-identity
|
||||
/// parity oracle for the streaming variant.</item>
|
||||
/// <item><see cref="WalkAsync(System.IO.Stream,System.Threading.CancellationToken)"/> — the streaming
|
||||
/// variant: walks the page structure from a forward stream in a bounded read buffer (one Ogg page at a
|
||||
/// time), so peak managed memory is O(buffer + seek-index + setup-header), independent of file size.</item>
|
||||
/// </list>
|
||||
/// </remarks>
|
||||
public static class OggOpusParser
|
||||
{
|
||||
/// <summary>
|
||||
/// The largest a single Ogg page can be: header(27) + a full 255-entry segment table + the maximum
|
||||
/// payload those segments can describe (255 × 255 bytes). The streaming read buffer is floored at this
|
||||
/// so a complete page always fits, which means a short read on a page can only mean a truncated stream.
|
||||
/// </summary>
|
||||
private const int MaxOggPageSize = OggOpusConstants.OggPageHeaderSize + 255 + 255 * 255; // 65307
|
||||
|
||||
/// <summary>
|
||||
/// Walks <paramref name="oggBytes"/> and produces the setup header + seek index, or null if the
|
||||
/// bytes are not a recognisable Ogg Opus stream (no setup header, no audio pages, or truncated
|
||||
/// structure). A null is the caller's signal to treat the transcode as failed and leave the track
|
||||
/// lossless-only (C6) — it does not throw for malformed input.
|
||||
/// </summary>
|
||||
public static OggOpusWalk? Walk(ReadOnlySpan<byte> oggBytes)
|
||||
{
|
||||
var state = new WalkState();
|
||||
|
||||
var offset = 0;
|
||||
while (offset + OggOpusConstants.OggPageHeaderSize <= oggBytes.Length)
|
||||
{
|
||||
var page = oggBytes.Slice(offset);
|
||||
if (!page[..4].SequenceEqual(OggOpusConstants.CapturePattern))
|
||||
{
|
||||
// Not on a page boundary — the encoder writes contiguous pages, so this means the
|
||||
// stream is malformed or we mis-stepped. Either way it is unrecoverable here.
|
||||
return null;
|
||||
}
|
||||
|
||||
var segmentCount = page[OggOpusConstants.PageSegmentCountOffset];
|
||||
var segmentTableEnd = OggOpusConstants.OggPageHeaderSize + segmentCount;
|
||||
if (segmentTableEnd > page.Length)
|
||||
return null; // truncated header
|
||||
|
||||
var payloadSize = 0;
|
||||
for (var i = 0; i < segmentCount; i++)
|
||||
payloadSize += page[OggOpusConstants.OggPageHeaderSize + i];
|
||||
|
||||
var pageTotalSize = segmentTableEnd + payloadSize;
|
||||
if (pageTotalSize > page.Length)
|
||||
return null; // truncated payload
|
||||
|
||||
var payload = page.Slice(segmentTableEnd, payloadSize);
|
||||
var granule = BinaryPrimitives.ReadUInt64LittleEndian(
|
||||
page.Slice(OggOpusConstants.GranulePositionOffset, 8));
|
||||
|
||||
state.AddPage(granule, payload, page.Slice(0, pageTotalSize), offset);
|
||||
|
||||
offset += pageTotalSize;
|
||||
}
|
||||
|
||||
return state.Finish((ulong)oggBytes.Length);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Streaming counterpart of <see cref="Walk(ReadOnlySpan{byte})"/>: walks the Ogg page structure from
|
||||
/// a forward <paramref name="stream"/> in a bounded read buffer (one Ogg page at a time), producing a
|
||||
/// byte-identical <see cref="OggOpusWalk"/> without ever holding the whole encoded file in memory.
|
||||
/// Returns null on the same malformed/truncated conditions as the buffer overload — it does not throw
|
||||
/// for bad input (only <see cref="OperationCanceledException"/> propagates on cancellation).
|
||||
/// </summary>
|
||||
public static Task<OggOpusWalk?> WalkAsync(Stream stream, CancellationToken cancellationToken = default)
|
||||
=> WalkAsync(stream, MaxOggPageSize, cancellationToken);
|
||||
|
||||
/// <summary>
|
||||
/// Buffer-size-parameterised core. <paramref name="bufferSize"/> is floored at
|
||||
/// <see cref="MaxOggPageSize"/> so any single page always fits in the buffer; the absolute byte
|
||||
/// cursor (<c>absoluteOffset</c>) advances as the window compacts, so recorded seek offsets stay
|
||||
/// absolute even though the buffer holds only a small window at any instant.
|
||||
/// </summary>
|
||||
internal static async Task<OggOpusWalk?> WalkAsync(Stream stream, int bufferSize, CancellationToken cancellationToken)
|
||||
{
|
||||
var state = new WalkState();
|
||||
var buffer = new byte[Math.Max(bufferSize, MaxOggPageSize)];
|
||||
var len = 0; // valid bytes held at buffer[0..len]
|
||||
long absoluteOffset = 0; // absolute stream position of buffer[0]
|
||||
|
||||
while (true)
|
||||
{
|
||||
// The buffer overload's loop guard requires a full fixed header before parsing a page; once
|
||||
// fewer than that remain (and the stream is drained) it is the natural end of the stream.
|
||||
if (len < OggOpusConstants.OggPageHeaderSize)
|
||||
{
|
||||
len += await FillAsync(stream, buffer, len, cancellationToken);
|
||||
if (len < OggOpusConstants.OggPageHeaderSize)
|
||||
break;
|
||||
}
|
||||
|
||||
if (!buffer.AsSpan(0, 4).SequenceEqual(OggOpusConstants.CapturePattern))
|
||||
return null;
|
||||
|
||||
var segmentCount = buffer[OggOpusConstants.PageSegmentCountOffset];
|
||||
var segmentTableEnd = OggOpusConstants.OggPageHeaderSize + segmentCount;
|
||||
if (len < segmentTableEnd)
|
||||
{
|
||||
len += await FillAsync(stream, buffer, len, cancellationToken);
|
||||
if (len < segmentTableEnd)
|
||||
return null; // truncated header
|
||||
}
|
||||
|
||||
var payloadSize = 0;
|
||||
for (var i = 0; i < segmentCount; i++)
|
||||
payloadSize += buffer[OggOpusConstants.OggPageHeaderSize + i];
|
||||
|
||||
var pageTotalSize = segmentTableEnd + payloadSize;
|
||||
if (len < pageTotalSize)
|
||||
{
|
||||
len += await FillAsync(stream, buffer, len, cancellationToken);
|
||||
if (len < pageTotalSize)
|
||||
return null; // truncated payload (page never fully arrived before EOF)
|
||||
}
|
||||
|
||||
var page = buffer.AsSpan(0, pageTotalSize);
|
||||
var granule = BinaryPrimitives.ReadUInt64LittleEndian(
|
||||
page.Slice(OggOpusConstants.GranulePositionOffset, 8));
|
||||
var payload = page.Slice(segmentTableEnd, payloadSize);
|
||||
|
||||
state.AddPage(granule, payload, page, absoluteOffset);
|
||||
|
||||
// Compact the consumed page off the front of the window; the absolute cursor advances by the
|
||||
// exact page size so every offset the state records remains an absolute stream position.
|
||||
var remaining = len - pageTotalSize;
|
||||
if (remaining > 0)
|
||||
Buffer.BlockCopy(buffer, pageTotalSize, buffer, 0, remaining);
|
||||
len = remaining;
|
||||
absoluteOffset += pageTotalSize;
|
||||
}
|
||||
|
||||
return state.Finish((ulong)absoluteOffset + (ulong)len);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Fills <paramref name="buffer"/> from <paramref name="offset"/> to its end, issuing as many reads as
|
||||
/// needed until the buffer is full or the stream is exhausted. Returns the number of bytes added.
|
||||
/// </summary>
|
||||
private static async Task<int> FillAsync(Stream stream, byte[] buffer, int offset, CancellationToken ct)
|
||||
{
|
||||
var added = 0;
|
||||
while (offset + added < buffer.Length)
|
||||
{
|
||||
var read = await stream.ReadAsync(buffer.AsMemory(offset + added, buffer.Length - offset - added), ct);
|
||||
if (read == 0)
|
||||
break;
|
||||
added += read;
|
||||
}
|
||||
return added;
|
||||
}
|
||||
|
||||
private static bool StartsWith(ReadOnlySpan<byte> payload, ReadOnlySpan<byte> signature) =>
|
||||
payload.Length >= signature.Length && payload[..signature.Length].SequenceEqual(signature);
|
||||
|
||||
/// <summary>
|
||||
/// The single page-processing core both <see cref="Walk(ReadOnlySpan{byte})"/> and
|
||||
/// <see cref="WalkAsync(Stream,int,CancellationToken)"/> drive, page by page, in stream order. Holding
|
||||
/// the setup-header + seek-index accumulation here is what makes the two entry points byte-identical
|
||||
/// by construction: there is exactly one copy of the OpusHead/OpusTags detection, pre-skip correction,
|
||||
/// t=0 anchoring, and 0.5 s bucketing logic.
|
||||
/// </summary>
|
||||
private sealed class WalkState
|
||||
{
|
||||
// The real setup (OpusHead + OpusTags pages) is a few KB; this cap bounds the streaming capture so
|
||||
// a malformed head-without-tags stream cannot grow it unboundedly. A stream that exceeds it has no
|
||||
// OpusTags within the cap, so no audio points are ever recorded and Finish returns null either way
|
||||
// — the cap never changes the output of a stream that produces a non-null result.
|
||||
private const int MaxSetupHeaderBytes = 8 * 1024 * 1024;
|
||||
|
||||
private bool _sawOpusHead;
|
||||
private bool _sawOpusTags;
|
||||
private ushort _preSkip;
|
||||
private int _setupHeaderEnd = -1;
|
||||
|
||||
private bool _capturingSetup = true;
|
||||
private readonly List<byte> _setupHeader = new();
|
||||
|
||||
private readonly List<OpusSeekPoint> _points = new();
|
||||
private ulong _lastGranule;
|
||||
private double _nextBucketTime;
|
||||
private bool _firstAudioPointTaken;
|
||||
|
||||
/// <summary>
|
||||
/// Processes one fully-framed page. <paramref name="pageBytes"/> is the whole page (header +
|
||||
/// segment table + payload) for verbatim setup capture; <paramref name="absoluteOffset"/> is the
|
||||
/// page's absolute start in the stream — the value recorded in the seek index.
|
||||
/// </summary>
|
||||
public void AddPage(ulong granule, ReadOnlySpan<byte> payload, ReadOnlySpan<byte> pageBytes, long absoluteOffset)
|
||||
{
|
||||
if (_capturingSetup)
|
||||
{
|
||||
if (_setupHeader.Count + pageBytes.Length > MaxSetupHeaderBytes)
|
||||
_capturingSetup = false; // malformed: give up capture (result will be null without tags)
|
||||
else
|
||||
_setupHeader.AddRange(pageBytes);
|
||||
}
|
||||
|
||||
// The setup pages carry no audio granule (OpusHead has granulepos 0; OpusTags too). They
|
||||
// are the leading pages whose payload opens with the Opus magic signatures.
|
||||
if (!_sawOpusHead && StartsWith(payload, OggOpusConstants.OpusHeadSignature))
|
||||
{
|
||||
_sawOpusHead = true;
|
||||
_setupHeaderEnd = (int)(absoluteOffset + pageBytes.Length);
|
||||
|
||||
// RFC 7845 §5.1 — OpusHead layout after the 8-byte "OpusHead" magic:
|
||||
// [0] version (1 byte), [1] channel count (1 byte),
|
||||
// [2-3] pre_skip (little-endian uint16) ← at packet bytes 10-11
|
||||
// pre_skip is the number of decoder samples to discard before presenting audio;
|
||||
// all granule→time conversions must subtract it (RFC 7845 §4.3).
|
||||
if (payload.Length >= OggOpusConstants.OpusHeadMinSize)
|
||||
_preSkip = BinaryPrimitives.ReadUInt16LittleEndian(
|
||||
payload.Slice(OggOpusConstants.OpusHeadPreSkipOffset, 2));
|
||||
}
|
||||
else if (_sawOpusHead && !_sawOpusTags && StartsWith(payload, OggOpusConstants.OpusTagsSignature))
|
||||
{
|
||||
_sawOpusTags = true;
|
||||
_setupHeaderEnd = (int)(absoluteOffset + pageBytes.Length);
|
||||
// The setup header ends at the OpusTags page; stop capturing so audio pages never grow it.
|
||||
_capturingSetup = false;
|
||||
}
|
||||
else if (_sawOpusHead && _sawOpusTags)
|
||||
{
|
||||
// Audio page. Record the first audio page unconditionally (the seek anchor at t=0),
|
||||
// then one entry per 0.5 s bucket. A page with no end-granule (mid-packet continuation,
|
||||
// granulepos == -1) is skipped for indexing — its time is unknown — but still advances
|
||||
// the byte cursor.
|
||||
if (granule != OggOpusConstants.NoGranulePosition)
|
||||
{
|
||||
// RFC 7845 §4.3: presentation time = max(0, granule − preSkip) / 48000.
|
||||
// Use this corrected time for bucketing so that a stream with pre-skip 3840 (~80 ms)
|
||||
// does not systematically offset every indexed time by that amount.
|
||||
var correctedTime = Math.Max(0.0,
|
||||
(granule - (double)_preSkip) / OggOpusConstants.OpusSampleRate);
|
||||
|
||||
if (!_firstAudioPointTaken)
|
||||
{
|
||||
// Anchor the first seek point at corrected time = 0 by storing the granule as
|
||||
// preSkip. This guarantees that a binary search for t=0 ("largest entry with
|
||||
// corrected time ≤ 0") always resolves to the first audio page's byte offset —
|
||||
// even when the real granule is slightly above preSkip due to encoder lead-in.
|
||||
_points.Add(new OpusSeekPoint(_preSkip, (ulong)absoluteOffset));
|
||||
_firstAudioPointTaken = true;
|
||||
_nextBucketTime = OggOpusConstants.SeekBucketSeconds;
|
||||
}
|
||||
else if (correctedTime >= _nextBucketTime)
|
||||
{
|
||||
_points.Add(new OpusSeekPoint(granule, (ulong)absoluteOffset));
|
||||
// Advance past every bucket this page crossed so a long page does not emit a
|
||||
// backlog of entries; the next bucket is the first boundary strictly after it.
|
||||
while (_nextBucketTime <= correctedTime)
|
||||
_nextBucketTime += OggOpusConstants.SeekBucketSeconds;
|
||||
}
|
||||
|
||||
_lastGranule = granule;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Produces the final walk, or null on the same conditions the buffer overload rejected:
|
||||
/// no OpusHead, no captured setup header, or no audio seek points. <paramref name="totalByteLength"/>
|
||||
/// is the full stream length, recorded for end-of-stream seek clamping.
|
||||
/// </summary>
|
||||
public OggOpusWalk? Finish(ulong totalByteLength)
|
||||
{
|
||||
if (!_sawOpusHead || _setupHeaderEnd < 0 || _points.Count == 0)
|
||||
return null;
|
||||
|
||||
var setupHeader = _setupHeader.ToArray();
|
||||
// RFC 7845 §4.3: total duration is also pre-skip-corrected, matching the time a listener
|
||||
// experiences (the last audio page's corrected time, clamped to ≥ 0).
|
||||
var totalDuration = Math.Max(0.0,
|
||||
(_lastGranule - (double)_preSkip) / OggOpusConstants.OpusSampleRate);
|
||||
var index = new OggOpusSeekIndex(_points, totalDuration, totalByteLength, _preSkip);
|
||||
return new OggOpusWalk(setupHeader, index);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,124 @@
|
||||
using System.Buffers.Binary;
|
||||
|
||||
namespace DeepDrftContent.Processors.Opus;
|
||||
|
||||
/// <summary>
|
||||
/// A single seek-index entry: an authoritative 48 kHz <see cref="GranulePosition"/> (Opus granule
|
||||
/// positions are always sample counts at 48 kHz) paired with the exact byte offset of the Ogg page that
|
||||
/// carries it. Every <see cref="ByteOffset"/> is a real page-start boundary, so a
|
||||
/// <c>Range: bytes={ByteOffset}-</c> fetch lands the decoder Ogg-sync-aligned.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Per RFC 7845 §4.3, the PCM presentation time is <c>(granulepos − preSkip) / 48000</c>. The raw
|
||||
/// <see cref="GranulePosition"/> is stored here as-is; callers should subtract the containing
|
||||
/// <see cref="OggOpusSeekIndex.PreSkip"/> before converting to a presentation time. Use
|
||||
/// <see cref="OggOpusSeekIndex.PresentationTimeSeconds"/> for the corrected value.
|
||||
/// </remarks>
|
||||
/// <param name="GranulePosition">The page's end granule position (48 kHz sample count).</param>
|
||||
/// <param name="ByteOffset">The byte offset of the page start in the Opus file.</param>
|
||||
public readonly record struct OpusSeekPoint(ulong GranulePosition, ulong ByteOffset)
|
||||
{
|
||||
/// <summary>
|
||||
/// Raw granule-position-to-time conversion (granulepos / 48 kHz). Does NOT subtract pre-skip — use
|
||||
/// <see cref="OggOpusSeekIndex.PresentationTimeSeconds"/> for the RFC 7845-correct presentation time.
|
||||
/// </summary>
|
||||
public double RawTimeSeconds => GranulePosition / OggOpusConstants.OpusSampleRate;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The accurate, precomputed transfer function from seek-time to true file byte offset for one Ogg
|
||||
/// Opus stream (§3.4a A). Built once at transcode time by walking the encoded stream; the client reads
|
||||
/// it back and binary-searches <see cref="Points"/> instead of doing inaccurate VBR byte-rate math.
|
||||
/// One entry per 0.5 s of audio (<see cref="OggOpusConstants.SeekBucketSeconds"/>), each snapped to the
|
||||
/// nearest enclosing page start, plus the totals needed to clamp a seek to range.
|
||||
/// </summary>
|
||||
/// <param name="Points">Ordered (granulepos, byteOffset) entries, ascending. The first entry always
|
||||
/// has <see cref="OpusSeekPoint.GranulePosition"/> == <paramref name="PreSkip"/> (corrected time = 0)
|
||||
/// and points at the first audio page start, ensuring a seek to t=0 always resolves.</param>
|
||||
/// <param name="TotalDurationSeconds">
|
||||
/// Pre-skip-corrected total stream duration: <c>max(0, lastGranule − preSkip) / 48000</c>.
|
||||
/// </param>
|
||||
/// <param name="TotalByteLength">Total Opus file byte length, for clamping a seek past the end.</param>
|
||||
/// <param name="PreSkip">
|
||||
/// The <c>pre_skip</c> value from the <c>OpusHead</c> identification header (RFC 7845 §5.1). Opus
|
||||
/// decoders must discard this many samples from the decoded start before presenting audio. The client
|
||||
/// (wave 18.4) needs this to trim the first decoded buffer; storing it here avoids a re-parse of the
|
||||
/// Ogg stream at delivery time.
|
||||
/// </param>
|
||||
public sealed record OggOpusSeekIndex(
|
||||
IReadOnlyList<OpusSeekPoint> Points,
|
||||
double TotalDurationSeconds,
|
||||
ulong TotalByteLength,
|
||||
ushort PreSkip)
|
||||
{
|
||||
/// <summary>
|
||||
/// Returns the RFC 7845-correct presentation time for a seek point: <c>max(0, granule − preSkip) / 48000</c>.
|
||||
/// Use this for all time comparisons; raw <see cref="OpusSeekPoint.RawTimeSeconds"/> omits the pre-skip.
|
||||
/// </summary>
|
||||
public double PresentationTimeSeconds(OpusSeekPoint point) =>
|
||||
Math.Max(0.0, (point.GranulePosition - (double)PreSkip) / OggOpusConstants.OpusSampleRate);
|
||||
|
||||
/// <summary>
|
||||
/// Serializes the index to the compact little-endian binary blob the sidecar stores. Layout:
|
||||
/// <c>[uint64 totalByteLength][double totalDurationSeconds][uint32 pointCount][uint16 preSkip][uint16 reserved]</c>
|
||||
/// then <c>pointCount × (uint64 granulepos, uint64 byteOffset)</c>. The four-byte preSkip+reserved
|
||||
/// region pads the header to 24 bytes, keeping the point table 8-byte-aligned.
|
||||
/// Fixed-width records keep the client parse to a single typed-array read.
|
||||
/// </summary>
|
||||
public byte[] ToBytes()
|
||||
{
|
||||
var size = OggOpusConstants.SeekIndexHeaderSize + Points.Count * OggOpusConstants.SeekPointSize;
|
||||
var bytes = new byte[size];
|
||||
var span = bytes.AsSpan();
|
||||
|
||||
BinaryPrimitives.WriteUInt64LittleEndian(span[..8], TotalByteLength);
|
||||
BinaryPrimitives.WriteDoubleLittleEndian(span.Slice(8, 8), TotalDurationSeconds);
|
||||
BinaryPrimitives.WriteUInt32LittleEndian(span.Slice(16, 4), (uint)Points.Count);
|
||||
BinaryPrimitives.WriteUInt16LittleEndian(span.Slice(20, 2), PreSkip);
|
||||
// bytes 22-23: reserved (zero-initialized by array allocation)
|
||||
|
||||
var cursor = OggOpusConstants.SeekIndexHeaderSize;
|
||||
foreach (var point in Points)
|
||||
{
|
||||
BinaryPrimitives.WriteUInt64LittleEndian(span.Slice(cursor, 8), point.GranulePosition);
|
||||
BinaryPrimitives.WriteUInt64LittleEndian(span.Slice(cursor + 8, 8), point.ByteOffset);
|
||||
cursor += OggOpusConstants.SeekPointSize;
|
||||
}
|
||||
|
||||
return bytes;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Parses a blob produced by <see cref="ToBytes"/>. Returns null if the blob is too short or its
|
||||
/// declared point count does not fit — the storage contract is exact, so a malformed blob is a
|
||||
/// corruption signal, not a recoverable shape. (Provided so tests and any future server-side reader
|
||||
/// share one codec with the writer.)
|
||||
/// </summary>
|
||||
public static OggOpusSeekIndex? FromBytes(ReadOnlySpan<byte> bytes)
|
||||
{
|
||||
if (bytes.Length < OggOpusConstants.SeekIndexHeaderSize)
|
||||
return null;
|
||||
|
||||
var totalByteLength = BinaryPrimitives.ReadUInt64LittleEndian(bytes[..8]);
|
||||
var totalDuration = BinaryPrimitives.ReadDoubleLittleEndian(bytes.Slice(8, 8));
|
||||
var count = BinaryPrimitives.ReadUInt32LittleEndian(bytes.Slice(16, 4));
|
||||
var preSkip = BinaryPrimitives.ReadUInt16LittleEndian(bytes.Slice(20, 2));
|
||||
// bytes 22-23: reserved — ignored on read for forward-compatibility
|
||||
|
||||
var expected = OggOpusConstants.SeekIndexHeaderSize + (long)count * OggOpusConstants.SeekPointSize;
|
||||
if (bytes.Length < expected)
|
||||
return null;
|
||||
|
||||
var points = new OpusSeekPoint[count];
|
||||
var cursor = OggOpusConstants.SeekIndexHeaderSize;
|
||||
for (var i = 0; i < count; i++)
|
||||
{
|
||||
var granule = BinaryPrimitives.ReadUInt64LittleEndian(bytes.Slice(cursor, 8));
|
||||
var offset = BinaryPrimitives.ReadUInt64LittleEndian(bytes.Slice(cursor + 8, 8));
|
||||
points[i] = new OpusSeekPoint(granule, offset);
|
||||
cursor += OggOpusConstants.SeekPointSize;
|
||||
}
|
||||
|
||||
return new OggOpusSeekIndex(points, totalDuration, totalByteLength, preSkip);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
using System.Buffers.Binary;
|
||||
|
||||
namespace DeepDrftContent.Processors.Opus;
|
||||
|
||||
/// <summary>
|
||||
/// The single derived sidecar artifact per track (§3.4a B, recommended design): the Opus setup header
|
||||
/// (<c>OpusHead</c> + <c>OpusTags</c>) followed by the granule→byte seek index. The client fetches this
|
||||
/// once on track load and parses it into its <c>OpusSeekData</c>, so it always has both the setup bytes
|
||||
/// (to prepend to any mid-stream slice) and the accurate seek transfer function before it ever issues a
|
||||
/// Range fetch — including a window that opens away from byte 0 (UC9).
|
||||
/// </summary>
|
||||
/// <param name="SetupHeaderBytes">The verbatim OpusHead + OpusTags pages.</param>
|
||||
/// <param name="SeekIndex">The bucketed granule→byte seek index.</param>
|
||||
public sealed record OpusSidecar(byte[] SetupHeaderBytes, OggOpusSeekIndex SeekIndex)
|
||||
{
|
||||
/// <summary>
|
||||
/// Serializes to <c>[uint32 setupHeaderLength][setup-header bytes][seek-index blob]</c>. The
|
||||
/// length prefix lets the client split the two regions with one read; the seek-index blob carries
|
||||
/// its own self-describing header (<see cref="OggOpusSeekIndex.ToBytes"/>), so it needs no trailing
|
||||
/// length.
|
||||
/// </summary>
|
||||
public byte[] ToBytes()
|
||||
{
|
||||
var indexBytes = SeekIndex.ToBytes();
|
||||
var bytes = new byte[4 + SetupHeaderBytes.Length + indexBytes.Length];
|
||||
var span = bytes.AsSpan();
|
||||
|
||||
BinaryPrimitives.WriteUInt32LittleEndian(span[..4], (uint)SetupHeaderBytes.Length);
|
||||
SetupHeaderBytes.CopyTo(span.Slice(4));
|
||||
indexBytes.CopyTo(span.Slice(4 + SetupHeaderBytes.Length));
|
||||
|
||||
return bytes;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Parses a blob produced by <see cref="ToBytes"/>. Returns null on any structural inconsistency
|
||||
/// (short blob, length prefix that overruns, or an unparseable index) — the format is exact, so a
|
||||
/// malformed blob is corruption.
|
||||
/// </summary>
|
||||
public static OpusSidecar? FromBytes(ReadOnlySpan<byte> bytes)
|
||||
{
|
||||
if (bytes.Length < 4)
|
||||
return null;
|
||||
|
||||
var setupLength = BinaryPrimitives.ReadUInt32LittleEndian(bytes[..4]);
|
||||
var indexStart = 4 + (long)setupLength;
|
||||
if (bytes.Length < indexStart)
|
||||
return null;
|
||||
|
||||
var setupHeader = bytes.Slice(4, (int)setupLength).ToArray();
|
||||
var index = OggOpusSeekIndex.FromBytes(bytes.Slice((int)indexStart));
|
||||
if (index is null)
|
||||
return null;
|
||||
|
||||
return new OpusSidecar(setupHeader, index);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
namespace DeepDrftContent.Processors.Opus;
|
||||
|
||||
/// <summary>
|
||||
/// Host-supplied configuration for the Opus transcode. The only operationally significant knob is
|
||||
/// <see cref="FfmpegPath"/> — the transcode shells out to FFmpeg (libopus), which must be present on the
|
||||
/// DeepDrftAPI host (see the wave handoff notes). Defaults target Ogg Opus fullband (48 kHz) at 320 kbps,
|
||||
/// the artifact the spec fixes (§1).
|
||||
/// </summary>
|
||||
public sealed class OpusTranscodeOptions
|
||||
{
|
||||
/// <summary>
|
||||
/// Path to the ffmpeg executable. Empty/null resolves to <c>"ffmpeg"</c> (found on PATH). Override
|
||||
/// with an absolute path when the binary is not on the host PATH.
|
||||
/// </summary>
|
||||
public string FfmpegPath { get; set; } = "ffmpeg";
|
||||
|
||||
/// <summary>Target Opus bitrate in kbps. 320 kbps fullband is the fixed artifact quality (§1).</summary>
|
||||
public int BitrateKbps { get; set; } = 320;
|
||||
|
||||
/// <summary>
|
||||
/// Directory for the transient source/output files the transcode stages. Defaults to the system
|
||||
/// temp path; the host overrides it to the data-disk upload-staging directory so large files never
|
||||
/// land on the small RAM-backed <c>/tmp</c> tmpfs (same constraint the upload path already honours).
|
||||
/// </summary>
|
||||
public string StagingPath { get; set; } = Path.GetTempPath();
|
||||
|
||||
/// <summary>
|
||||
/// Hard ceiling on a single transcode, in seconds. A run that exceeds it is killed and the track
|
||||
/// stays lossless-only (C6). Generous by default — a 1 GB mix is CPU-expensive (§3.1) — but bounded
|
||||
/// so a hung ffmpeg never wedges the background worker.
|
||||
/// </summary>
|
||||
public int TimeoutSeconds { get; set; } = 3600;
|
||||
}
|
||||
@@ -0,0 +1,189 @@
|
||||
using DeepDrftContent.Constants;
|
||||
using DeepDrftContent.FileDatabase.Models;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
using FileDb = DeepDrftContent.FileDatabase.Services.FileDatabase;
|
||||
|
||||
namespace DeepDrftContent.Processors.Opus;
|
||||
|
||||
/// <summary>
|
||||
/// Derives and persists a track's low-data Ogg Opus artifacts (Phase 18.1). Mirrors
|
||||
/// <see cref="WaveformProfileService"/>'s derived-artifact lifecycle: compute from the stored source,
|
||||
/// store in a dedicated vault keyed by <c>EntryKey</c>, regenerable, failure-tolerant. For one track it
|
||||
/// produces two entries in the <see cref="VaultConstants.TrackOpus"/> vault — the Opus audio bytes and a
|
||||
/// combined setup-header + seek-index sidecar (§3.4a). Strictly additive: the source <c>tracks</c> vault
|
||||
/// is never touched, and a failure here leaves the track lossless-only and eligible for backfill (C2/C6).
|
||||
/// </summary>
|
||||
public sealed class OpusTranscodeService
|
||||
{
|
||||
private readonly FileDb _fileDatabase;
|
||||
private readonly TrackContentService _trackContent;
|
||||
private readonly FfmpegOpusEncoder _encoder;
|
||||
private readonly OpusTranscodeOptions _options;
|
||||
private readonly ILogger<OpusTranscodeService> _logger;
|
||||
|
||||
public OpusTranscodeService(
|
||||
FileDb fileDatabase,
|
||||
TrackContentService trackContent,
|
||||
FfmpegOpusEncoder encoder,
|
||||
IOptions<OpusTranscodeOptions> options,
|
||||
ILogger<OpusTranscodeService> logger)
|
||||
{
|
||||
_fileDatabase = fileDatabase;
|
||||
_trackContent = trackContent;
|
||||
_encoder = encoder;
|
||||
_options = options.Value;
|
||||
_logger = logger;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Reads the source audio for <paramref name="entryKey"/> from the <c>tracks</c> vault, transcodes it
|
||||
/// to Ogg Opus 320, walks the encoded stream to build the seek index + capture the setup header, and
|
||||
/// stores the Opus bytes and the sidecar in the <see cref="VaultConstants.TrackOpus"/> vault under the
|
||||
/// same key. Re-runnable — a second call overwrites the prior artifacts (backfill / replace-audio).
|
||||
/// Returns false (logged) on any failure; never throws for expected failure modes (C6). The only
|
||||
/// propagated exception is <see cref="OperationCanceledException"/> on genuine shutdown.
|
||||
/// </summary>
|
||||
public async Task<bool> TranscodeAndStoreAsync(string entryKey, CancellationToken ct)
|
||||
{
|
||||
// Read the source extension + duration from the vault index (no body load) and open a streamed
|
||||
// read over the source bytes — never the whole-buffer AudioBinary. A 92-min mix source is ~970 MB;
|
||||
// buffering it (and the encoded output below) was the last unconverted store-path OOM violation.
|
||||
var trackDuration = await _trackContent.GetAudioDurationAsync(entryKey) ?? 0.0;
|
||||
var sourceMedia = await _trackContent.OpenAudioMediaStreamAsync(entryKey);
|
||||
if (sourceMedia is null)
|
||||
{
|
||||
_logger.LogWarning("Opus transcode: no source audio in vault for {EntryKey}; skipping.", entryKey);
|
||||
return false;
|
||||
}
|
||||
|
||||
string? sourcePath = null;
|
||||
string? opusPath = null;
|
||||
try
|
||||
{
|
||||
// Stage the source to disk in bounded chunks so ffmpeg can read it by file path/extension.
|
||||
// The inner finally disposes the source stream as soon as the copy is done — the read handle
|
||||
// is not held across the (long) encode — and guarantees disposal even if staging setup throws.
|
||||
try
|
||||
{
|
||||
Directory.CreateDirectory(_options.StagingPath);
|
||||
sourcePath = Path.Combine(_options.StagingPath, $"opus-src-{Guid.NewGuid():N}{sourceMedia.Extension}");
|
||||
opusPath = Path.Combine(_options.StagingPath, $"opus-out-{Guid.NewGuid():N}{OggOpusConstants.OpusExtension}");
|
||||
|
||||
await using var staging = new FileStream(
|
||||
sourcePath, FileMode.Create, FileAccess.Write, FileShare.None,
|
||||
bufferSize: 81920, useAsync: true);
|
||||
await sourceMedia.Stream.CopyToAsync(staging, bufferSize: 81920, ct);
|
||||
}
|
||||
finally
|
||||
{
|
||||
await sourceMedia.DisposeAsync();
|
||||
}
|
||||
|
||||
if (!await _encoder.EncodeAsync(sourcePath, opusPath, ct))
|
||||
return false; // encoder already logged the cause
|
||||
|
||||
// Walk the encoded output from a streamed read in a bounded buffer (no whole-file load). The
|
||||
// seek index and setup header are byte-identical to the buffer walk (parity-tested).
|
||||
OggOpusWalk? walk;
|
||||
await using (var opusIn = new FileStream(
|
||||
opusPath, FileMode.Open, FileAccess.Read, FileShare.Read,
|
||||
bufferSize: 81920, useAsync: true))
|
||||
{
|
||||
walk = await OggOpusParser.WalkAsync(opusIn, ct);
|
||||
}
|
||||
if (walk is null)
|
||||
{
|
||||
_logger.LogError(
|
||||
"Opus transcode: ffmpeg produced output but the Ogg stream could not be walked for {EntryKey}; " +
|
||||
"no artifacts stored.", entryKey);
|
||||
return false;
|
||||
}
|
||||
|
||||
await EnsureVaultAsync();
|
||||
|
||||
// Bitrate from the output file length + duration — both available without buffering the bytes.
|
||||
var opusLength = new FileInfo(opusPath).Length;
|
||||
var opusBitrate = trackDuration > 0
|
||||
? (int)(opusLength * 8 / trackDuration / 1000)
|
||||
: _options.BitrateKbps;
|
||||
|
||||
// Store the audio first, then the sidecar. If the sidecar write fails the Opus bytes are
|
||||
// present but unseekable — treat that as a failed derive (return false) so a backfill re-runs
|
||||
// it; do not leave a half-derived track that the delivery layer would treat as complete.
|
||||
var audioMeta = MetaDataFactory.CreateAudioMetaData(
|
||||
OpusAudioKey(entryKey), OggOpusConstants.OpusExtension, trackDuration, opusBitrate);
|
||||
var stagedOpusPath = opusPath;
|
||||
var audioStored = await _fileDatabase.RegisterResourceStreamingAsync(
|
||||
VaultConstants.TrackOpus, OpusAudioKey(entryKey), audioMeta,
|
||||
(destination, token) => AudioStoreStream.CopyFileAsync(stagedOpusPath, destination, token), ct);
|
||||
if (!audioStored)
|
||||
{
|
||||
_logger.LogError("Opus transcode: vault write of Opus audio failed for {EntryKey}.", entryKey);
|
||||
return false;
|
||||
}
|
||||
|
||||
// The sidecar is the setup header (a few KB) plus the seek index (~16 bytes per 0.5 s bucket);
|
||||
// it is inherently bounded and already in managed memory, so the whole-buffer write is correct.
|
||||
var sidecar = new OpusSidecar(walk.SetupHeaderBytes, walk.SeekIndex).ToBytes();
|
||||
var sidecarBinary = new MediaBinary(new MediaBinaryParams(
|
||||
sidecar, sidecar.Length, OggOpusConstants.SidecarExtension));
|
||||
var sidecarStored = await _fileDatabase.RegisterResourceAsync(
|
||||
VaultConstants.TrackOpus, OpusSidecarKey(entryKey), sidecarBinary);
|
||||
if (!sidecarStored)
|
||||
{
|
||||
_logger.LogError("Opus transcode: vault write of sidecar failed for {EntryKey}.", entryKey);
|
||||
return false;
|
||||
}
|
||||
|
||||
_logger.LogInformation(
|
||||
"Opus transcode complete for {EntryKey}: {OpusBytes} bytes, {Points} seek points, {Duration:F1}s.",
|
||||
entryKey, opusLength, walk.SeekIndex.Points.Count, walk.SeekIndex.TotalDurationSeconds);
|
||||
return true;
|
||||
}
|
||||
catch (OperationCanceledException) when (ct.IsCancellationRequested)
|
||||
{
|
||||
throw;
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Opus transcode failed for {EntryKey}; track stays lossless-only.", entryKey);
|
||||
return false;
|
||||
}
|
||||
finally
|
||||
{
|
||||
if (sourcePath is not null)
|
||||
TryDelete(sourcePath);
|
||||
if (opusPath is not null)
|
||||
TryDelete(opusPath);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>The vault entry key under which a track's Opus audio bytes are stored.</summary>
|
||||
public static string OpusAudioKey(string entryKey) => entryKey;
|
||||
|
||||
/// <summary>The vault entry key under which a track's setup-header + seek-index sidecar is stored.</summary>
|
||||
public static string OpusSidecarKey(string entryKey) => $"{entryKey}-sidecar";
|
||||
|
||||
private async Task EnsureVaultAsync()
|
||||
{
|
||||
// The TrackOpus vault is created at host startup (Startup.cs), so this guard is normally a
|
||||
// no-op for the upload path. It is retained for the backfill path, which may run via a
|
||||
// standalone CLI or a host that skips vault pre-creation, where the vault might not exist.
|
||||
if (!_fileDatabase.HasVault(VaultConstants.TrackOpus))
|
||||
await _fileDatabase.CreateVaultAsync(VaultConstants.TrackOpus, MediaVaultType.Audio);
|
||||
}
|
||||
|
||||
private void TryDelete(string path)
|
||||
{
|
||||
try
|
||||
{
|
||||
if (File.Exists(path))
|
||||
File.Delete(path);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogWarning(ex, "Opus transcode: failed to delete staging file {Path}.", path);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
using DeepDrftModels.Enums;
|
||||
|
||||
namespace DeepDrftContent.Processors.Opus;
|
||||
|
||||
/// <summary>
|
||||
/// The outcome of resolving a track + requested <see cref="AudioFormat"/> to a concrete artifact
|
||||
/// (Phase 18.2; read-path streaming). Carries an <em>open, seekable, disk-backed</em> <see cref="Stream"/>
|
||||
/// over the artifact's bytes — never a buffered <c>byte[]</c>, so a ~220 MB Opus file or ~970 MB lossless
|
||||
/// source is never materialized in a managed array per request. Also carries the content-type that matches
|
||||
/// <em>what was actually returned</em>, and the format actually served — which may differ from the requested
|
||||
/// one when the C2 fallback fires (Opus requested, no Opus artifact → the lossless artifact + its
|
||||
/// content-type). The delivery layer (18.3) sets the response <c>Content-Type</c> from
|
||||
/// <see cref="ContentType"/> so the eventual decoder picks the right decoder for the bytes it receives.
|
||||
/// <para>
|
||||
/// Ownership: the resolver opens the stream; the caller takes ownership. On the success path the caller hands
|
||||
/// <see cref="Stream"/> to <c>File(..., enableRangeProcessing: true)</c>, which disposes it after the
|
||||
/// response. On any pre-handoff throw the caller disposes this instance (which disposes the stream) so the
|
||||
/// underlying <see cref="FileStream"/> never leaks — mirroring the lossless disk-stream path's catch-path
|
||||
/// disposal.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
/// <param name="Stream">An open, seekable, disk-backed stream over the resolved artifact. The caller owns it.</param>
|
||||
/// <param name="ContentType">The MIME type of the bytes in <paramref name="Stream"/> (e.g. <c>audio/ogg</c>
|
||||
/// for Opus, or the source's real MIME for lossless).</param>
|
||||
/// <param name="ResolvedFormat">The format actually returned. Equal to the requested format on a direct
|
||||
/// hit; <see cref="AudioFormat.Lossless"/> when an Opus request fell back.</param>
|
||||
public sealed record ResolvedAudio(Stream Stream, string ContentType, AudioFormat ResolvedFormat)
|
||||
: IDisposable, IAsyncDisposable
|
||||
{
|
||||
/// <summary>True when an Opus request was served the lossless artifact because no Opus existed (C2).</summary>
|
||||
public bool DidFallBack(AudioFormat requested) => requested != ResolvedFormat;
|
||||
|
||||
public void Dispose() => Stream.Dispose();
|
||||
|
||||
public ValueTask DisposeAsync() => Stream.DisposeAsync();
|
||||
}
|
||||
@@ -0,0 +1,129 @@
|
||||
using DeepDrftContent.Constants;
|
||||
using DeepDrftContent.FileDatabase.Models;
|
||||
using DeepDrftModels.Enums;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using FileDb = DeepDrftContent.FileDatabase.Services.FileDatabase;
|
||||
|
||||
namespace DeepDrftContent.Processors.Opus;
|
||||
|
||||
/// <summary>
|
||||
/// The server-side format resolution + sidecar lookup seam (Phase 18.2). Given a track's
|
||||
/// <c>EntryKey</c> and a requested <see cref="AudioFormat"/>, returns the correct audio artifact and the
|
||||
/// content-type that matches it; given an <c>EntryKey</c>, returns the Opus seek/setup sidecar bytes.
|
||||
/// Downstream waves call this — 18.3 wires it behind the <c>?format=</c> stream param and serves the
|
||||
/// sidecar over HTTP; this wave delivers only the seam, not the HTTP surface.
|
||||
/// <para>
|
||||
/// Additive and non-breaking (C2): the lossless branch streams the source exactly as the existing read
|
||||
/// path does (via <see cref="TrackContentService.OpenAudioMediaStreamAsync"/>, a non-buffering disk
|
||||
/// stream), and an Opus request for a track with no Opus artifact falls back to lossless rather than
|
||||
/// failing. Mirrors the <see cref="WaveformProfileService"/> derived-artifact lookup precedent: read from
|
||||
/// the dedicated vault, swallow misses to null (FileDatabase convention), let the caller decide.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// Read-path streaming: artifacts are resolved as open, seekable, disk-backed <see cref="ResolvedAudio"/>
|
||||
/// handles — never whole-file <c>byte[]</c> loads — so the delivery layer streams them straight to the
|
||||
/// response (Range/206 honoured by the seekable <c>FileStream</c>) without buffering a ~220 MB Opus file
|
||||
/// or a ~970 MB lossless source per request.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public sealed class TrackFormatResolver
|
||||
{
|
||||
private readonly FileDb _fileDatabase;
|
||||
private readonly TrackContentService _trackContentService;
|
||||
private readonly ILogger<TrackFormatResolver> _logger;
|
||||
|
||||
public TrackFormatResolver(
|
||||
FileDb fileDatabase,
|
||||
TrackContentService trackContentService,
|
||||
ILogger<TrackFormatResolver> logger)
|
||||
{
|
||||
_fileDatabase = fileDatabase;
|
||||
_trackContentService = trackContentService;
|
||||
_logger = logger;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Resolves <paramref name="entryKey"/> + <paramref name="requestedFormat"/> to the audio artifact to
|
||||
/// serve plus its content-type. <see cref="AudioFormat.Lossless"/> resolves the source artifact in the
|
||||
/// <c>tracks</c> vault with its real MIME (WAV/MP3/FLAC). <see cref="AudioFormat.Opus"/> resolves the
|
||||
/// derived Opus artifact (<c>audio/ogg</c>) when present, and <strong>falls back to lossless</strong>
|
||||
/// when it is not (C2). Returns null only when even the lossless source is missing — i.e. the track has
|
||||
/// no audio at all (an unknown key or a genuinely empty vault), the one case the caller treats as 404.
|
||||
/// </summary>
|
||||
public async Task<ResolvedAudio?> ResolveAsync(string entryKey, AudioFormat requestedFormat)
|
||||
{
|
||||
if (requestedFormat == AudioFormat.Opus)
|
||||
{
|
||||
var opusVault = _fileDatabase.GetVault(VaultConstants.TrackOpus);
|
||||
if (opusVault is not null)
|
||||
{
|
||||
// Disk-backed, seekable stream over the Opus artifact — no whole-file buffer. The caller
|
||||
// owns the stream (hands it to File(...) on success, disposes on a pre-handoff throw).
|
||||
var opus = await opusVault.GetEntryStreamAsync(OpusTranscodeService.OpusAudioKey(entryKey));
|
||||
if (opus is not null)
|
||||
return new ResolvedAudio(
|
||||
opus.Stream, MimeTypeExtensions.GetMimeType(opus.Extension), AudioFormat.Opus);
|
||||
}
|
||||
|
||||
// C2 fallback: no Opus artifact yet (legacy row, not backfilled, or transcode failed). Degrade
|
||||
// to lossless rather than 404 — Opus is strictly additive; its absence never means "no audio".
|
||||
_logger.LogInformation(
|
||||
"Opus requested for {EntryKey} but no Opus artifact exists; falling back to lossless.", entryKey);
|
||||
}
|
||||
|
||||
return await ResolveLosslessAsync(entryKey);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Resolves the lossless source artifact and its real MIME as a non-buffering disk stream — the existing
|
||||
/// read path. Shared by the explicit-lossless branch and the Opus fallback so both produce identical
|
||||
/// bytes + content-type. The returned stream is seekable, so the delivery layer's Range→206 still works.
|
||||
/// </summary>
|
||||
private async Task<ResolvedAudio?> ResolveLosslessAsync(string entryKey)
|
||||
{
|
||||
var source = await _trackContentService.OpenAudioMediaStreamAsync(entryKey);
|
||||
if (source is null)
|
||||
return null;
|
||||
|
||||
return new ResolvedAudio(
|
||||
source.Stream, MimeTypeExtensions.GetMimeType(source.Extension), AudioFormat.Lossless);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Returns the Opus setup-header + seek-index sidecar bytes for <paramref name="entryKey"/>, or null
|
||||
/// when no sidecar is stored (no Opus artifact yet, or an older derive predating the sidecar). 18.3
|
||||
/// serves these on their own path; 18.4 fetches them once on track load and parses them into the
|
||||
/// client's <c>OpusSeekData</c>. The bytes are the raw <see cref="OpusSidecar"/> blob
|
||||
/// (<c>[uint32 setupHeaderLength][setup-header][seek-index]</c>) exactly as 18.1 stored them.
|
||||
/// </summary>
|
||||
public async Task<byte[]?> GetOpusSidecarAsync(string entryKey)
|
||||
{
|
||||
var sidecar = await _fileDatabase.LoadResourceAsync<MediaBinary>(
|
||||
VaultConstants.TrackOpus, OpusTranscodeService.OpusSidecarKey(entryKey));
|
||||
return sidecar?.Buffer;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Reports whether <paramref name="entryKey"/> already has a complete Opus derive — both the audio bytes
|
||||
/// AND the seek/setup sidecar present in the <c>track-opus</c> vault. The Backfill-Opus pass (18.5) uses
|
||||
/// this to enqueue only tracks that are missing or half-derived (audio without sidecar = unseekable, so
|
||||
/// treated as incomplete and re-derived). Both halves are required because the transcode stores them in
|
||||
/// sequence and a sidecar-write failure leaves a track the delivery layer must not treat as Opus-ready.
|
||||
/// </summary>
|
||||
public async Task<bool> HasOpusAsync(string entryKey)
|
||||
{
|
||||
// Index-only existence — never read a file body. The opus-status admin endpoint calls this in a loop
|
||||
// over the entire catalogue, so a body load here would stream the whole library's audio sequentially.
|
||||
// HasIndexEntry is a pure in-memory index lookup (no disk read, no allocation per track).
|
||||
var opusVault = _fileDatabase.GetVault(VaultConstants.TrackOpus);
|
||||
if (opusVault is null)
|
||||
return false;
|
||||
|
||||
if (!await opusVault.HasIndexEntry(OpusTranscodeService.OpusAudioKey(entryKey)))
|
||||
return false;
|
||||
|
||||
// Both halves required: audio without the seek/setup sidecar is unseekable, so a half-derived track
|
||||
// counts as not-having-Opus (the same completeness rule the Backfill-Opus pass enqueues against).
|
||||
return await opusVault.HasIndexEntry(OpusTranscodeService.OpusSidecarKey(entryKey));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,68 @@
|
||||
namespace DeepDrftContent.Processors;
|
||||
|
||||
/// <summary>
|
||||
/// The product of processing an uploaded audio file on the store path: the metadata SQL and the
|
||||
/// vault index need, plus a streamed writer that emits the canonical vault bytes to a destination
|
||||
/// stream without ever materializing the whole file in a managed <c>byte[]</c>.
|
||||
///
|
||||
/// This replaces the former whole-file <c>AudioBinary</c> as the processor output for upload /
|
||||
/// replace-audio (Wave 1 OOM fix): passthrough formats (standard-PCM WAV, MP3, FLAC) stream the
|
||||
/// source file straight to the destination, and EXTENSIBLE WAVs stream their normalization to
|
||||
/// standard PCM. The vault <em>load</em> path still uses <c>AudioBinary</c> (a full buffer) — that
|
||||
/// is the Wave 2 read path and is out of scope here.
|
||||
///
|
||||
/// <see cref="WriteToAsync"/> is invoked exactly once by the streaming vault register, against the
|
||||
/// freshly opened backing <see cref="System.IO.FileStream"/>. The writer re-opens the source file
|
||||
/// itself, so the source (a staging file) must still exist when the register runs — it does, because
|
||||
/// processing and registration are sequential within the store call, before the staging-file
|
||||
/// <c>finally</c> cleanup.
|
||||
/// </summary>
|
||||
public sealed class ProcessedAudio
|
||||
{
|
||||
/// <summary>The stored file extension (e.g. <c>.wav</c>, <c>.mp3</c>, <c>.flac</c>).</summary>
|
||||
public string Extension { get; }
|
||||
|
||||
/// <summary>Audio duration in seconds, extracted from the header.</summary>
|
||||
public double Duration { get; }
|
||||
|
||||
/// <summary>Audio bitrate in kbps, extracted from (or estimated for) the header.</summary>
|
||||
public int Bitrate { get; }
|
||||
|
||||
/// <summary>
|
||||
/// The canonical stored byte count — computed from the header and file length, never by
|
||||
/// buffering the body. Used only for diagnostics (confirming the streamed path was taken).
|
||||
/// </summary>
|
||||
public long Size { get; }
|
||||
|
||||
private readonly Func<Stream, CancellationToken, Task> _writeTo;
|
||||
|
||||
public ProcessedAudio(
|
||||
string extension,
|
||||
double duration,
|
||||
int bitrate,
|
||||
long size,
|
||||
Func<Stream, CancellationToken, Task> writeTo)
|
||||
{
|
||||
Extension = extension;
|
||||
Duration = duration;
|
||||
Bitrate = bitrate;
|
||||
Size = size;
|
||||
_writeTo = writeTo;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Streams the canonical vault bytes to <paramref name="destination"/>. Bounded-buffer — peak
|
||||
/// managed memory is O(buffer), not O(filesize).
|
||||
/// </summary>
|
||||
public Task WriteToAsync(Stream destination, CancellationToken cancellationToken = default)
|
||||
=> _writeTo(destination, cancellationToken);
|
||||
|
||||
/// <summary>
|
||||
/// Builds a passthrough plan: the stored bytes are byte-identical to the source file (standard
|
||||
/// PCM WAV, MP3, FLAC — no transcoding). The writer is a bounded disk-to-disk copy.
|
||||
/// </summary>
|
||||
public static ProcessedAudio Passthrough(
|
||||
string sourcePath, string extension, double duration, int bitrate, long sourceLength)
|
||||
=> new(extension, duration, bitrate, sourceLength,
|
||||
(destination, ct) => AudioStoreStream.CopyFileAsync(sourcePath, destination, ct));
|
||||
}
|
||||
@@ -18,100 +18,27 @@ public class RmsLoudnessAlgorithm : ILoudnessAlgorithm
|
||||
/// </summary>
|
||||
public const double SmoothingTimeConstantSeconds = 0.005;
|
||||
|
||||
/// <summary>
|
||||
/// Whole-buffer reduction. Defined in terms of <see cref="CreateAccumulator"/> so the streaming and
|
||||
/// whole-buffer paths share one decode + finalize implementation — byte-identical output by
|
||||
/// construction, not by parallel maintenance.
|
||||
/// </summary>
|
||||
public double[] Compute(ReadOnlySpan<byte> pcmData, int channels, int sampleRate, int bitsPerSample, int bucketCount)
|
||||
{
|
||||
var accumulator = CreateAccumulator(pcmData.Length, channels, sampleRate, bitsPerSample, bucketCount);
|
||||
accumulator.Add(pcmData);
|
||||
return accumulator.Finish();
|
||||
}
|
||||
|
||||
public ILoudnessAccumulator CreateAccumulator(
|
||||
long pcmByteLength, int channels, int sampleRate, int bitsPerSample, int bucketCount)
|
||||
{
|
||||
if (bucketCount <= 0)
|
||||
{
|
||||
throw new ArgumentOutOfRangeException(nameof(bucketCount), "Bucket count must be positive.");
|
||||
}
|
||||
|
||||
var result = new double[bucketCount];
|
||||
|
||||
if (channels <= 0)
|
||||
{
|
||||
return result;
|
||||
}
|
||||
|
||||
var bytesPerSample = bitsPerSample / 8;
|
||||
if (bytesPerSample <= 0)
|
||||
{
|
||||
return result;
|
||||
}
|
||||
|
||||
var bytesPerFrame = bytesPerSample * channels;
|
||||
var frameCount = pcmData.Length / bytesPerFrame;
|
||||
if (frameCount == 0)
|
||||
{
|
||||
return result;
|
||||
}
|
||||
|
||||
// Sum of squared mono amplitudes and the frame count, per bucket. A frame's bucket is
|
||||
// determined by its position in the timeline so buckets are equal-duration slices.
|
||||
var sumSquares = new double[bucketCount];
|
||||
var counts = new long[bucketCount];
|
||||
|
||||
for (var frame = 0; frame < frameCount; frame++)
|
||||
{
|
||||
var frameStart = frame * bytesPerFrame;
|
||||
|
||||
double channelSum = 0;
|
||||
for (var ch = 0; ch < channels; ch++)
|
||||
{
|
||||
var sampleStart = frameStart + ch * bytesPerSample;
|
||||
channelSum += ReadSampleNormalized(pcmData, sampleStart, bitsPerSample);
|
||||
}
|
||||
|
||||
var mono = channelSum / channels;
|
||||
|
||||
// long math avoids overflow on large files before the divide back into bucket index.
|
||||
var bucket = (int)((long)frame * bucketCount / frameCount);
|
||||
if (bucket >= bucketCount)
|
||||
{
|
||||
bucket = bucketCount - 1;
|
||||
}
|
||||
|
||||
sumSquares[bucket] += mono * mono;
|
||||
counts[bucket]++;
|
||||
}
|
||||
|
||||
for (var i = 0; i < bucketCount; i++)
|
||||
{
|
||||
if (counts[i] > 0)
|
||||
{
|
||||
result[i] = Math.Sqrt(sumSquares[i] / counts[i]);
|
||||
}
|
||||
}
|
||||
|
||||
// Envelope smoothing (~15 ms): round the spikey per-bucket RMS into a smooth contour before
|
||||
// peak-normalization, so the rendered ribbon reads as a continuous curve, not faceted polygons.
|
||||
// Each bucket spans (totalSeconds / bucketCount) of audio; the filter coefficient is derived
|
||||
// from that against the time constant so the smoothing is duration-aware, not a fixed window.
|
||||
var totalSeconds = (double)frameCount / sampleRate;
|
||||
var bucketSeconds = totalSeconds / bucketCount;
|
||||
SmoothEnvelope(result, bucketSeconds);
|
||||
|
||||
var peak = 0.0;
|
||||
for (var i = 0; i < bucketCount; i++)
|
||||
{
|
||||
if (result[i] > peak)
|
||||
{
|
||||
peak = result[i];
|
||||
}
|
||||
}
|
||||
|
||||
if (peak <= 0)
|
||||
{
|
||||
// Silence — return all zeros (Array is already zero-initialized).
|
||||
Array.Clear(result);
|
||||
return result;
|
||||
}
|
||||
|
||||
for (var i = 0; i < bucketCount; i++)
|
||||
{
|
||||
result[i] /= peak;
|
||||
}
|
||||
|
||||
return result;
|
||||
return new RmsLoudnessAccumulator(pcmByteLength, channels, sampleRate, bitsPerSample, bucketCount);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -122,7 +49,7 @@ public class RmsLoudnessAlgorithm : ILoudnessAlgorithm
|
||||
/// each bucket blends <c>(1 − a)</c> of itself with <c>a</c> of the running envelope. A near-zero
|
||||
/// or non-finite bucket duration leaves the data untouched (nothing to smooth meaningfully).
|
||||
/// </summary>
|
||||
private static void SmoothEnvelope(double[] data, double bucketSeconds)
|
||||
internal static void SmoothEnvelope(double[] data, double bucketSeconds)
|
||||
{
|
||||
if (data.Length < 2 || bucketSeconds <= 0 || !double.IsFinite(bucketSeconds))
|
||||
{
|
||||
@@ -154,7 +81,7 @@ public class RmsLoudnessAlgorithm : ILoudnessAlgorithm
|
||||
/// Decodes one PCM sample at <paramref name="offset"/> to a normalized amplitude in [-1, 1].
|
||||
/// 8-bit is unsigned (0..255, centered at 128); 16/24/32-bit are signed little-endian.
|
||||
/// </summary>
|
||||
private static double ReadSampleNormalized(ReadOnlySpan<byte> data, int offset, int bitsPerSample)
|
||||
internal static double ReadSampleNormalized(ReadOnlySpan<byte> data, int offset, int bitsPerSample)
|
||||
{
|
||||
switch (bitsPerSample)
|
||||
{
|
||||
@@ -194,3 +121,167 @@ public class RmsLoudnessAlgorithm : ILoudnessAlgorithm
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Single-pass RMS accumulator backing <see cref="RmsLoudnessAlgorithm"/>. Frames are fed via
|
||||
/// <see cref="Add"/> in arbitrary chunks; a partial frame straddling a chunk boundary is carried in a
|
||||
/// one-frame buffer. The per-frame decode, bucket assignment, and per-bucket accumulation are the exact
|
||||
/// arithmetic the former whole-buffer loop used, in the same frame order, so the floating-point result
|
||||
/// is bit-identical whether the PCM arrives in one span or many. <see cref="Finish"/> applies the same
|
||||
/// envelope smoothing and peak-normalization as before. Memory is O(bucketCount + one frame).
|
||||
/// </summary>
|
||||
public sealed class RmsLoudnessAccumulator : ILoudnessAccumulator
|
||||
{
|
||||
private readonly int _channels;
|
||||
private readonly int _sampleRate;
|
||||
private readonly int _bitsPerSample;
|
||||
private readonly int _bucketCount;
|
||||
private readonly int _bytesPerSample;
|
||||
private readonly int _bytesPerFrame;
|
||||
private readonly long _frameCount;
|
||||
|
||||
private readonly double[] _sumSquares;
|
||||
private readonly long[] _counts;
|
||||
private readonly byte[] _carry;
|
||||
private int _carryLen;
|
||||
private long _frameIndex;
|
||||
|
||||
internal RmsLoudnessAccumulator(long pcmByteLength, int channels, int sampleRate, int bitsPerSample, int bucketCount)
|
||||
{
|
||||
_channels = channels;
|
||||
_sampleRate = sampleRate;
|
||||
_bitsPerSample = bitsPerSample;
|
||||
_bucketCount = bucketCount;
|
||||
_sumSquares = new double[bucketCount];
|
||||
_counts = new long[bucketCount];
|
||||
|
||||
// Guards mirror the former whole-buffer Compute exactly: any degenerate parameter leaves
|
||||
// _frameCount at 0, so Add is a no-op and Finish returns the zero-initialized profile.
|
||||
_bytesPerSample = bitsPerSample / 8;
|
||||
if (channels <= 0 || _bytesPerSample <= 0)
|
||||
{
|
||||
_bytesPerFrame = 0;
|
||||
_frameCount = 0;
|
||||
_carry = [];
|
||||
return;
|
||||
}
|
||||
|
||||
_bytesPerFrame = _bytesPerSample * channels;
|
||||
_frameCount = pcmByteLength / _bytesPerFrame;
|
||||
_carry = new byte[_bytesPerFrame];
|
||||
}
|
||||
|
||||
public void Add(ReadOnlySpan<byte> pcmChunk)
|
||||
{
|
||||
if (_frameIndex >= _frameCount)
|
||||
{
|
||||
return; // degenerate input, or every expected frame already consumed
|
||||
}
|
||||
|
||||
var pos = 0;
|
||||
|
||||
// Complete a frame carried from the previous chunk first.
|
||||
if (_carryLen > 0)
|
||||
{
|
||||
var need = _bytesPerFrame - _carryLen;
|
||||
var take = Math.Min(need, pcmChunk.Length);
|
||||
pcmChunk.Slice(0, take).CopyTo(_carry.AsSpan(_carryLen));
|
||||
_carryLen += take;
|
||||
pos += take;
|
||||
|
||||
if (_carryLen < _bytesPerFrame)
|
||||
{
|
||||
return; // still not a full frame
|
||||
}
|
||||
|
||||
ProcessFrame(_carry);
|
||||
_carryLen = 0;
|
||||
if (_frameIndex >= _frameCount)
|
||||
{
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// Whole frames directly from the chunk.
|
||||
while (pos + _bytesPerFrame <= pcmChunk.Length && _frameIndex < _frameCount)
|
||||
{
|
||||
ProcessFrame(pcmChunk.Slice(pos, _bytesPerFrame));
|
||||
pos += _bytesPerFrame;
|
||||
}
|
||||
|
||||
// Stash a trailing partial frame for the next chunk — but only while frames are still expected.
|
||||
// A trailing partial frame on the final chunk is dropped, matching the whole-buffer path.
|
||||
if (_frameIndex < _frameCount && pos < pcmChunk.Length)
|
||||
{
|
||||
var remainder = pcmChunk.Slice(pos);
|
||||
remainder.CopyTo(_carry);
|
||||
_carryLen = remainder.Length;
|
||||
}
|
||||
}
|
||||
|
||||
private void ProcessFrame(ReadOnlySpan<byte> frame)
|
||||
{
|
||||
double channelSum = 0;
|
||||
for (var ch = 0; ch < _channels; ch++)
|
||||
{
|
||||
channelSum += RmsLoudnessAlgorithm.ReadSampleNormalized(frame, ch * _bytesPerSample, _bitsPerSample);
|
||||
}
|
||||
|
||||
var mono = channelSum / _channels;
|
||||
|
||||
// long math avoids overflow on large files before the divide back into bucket index.
|
||||
var bucket = (int)(_frameIndex * _bucketCount / _frameCount);
|
||||
if (bucket >= _bucketCount)
|
||||
{
|
||||
bucket = _bucketCount - 1;
|
||||
}
|
||||
|
||||
_sumSquares[bucket] += mono * mono;
|
||||
_counts[bucket]++;
|
||||
_frameIndex++;
|
||||
}
|
||||
|
||||
public double[] Finish()
|
||||
{
|
||||
var result = new double[_bucketCount];
|
||||
if (_frameCount == 0)
|
||||
{
|
||||
return result; // degenerate input — all zeros, as the whole-buffer guards returned
|
||||
}
|
||||
|
||||
for (var i = 0; i < _bucketCount; i++)
|
||||
{
|
||||
if (_counts[i] > 0)
|
||||
{
|
||||
result[i] = Math.Sqrt(_sumSquares[i] / _counts[i]);
|
||||
}
|
||||
}
|
||||
|
||||
// Envelope smoothing (~15 ms) then peak-normalization — identical to the whole-buffer finalize.
|
||||
var totalSeconds = (double)_frameCount / _sampleRate;
|
||||
var bucketSeconds = totalSeconds / _bucketCount;
|
||||
RmsLoudnessAlgorithm.SmoothEnvelope(result, bucketSeconds);
|
||||
|
||||
var peak = 0.0;
|
||||
for (var i = 0; i < _bucketCount; i++)
|
||||
{
|
||||
if (result[i] > peak)
|
||||
{
|
||||
peak = result[i];
|
||||
}
|
||||
}
|
||||
|
||||
if (peak <= 0)
|
||||
{
|
||||
Array.Clear(result);
|
||||
return result;
|
||||
}
|
||||
|
||||
for (var i = 0; i < _bucketCount; i++)
|
||||
{
|
||||
result[i] /= peak;
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -17,6 +17,10 @@ public class WaveformProfileService
|
||||
{
|
||||
private const string ProfileExtension = ".wfp";
|
||||
|
||||
/// <summary>Bounded read-buffer size for the streaming PCM pass — the only filesize-independent
|
||||
/// allocation on the streaming path (matches the store path's 80 KB copy buffer).</summary>
|
||||
private const int StreamReadBufferSize = 81920;
|
||||
|
||||
private readonly FileDb _fileDatabase;
|
||||
private readonly AudioProcessor _audioProcessor;
|
||||
private readonly ILoudnessAlgorithm _loudnessAlgorithm;
|
||||
@@ -117,6 +121,161 @@ public class WaveformProfileService
|
||||
return ComputeAndStoreAsync(wavBytes, entryKey, bucketCount, VaultConstants.TrackWaveforms);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Streaming counterpart of <see cref="ComputeAndStoreAsync"/>: computes and stores the fixed
|
||||
/// 512-bucket player-bar profile by reading the WAV from <paramref name="openWavStream"/> in bounded
|
||||
/// chunks, never materializing the whole file in a managed <c>byte[]</c>. Tri-state result matches
|
||||
/// the <c>RemoveResourceAsync</c> idiom so callers can map outcomes precisely: <c>null</c> = no audio
|
||||
/// stream available (the entry has no backing audio); <c>false</c> = audio present but no profile
|
||||
/// computable (non-WAV / float / padded) or the vault write failed; <c>true</c> = stored. Output is
|
||||
/// byte-identical to the whole-buffer path for the same WAV.
|
||||
/// </summary>
|
||||
public Task<bool?> ComputeAndStoreProfileStreamingAsync(
|
||||
Func<CancellationToken, Task<Stream?>> openWavStream,
|
||||
string entryKey,
|
||||
CancellationToken ct = default) =>
|
||||
RunStreamingAsync(
|
||||
openWavStream, entryKey,
|
||||
[(_options.BucketCount, VaultConstants.WaveformProfiles)], ct);
|
||||
|
||||
/// <summary>
|
||||
/// Streaming counterpart of <see cref="ComputeAndStoreHighResAsync"/>: computes and stores the
|
||||
/// duration-derived high-res datum (<see cref="VaultConstants.TrackWaveforms"/>) by streaming the WAV
|
||||
/// from <paramref name="openWavStream"/>. <paramref name="durationSeconds"/> drives the bucket count
|
||||
/// exactly as the whole-buffer path's <c>audio.Duration</c> did — pass the same vault-metadata
|
||||
/// duration to keep the stored bytes identical. Tri-state result as in
|
||||
/// <see cref="ComputeAndStoreProfileStreamingAsync"/>.
|
||||
/// </summary>
|
||||
public Task<bool?> ComputeAndStoreHighResStreamingAsync(
|
||||
Func<CancellationToken, Task<Stream?>> openWavStream,
|
||||
string entryKey,
|
||||
double durationSeconds,
|
||||
CancellationToken ct = default) =>
|
||||
RunStreamingAsync(
|
||||
openWavStream, entryKey,
|
||||
[(WaveformResolution.BucketCountForDuration(durationSeconds), VaultConstants.TrackWaveforms)], ct);
|
||||
|
||||
/// <summary>
|
||||
/// Computes and stores BOTH datums a track carries — the 512-bucket profile and the duration-derived
|
||||
/// high-res datum — from a SINGLE streaming pass over the WAV. One sequential read of the (possibly
|
||||
/// ~GB) audio feeds two independent accumulators, so memory stays O(bucket arrays + read buffer) and
|
||||
/// disk I/O is halved versus two separate passes. This is the upload / replace-audio hot path. Each
|
||||
/// datum's stored bytes are byte-identical to its whole-buffer counterpart. Tri-state: <c>null</c> =
|
||||
/// no audio stream; <c>false</c> = not WAV-decodable or a vault write failed; <c>true</c> = both
|
||||
/// datums stored. Best-effort callers ignore the result.
|
||||
/// </summary>
|
||||
public Task<bool?> ComputeAndStoreAllStreamingAsync(
|
||||
Func<CancellationToken, Task<Stream?>> openWavStream,
|
||||
string entryKey,
|
||||
double durationSeconds,
|
||||
CancellationToken ct = default) =>
|
||||
RunStreamingAsync(
|
||||
openWavStream, entryKey,
|
||||
[
|
||||
(_options.BucketCount, VaultConstants.WaveformProfiles),
|
||||
(WaveformResolution.BucketCountForDuration(durationSeconds), VaultConstants.TrackWaveforms),
|
||||
],
|
||||
ct);
|
||||
|
||||
/// <summary>
|
||||
/// Core streaming reduction: opens the WAV once, parses its header (bounded), then streams the PCM
|
||||
/// data region through one loudness accumulator per requested target, storing each datum. All
|
||||
/// targets are computed in the single pass. See the tri-state contract on the public wrappers.
|
||||
/// </summary>
|
||||
private async Task<bool?> RunStreamingAsync(
|
||||
Func<CancellationToken, Task<Stream?>> openWavStream,
|
||||
string entryKey,
|
||||
IReadOnlyList<(int BucketCount, string VaultName)> targets,
|
||||
CancellationToken ct)
|
||||
{
|
||||
try
|
||||
{
|
||||
await using var stream = await openWavStream(ct);
|
||||
if (stream is null)
|
||||
{
|
||||
// No backing audio for this entry — distinct from "present but undecodable".
|
||||
return null;
|
||||
}
|
||||
|
||||
var info = await _audioProcessor.TryReadPcmStreamInfoAsync(stream, stream.Length, ct);
|
||||
if (info is null)
|
||||
{
|
||||
_logger.LogWarning(
|
||||
"Waveform profile not computed for {EntryKey}: WAV PCM could not be extracted (streaming).",
|
||||
entryKey);
|
||||
return false;
|
||||
}
|
||||
|
||||
var v = info.Value;
|
||||
var accumulators = new ILoudnessAccumulator[targets.Count];
|
||||
for (var i = 0; i < targets.Count; i++)
|
||||
{
|
||||
accumulators[i] = _loudnessAlgorithm.CreateAccumulator(
|
||||
v.DataLength, v.Channels, v.SampleRate, v.BitsPerSample, targets[i].BucketCount);
|
||||
}
|
||||
|
||||
await StreamPcmThroughAsync(stream, v.DataStart, v.DataLength, accumulators, ct);
|
||||
|
||||
_logger.LogInformation(
|
||||
"Streaming waveform compute for {EntryKey}: {DataLength} PCM bytes, {TargetCount} datum(s), " +
|
||||
"{BufferSize}B read buffer — no whole-file load.",
|
||||
entryKey, v.DataLength, targets.Count, StreamReadBufferSize);
|
||||
|
||||
var allStored = true;
|
||||
for (var i = 0; i < targets.Count; i++)
|
||||
{
|
||||
var profile = accumulators[i].Finish();
|
||||
var quantized = Quantize(profile);
|
||||
|
||||
await EnsureVaultAsync(targets[i].VaultName);
|
||||
var binary = new MediaBinary(new MediaBinaryParams(quantized, quantized.Length, ProfileExtension));
|
||||
var stored = await _fileDatabase.RegisterResourceAsync(targets[i].VaultName, entryKey, binary);
|
||||
if (!stored)
|
||||
{
|
||||
_logger.LogWarning(
|
||||
"Waveform vault write failed for {EntryKey} in {VaultName}.", entryKey, targets[i].VaultName);
|
||||
allStored = false;
|
||||
}
|
||||
}
|
||||
|
||||
return allStored;
|
||||
}
|
||||
catch (Exception ex) when (ex is not OperationCanceledException)
|
||||
{
|
||||
_logger.LogError(ex, "Streaming waveform computation failed for {EntryKey}.", entryKey);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Seeks to the PCM data region and streams exactly <paramref name="dataLength"/> bytes through each
|
||||
/// accumulator in bounded reads. The accumulators carry partial frames internally, so the read
|
||||
/// boundaries need not align to frames. Peak memory is one read buffer — independent of file size.
|
||||
/// </summary>
|
||||
private static async Task StreamPcmThroughAsync(
|
||||
Stream stream, long dataStart, long dataLength, ILoudnessAccumulator[] accumulators, CancellationToken ct)
|
||||
{
|
||||
stream.Seek(dataStart, SeekOrigin.Begin);
|
||||
|
||||
var buffer = new byte[StreamReadBufferSize];
|
||||
var remaining = dataLength;
|
||||
while (remaining > 0)
|
||||
{
|
||||
var want = (int)Math.Min(buffer.Length, remaining);
|
||||
var read = await stream.ReadAsync(buffer.AsMemory(0, want), ct);
|
||||
if (read == 0)
|
||||
break;
|
||||
|
||||
var span = buffer.AsSpan(0, read);
|
||||
foreach (var accumulator in accumulators)
|
||||
{
|
||||
accumulator.Add(span);
|
||||
}
|
||||
|
||||
remaining -= read;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Returns the stored quantized profile bytes for a track from <paramref name="vaultName"/>
|
||||
/// (defaults to <see cref="VaultConstants.WaveformProfiles"/> when null), or null if no profile
|
||||
|
||||
@@ -40,13 +40,15 @@ public class TrackContentService
|
||||
string? album = null,
|
||||
string? genre = null,
|
||||
DateOnly? releaseDate = null,
|
||||
string? originalFileName = null)
|
||||
string? originalFileName = null,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
try
|
||||
{
|
||||
// Process the audio file (routed by extension)
|
||||
var audioBinary = await _audioProcessorRouter.ProcessAudioFileAsync(audioFilePath);
|
||||
if (audioBinary == null)
|
||||
// Process the audio file (routed by extension). The returned plan carries metadata plus a
|
||||
// streamed writer — no whole-file buffer (the store-path OOM fix).
|
||||
var processed = await _audioProcessorRouter.ProcessAudioFileAsync(audioFilePath, cancellationToken);
|
||||
if (processed == null)
|
||||
{
|
||||
throw new InvalidOperationException("Failed to process audio file");
|
||||
}
|
||||
@@ -60,8 +62,11 @@ public class TrackContentService
|
||||
await _fileDatabase.CreateVaultAsync(VaultConstants.Tracks, MediaVaultType.Audio);
|
||||
}
|
||||
|
||||
// Store the audio in FileDatabase
|
||||
var success = await _fileDatabase.RegisterResourceAsync(VaultConstants.Tracks, trackId, audioBinary);
|
||||
// Stream the audio into the vault. The metadata is supplied directly (there is no in-memory
|
||||
// AudioBinary on this path), and the bytes are written progressively from the staging file.
|
||||
var metaData = MetaDataFactory.CreateAudioMetaData(trackId, processed.Extension, processed.Duration, processed.Bitrate);
|
||||
var success = await _fileDatabase.RegisterResourceStreamingAsync(
|
||||
VaultConstants.Tracks, trackId, metaData, processed.WriteToAsync, cancellationToken);
|
||||
if (!success)
|
||||
{
|
||||
throw new InvalidOperationException("Failed to store audio in FileDatabase");
|
||||
@@ -77,7 +82,7 @@ public class TrackContentService
|
||||
OriginalFileName = originalFileName,
|
||||
// Persist the processor-extracted runtime to SQL so aggregate queries (total mix runtime)
|
||||
// need not touch the vault. Same value the high-res waveform compute reads downstream.
|
||||
DurationSeconds = audioBinary.Duration
|
||||
DurationSeconds = processed.Duration
|
||||
};
|
||||
|
||||
return trackEntity;
|
||||
@@ -100,34 +105,37 @@ public class TrackContentService
|
||||
string? album = null,
|
||||
string? genre = null,
|
||||
DateOnly? releaseDate = null,
|
||||
string? originalFileName = null) =>
|
||||
AddTrackAsync(wavFilePath, trackName, artist, album, genre, releaseDate, originalFileName);
|
||||
string? originalFileName = null,
|
||||
CancellationToken cancellationToken = default) =>
|
||||
AddTrackAsync(wavFilePath, trackName, artist, album, genre, releaseDate, originalFileName, cancellationToken);
|
||||
|
||||
/// <summary>
|
||||
/// Swaps the audio bytes for an existing track in place: processes a new audio file and
|
||||
/// re-registers it under the SAME <paramref name="entryKey"/> in the tracks vault. The track's
|
||||
/// vault key — and therefore its SQL link, release membership, position, and metadata — is
|
||||
/// untouched; only the binary changes. The new audio is written first; only on confirmed success
|
||||
/// is a stale old backing file cleaned up. A cross-format replacement (e.g. .wav → .flac) leaves
|
||||
/// the old file on disk under its former filename once the index is updated; the post-success
|
||||
/// cleanup removes it. For a same-extension overwrite the register alone suffices — the file is
|
||||
/// written in place. If the register fails the original audio is left intact and null is returned,
|
||||
/// so the track remains playable. Returns the freshly stored <see cref="AudioBinary"/> on success
|
||||
/// (so the caller can regenerate waveform data from the same bytes) — matching the FileDatabase
|
||||
/// swallow-and-return-null contract.
|
||||
/// untouched; only the binary changes. The new audio is streamed to the vault first; only on
|
||||
/// confirmed success is a stale old backing file cleaned up. A cross-format replacement (e.g.
|
||||
/// .wav → .flac) leaves the old file on disk under its former filename once the index is updated;
|
||||
/// the post-success cleanup removes it. For a same-extension overwrite the register alone suffices.
|
||||
/// If the register fails the original audio is left intact and null is returned, so the track
|
||||
/// remains playable. Returns the freshly stored audio's <b>duration</b> on success (the caller
|
||||
/// re-reads the vault for waveform regen and uses this for the SQL duration write) — matching the
|
||||
/// FileDatabase swallow-and-return-null contract. The new bytes are never materialized in memory.
|
||||
/// </summary>
|
||||
public async Task<AudioBinary?> ReplaceTrackAudioAsync(string entryKey, string audioFilePath)
|
||||
public async Task<double?> ReplaceTrackAudioAsync(string entryKey, string audioFilePath, CancellationToken cancellationToken = default)
|
||||
{
|
||||
try
|
||||
{
|
||||
// Capture the old extension before touching the vault. After register the index
|
||||
// will point to the new extension, so we need the old value now to detect a
|
||||
// cross-format swap and clean up the stale file post-success.
|
||||
var existing = await _fileDatabase.LoadResourceAsync<AudioBinary>(VaultConstants.Tracks, entryKey);
|
||||
var oldExtension = existing?.Extension;
|
||||
// Capture the old extension from the index metadata (not by loading the file — that would
|
||||
// pull the whole old audio into memory). After register the index points to the new
|
||||
// extension, so we need the old value now to detect a cross-format swap and clean up the
|
||||
// stale file post-success.
|
||||
var trackVault = _fileDatabase.GetVault(VaultConstants.Tracks);
|
||||
var existingMeta = trackVault is null ? null : await trackVault.GetEntryMetadata(entryKey);
|
||||
var oldExtension = existingMeta?.Extension;
|
||||
|
||||
var audioBinary = await _audioProcessorRouter.ProcessAudioFileAsync(audioFilePath);
|
||||
if (audioBinary == null)
|
||||
var processed = await _audioProcessorRouter.ProcessAudioFileAsync(audioFilePath, cancellationToken);
|
||||
if (processed == null)
|
||||
{
|
||||
Console.WriteLine($"TrackContentService.ReplaceTrackAudioAsync: processing returned null for {entryKey}");
|
||||
return null;
|
||||
@@ -138,9 +146,11 @@ public class TrackContentService
|
||||
await _fileDatabase.CreateVaultAsync(VaultConstants.Tracks, MediaVaultType.Audio);
|
||||
}
|
||||
|
||||
// Register the new audio. This upserts the index entry (new extension recorded) and
|
||||
// writes the new file to disk. If this fails the original entry and file are untouched.
|
||||
var success = await _fileDatabase.RegisterResourceAsync(VaultConstants.Tracks, entryKey, audioBinary);
|
||||
// Stream the new audio in. This upserts the index entry (new extension recorded) and writes
|
||||
// the new file to disk. If this fails the original entry and file are untouched.
|
||||
var metaData = MetaDataFactory.CreateAudioMetaData(entryKey, processed.Extension, processed.Duration, processed.Bitrate);
|
||||
var success = await _fileDatabase.RegisterResourceStreamingAsync(
|
||||
VaultConstants.Tracks, entryKey, metaData, processed.WriteToAsync, cancellationToken);
|
||||
if (!success)
|
||||
{
|
||||
Console.WriteLine($"TrackContentService.ReplaceTrackAudioAsync: vault write failed for {entryKey}; original audio preserved");
|
||||
@@ -153,7 +163,7 @@ public class TrackContentService
|
||||
// old path — RemoveResourceAsync would now resolve to the new extension and delete the
|
||||
// wrong file. Non-fatal: an orphaned old file is a disk-hygiene concern, not a
|
||||
// playback issue (the index no longer references it).
|
||||
if (oldExtension != null && oldExtension != audioBinary.Extension)
|
||||
if (oldExtension != null && oldExtension != processed.Extension)
|
||||
{
|
||||
var vault = _fileDatabase.GetVault(VaultConstants.Tracks);
|
||||
if (vault != null)
|
||||
@@ -172,7 +182,7 @@ public class TrackContentService
|
||||
}
|
||||
}
|
||||
|
||||
return audioBinary;
|
||||
return processed.Duration;
|
||||
}
|
||||
catch (Exception ex) when (ex is not OperationCanceledException)
|
||||
{
|
||||
@@ -191,6 +201,65 @@ public class TrackContentService
|
||||
return await _fileDatabase.LoadResourceAsync<AudioBinary>(VaultConstants.Tracks, trackId);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Opens a read-only, seekable stream over a track's vault audio, or null if the entry has no
|
||||
/// backing file. The caller owns the stream and must dispose it. Unlike <see cref="GetAudioBinaryAsync"/>
|
||||
/// this never buffers the whole file — it is the source for the streaming waveform compute. Follows
|
||||
/// the FileDatabase swallow-and-return-null contract.
|
||||
/// </summary>
|
||||
/// <param name="trackId">Track ID (EntryKey)</param>
|
||||
public async Task<Stream?> OpenAudioStreamAsync(string trackId)
|
||||
{
|
||||
var vault = _fileDatabase.GetVault(VaultConstants.Tracks);
|
||||
if (vault is null)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
var media = await vault.GetEntryStreamAsync(trackId);
|
||||
return media?.Stream;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Opens a read-only stream over a track's vault audio together with its stored extension, or null if
|
||||
/// the entry has no backing file. Same non-buffering contract as <see cref="OpenAudioStreamAsync"/>,
|
||||
/// but keeps the <see cref="MediaStream.Extension"/> the caller needs to name a staging file for a
|
||||
/// format-detecting consumer (the Opus transcode reopens the source by extension for ffmpeg). The
|
||||
/// caller owns the returned <see cref="MediaStream"/> and must dispose it. Follows the FileDatabase
|
||||
/// swallow-and-return-null contract.
|
||||
/// </summary>
|
||||
/// <param name="trackId">Track ID (EntryKey)</param>
|
||||
public async Task<MediaStream?> OpenAudioMediaStreamAsync(string trackId)
|
||||
{
|
||||
var vault = _fileDatabase.GetVault(VaultConstants.Tracks);
|
||||
if (vault is null)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
return await vault.GetEntryStreamAsync(trackId);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Reads a track's stored audio duration from the vault index metadata WITHOUT loading the audio
|
||||
/// body — the cheap counterpart of <c>GetAudioBinaryAsync(...).Duration</c>. Returns null if the
|
||||
/// entry is unknown or carries no audio metadata. The streaming high-res waveform path uses this to
|
||||
/// derive the duration-based bucket count, matching the value the whole-buffer path read off
|
||||
/// <see cref="AudioBinary.Duration"/> so the stored datum is byte-identical.
|
||||
/// </summary>
|
||||
/// <param name="trackId">Track ID (EntryKey)</param>
|
||||
public async Task<double?> GetAudioDurationAsync(string trackId)
|
||||
{
|
||||
var vault = _fileDatabase.GetVault(VaultConstants.Tracks);
|
||||
if (vault is null)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
var metaData = await vault.GetEntryMetadata(trackId);
|
||||
return metaData is AudioMetaData audio ? audio.Duration : null;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Checks if FileDatabase is available and tracks vault exists
|
||||
/// </summary>
|
||||
|
||||
@@ -518,7 +518,10 @@
|
||||
{
|
||||
var row = _tracks[i];
|
||||
|
||||
if (row.Status == BatchRowStatus.Done)
|
||||
// Skip rows already processed in a prior submit attempt. PostProcessing counts as processed:
|
||||
// the track persisted successfully (only its background Opus derive is still settling), so a
|
||||
// re-submit after a partial failure must NOT re-upload it and mint a duplicate.
|
||||
if (row.Status is BatchRowStatus.Done or BatchRowStatus.PostProcessing)
|
||||
{
|
||||
_processedCount++;
|
||||
continue;
|
||||
@@ -638,7 +641,12 @@
|
||||
}
|
||||
}
|
||||
|
||||
row.Status = BatchRowStatus.Done;
|
||||
// §3.1a: a new-track upload persists the track (live + lossless) and the server
|
||||
// derives Opus in the background, so the row enters the visible Post-Processing
|
||||
// phase — same as BatchUpload. (The metadata-only update path above stays Done: it
|
||||
// changes no audio, so it triggers no transcode.) Non-blocking; the Releases view
|
||||
// polls the durable Opus status to settle it.
|
||||
row.Status = BatchRowStatus.PostProcessing;
|
||||
succeeded++;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -40,4 +40,9 @@ public class BatchRowModel
|
||||
: 0;
|
||||
}
|
||||
|
||||
public enum BatchRowStatus { Queued, Uploading, Done, Failed }
|
||||
// Done is the terminal success state (track persisted + playable losslessly). PostProcessing is the
|
||||
// visible third upload phase (§3.1a): the byte transfer and server persist are finished and the track is
|
||||
// live, but the server-side background Opus transcode is still running. It is NOT a failure and never
|
||||
// blocks completion — the form may navigate away while a row sits in PostProcessing; the Releases browse
|
||||
// view polls the durable Opus status from there.
|
||||
public enum BatchRowStatus { Queued, Uploading, PostProcessing, Done, Failed }
|
||||
|
||||
@@ -126,6 +126,8 @@
|
||||
{
|
||||
BatchRowStatus.Uploading => @<MudChip T="string" Size="Size.Small" Color="Color.Info" Variant="Variant.Text">
|
||||
<MudProgressCircular Indeterminate="true" Size="Size.Small" Class="mr-1" />Uploading</MudChip>,
|
||||
BatchRowStatus.PostProcessing => @<MudChip T="string" Size="Size.Small" Color="Color.Info" Variant="Variant.Text">
|
||||
<MudProgressCircular Indeterminate="true" Size="Size.Small" Class="mr-1" />Post-Processing</MudChip>,
|
||||
BatchRowStatus.Done => @<MudChip T="string" Size="Size.Small" Color="Color.Success" Variant="Variant.Text" Icon="@Icons.Material.Filled.CheckCircle">Done</MudChip>,
|
||||
BatchRowStatus.Failed => @<MudChip T="string" Size="Size.Small" Color="Color.Error" Variant="Variant.Text" Icon="@Icons.Material.Filled.Error">Failed</MudChip>,
|
||||
_ => @<MudChip T="string" Size="Size.Small" Color="Color.Default" Variant="Variant.Text">Queued</MudChip>
|
||||
|
||||
@@ -69,6 +69,15 @@
|
||||
Value="@_tracks[0].UploadPercent"
|
||||
aria-label="Uploading track" />
|
||||
}
|
||||
else if (_tracks[0].Status == BatchRowStatus.PostProcessing)
|
||||
{
|
||||
@* §3.1a: track is live + plays lossless; the Opus transcode runs in the background.
|
||||
Indeterminate (no client-side progress for a server-side job) and non-blocking. *@
|
||||
<MudStack Row="true" AlignItems="AlignItems.Center" Spacing="2">
|
||||
<MudProgressCircular Indeterminate="true" Size="Size.Small" />
|
||||
<MudText Typo="Typo.caption">Post-Processing (deriving Opus)…</MudText>
|
||||
</MudStack>
|
||||
}
|
||||
}
|
||||
</MudStack>
|
||||
</MudPaper>
|
||||
@@ -519,7 +528,13 @@
|
||||
}
|
||||
}
|
||||
|
||||
row.Status = BatchRowStatus.Done;
|
||||
// §3.1a: the byte transfer + server persist are done and the track is live and plays
|
||||
// losslessly — the upload is successful HERE. The server then derives Opus on a
|
||||
// background worker, so the row enters the visible Post-Processing phase rather than
|
||||
// jumping straight to Done. This never blocks: the loop continues and the form may
|
||||
// navigate away while rows sit in Post-Processing; the Releases view polls the durable
|
||||
// Opus status to settle each one.
|
||||
row.Status = BatchRowStatus.PostProcessing;
|
||||
succeeded++;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
@inject ISnackbar Snackbar
|
||||
@inject ILogger<Releases> Logger
|
||||
@inject NavigationManager NavigationManager
|
||||
@implements IDisposable
|
||||
@attribute [Authorize]
|
||||
|
||||
<PageTitle>Releases — Deep DRFT Management</PageTitle>
|
||||
@@ -51,6 +52,28 @@
|
||||
<span>Backfill High-res (@MissingHighResCount)</span>
|
||||
}
|
||||
</MudButton>
|
||||
@* Backfill-Opus (Phase 18.5 + 18.6 badge). The Opus derive runs on a server-side background
|
||||
worker: pressing the button enqueues every track lacking a complete Opus artifact and reports
|
||||
the (enqueued / skipped) outcome. The "missing N" badge (18.6) reads the same opus-status map
|
||||
the page polls, giving visual parity with the two waveform backfill buttons — but unlike them,
|
||||
the count is informational, not a per-track client loop (the work is scheduled, not driven from
|
||||
here). The button stays pressable at N=0 (a no-op re-run is harmless); it only disables while a
|
||||
press is in flight or another bulk run holds the page. *@
|
||||
<MudButton Variant="Variant.Outlined"
|
||||
Color="Color.Primary"
|
||||
StartIcon="@Icons.Material.Filled.GraphicEq"
|
||||
Disabled="@(_bulkRunning || _highResBulkRunning || _opusBackfillRunning)"
|
||||
OnClick="BackfillOpusAsync">
|
||||
@if (_opusBackfillRunning)
|
||||
{
|
||||
<MudProgressCircular Class="mr-2" Size="Size.Small" Indeterminate="true" />
|
||||
<span>Scheduling…</span>
|
||||
}
|
||||
else
|
||||
{
|
||||
<span>Backfill Opus (@MissingOpusCount)</span>
|
||||
}
|
||||
</MudButton>
|
||||
</MudStack>
|
||||
</MudStack>
|
||||
|
||||
@@ -140,6 +163,22 @@
|
||||
private int MissingProfileCount => _waveformStatus.Count(s => !s.HasProfile);
|
||||
private int MissingHighResCount => _waveformStatus.Count(s => !s.HasHighRes);
|
||||
|
||||
// EntryKey → HasOpus (a complete audio+sidecar derive). Loaded alongside the waveform status on init and
|
||||
// re-read after a Backfill-Opus run so the "missing N" badge settles. Also the source the Post-Processing
|
||||
// poll watches: a freshly uploaded track lands here with HasOpus=false and flips to true once the
|
||||
// server-side background transcode finishes — the durable surface for the upload meter's Post-Processing
|
||||
// phase after the form returns the admin to this view (§3.1a).
|
||||
private IReadOnlyList<OpusStatusDto> _opusStatus = Array.Empty<OpusStatusDto>();
|
||||
|
||||
private int MissingOpusCount => _opusStatus.Count(s => !s.HasOpus);
|
||||
|
||||
// Post-Processing poll: while any track is still missing Opus (a transcode in flight or not yet
|
||||
// backfilled), re-read the opus-status map on an interval so the "missing N" badge and any per-track
|
||||
// Post-Processing indicator settle without a manual refresh. Stops itself once nothing is missing, and is
|
||||
// torn down on dispose. Non-blocking — it never gates an upload or a button; it only refreshes a count.
|
||||
private const int OpusPollIntervalMs = 4000;
|
||||
private CancellationTokenSource? _opusPollCts;
|
||||
|
||||
// Local state for the parent-owned "Generate All Profiles" bulk run.
|
||||
private bool _bulkRunning;
|
||||
private int _bulkTotal;
|
||||
@@ -150,6 +189,11 @@
|
||||
private int _highResBulkTotal;
|
||||
private int _highResBulkDone;
|
||||
|
||||
// Local state for the "Backfill Opus" action. The Opus derive is server-side and background-queued, so
|
||||
// there is no client-side per-track loop or progress total — this flag only guards the button while the
|
||||
// single scheduling call is in flight.
|
||||
private bool _opusBackfillRunning;
|
||||
|
||||
protected override async Task OnInitializedAsync()
|
||||
{
|
||||
// Seed the active tab from ?medium= so a catalogue card deep-links straight to its medium. Panel 0
|
||||
@@ -170,9 +214,81 @@
|
||||
_waveformStatus = result.Success && result.Value is not null
|
||||
? result.Value
|
||||
: Array.Empty<WaveformStatusDto>();
|
||||
|
||||
await RefreshOpusStatusAsync();
|
||||
StateHasChanged();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Re-read the per-track Opus derive status and (re)arm the Post-Processing poll when work is still
|
||||
/// pending. Called on init, after a Backfill-Opus run, and from the poll itself. Best-effort: a failed
|
||||
/// fetch leaves the previous map in place rather than zeroing the badge on a transient API blip. Does not
|
||||
/// call StateHasChanged itself — callers batch it with their own render (the poll path renders explicitly).
|
||||
/// </summary>
|
||||
private async Task RefreshOpusStatusAsync()
|
||||
{
|
||||
var opusResult = await CmsTrackService.GetOpusStatusAsync();
|
||||
if (opusResult.Success && opusResult.Value is not null)
|
||||
{
|
||||
_opusStatus = opusResult.Value;
|
||||
}
|
||||
|
||||
if (MissingOpusCount > 0)
|
||||
{
|
||||
EnsureOpusPollRunning();
|
||||
}
|
||||
}
|
||||
|
||||
// Start the Post-Processing poll if it is not already running. The loop re-reads opus-status every
|
||||
// OpusPollIntervalMs and renders; it exits as soon as nothing is missing (or on cancel/dispose). Guarded
|
||||
// by a non-null CTS so overlapping callers (init + backfill) cannot start two loops.
|
||||
private void EnsureOpusPollRunning()
|
||||
{
|
||||
if (_opusPollCts is not null)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
_opusPollCts = new CancellationTokenSource();
|
||||
_ = PollOpusStatusAsync(_opusPollCts.Token);
|
||||
}
|
||||
|
||||
private async Task PollOpusStatusAsync(CancellationToken ct)
|
||||
{
|
||||
try
|
||||
{
|
||||
while (!ct.IsCancellationRequested && MissingOpusCount > 0)
|
||||
{
|
||||
await Task.Delay(OpusPollIntervalMs, ct);
|
||||
|
||||
var result = await CmsTrackService.GetOpusStatusAsync();
|
||||
if (result.Success && result.Value is not null)
|
||||
{
|
||||
_opusStatus = result.Value;
|
||||
await InvokeAsync(StateHasChanged);
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (OperationCanceledException)
|
||||
{
|
||||
// Expected on dispose / navigation away — nothing to do.
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
Logger.LogWarning(ex, "Opus Post-Processing poll stopped on an unexpected error.");
|
||||
}
|
||||
finally
|
||||
{
|
||||
_opusPollCts?.Dispose();
|
||||
_opusPollCts = null;
|
||||
}
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
_opusPollCts?.Cancel();
|
||||
}
|
||||
|
||||
// Invalidates the cached per-track waveform status on all embedded grids so the next row expand
|
||||
// re-fetches fresh data. Called after each catalogue-wide bulk run so already-expanded rows
|
||||
// reflect the new waveform state on the next expand interaction.
|
||||
@@ -291,4 +407,54 @@
|
||||
Snackbar.Add($"Backfilled {succeeded} high-res datum(s); {failures} failed.", Severity.Warning);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Kick off the catalogue-wide Backfill-Opus pass. The API enumerates the tracks lacking a complete Opus
|
||||
/// artifact, enqueues a background derive for each, and returns the (enqueued, skipped) counts. This is a
|
||||
/// single scheduling call — the transcodes run server-side afterward — so there is no per-track progress
|
||||
/// to render here, just a busy flag and a result snackbar. Re-runnable: a second press only schedules
|
||||
/// tracks still missing Opus.
|
||||
/// </summary>
|
||||
private async Task BackfillOpusAsync()
|
||||
{
|
||||
_opusBackfillRunning = true;
|
||||
StateHasChanged();
|
||||
try
|
||||
{
|
||||
var result = await CmsTrackService.BackfillOpusAsync();
|
||||
if (!result.Success)
|
||||
{
|
||||
var error = result.Messages.FirstOrDefault()?.Message ?? "Failed to start the Opus backfill.";
|
||||
Snackbar.Add(error, Severity.Error);
|
||||
return;
|
||||
}
|
||||
|
||||
// Re-read the status map so the "missing N" badge reflects the just-enqueued work and the
|
||||
// Post-Processing poll arms to watch the transcodes settle from N→0 as each finishes.
|
||||
await RefreshOpusStatusAsync();
|
||||
|
||||
var (enqueued, skipped) = (result.Value.Enqueued, result.Value.Skipped);
|
||||
if (enqueued == 0)
|
||||
{
|
||||
Snackbar.Add($"All {skipped} track(s) already have Opus — nothing to backfill.", Severity.Info);
|
||||
}
|
||||
else
|
||||
{
|
||||
Snackbar.Add(
|
||||
$"Scheduled {enqueued} Opus transcode(s) in the background ({skipped} already had Opus). " +
|
||||
"They will appear as each finishes.",
|
||||
Severity.Success);
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
Logger.LogError(ex, "Opus backfill failed to start");
|
||||
Snackbar.Add("Failed to start the Opus backfill.", Severity.Error);
|
||||
}
|
||||
finally
|
||||
{
|
||||
_opusBackfillRunning = false;
|
||||
StateHasChanged();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -765,6 +765,89 @@ public class CmsTrackService : ICmsTrackService
|
||||
}
|
||||
}
|
||||
|
||||
public async Task<ResultContainer<OpusBackfillResult>> BackfillOpusAsync(CancellationToken ct = default)
|
||||
{
|
||||
var client = _httpClientFactory.CreateClient(ContentCmsClientName);
|
||||
|
||||
HttpResponseMessage response;
|
||||
try
|
||||
{
|
||||
response = await client.PostAsync("api/track/opus/backfill", null, ct);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Content API call failed for Opus backfill");
|
||||
return ResultContainer<OpusBackfillResult>.CreateFailResult("Content API is unreachable.");
|
||||
}
|
||||
|
||||
using (response)
|
||||
{
|
||||
if (!response.IsSuccessStatusCode)
|
||||
{
|
||||
var body = await response.Content.ReadAsStringAsync(ct);
|
||||
_logger.LogError("Content API Opus backfill failed: {Status} {Body}", (int)response.StatusCode, body);
|
||||
return ResultContainer<OpusBackfillResult>.CreateFailResult("Failed to start the Opus backfill.");
|
||||
}
|
||||
|
||||
OpusBackfillResult payload;
|
||||
try
|
||||
{
|
||||
payload = await response.Content.ReadFromJsonAsync<OpusBackfillResult>(ct);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Failed to deserialize Opus backfill response from Content API");
|
||||
return ResultContainer<OpusBackfillResult>.CreateFailResult("Content API returned an unexpected response.");
|
||||
}
|
||||
|
||||
return ResultContainer<OpusBackfillResult>.CreatePassResult(payload);
|
||||
}
|
||||
}
|
||||
|
||||
public async Task<ResultContainer<OpusStatusDto[]>> GetOpusStatusAsync(CancellationToken ct = default)
|
||||
{
|
||||
var client = _httpClientFactory.CreateClient(ContentCmsClientName);
|
||||
|
||||
HttpResponseMessage response;
|
||||
try
|
||||
{
|
||||
response = await client.GetAsync("api/track/opus-status", ct);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Content API call failed for Opus status");
|
||||
return ResultContainer<OpusStatusDto[]>.CreateFailResult("Content API is unreachable.");
|
||||
}
|
||||
|
||||
using (response)
|
||||
{
|
||||
if (!response.IsSuccessStatusCode)
|
||||
{
|
||||
_logger.LogError("Content API Opus status failed: {Status}", (int)response.StatusCode);
|
||||
return ResultContainer<OpusStatusDto[]>.CreateFailResult("Failed to load Opus status.");
|
||||
}
|
||||
|
||||
OpusStatusDto[]? status;
|
||||
try
|
||||
{
|
||||
status = await response.Content.ReadFromJsonAsync<OpusStatusDto[]>(ct);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Failed to deserialize Opus status from Content API response");
|
||||
return ResultContainer<OpusStatusDto[]>.CreateFailResult("Content API returned an unexpected response.");
|
||||
}
|
||||
|
||||
if (status is null)
|
||||
{
|
||||
_logger.LogError("Content API returned a null Opus status list");
|
||||
return ResultContainer<OpusStatusDto[]>.CreateFailResult("Content API returned an empty response.");
|
||||
}
|
||||
|
||||
return ResultContainer<OpusStatusDto[]>.CreatePassResult(status);
|
||||
}
|
||||
}
|
||||
|
||||
public async Task<ResultContainer<List<ReleaseDto>>> GetReleasesAsync(CancellationToken ct = default)
|
||||
{
|
||||
var client = _httpClientFactory.CreateClient(ContentCmsClientName);
|
||||
|
||||
@@ -152,6 +152,23 @@ public interface ICmsTrackService
|
||||
/// </summary>
|
||||
Task<Result> GenerateHighResWaveformAsync(string entryKey, CancellationToken ct = default);
|
||||
|
||||
/// <summary>
|
||||
/// Trigger the catalogue-wide Backfill-Opus pass via <c>POST api/track/opus/backfill</c> (Phase 18.5).
|
||||
/// The API enqueues a background Opus derive for every track lacking a complete Opus artifact and returns
|
||||
/// the (enqueued, skipped) counts. Enqueue-only — the transcodes run server-side on a serial background
|
||||
/// worker, so this call returns as soon as the work is scheduled, not when transcoding finishes. The
|
||||
/// <c>Enqueued</c> count is how many derives were scheduled; <c>Skipped</c> is how many already had Opus.
|
||||
/// </summary>
|
||||
Task<ResultContainer<OpusBackfillResult>> BackfillOpusAsync(CancellationToken ct = default);
|
||||
|
||||
/// <summary>
|
||||
/// Fetch per-track Opus derive status from <c>GET api/track/opus-status</c> (Phase 18.6) for the CMS
|
||||
/// Post-Processing surfaces. Unpaged — the admin catalogue is small. Each row's <c>HasOpus</c> is true only
|
||||
/// when the track carries a complete Opus artifact (audio + sidecar). Drives the Backfill-Opus "missing N"
|
||||
/// badge and the post-upload Post-Processing poll. Idempotent read — safe to poll on an interval.
|
||||
/// </summary>
|
||||
Task<ResultContainer<OpusStatusDto[]>> GetOpusStatusAsync(CancellationToken ct = default);
|
||||
|
||||
/// <summary>Returns all releases with track counts from GET api/track/albums.</summary>
|
||||
Task<ResultContainer<List<ReleaseDto>>> GetReleasesAsync(CancellationToken ct = default);
|
||||
|
||||
@@ -160,3 +177,11 @@ public interface ICmsTrackService
|
||||
/// </summary>
|
||||
Task<ResultContainer<int>> GetTrackCountAsync(CancellationToken ct = default);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Outcome of a Backfill-Opus pass (Phase 18.5): how many tracks had a background derive scheduled
|
||||
/// (<paramref name="Enqueued"/>) and how many were skipped because they already carry a complete Opus
|
||||
/// artifact (<paramref name="Skipped"/>). Both are counts of tracks, not finished transcodes — the work
|
||||
/// runs asynchronously on the API's background worker after this returns.
|
||||
/// </summary>
|
||||
public readonly record struct OpusBackfillResult(int Enqueued, int Skipped);
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
{
|
||||
"Logging": {
|
||||
"LogLevel": {
|
||||
"Default": "Information",
|
||||
"Microsoft.AspNetCore": "Warning"
|
||||
"Default": "Warning"
|
||||
}
|
||||
},
|
||||
"AllowedHosts": "*",
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
namespace DeepDrftModels.DTOs;
|
||||
|
||||
/// <summary>
|
||||
/// Per-track Opus derive status for the CMS Post-Processing surfaces (Phase 18.6). Mirrors
|
||||
/// <see cref="WaveformStatusDto"/>: one row per track, flagging whether the track already carries a
|
||||
/// <strong>complete</strong> Opus artifact. "Complete" means BOTH the Opus audio bytes AND the seek/setup
|
||||
/// sidecar are present in the <c>track-opus</c> vault — a half-derived track (audio without sidecar) is
|
||||
/// unseekable and counts as missing, so the Backfill-Opus pass re-derives it. <see cref="EntryKey"/> is the
|
||||
/// vault key the per-track enqueue trigger and the polling Post-Processing affordance key on.
|
||||
/// </summary>
|
||||
public class OpusStatusDto
|
||||
{
|
||||
public long TrackId { get; set; }
|
||||
public string EntryKey { get; set; } = string.Empty;
|
||||
public string TrackName { get; set; } = string.Empty;
|
||||
|
||||
/// <summary>True only when both the Opus audio and the seek/setup sidecar are stored (a complete derive).</summary>
|
||||
public bool HasOpus { get; set; }
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
namespace DeepDrftModels.Enums;
|
||||
|
||||
/// <summary>
|
||||
/// The delivery format a listener requests for a track's audio (Phase 18). One <c>TrackEntity</c> /
|
||||
/// <c>EntryKey</c> addresses both renderings — "one source, multiple views" applied to delivery (C5).
|
||||
/// Lives here, not in the content library, because it is a cross-boundary contract: the API stream
|
||||
/// endpoint (18.3) parses it off the <c>?format=</c> query param, the WASM client (18.4 / 18.6) selects
|
||||
/// it, and the content-side resolver (18.2) resolves it to bytes — all three reference one enum.
|
||||
/// </summary>
|
||||
public enum AudioFormat
|
||||
{
|
||||
/// <summary>
|
||||
/// The existing source artifact in the <c>tracks</c> vault, served byte-for-byte with its real MIME
|
||||
/// (WAV/MP3/FLAC — do not assume WAV). The universal, always-present rendering.
|
||||
/// </summary>
|
||||
Lossless,
|
||||
|
||||
/// <summary>
|
||||
/// The derived low-data Ogg Opus 320 artifact in the <c>track-opus</c> vault (<c>audio/ogg</c>). A
|
||||
/// best-effort derived artifact: when absent (not yet transcoded, or transcode failed) a request for
|
||||
/// it falls back to <see cref="Lossless"/> rather than 404ing (C2).
|
||||
/// </summary>
|
||||
Opus
|
||||
}
|
||||
@@ -51,12 +51,13 @@ All interactive UI for the site. Blazor WebAssembly. Pages, controls, the stream
|
||||
- `StreamingAudioPlayerService`: Production implementation. Chunked stream from `TrackMediaClient`, adaptive 16–64 KB buffer, early-playback, **seek-beyond-buffer** via offset request to the content API.
|
||||
- `AudioInteropService`: JS interop wrapper over `window.DeepDrftAudio`. Manages `DotNetObjectReference` lifetimes for progress, end-of-playback, spectrum callbacks.
|
||||
- Dark-mode services: `DarkModeServiceBase` (cookie name constant), `DarkModeCookieService` (JS cookie read/write).
|
||||
- `WaveformVisualizerControlState`: Scoped session-persistent holder for the visualizer's **eight** continuous control positions, **two subsystem on/off toggles** (Phase 15), and one **Theater-Mode flag** (Phase 20): `ScrollSpeed`, `GradientRotationSpeed`, `LavaGravity`, `LavaHeat`, `FluidAmount` (wax count/volume), `FluidViscosity` (cohesion — the second half of the Phase 10 "bubbles" split; `BlobDensity` is gone), `CollisionStrength`, `WaveformWidth`, `LavaEnabled` (bool, default `true`), `WaveformEnabled` (bool, default `true`), `TheaterMode` (bool, default `false` — `DefaultTheaterMode`). Each has a matching `Default*` const. `Changed` event is the decoupling seam — controls mutate state + raise `Changed`; the bridge (`WaveformVisualizer`) subscribes and pushes the affected uniform or subsystem-enable; the Theater observers (the three detail pages and `AudioPlayerBar`) subscribe to react to `TheaterMode`. **`CoerceTheaterMode()`**: enforces the invariant that Theater Mode cannot remain on when both subsystems are off — called from `WaveformVisualizerControls.ToggleLava`/`ToggleWaveform` **before** `NotifyChanged()` so all observers see a consistent, coerced state in the same `Changed` cycle. `TheaterMode` is a page-chrome presentation flag; the visualizer bridge ignores it. Scoped DI so state survives SPA nav within a session and resets on fresh page load. **Phase 20 Wave 2 — playing-release predicates** live in `ReleaseDetailBase` / `CutDetailBase` (not in this state holder): `IsThisReleasePlaying` (`PlayerService?.CurrentTrack?.Release?.EntryKey == EntryKey`), `IsContentHidden` (`TheaterMode && IsThisReleasePlaying`), `ShowTheaterToggle` (`(LavaEnabled || WaveformEnabled) && IsThisReleasePlaying`). Both base classes also subscribe to `IStreamingPlayerService.StateChanged` (idempotent, reference-guarded, disposed) so the predicates re-evaluate live when playback moves between releases.
|
||||
- `WaveformVisualizerControlState`: Scoped session-persistent holder for the visualizer's **eight** continuous control positions, **two subsystem on/off toggles** (Phase 15), and one **Theater-Mode flag** (Phase 20): `ScrollSpeed`, `GradientRotationSpeed`, `LavaGravity`, `LavaHeat`, `FluidAmount` (wax count/volume), `FluidViscosity` (cohesion — the second half of the Phase 10 "bubbles" split; `BlobDensity` is gone), `CollisionStrength`, `WaveformWidth`, `LavaEnabled` (bool, default `true`), `WaveformEnabled` (bool, default `true`), `TheaterMode` (bool, default `false` — `DefaultTheaterMode`). Each has a matching `Default*` const. `Changed` event is the decoupling seam — controls mutate state + raise `Changed`; the bridge (`WaveformVisualizer`) subscribes and pushes the affected uniform or subsystem-enable; the Theater observers (the three detail pages and `AudioPlayerBar`) subscribe to react to `TheaterMode`. **`CoerceTheaterMode()`**: enforces the invariant that Theater Mode cannot remain on when both subsystems are off — called from `WaveformVisualizerControls.ToggleLava`/`ToggleWaveform` **before** `NotifyChanged()` so all observers see a consistent, coerced state in the same `Changed` cycle. **`ApplyCapabilityDefault(bool hardwareAccelerated)`**: one-time scoped capability default (guarded by `_capabilityDefaultApplied`; never re-applies on SPA navigation, never overrides an explicit in-session toggle). When `hardwareAccelerated` is false (positive software-renderer match from `hwAccel.ts`'s `UNMASKED_RENDERER_WEBGL` probe, or total WebGL failure) sets `LavaEnabled = false` while leaving `WaveformEnabled` at its default on, then calls `CoerceTheaterMode()` + `NotifyChanged()` once so all observers see the default in a single cycle. Called by the visualizer bridge on first interactive render once JS interop (the HW-accel probe via `detectHardwareAcceleration()` exported from `WaveformVisualizer.ts`) is available; a no-op when HW accel is present. `TheaterMode` is a page-chrome presentation flag; the visualizer bridge ignores it. Scoped DI so state survives SPA nav within a session and resets on fresh page load. **Phase 20 Wave 2 — playing-release predicates** live in `ReleaseDetailBase` / `CutDetailBase` (not in this state holder): `IsThisReleasePlaying` (`PlayerService?.CurrentTrack?.Release?.EntryKey == EntryKey`), `IsContentHidden` (`TheaterMode && IsThisReleasePlaying`), `ShowTheaterToggle` (`(LavaEnabled || WaveformEnabled) && IsThisReleasePlaying`). Both base classes also subscribe to `IStreamingPlayerService.StateChanged` (idempotent, reference-guarded, disposed) so the predicates re-evaluate live when playback moves between releases.
|
||||
- `PlayTracker`: Per-session play-session tracker (Phase 16 wave 16.1). Opens on playback start, advances a high-water position on each progress tick (from `StreamingAudioPlayerService` — not the HTTP layer, so seek-beyond-buffer re-fetches are the same play), closes on track-switch / stop / organic-end / page-unload. Engagement floor: ≥3 s OR ≥5% of duration. Three-bucket classification (`partial`/`sampled`/`complete`). Emits at most one event per session via `IPlayEventSink`. No player or JS dependency — testable against a fake sink.
|
||||
- `ShareTracker`: Per-session share tracker (Phase 16 wave 16.1). Called by `SharePopover` after a successful clipboard write; applies a 60-second per-(target, channel) debounce. Sends via `BeaconInterop`. Scoped so debounce memory resets on fresh page load. **Wave 16.3:** injects `IAnonIdProvider`; attaches `_anonId.Current` to `ShareEventDto.AnonId` (omitted when null).
|
||||
- `BeaconInterop`: `navigator.sendBeacon` JS interop wrapper (Phase 16 wave 16.1). Fires JSON payloads to `api/event/{play,share}` fire-and-forget. Also wires a page-unload handler that flushes any pending play event when the page is torn down.
|
||||
- `BeaconPlayEventSink`: Production `IPlayEventSink` (Phase 16 wave 16.1). Serializes the play classification and fires it via `BeaconInterop` to `api/event/play`. Synchronous (`EmitPlay` cannot await — it is called from the player close path and the page-unload handler). **Wave 16.3:** injects `IAnonIdProvider`; reads `_anonId.Current` synchronously at emit time and sets `PlayEventDto.AnonId` (omitted when null via `WhenWritingNull`).
|
||||
- `IAnonIdProvider` / `AnonIdProvider`: Wave 16.3 anonymous-listener id seam. `IAnonIdProvider` exposes `string? Current` (synchronous cached read, safe on the unload path) and `ValueTask EnsureLoadedAsync()` (warms the cache from `localStorage` via `window.DeepDrftAnonId.get` JS interop — idempotent, never throws). `AnonIdProvider` is the production implementation; degrades to null when `localStorage` is unavailable (private mode / blocked storage). The token itself outlives the session in `localStorage`; the in-process cache is scoped (resets on fresh page load). Callers warm the cache when going interactive, then read `Current` synchronously on the close/unload path with no extra JS hop. TypeScript interop: `DeepDrftPublic/Interop/telemetry/anonid.ts` (mints GUID on first visit, returns null without throwing when storage is unavailable).
|
||||
- `ShareTracker`: Per-session share tracker (Phase 16 wave 16.1). Called by `SharePopover` after a successful clipboard write; applies a 60-second per-(target, channel) debounce. A share is always a live-page user interaction (never a tab-unload), so it sends via the first-party `IEventPoster` fetch **only** — no `sendBeacon` arm. Scoped so debounce memory resets on fresh page load. **Wave 16.3:** injects `IAnonIdProvider`; attaches `_anonId.Current` to `ShareEventDto.AnonId` (omitted when null).
|
||||
- `IEventPoster` / `HttpEventPoster`: First-party same-origin event transport (telemetry transport-resilience). `PostAsync(url, json)` POSTs an `application/json` body to the host's own `api/event/*` proxy via a default `IHttpClientFactory` client, best-effort and non-throwing. A first-party fetch is not name-matched by tracking/fingerprinting heuristics the way a `telemetry/beacon` `sendBeacon` module is — this is the transport for normal play closes (end/switch/stop) and every share. One seam so the play sink and share tracker share it and tests capture the wire payload behind a fake.
|
||||
- `BeaconInterop`: `navigator.sendBeacon` JS interop wrapper over `window.DeepDrftLifecycle` (served from `js/session/lifecycle.js`). After the transport-resilience split it is the **unload-edge transport only**: it fires the play payload via `sendBeacon` on page tear-down (where an awaited fetch would be cancelled) and wires the page-unload handler. Normal closes go over `IEventPoster`. Named off the former `telemetry/beacon` path so the retained fallback isn't name-matched either.
|
||||
- `BeaconPlayEventSink`: Production `IPlayEventSink` (Phase 16 wave 16.1; transport-resilience split). Serializes the play classification once and dispatches down the arm the close chose: `EmitPlayAsync` over the first-party `IEventPoster` (normal close: organic end / track-switch / stop, page alive) and `EmitPlayOnUnload` over `BeaconInterop` `sendBeacon` (tab-unload edge). Both arms send byte-identical payloads. `PlayTracker.Close(bool viaUnload)` selects the arm — `OnPageUnload` passes `viaUnload: true`, every other close defaults to fetch. **Wave 16.3:** injects `IAnonIdProvider`; reads `_anonId.Current` synchronously at emit time and sets `PlayEventDto.AnonId` (omitted when null via `WhenWritingNull`).
|
||||
- `IAnonIdProvider` / `AnonIdProvider`: Wave 16.3 anonymous-listener id seam. `IAnonIdProvider` exposes `string? Current` (synchronous cached read, safe on the unload path) and `ValueTask EnsureLoadedAsync()` (warms the cache from `localStorage` via `window.DeepDrftAnonId.get` JS interop — idempotent, never throws). `AnonIdProvider` is the production implementation; degrades to null when `localStorage` is unavailable (private mode / blocked storage). The token itself outlives the session in `localStorage`; the in-process cache is scoped (resets on fresh page load). Callers warm the cache when going interactive, then read `Current` synchronously on the close/unload path with no extra JS hop. TypeScript interop: `DeepDrftPublic/Interop/session/anonid.ts` (exposes `window.DeepDrftAnonId`, served from `js/session/anonid.js`; mints GUID on first visit, returns null without throwing when storage is unavailable).
|
||||
- `IQueueService` / `QueueService`: **Two-level deque** orchestrator above the single-slot player. The deque has two entry ends. **PLAY (manual)** enters the FRONT: `PlayTrack(track)` and `PlayRelease(tracks, startIndex)` prepend the played track/release in order, **remove the previously-current track**, make the new front current, start streaming it, and leave whatever sat after the old current intact behind the prepend (a whole release prepends in order in one op). The detail pages (Cut header/row, Session/Mix hero) and `StreamNowButton` route their PLAY through these. **Add-to-queue** enters the BACK: `Enqueue`/`EnqueueRange` append to the end without interrupting the current track (`AddToQueueButton`). `Next`/`Previous` advance or step back, walking `CurrentIndex` and leaving played tracks behind so `Previous` can reach them; `JumpTo(index)` moves the pointer to a queued row and streams it once (the playlist panel's row-jump — it does NOT prepend or stream the intervening rows). **End-of-track:** auto-advance (`TrackEnded`) advances when there is a next track; when the **last** track ends naturally the queue **empties** and goes dormant (bug #2) rather than stranding the finished track. `Clear` empties the queue. **Bug #3 (dormant-seed):** the first `Enqueue`/`EnqueueRange` into a dormant queue while a track is already playing externally (via the attached player, not through the queue) seeds the head with that now-playing track and then appends — yielding `[now-playing, added]` (even when adding the same track). The queue learns the externally-playing track through the existing `Attach(player)` seam (`_player.CurrentTrack`) — no new dependency, no `IServiceProvider`. **Armed-idle state** (prerender-safe release embeds): `Arm(tracks)` replaces the queue at index 0 with no JS interop; `IsArmed` signals armed-but-not-streaming; `Start()` streams the current track and clears `IsArmed`. `AudioPlayerBar` reads `IsArmed` to route the embed's first play gesture through `Start()`. `QueueChanged` fires on all list/position changes; cascaded via `AudioPlayerProvider`. `Move`/`RemoveAt` are interop-free reorder/remove mutations that adjust `CurrentIndex` and never re-stream. `ClearUpcoming()` keeps the current track and drops the up-next. **Bug #4 (reactivity):** `AudioPlayerBar.QueueItems` caches `QueueService.Items` as a `_queueItemsCache` snapshot (the service exposes its backing list by reference); the cache is invalidated and set to `null` in `OnQueueChanged`, so every real mutation hands `QueueList` a new list reference while frequent progress-tick re-renders reuse the cached one without allocating. `QueueList.OnParametersSet` calls `_dropContainer?.Refresh()` so the `MudDropContainer` re-reads the new list and the open panel re-flows immediately. **Bug #1 (label):** the docked `QueueOverlay` panel header reads **"Playlist"** (the current track stays listed). `PlayRelease` materializes `tracks.ToList()` before mutating so it can never alias the service's own `Items` list.
|
||||
- `Clients/`: HTTP API clients (both target DeepDrftAPI).
|
||||
- `TrackClient`: SQL metadata API. Uses named `IHttpClientFactory` client `"DeepDrft.API"`. Sends `page` param (not `pageNumber`). Deserializes response as bare `PagedResult<TrackDto>` (not wrapped in ApiResultDto envelope).
|
||||
|
||||
@@ -2,6 +2,8 @@ using System.Net;
|
||||
using System.Net.Http.Headers;
|
||||
using System.Net.Http.Json;
|
||||
using DeepDrftModels.DTOs;
|
||||
using DeepDrftModels.Enums;
|
||||
using Microsoft.AspNetCore.Components.WebAssembly.Http;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using NetBlocks.Models;
|
||||
|
||||
@@ -18,13 +20,24 @@ public class TrackMediaResponse : IDisposable
|
||||
/// </summary>
|
||||
public string ContentType { get; }
|
||||
|
||||
/// <summary>
|
||||
/// The total file length in bytes, parsed from the 206 response's <c>Content-Range:
|
||||
/// bytes start-end/TOTAL</c> header (Phase 21 Direction B). Null when the server returned
|
||||
/// 200 (no Content-Range) — callers fall back to <see cref="ContentLength"/> as the total.
|
||||
/// This is the EOF boundary the segment loop advances its cursor toward, and the full
|
||||
/// logical length the JS decoder must see (so a bounded segment's small Content-Length
|
||||
/// never trips the decoder's byte-count completion early).
|
||||
/// </summary>
|
||||
public long? TotalLength { get; }
|
||||
|
||||
private readonly HttpResponseMessage _response;
|
||||
|
||||
public TrackMediaResponse(Stream stream, long contentLength, string contentType, HttpResponseMessage response)
|
||||
public TrackMediaResponse(Stream stream, long contentLength, string contentType, long? totalLength, HttpResponseMessage response)
|
||||
{
|
||||
Stream = stream;
|
||||
ContentLength = contentLength;
|
||||
ContentType = contentType;
|
||||
TotalLength = totalLength;
|
||||
_response = response;
|
||||
}
|
||||
|
||||
@@ -45,24 +58,61 @@ public class TrackMediaClient
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Fetches the WAV stream for a track via an HTTP Range request starting at a
|
||||
/// Fetches the audio stream for a track via an HTTP Range request starting at a
|
||||
/// file-absolute byte offset. <paramref name="byteOffset"/> is the position from
|
||||
/// the start of the file on disk (including the WAV header) — callers seeking into
|
||||
/// audio data must add the header size themselves. The cancellation token aborts
|
||||
/// the in-flight server connection rather than leaving the server draining bytes
|
||||
/// into a dead socket.
|
||||
/// the start of the file on disk (including any container/header bytes) — callers
|
||||
/// seeking into audio data must add the header size themselves. The cancellation
|
||||
/// token aborts the in-flight server connection rather than leaving the server
|
||||
/// draining bytes into a dead socket.
|
||||
/// <para>
|
||||
/// <paramref name="byteEnd"/> (Phase 21 Direction B) bounds the request to a single
|
||||
/// segment: when set, the Range header is <c>bytes={byteOffset}-{byteEnd}</c> (inclusive),
|
||||
/// so the browser holds at most ~one segment of raw bytes regardless of file size — the
|
||||
/// network-memory bound this phase exists for. When null the request is open-ended
|
||||
/// (<c>bytes={byteOffset}-</c>), the pre-Direction-B behaviour. Either way the response's
|
||||
/// <c>Content-Range</c> total is surfaced via <see cref="TrackMediaResponse.TotalLength"/>
|
||||
/// so the caller knows the EOF boundary and the full logical length the decoder must see.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <paramref name="format"/> selects the delivery rendering (Phase 18): the default
|
||||
/// <see cref="AudioFormat.Lossless"/> sends no <c>format</c> query param, so existing
|
||||
/// callers hit the byte-identical pre-Phase-18 endpoint; <see cref="AudioFormat.Opus"/>
|
||||
/// requests the low-data Ogg Opus artifact, which the server resolves and falls back to
|
||||
/// lossless when absent (C2). The response <see cref="TrackMediaResponse.ContentType"/>
|
||||
/// reports the format actually served, so the JS decoder dispatches on the real bytes.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public async Task<ApiResult<TrackMediaResponse>> GetTrackMedia(
|
||||
string trackId,
|
||||
long byteOffset = 0,
|
||||
long? byteEnd = null,
|
||||
AudioFormat format = AudioFormat.Lossless,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
try
|
||||
{
|
||||
// Same URL for every seek — only the Range header differs. byteOffset 0 is
|
||||
// Same URL for every fetch — only the Range header differs. byteOffset 0 is
|
||||
// not special-cased: "bytes=0-" requests the whole file from the start.
|
||||
using var request = new HttpRequestMessage(HttpMethod.Get, $"api/track/{trackId}");
|
||||
request.Headers.Range = new RangeHeaderValue(byteOffset, null);
|
||||
// Lossless omits the format param entirely so the request is byte-identical to
|
||||
// the pre-Phase-18 endpoint; only Opus appends ?format=opus.
|
||||
var uri = format == AudioFormat.Lossless
|
||||
? $"api/track/{trackId}"
|
||||
: $"api/track/{trackId}?format={format.ToString().ToLowerInvariant()}";
|
||||
using var request = new HttpRequestMessage(HttpMethod.Get, uri);
|
||||
// Bounded (byteEnd set) → "bytes=start-end" so the server returns a finite 206
|
||||
// slice and the browser buffers only that segment; open-ended (byteEnd null) →
|
||||
// "bytes=start-". The server honours both via File(..., enableRangeProcessing: true),
|
||||
// which parses the full RFC 7233 range grammar and slices accordingly.
|
||||
request.Headers.Range = new RangeHeaderValue(byteOffset, byteEnd);
|
||||
|
||||
// Stream the response body incrementally instead of buffering it whole (Phase 21.4 fix).
|
||||
// In Blazor WebAssembly the HttpClient is backed by the browser fetch API; without this the
|
||||
// browser buffers the ENTIRE body before the response stream yields a byte. With Direction B
|
||||
// each request is already bounded to one segment, so the body is small regardless — but
|
||||
// streaming still lets us read it incrementally and is harmless on the SSR server-to-server
|
||||
// path (SocketsHttpHandler ignores the unknown option). Kept for both the initial and the
|
||||
// seek/refill paths since both share this method.
|
||||
request.SetBrowserResponseStreamingEnabled(true);
|
||||
|
||||
// Use HttpCompletionOption.ResponseHeadersRead to get stream immediately
|
||||
var response = await _http.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, cancellationToken);
|
||||
@@ -72,11 +122,15 @@ public class TrackMediaClient
|
||||
// Default to WAV when the server omits the header — the only format shipping
|
||||
// today — so the JS factory always receives a usable media type.
|
||||
var contentType = response.Content.Headers.ContentType?.MediaType ?? "audio/wav";
|
||||
// Content-Range "bytes start-end/TOTAL" carries the full file length on a 206; on a 200
|
||||
// there is no Content-Range, so TotalLength is null and callers use ContentLength.
|
||||
var totalLength = response.Content.Headers.ContentRange?.Length;
|
||||
var stream = await response.Content.ReadAsStreamAsync(cancellationToken);
|
||||
|
||||
// TrackMediaResponse takes ownership of both stream and response;
|
||||
// do NOT dispose response here — the caller disposes via TrackMediaResponse.Dispose().
|
||||
return ApiResult<TrackMediaResponse>.CreatePassResult(new TrackMediaResponse(stream, contentLength, contentType, response));
|
||||
return ApiResult<TrackMediaResponse>.CreatePassResult(
|
||||
new TrackMediaResponse(stream, contentLength, contentType, totalLength, response));
|
||||
}
|
||||
catch (Exception e)
|
||||
{
|
||||
@@ -115,4 +169,33 @@ public class TrackMediaClient
|
||||
return ApiResult<WaveformProfileDto>.CreateFailResult(e.Message);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Fetches a track's Opus seek/setup sidecar — the combined OpusHead/OpusTags setup header plus the
|
||||
/// granule→byte seek index (Phase 18). The caller (18.5 player wiring) fetches this once on track load
|
||||
/// and parses it into the JS-side OpusSeekData before issuing any Opus seek. A 404 means no Opus
|
||||
/// artifact / sidecar exists for the track (legacy row, not backfilled, or transcode failed); callers
|
||||
/// treat that as "this track has no Opus seek data — stay on lossless" rather than an error, so it
|
||||
/// surfaces as a fail result with a stable message rather than throwing (mirrors GetWaveformProfileAsync).
|
||||
/// </summary>
|
||||
public async Task<ApiResult<byte[]>> GetOpusSidecarAsync(string trackId, CancellationToken cancellationToken = default)
|
||||
{
|
||||
try
|
||||
{
|
||||
var response = await _http.GetAsync($"api/track/{trackId}/opus/seekdata", cancellationToken);
|
||||
if (response.StatusCode == HttpStatusCode.NotFound)
|
||||
{
|
||||
return ApiResult<byte[]>.CreateFailResult("No Opus sidecar available");
|
||||
}
|
||||
|
||||
response.EnsureSuccessStatusCode();
|
||||
|
||||
var bytes = await response.Content.ReadAsByteArrayAsync(cancellationToken);
|
||||
return ApiResult<byte[]>.CreatePassResult(bytes);
|
||||
}
|
||||
catch (Exception e)
|
||||
{
|
||||
return ApiResult<byte[]>.CreateFailResult(e.Message);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
using Microsoft.AspNetCore.Components;
|
||||
|
||||
namespace DeepDrftPublic.Client.Common;
|
||||
|
||||
/// <summary>
|
||||
/// The single public-site listener-settings object (Phase 18 wave 18.6, §4a). The generalized analogue of
|
||||
/// <see cref="DarkModeSettings"/>: one scoped holder for every remembered listener preference, seeded at
|
||||
/// server prerender, carried into WASM via <see cref="PersistentState"/>, and persisted to a cookie on
|
||||
/// change. Today it carries one preference — streaming quality; tomorrow dark mode (and whatever follows)
|
||||
/// folds in here as another property without disturbing the menu that reads it.
|
||||
/// <para>
|
||||
/// Built design-for-adaptability per §4a: a new preference is a new <c>[PersistentState]</c> property here
|
||||
/// plus a new <see cref="Components.SettingsItem"/> in the menu — not a rewire. Dark mode is intentionally
|
||||
/// <em>not</em> migrated in now (it keeps its own <see cref="DarkModeSettings"/> seam); this object is shaped
|
||||
/// so that consolidation is later a merge of two identical seams, not a reconciliation of two different ones.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public class PublicSiteSettings
|
||||
{
|
||||
/// <summary>
|
||||
/// The listener's streaming-quality preference. Defaults to <see cref="StreamQuality.LowData"/> (Opus,
|
||||
/// capability-gated — OQ2). Seeded from the <c>streamQuality</c> cookie at prerender; persisted on change
|
||||
/// by the client cookie service. The player reads this to decide which <c>?format=</c> to request, but
|
||||
/// the capability gate and C2 fallback still apply on top, so a <see cref="StreamQuality.LowData"/>
|
||||
/// preference never forces an unplayable stream.
|
||||
/// </summary>
|
||||
[PersistentState]
|
||||
public StreamQuality StreamQuality { get; set; } = StreamQuality.LowData;
|
||||
}
|
||||
@@ -0,0 +1,12 @@
|
||||
using Microsoft.AspNetCore.Components;
|
||||
|
||||
namespace DeepDrftPublic.Client.Common;
|
||||
|
||||
/// <summary>
|
||||
/// One entry in the public-site Settings menu (Phase 18 wave 18.6, §4a). The settings-item abstraction the
|
||||
/// menu renders instead of a hard-coded control list: a <see cref="Label"/> plus a <see cref="Control"/>
|
||||
/// fragment bound to a persisted preference. Adding a future tenant (e.g. dark mode) is appending one of
|
||||
/// these — not rewiring the menu. The control fragment owns its own binding to <see cref="PublicSiteSettings"/>
|
||||
/// and its own persistence call, so each item is self-contained and the menu stays preference-agnostic.
|
||||
/// </summary>
|
||||
public sealed record SettingsItem(string Label, RenderFragment Control);
|
||||
@@ -0,0 +1,18 @@
|
||||
namespace DeepDrftPublic.Client.Common;
|
||||
|
||||
/// <summary>
|
||||
/// The listener's streaming-quality preference (Phase 18 wave 18.6, §4). This is the user's <em>intent</em>,
|
||||
/// not the wire format that ultimately gets served: <see cref="LowData"/> means "give me Opus if you can,"
|
||||
/// but the player still capability-gates and C2-falls-back to lossless when Opus can't play (a browser that
|
||||
/// can't decode Ogg Opus, or a track with no Opus artifact). It is therefore deliberately distinct from
|
||||
/// <c>DeepDrftModels.Enums.AudioFormat</c> (the delivery rendering resolved per request): one is the
|
||||
/// remembered preference, the other is what a given stream request actually asks for.
|
||||
/// </summary>
|
||||
public enum StreamQuality
|
||||
{
|
||||
/// <summary>Bandwidth-friendly Opus (capability-gated; the default before any choice — OQ2).</summary>
|
||||
LowData,
|
||||
|
||||
/// <summary>The lossless WAV path, always playable everywhere.</summary>
|
||||
Lossless
|
||||
}
|
||||
@@ -36,10 +36,9 @@ public partial class AudioPlayerBar : ComponentBase, IAsyncDisposable
|
||||
// error banner.
|
||||
//
|
||||
// _miniDock is the minimized FAB container. We observe it in minimized state so
|
||||
// --player-height stays non-zero (the FAB's actual height) and the WaveformVisualizer
|
||||
// clips to the top of the FAB rather than extending to the viewport bottom (fix §1).
|
||||
// The player-spacer's .minimized class uses a hardcoded 60px and ignores the var,
|
||||
// so publishing the FAB height here does not regress the spacer.
|
||||
// --player-height stays non-zero (the FAB's actual height). The player-spacer's
|
||||
// .minimized class uses a hardcoded 60px and ignores the var, so this is belt-and-
|
||||
// braces; the var's sole live consumer is the spacer's .expanded height.
|
||||
private ElementReference _playerRoot;
|
||||
private ElementReference _miniDock;
|
||||
private ElementReference _lastObservedElement;
|
||||
@@ -247,13 +246,15 @@ public partial class AudioPlayerBar : ComponentBase, IAsyncDisposable
|
||||
}
|
||||
|
||||
// For the docked player: we observe in BOTH expanded and minimized states
|
||||
// so --player-height always reflects the live height of whichever element
|
||||
// is visible. This keeps the WaveformVisualizer clipped to the top of
|
||||
// the footer in both states (fix §1).
|
||||
// so --player-height stays non-zero and always reflects the live height of
|
||||
// whichever element is visible. The var's sole live consumer is the
|
||||
// player-spacer's .expanded height (keeps the spacer sized correctly across
|
||||
// breakpoints and banner reflows).
|
||||
// expanded → observe _playerRoot (full player bar, reflows across breakpoints)
|
||||
// minimized → observe _miniDock (floating FAB container, ~56–60px)
|
||||
// The player-spacer's .minimized class uses a hardcoded height and ignores
|
||||
// the var, so publishing the FAB height here does not regress the spacer.
|
||||
// The player-spacer's .minimized class uses a hardcoded 60px and ignores
|
||||
// the var, so observing in minimized state is belt-and-braces; it does not
|
||||
// regress the spacer.
|
||||
|
||||
var elementToObserve = _isMinimized ? _miniDock : _playerRoot;
|
||||
var alreadyOnThisElement = _spacerObserved && elementToObserve.Id == _lastObservedElement.Id;
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
using DeepDrftPublic.Client.Common;
|
||||
using DeepDrftPublic.Client.Services;
|
||||
using DeepDrftPublic.Client.Clients;
|
||||
using Microsoft.AspNetCore.Components;
|
||||
@@ -13,6 +14,7 @@ public partial class AudioPlayerProvider : ComponentBase, IAsyncDisposable
|
||||
[Inject] public required BeaconInterop Beacon { get; set; }
|
||||
[Inject] public required IPlayEventSink PlayEventSink { get; set; }
|
||||
[Inject] public required IAnonIdProvider AnonId { get; set; }
|
||||
[Inject] public required PublicSiteSettings Settings { get; set; }
|
||||
|
||||
private IStreamingPlayerService? _audioPlayerService;
|
||||
private QueueService? _queueService;
|
||||
@@ -26,7 +28,12 @@ public partial class AudioPlayerProvider : ComponentBase, IAsyncDisposable
|
||||
// EnsureInitializedAsync — that path is correct because audio contexts
|
||||
// require a user gesture anyway. Initializing eagerly here causes 4+
|
||||
// SignalR round-trips before any content is stable.
|
||||
var player = new StreamingAudioPlayerService(AudioInterop, TrackMediaClient, Logger);
|
||||
// Construct the preference-aware player (Phase 18 wave 18.6): it honours the listener's streaming-
|
||||
// quality choice via the ResolveStreamFormatAsync seam while inheriting the 18.5 capability gate and
|
||||
// C2 fallback. PublicSiteSettings is scoped data (already prerender-seeded + WASM-bridged), so passing
|
||||
// it through the constructor is cheap and carries no lifecycle — the telemetry tracker still binds
|
||||
// post-construction below, exactly as before.
|
||||
var player = new PreferenceAwareStreamingPlayerService(AudioInterop, TrackMediaClient, Logger, Settings);
|
||||
|
||||
// Phase 16: bind the play-session tracker to the player after construction, the same way the
|
||||
// queue binds — the player is built with `new`, not DI, so threading telemetry through its
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
@using DeepDrftPublic.Client.Common
|
||||
@using DeepDrftPublic.Client.Controls.Settings
|
||||
@using DeepDrftPublic.Client.Services
|
||||
|
||||
@*
|
||||
The public-site Settings menu (Phase 18 wave 18.6, §4a). An app-bar trigger opening a MudMenu that renders
|
||||
a settings-item list — NOT a hard-coded control stack. Each entry is a SettingsItem (label + a control
|
||||
fragment bound to a persisted preference), so a future tenant (dark mode) plugs in as a new list entry, not
|
||||
a menu rewire. Today the list holds one item: the streaming-quality toggle.
|
||||
|
||||
The MudMenu items carry OnClick="@(() => {})" + OnTouch so a click inside a control row does not dismiss the
|
||||
menu (MudMenu auto-closes on item activation otherwise), keeping the radio group usable.
|
||||
*@
|
||||
<div class="dd-accent-icon">
|
||||
<MudMenu Icon="@Icons.Material.Filled.Settings"
|
||||
Color="Color.Inherit"
|
||||
AnchorOrigin="Origin.BottomRight"
|
||||
TransformOrigin="Origin.TopRight"
|
||||
AriaLabel="Settings"
|
||||
Class="dd-settings-menu">
|
||||
<div class="dd-settings-panel">
|
||||
<div class="dd-settings-heading">Settings</div>
|
||||
@foreach (var item in _items)
|
||||
{
|
||||
<div class="dd-settings-item">
|
||||
<div class="dd-settings-item-label">@item.Label</div>
|
||||
@item.Control
|
||||
</div>
|
||||
}
|
||||
</div>
|
||||
</MudMenu>
|
||||
</div>
|
||||
|
||||
@code {
|
||||
// The active player, cascaded by AudioPlayerProvider. SettingsMenu sits in the app bar inside the
|
||||
// provider, so it receives the cascade here — but the MudMenu PANEL content below is portaled to
|
||||
// <MudPopoverProvider> (outside the provider), so a cascade cannot reach it. We thread the player into
|
||||
// the setting as an explicit parameter instead: an explicit value captured in the item fragment flows
|
||||
// into portaled content fine, where a [CascadingParameter] would land null.
|
||||
[CascadingParameter] public IStreamingPlayerService? Player { get; set; }
|
||||
|
||||
// The settings-item list. Built once; adding a preference is appending one SettingsItem with its control
|
||||
// fragment — the menu body above renders whatever is here without knowing what each item is. The item
|
||||
// fragment reads Player at render time (when the menu opens), so it picks up the cascaded instance even
|
||||
// though the list itself is initialized before the cascade is set.
|
||||
private readonly List<SettingsItem> _items;
|
||||
|
||||
public SettingsMenu()
|
||||
{
|
||||
_items =
|
||||
[
|
||||
new SettingsItem("Streaming quality", @<StreamQualitySetting Player="@Player" />)
|
||||
];
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,97 @@
|
||||
@using DeepDrftPublic.Client.Common
|
||||
@using DeepDrftPublic.Client.Services
|
||||
|
||||
@*
|
||||
The streaming-quality control (Phase 18 wave 18.6, §4) — the first occupant of the Settings menu. Binds
|
||||
the listener's choice to PublicSiteSettings.StreamQuality and persists it via the cookie seam. Honest
|
||||
capability gate (OQ2 / AC7): on a browser that cannot decode Ogg Opus the Low-data option still selects,
|
||||
but a note tells the listener the effective stream is lossless — we never let the choice silently imply a
|
||||
format that can't play.
|
||||
*@
|
||||
<div class="dd-setting-control">
|
||||
<MudRadioGroup T="StreamQuality" Value="_quality" ValueChanged="OnQualityChanged">
|
||||
<MudRadio T="StreamQuality" Value="StreamQuality.LowData" Color="Color.Primary" Dense="true">
|
||||
Low-data (Opus)
|
||||
</MudRadio>
|
||||
<MudRadio T="StreamQuality" Value="StreamQuality.Lossless" Color="Color.Primary" Dense="true">
|
||||
Lossless (WAV)
|
||||
</MudRadio>
|
||||
</MudRadioGroup>
|
||||
|
||||
@if (_opusUnavailable && _quality == StreamQuality.LowData)
|
||||
{
|
||||
<div class="dd-setting-note">
|
||||
This browser can't decode Opus — you'll stream lossless.
|
||||
</div>
|
||||
}
|
||||
<div class="d-flex flex-align-right">
|
||||
<MudButton Disabled="!IsApplyEnabled"
|
||||
Color="Color.Primary"
|
||||
Variant="Variant.Filled"
|
||||
OnClick="@ApplyStreamQualitySetting">
|
||||
APPLY
|
||||
</MudButton>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@code {
|
||||
[Inject] public required PublicSiteSettings Settings { get; set; }
|
||||
[Inject] public required SettingsCookieService CookieService { get; set; }
|
||||
[Inject] public required AudioInteropService AudioInterop { get; set; }
|
||||
|
||||
// The active player, threaded in from SettingsMenu (which reads it off the AudioPlayerProvider cascade).
|
||||
// It is an explicit [Parameter], NOT a [CascadingParameter], because this control renders inside the
|
||||
// MudMenu panel, which MudBlazor portals to <MudPopoverProvider> — outside AudioPlayerProvider's cascade
|
||||
// scope, so a cascade would land null here. Null during prerender or when no provider is present — Apply
|
||||
// then just persists the preference, with no live track to restart.
|
||||
[Parameter] public IStreamingPlayerService? Player { get; set; }
|
||||
|
||||
private StreamQuality _quality;
|
||||
|
||||
public bool IsApplyEnabled => _quality != Settings.StreamQuality;
|
||||
|
||||
// Null until the capability probe runs (post-render JS interop). false → can decode Opus; true → cannot.
|
||||
private bool _opusUnavailable;
|
||||
|
||||
protected override void OnInitialized()
|
||||
{
|
||||
// Read the current preference (already seeded at prerender + bridged into WASM).
|
||||
_quality = Settings.StreamQuality;
|
||||
}
|
||||
|
||||
protected override async Task OnAfterRenderAsync(bool firstRender)
|
||||
{
|
||||
if (!firstRender) return;
|
||||
|
||||
// Capability probe is JS interop — only valid once interactive. Surfaces the honest note when the
|
||||
// browser can't decode Ogg Opus, so a Low-data pick reads as "effectively lossless" rather than
|
||||
// silently failing. The player applies the same gate independently; this is purely the UI honesty.
|
||||
var canDecodeOpus = await AudioInterop.CanDecodeOggOpus();
|
||||
if (canDecodeOpus == _opusUnavailable)
|
||||
{
|
||||
_opusUnavailable = !canDecodeOpus;
|
||||
StateHasChanged();
|
||||
}
|
||||
}
|
||||
|
||||
private async Task OnQualityChanged(StreamQuality quality)
|
||||
{
|
||||
_quality = quality;
|
||||
}
|
||||
|
||||
private async Task ApplyStreamQualitySetting(MouseEventArgs arg)
|
||||
{
|
||||
// Persist the choice first so the cookie + in-memory PublicSiteSettings.StreamQuality both reflect
|
||||
// the new value BEFORE the restart: the reload re-resolves the delivery format via
|
||||
// PreferenceAwareStreamingPlayerService, which reads PublicSiteSettings.StreamQuality fresh.
|
||||
await CookieService.SetStreamQualityAsync(_quality);
|
||||
|
||||
// Switch the currently-playing track to the new format immediately, preserving the listener's
|
||||
// position. No-op inside the player when nothing is loaded — the new preference then simply
|
||||
// applies to the next track played. Fire-and-forget: the reload runs the streaming loop for the
|
||||
// life of the track, so awaiting it would pin this handler open; UI updates flow through the
|
||||
// player's state notifications, not this await.
|
||||
_ = Player?.ReloadPreservingPositionAsync();
|
||||
}
|
||||
|
||||
}
|
||||
@@ -265,6 +265,24 @@ public partial class WaveformVisualizer : ComponentBase, IAsyncDisposable
|
||||
return;
|
||||
}
|
||||
|
||||
// Apply the hardware-capability default ONCE per session before seeding the controls: if the
|
||||
// browser has no WebGL hardware acceleration, the lava subsystem (which software-renders on
|
||||
// the main thread and starves audio decode) defaults off while the waveform stays on. The
|
||||
// probe lives in JS (it needs a real WebGL context); the scoped state guards the one-time
|
||||
// application, so a remounted visualizer never re-applies and a later explicit toggle is
|
||||
// never clobbered. Sequenced before PushControlsAsync so the seed already carries the
|
||||
// corrected enables; ApplyCapabilityDefault also raises Changed for the controls UI.
|
||||
try
|
||||
{
|
||||
var hardwareAccelerated = await _module.InvokeAsync<bool>("detectHardwareAcceleration");
|
||||
ControlState.ApplyCapabilityDefault(hardwareAccelerated);
|
||||
}
|
||||
catch (JSException ex)
|
||||
{
|
||||
// A probe failure must not regress the HW-accel majority: leave the defaults (lava on).
|
||||
Logger.LogWarning(ex, "WaveformVisualizer: hardware-acceleration probe failed; leaving lava at its default.");
|
||||
}
|
||||
|
||||
// Seed the module with the current state now that it exists. All control values (the eight
|
||||
// dials + the two Phase 15 subsystem enables) come from the shared (session-persisted) state,
|
||||
// so a mix opened mid-session seeds the module with the knob/toggle positions the listener
|
||||
|
||||
@@ -1,19 +1,20 @@
|
||||
/* Full-viewport fixed backdrop. Sits behind the detail content (.mix-detail-foreground is z-index:1)
|
||||
and never intercepts pointer events — except the zoom slider, which re-enables them on itself.
|
||||
|
||||
Footer clip (Phase 10 W1, spec §2c): the backdrop must stop cleanly ABOVE the audio player bar so
|
||||
no lava/waveform pixel paints over or under it. `overflow: hidden` clips the canvas to this box, and
|
||||
`bottom` is inset by `--player-height`, which AudioPlayerBar publishes on :root via its ResizeObserver
|
||||
(Interop/layout/spacer.ts). The observer now points at whichever element is live:
|
||||
expanded → the full player dock (tracks breakpoint reflow + error-banner growth)
|
||||
minimized → the minimized-dock FAB container (~56–60 px)
|
||||
so --player-height is always non-zero while the player is mounted and the clip line follows the bar in
|
||||
BOTH states (fix §1 / p10-reframe-w1-fix). The 0px fallback keeps the backdrop full-height on any
|
||||
page that does not host the player. */
|
||||
Anchored to the viewport bottom (`inset: 0` — fills the whole screen). The chrome that must occlude
|
||||
it paints OVER it on z-index, not by clipping the box: the app bar (z 100), the docked player bar
|
||||
(z 1200/1300), the site footer (z 1 stacking context), and the layout spacer (z 1 stacking context,
|
||||
`--deepdrft-page-surface` background) all sit above this z-0 layer. Where those elements are inset
|
||||
from the screen edges or scrolled below the fold, the visualizer fills the gap continuously — no
|
||||
page-background strip around the inset player bar. `overflow: hidden` clips the canvas to this box.
|
||||
|
||||
NOTE: this box is decoupled from `--player-height` (it no longer reads that var). The renderer's own
|
||||
ResizeObserver therefore never fires on a player-bar height change, so an eased Theater-Mode collapse
|
||||
can no longer clear the GL backing store mid-ease (the former theater-flash source). See
|
||||
Interop/layout/spacer.ts. */
|
||||
.mix-waveform-bg {
|
||||
position: fixed;
|
||||
inset: 0;
|
||||
bottom: var(--player-height, 0px);
|
||||
z-index: 0;
|
||||
pointer-events: none;
|
||||
overflow: hidden;
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
@using DeepDrftPublic.Client.Common
|
||||
@using DeepDrftPublic.Client.Controls
|
||||
@using DeepDrftPublic.Client.Controls.Settings
|
||||
@using DeepDrftPublic.Client.Services
|
||||
|
||||
@* Desktop Menu *@
|
||||
@@ -42,6 +43,7 @@
|
||||
|
||||
<div class="dd-nav-actions">
|
||||
<StreamNowButton ButtonClass="dd-nav-cta" ButtonLabel="Stream Now ▶"/>
|
||||
<SettingsMenu />
|
||||
<MudIconButton Icon="@(DarkLightModeButtonIcon)" Color="Color.Inherit" OnClick="@DarkModeToggle"/>
|
||||
</div>
|
||||
</nav>
|
||||
@@ -74,7 +76,8 @@
|
||||
@onclick="ToggleMobileMenu">
|
||||
<span></span><span></span><span></span>
|
||||
</button>
|
||||
|
||||
|
||||
<SettingsMenu />
|
||||
<MudIconButton Icon="@(DarkLightModeButtonIcon)" Color="Color.Inherit" OnClick="@DarkModeToggle"/>
|
||||
</div>
|
||||
|
||||
|
||||
@@ -42,6 +42,7 @@
|
||||
@code {
|
||||
private string _audioPlayerClass = "minimized";
|
||||
private const string DarkModeKey = "darkMode";
|
||||
private const string StreamQualityKey = "streamQuality";
|
||||
private bool _isDarkMode = false;
|
||||
private bool? _lastAppliedDarkMode = null;
|
||||
private PersistingComponentStateSubscription _persistingSubscription;
|
||||
@@ -49,6 +50,7 @@
|
||||
|
||||
[Inject] public required PersistentComponentState PersistentState { get; set; }
|
||||
[Inject] public required DarkModeSettings DarkModeSettings { get; set; }
|
||||
[Inject] public required PublicSiteSettings PublicSiteSettings { get; set; }
|
||||
[Inject] public required IJSRuntime JS { get; set; }
|
||||
|
||||
protected override void OnInitialized()
|
||||
@@ -66,8 +68,17 @@
|
||||
_isDarkMode = DarkModeSettings.IsDarkMode;
|
||||
}
|
||||
|
||||
// Restore the prerender-seeded streaming-quality preference (Phase 18 wave 18.6). Same bridge dark
|
||||
// mode uses: the server SettingsService seeded PublicSiteSettings from the streamQuality cookie, and
|
||||
// this carries it into WASM so the client boots already knowing the preference (no re-read flash, no
|
||||
// wrong default before the first stream).
|
||||
if (PersistentState.TryTakeFromJson<StreamQuality>(StreamQualityKey, out var restoredQuality))
|
||||
{
|
||||
PublicSiteSettings.StreamQuality = restoredQuality;
|
||||
}
|
||||
|
||||
// Register to persist state when prerendering completes
|
||||
_persistingSubscription = PersistentState.RegisterOnPersisting(PersistDarkMode);
|
||||
_persistingSubscription = PersistentState.RegisterOnPersisting(PersistState);
|
||||
}
|
||||
|
||||
// Sync dark mode class on <body> so portaled MudBlazor elements (popovers, menus, selects)
|
||||
@@ -91,9 +102,10 @@
|
||||
// Theme wrapper class for CSS targeting
|
||||
private string ThemeWrapperClass => _isDarkMode ? "deepdrft-theme-dark" : "deepdrft-theme-light";
|
||||
|
||||
private Task PersistDarkMode()
|
||||
private Task PersistState()
|
||||
{
|
||||
PersistentState.PersistAsJson(DarkModeKey, _isDarkMode);
|
||||
PersistentState.PersistAsJson(StreamQualityKey, PublicSiteSettings.StreamQuality);
|
||||
return Task.CompletedTask;
|
||||
}
|
||||
|
||||
|
||||
@@ -1,7 +1,13 @@
|
||||
/* Spacer to prevent content overlap */
|
||||
/* Spacer to prevent content overlap. position:relative + z-index:1 establishes a stacking context
|
||||
that paints above the WaveformVisualizer backdrop (fixed, z-index:0), and the opaque page-surface
|
||||
background makes the spacer read as solid page — occluding the visualizer where it sits in flow,
|
||||
the same way the app bar covers the top. Theme-aware alias, so it inverts for free. */
|
||||
.player-spacer {
|
||||
width: 100%;
|
||||
flex-shrink: 0;
|
||||
position: relative;
|
||||
z-index: 1;
|
||||
background: var(--deepdrft-page-surface);
|
||||
}
|
||||
|
||||
.player-spacer.expanded {
|
||||
|
||||
@@ -70,6 +70,37 @@ public class AudioInteropService : IAsyncDisposable
|
||||
return await InvokeJsAsync<AudioOperationResult>("DeepDrftAudio.initializeStreaming", playerId, totalStreamLength, contentType);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Probes whether this browser can stream-decode Ogg Opus via WebCodecs (<c>AudioDecoder</c> +
|
||||
/// <c>codec:'opus'</c>; Safari < 16.4 / older Firefox cannot). Phase 18 capability gate (OQ2): the
|
||||
/// player only requests Opus when this returns true, otherwise it stays on the universal lossless path
|
||||
/// (AC7 — no listener ever gets silence over a codec gap). Probe failures degrade to <c>false</c>
|
||||
/// (assume incapable) so an interop error can never silence playback.
|
||||
/// </summary>
|
||||
public async Task<bool> CanDecodeOggOpus()
|
||||
{
|
||||
try
|
||||
{
|
||||
return await _jsRuntime.InvokeAsync<bool>("DeepDrftAudio.canDecodeOggOpus");
|
||||
}
|
||||
catch
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Hands the raw Opus seek/setup sidecar bytes (setup header + granule→byte seek index) to the JS player
|
||||
/// so the next Opus stream's decoder has them BEFORE init (the 18.4 set-before-init contract). The player
|
||||
/// parses and stashes them; <see cref="InitializeStreaming"/> applies them when it builds the Opus decoder.
|
||||
/// Must be called before <see cref="InitializeStreaming"/> on an Opus stream. Returns the parse result —
|
||||
/// a failure means the bytes were not a valid sidecar, and the caller falls back to lossless.
|
||||
/// </summary>
|
||||
public async Task<AudioOperationResult> SetOpusSidecar(string playerId, byte[] sidecarBytes)
|
||||
{
|
||||
return await InvokeJsAsync<AudioOperationResult>("DeepDrftAudio.setOpusSidecar", playerId, sidecarBytes);
|
||||
}
|
||||
|
||||
public async Task<StreamingResult> ProcessStreamingChunk(string playerId, byte[] audioChunk)
|
||||
{
|
||||
return await InvokeJsAsync<StreamingResult>("DeepDrftAudio.processStreamingChunk", playerId, audioChunk);
|
||||
@@ -115,6 +146,18 @@ public class AudioInteropService : IAsyncDisposable
|
||||
return await InvokeJsAsync<SeekResult>("DeepDrftAudio.seek", playerId, position);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Resolve the file-absolute byte offset to begin a stream at <paramref name="position"/> with no
|
||||
/// active playback or buffered audio — the "load at timestamp" seam (Phase 18 wave 18.6 format switch).
|
||||
/// Returns <see cref="SeekResult.ByteOffset"/> on success; <see cref="AudioOperationResult.Success"/> is
|
||||
/// false when the decoder cannot yet resolve an offset (e.g. a WAV stream whose header has not been
|
||||
/// parsed), so the caller can feed header bytes and retry.
|
||||
/// </summary>
|
||||
public async Task<SeekResult> ResolveStreamOffsetAsync(string playerId, double position)
|
||||
{
|
||||
return await InvokeJsAsync<SeekResult>("DeepDrftAudio.resolveStreamOffset", playerId, position);
|
||||
}
|
||||
|
||||
// New methods for seek-beyond-buffer support
|
||||
public async Task<double> GetBufferedDuration(string playerId)
|
||||
{
|
||||
@@ -128,11 +171,42 @@ public class AudioInteropService : IAsyncDisposable
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Phase 21.2a back-pressure poll: ask whether the scheduler is still over its forward
|
||||
/// high/low-water band. The read loop calls this only WHILE already throttled, to learn when it
|
||||
/// may resume reading — the steady-state loop reads the piggybacked <c>ProductionPaused</c> flag
|
||||
/// off each chunk result instead. Defaults to false on any interop failure so a torn-down player
|
||||
/// never wedges a loop that is exiting anyway.
|
||||
/// </summary>
|
||||
public async Task<bool> IsProductionPaused(string playerId)
|
||||
{
|
||||
try
|
||||
{
|
||||
return await _jsRuntime.InvokeAsync<bool>("DeepDrftAudio.isProductionPaused", playerId);
|
||||
}
|
||||
catch
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
public async Task<AudioOperationResult> ReinitializeFromOffset(string playerId, long totalStreamLength, double seekPosition)
|
||||
{
|
||||
return await InvokeJsAsync<AudioOperationResult>("DeepDrftAudio.reinitializeFromOffset", playerId, totalStreamLength, seekPosition);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Phase 21.3 / AC6 clean-failure recovery: after a window-miss refill (seek-back past the
|
||||
/// retained tail) fails its Range fetch or reinit, halt the starved scheduler and leave the
|
||||
/// player paused-but-loaded at <paramref name="seekPosition"/> so no silent false end fires and a
|
||||
/// retry is possible. Routes through <see cref="InvokeJsAsync{T}"/> so an interop failure during
|
||||
/// recovery still yields a failure result rather than throwing into the seek path.
|
||||
/// </summary>
|
||||
public async Task<AudioOperationResult> RecoverFromFailedRefill(string playerId, double seekPosition)
|
||||
{
|
||||
return await InvokeJsAsync<AudioOperationResult>("DeepDrftAudio.recoverFromFailedRefill", playerId, seekPosition);
|
||||
}
|
||||
|
||||
public async Task<AudioOperationResult> SetVolumeAsync(string playerId, double volume)
|
||||
{
|
||||
return await InvokeJsAsync<AudioOperationResult>("DeepDrftAudio.setVolume", playerId, volume);
|
||||
@@ -388,6 +462,11 @@ public class StreamingResult : AudioOperationResult
|
||||
public bool HeaderParsed { get; set; }
|
||||
public int BufferCount { get; set; }
|
||||
public double? Duration { get; set; } // Duration in seconds calculated from WAV header
|
||||
|
||||
// Phase 21.2a back-pressure: true when the scheduler's forward decoded fill is over the
|
||||
// high-water mark and the C# read loop should stop calling ReadAsync until it drains. Read off
|
||||
// the chunk result the loop already awaits — no extra interop hop in the unthrottled steady state.
|
||||
public bool ProductionPaused { get; set; }
|
||||
}
|
||||
|
||||
public class AudioPlayerState
|
||||
|
||||
@@ -118,6 +118,16 @@ public abstract class AudioPlayerService : IPlayerService, IAsyncDisposable
|
||||
IsPlaying = true;
|
||||
IsPaused = false;
|
||||
}
|
||||
else if (IsPaused)
|
||||
{
|
||||
// Play failed while the player is paused — the scheduler may be empty after a
|
||||
// failed refill (AC6 recovery). Re-issue a seek at the current position: the
|
||||
// seek path routes to seekBeyondBuffer when the scheduler is empty (Phase 21.3
|
||||
// fix), triggering a real refetch rather than returning "Streaming not ready".
|
||||
// We return early here; Seek owns its own state mutations and NotifyStateChanged.
|
||||
await Seek(CurrentTime);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
if (!result.Success)
|
||||
@@ -128,7 +138,7 @@ public abstract class AudioPlayerService : IPlayerService, IAsyncDisposable
|
||||
{
|
||||
ErrorMessage = null;
|
||||
}
|
||||
|
||||
|
||||
await NotifyStateChanged();
|
||||
}
|
||||
catch (Exception ex)
|
||||
@@ -298,7 +308,6 @@ public abstract class AudioPlayerService : IPlayerService, IAsyncDisposable
|
||||
/// </summary>
|
||||
protected virtual void OnPlaybackEnded() { }
|
||||
|
||||
|
||||
protected async Task EnsureInitializedAsync()
|
||||
{
|
||||
if (!IsInitialized)
|
||||
|
||||
@@ -3,11 +3,16 @@ using Microsoft.JSInterop;
|
||||
namespace DeepDrftPublic.Client.Services;
|
||||
|
||||
/// <summary>
|
||||
/// Thin C# wrapper over the <c>window.DeepDrftBeacon</c> TS interop (Phase 16 §2.2). Wraps the
|
||||
/// <c>navigator.sendBeacon</c> POST and the page-unload registration so the rest of the client never
|
||||
/// touches <see cref="IJSRuntime"/> string identifiers directly. All calls are best-effort: a JS
|
||||
/// failure (module not yet loaded, interop unavailable during prerender) is swallowed — telemetry must
|
||||
/// never throw into the UI or the playback path.
|
||||
/// Thin C# wrapper over the <c>window.DeepDrftLifecycle</c> TS interop. Wraps the <c>navigator.sendBeacon</c>
|
||||
/// POST and the page-unload registration so the rest of the client never touches <see cref="IJSRuntime"/>
|
||||
/// string identifiers directly. After the transport-resilience split this is the <b>unload-edge transport
|
||||
/// only</b>: normal play closes and shares go over the first-party <see cref="IEventPoster"/> fetch, and
|
||||
/// <c>sendBeacon</c> is retained solely for the page-unload path (pagehide / visibility→hidden) where an
|
||||
/// awaited fetch would be cancelled. The module is named off the former <c>telemetry/beacon</c> path
|
||||
/// (<c>DeepDrftLifecycle</c>, served from <c>js/session/lifecycle.js</c>) so even this retained fallback is
|
||||
/// not caught by name-based tracking/fingerprinting blockers. All calls are best-effort: a JS failure
|
||||
/// (module not yet loaded, interop unavailable during prerender) is swallowed — telemetry must never throw
|
||||
/// into the UI or the playback path.
|
||||
/// </summary>
|
||||
public sealed class BeaconInterop
|
||||
{
|
||||
@@ -23,7 +28,7 @@ public sealed class BeaconInterop
|
||||
{
|
||||
try
|
||||
{
|
||||
await _js.InvokeAsync<bool>("DeepDrftBeacon.send", url, json);
|
||||
await _js.InvokeAsync<bool>("DeepDrftLifecycle.send", url, json);
|
||||
}
|
||||
catch
|
||||
{
|
||||
@@ -37,7 +42,7 @@ public sealed class BeaconInterop
|
||||
{
|
||||
try
|
||||
{
|
||||
await _js.InvokeVoidAsync("DeepDrftBeacon.registerUnload", key, dotNetRef, methodName);
|
||||
await _js.InvokeVoidAsync("DeepDrftLifecycle.registerUnload", key, dotNetRef, methodName);
|
||||
}
|
||||
catch
|
||||
{
|
||||
@@ -50,7 +55,7 @@ public sealed class BeaconInterop
|
||||
{
|
||||
try
|
||||
{
|
||||
await _js.InvokeVoidAsync("DeepDrftBeacon.unregisterUnload", key);
|
||||
await _js.InvokeVoidAsync("DeepDrftLifecycle.unregisterUnload", key);
|
||||
}
|
||||
catch
|
||||
{
|
||||
|
||||
@@ -7,28 +7,38 @@ using Microsoft.AspNetCore.Components;
|
||||
namespace DeepDrftPublic.Client.Services;
|
||||
|
||||
/// <summary>
|
||||
/// Production <see cref="IPlayEventSink"/> (Phase 16 §2.2): serializes the play classification and fires
|
||||
/// it via <c>navigator.sendBeacon</c> to the proxied <c>api/event/play</c> route. Fire-and-forget by
|
||||
/// design — <see cref="IPlayEventSink.EmitPlay"/> is synchronous (it is called from the player's close
|
||||
/// path and the unload handler, neither of which can await), so the beacon is dispatched without
|
||||
/// awaiting and its failure is irrelevant. The current <c>anonId</c> (wave 16.3) is read synchronously
|
||||
/// from the warmed <see cref="IAnonIdProvider"/> cache and omitted when null (storage unavailable / not
|
||||
/// yet warmed) — an anonId-less play still counts, it just doesn't contribute to the listener tally.
|
||||
/// Production <see cref="IPlayEventSink"/> (Phase 16 §2.2; telemetry transport-resilience). Serializes the
|
||||
/// play classification once and dispatches it down the arm the close chose:
|
||||
/// <list type="bullet">
|
||||
/// <item><see cref="EmitPlayAsync"/> posts via the first-party <see cref="IEventPoster"/> — a same-origin
|
||||
/// <c>HttpClient</c> fetch to <c>api/event/play</c>, used for the normal close paths (organic end /
|
||||
/// track-switch / stop) that a privacy-hardened browser would block if they used a name-matched
|
||||
/// <c>sendBeacon</c> module.</item>
|
||||
/// <item><see cref="EmitPlayOnUnload"/> fires <see cref="BeaconInterop"/> (<c>sendBeacon</c>) for the
|
||||
/// tab-unload edge, where an awaited fetch would be cancelled.</item>
|
||||
/// </list>
|
||||
/// Both arms send byte-identical payloads (same DTO, same anonId, same JSON options). The current
|
||||
/// <c>anonId</c> (wave 16.3) is read synchronously from the warmed <see cref="IAnonIdProvider"/> cache and
|
||||
/// omitted when null (storage unavailable / not yet warmed) — an anonId-less play still counts, it just
|
||||
/// doesn't contribute to the listener tally.
|
||||
/// </summary>
|
||||
public sealed class BeaconPlayEventSink : IPlayEventSink
|
||||
{
|
||||
// Omit a null anonId from the wire payload (§2.2 — "omitted entirely" when absent) rather than
|
||||
// sending "anonId":null. The API treats absent and null identically, so this is cosmetic minimalism;
|
||||
// it does not change the integer enum encoding the 16.1 contract already relies on.
|
||||
private static readonly JsonSerializerOptions BeaconJson =
|
||||
private static readonly JsonSerializerOptions EventJson =
|
||||
new() { DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull };
|
||||
|
||||
private readonly IEventPoster _poster;
|
||||
private readonly BeaconInterop _beacon;
|
||||
private readonly IAnonIdProvider _anonId;
|
||||
private readonly string _playUrl;
|
||||
|
||||
public BeaconPlayEventSink(BeaconInterop beacon, IAnonIdProvider anonId, NavigationManager navigation)
|
||||
public BeaconPlayEventSink(
|
||||
IEventPoster poster, BeaconInterop beacon, IAnonIdProvider anonId, NavigationManager navigation)
|
||||
{
|
||||
_poster = poster;
|
||||
_beacon = beacon;
|
||||
_anonId = anonId;
|
||||
// The WASM client posts to its own host, which proxies to DeepDrftAPI. BaseUri carries a
|
||||
@@ -36,17 +46,19 @@ public sealed class BeaconPlayEventSink : IPlayEventSink
|
||||
_playUrl = $"{navigation.BaseUri}api/event/play";
|
||||
}
|
||||
|
||||
public void EmitPlay(string trackEntryKey, PlayBucket bucket)
|
||||
{
|
||||
var json = JsonSerializer.Serialize(new PlayEventDto
|
||||
public Task EmitPlayAsync(string trackEntryKey, PlayBucket bucket)
|
||||
=> _poster.PostAsync(_playUrl, Serialize(trackEntryKey, bucket));
|
||||
|
||||
public void EmitPlayOnUnload(string trackEntryKey, PlayBucket bucket)
|
||||
// Fire-and-forget: the beacon survives unload; the C# task may not, and we do not act on the
|
||||
// result either way.
|
||||
=> _ = _beacon.SendAsync(_playUrl, Serialize(trackEntryKey, bucket));
|
||||
|
||||
private string Serialize(string trackEntryKey, PlayBucket bucket)
|
||||
=> JsonSerializer.Serialize(new PlayEventDto
|
||||
{
|
||||
TrackEntryKey = trackEntryKey,
|
||||
Bucket = bucket,
|
||||
AnonId = _anonId.Current,
|
||||
}, BeaconJson);
|
||||
|
||||
// Fire-and-forget: do not await. The beacon survives unload; the C# task may not, and we do not
|
||||
// act on the result either way.
|
||||
_ = _beacon.SendAsync(_playUrl, json);
|
||||
}
|
||||
}, EventJson);
|
||||
}
|
||||
|
||||
@@ -14,9 +14,7 @@ public class DarkModeCookieService(DarkModeSettings darkModeSetting, IJSRuntime
|
||||
|
||||
public async ValueTask SetDarkModeAsync(bool isDarkMode)
|
||||
{
|
||||
var expires = DateTime.UtcNow.AddDays(EXPIRY_DAYS).ToString("R");
|
||||
await js.InvokeVoidAsync("eval",
|
||||
$"document.cookie = '{COOKIE_NAME}={isDarkMode.ToString().ToLower()}; expires={expires}; path=/; SameSite=Lax'");
|
||||
await js.InvokeVoidAsync("DeepDrftSettings.setCookie", COOKIE_NAME, isDarkMode.ToString().ToLower(), EXPIRY_DAYS);
|
||||
darkModeSetting.IsDarkMode = isDarkMode;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
using System.Text;
|
||||
|
||||
namespace DeepDrftPublic.Client.Services;
|
||||
|
||||
/// <summary>
|
||||
/// Production <see cref="IEventPoster"/>: a first-party, same-origin <see cref="System.Net.Http.HttpClient"/>
|
||||
/// POST to the site's own <c>api/event/*</c> proxy (telemetry transport-resilience). A fetch to the host's
|
||||
/// own origin is not third-party tracking and is not caught by the name-based heuristics (Firefox
|
||||
/// Fingerprinting / Tracking Protection) that block a <c>telemetry/beacon</c>-named <c>sendBeacon</c>
|
||||
/// module. Used for the awaitable play-close paths (organic end / track-switch / stop) and every share
|
||||
/// event; only the rare tab-unload edge still goes through <see cref="BeaconInterop"/>, where an awaited
|
||||
/// fetch would be cancelled as the page freezes.
|
||||
///
|
||||
/// <para>Best-effort and non-throwing by contract: a failed POST (offline, blocked, server error) is
|
||||
/// swallowed so a dropped telemetry event never throws into the UI or the playback path — identical
|
||||
/// posture to the beacon transport.</para>
|
||||
/// </summary>
|
||||
public sealed class HttpEventPoster : IEventPoster
|
||||
{
|
||||
// The default factory client carries no base address; the sink/tracker pass an absolute same-origin
|
||||
// URL built from NavigationManager.BaseUri, so the POST targets the host proxy regardless of how the
|
||||
// named clients are configured. The default client uses the browser fetch handler in WASM, which is
|
||||
// exactly the first-party request the heuristic blockers permit.
|
||||
private readonly IHttpClientFactory _httpClientFactory;
|
||||
|
||||
public HttpEventPoster(IHttpClientFactory httpClientFactory)
|
||||
{
|
||||
_httpClientFactory = httpClientFactory;
|
||||
}
|
||||
|
||||
public async Task PostAsync(string url, string json)
|
||||
{
|
||||
try
|
||||
{
|
||||
using var client = _httpClientFactory.CreateClient();
|
||||
using var content = new StringContent(json, Encoding.UTF8, "application/json");
|
||||
// Best-effort: the server records the event; we do not act on the status either way.
|
||||
using var response = await client.PostAsync(url, content);
|
||||
}
|
||||
catch
|
||||
{
|
||||
// Swallow — a dropped telemetry event is acceptable; telemetry must never throw into the UI
|
||||
// or the playback path. Mirrors the beacon's fire-and-forget contract.
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
namespace DeepDrftPublic.Client.Services;
|
||||
|
||||
/// <summary>
|
||||
/// The first-party event transport seam (telemetry transport-resilience). Sends a serialized event body
|
||||
/// to a same-origin <c>api/event/*</c> route over <see cref="System.Net.Http.HttpClient"/> — a fetch to
|
||||
/// the site's own host proxy, which privacy / tracking heuristics do not block the way they name-match a
|
||||
/// <c>sendBeacon</c> module. Abstracted so the play sink and the share tracker share one best-effort,
|
||||
/// non-throwing POST, and so tests can capture the wire payload behind a fake with no HTTP — the same
|
||||
/// one-seam pattern as <see cref="IPlayEventSink"/>.
|
||||
/// </summary>
|
||||
public interface IEventPoster
|
||||
{
|
||||
/// <summary>
|
||||
/// Best-effort POST of <paramref name="json"/> (an <c>application/json</c> body) to
|
||||
/// <paramref name="url"/>. Never throws: a failed POST is swallowed so telemetry cannot break the UI
|
||||
/// or the playback path. Awaitable, but safe to fire-and-forget on a live page.
|
||||
/// </summary>
|
||||
Task PostAsync(string url, string json);
|
||||
}
|
||||
@@ -5,12 +5,30 @@ namespace DeepDrftPublic.Client.Services;
|
||||
/// <summary>
|
||||
/// The emit seam for the <see cref="PlayTracker"/> (Phase 16 §2.1). The tracker owns the session
|
||||
/// lifecycle, the engagement floor, and the bucket classification but knows nothing about transport —
|
||||
/// it hands a finished classification to a sink. The production sink fires a <c>sendBeacon</c> POST to
|
||||
/// <c>api/event/play</c>; tests substitute a fake sink to assert floor and bucket behaviour with no
|
||||
/// JS interop. This keeps the tracker's logic testable behind one seam, as the spec calls for.
|
||||
/// it hands a finished classification to a sink, choosing only which arm fits the close that triggered
|
||||
/// it. Two arms exist because the close paths differ in whether the page survives the call (telemetry
|
||||
/// transport-resilience):
|
||||
/// <list type="bullet">
|
||||
/// <item><see cref="EmitPlayAsync"/> — normal closes (organic end / track-switch / stop), where the page
|
||||
/// stays alive, go over a first-party <c>HttpClient</c> POST to <c>api/event/play</c>. A first-party
|
||||
/// fetch is not name-matched by tracking/fingerprinting heuristics the way a <c>sendBeacon</c> module is.</item>
|
||||
/// <item><see cref="EmitPlayOnUnload"/> — the page-unload edge (pagehide / visibility→hidden), where an
|
||||
/// awaited fetch would be cancelled, still goes over <c>navigator.sendBeacon</c>.</item>
|
||||
/// </list>
|
||||
/// Tests substitute a fake sink to assert floor and bucket behaviour with no transport.
|
||||
/// </summary>
|
||||
public interface IPlayEventSink
|
||||
{
|
||||
/// <summary>Emit one recorded play. Called at most once per session, only when the floor is crossed.</summary>
|
||||
void EmitPlay(string trackEntryKey, PlayBucket bucket);
|
||||
/// <summary>
|
||||
/// Emit one recorded play over the first-party fetch transport (normal close: end / switch / stop).
|
||||
/// Called at most once per session, only when the floor is crossed. Awaitable but safe to
|
||||
/// fire-and-forget on a live page; never throws.
|
||||
/// </summary>
|
||||
Task EmitPlayAsync(string trackEntryKey, PlayBucket bucket);
|
||||
|
||||
/// <summary>
|
||||
/// Emit one recorded play over <c>sendBeacon</c> for the page-unload edge, where an awaited fetch
|
||||
/// would be cancelled as the page freezes. Synchronous and fire-and-forget; never throws.
|
||||
/// </summary>
|
||||
void EmitPlayOnUnload(string trackEntryKey, PlayBucket bucket);
|
||||
}
|
||||
|
||||
@@ -91,4 +91,16 @@ public interface IStreamingPlayerService : IPlayerService
|
||||
/// <see cref="IPlayerService.CurrentTrack"/> and notifies; performs no JS interop.
|
||||
/// </summary>
|
||||
Task StageTrack(TrackDto track);
|
||||
|
||||
/// <summary>
|
||||
/// Re-streams the current track in the freshly-resolved delivery format while preserving the
|
||||
/// listener's playback position (Phase 18 wave 18.6 — the Settings "Apply" restart). The format is
|
||||
/// re-resolved on the new load via the <c>ResolveStreamFormatAsync</c> seam, so this picks up a just-
|
||||
/// changed streaming-quality preference. A cross-format byte offset can only be resolved once the NEW
|
||||
/// format's decoder has parsed its header, so the reload runs a fresh load from byte 0 to initialize
|
||||
/// that decoder, then seeks back to the saved position through the existing seek-beyond-buffer path.
|
||||
/// No-op when no track is loaded (nothing playing to switch). Safe to call while a track is playing;
|
||||
/// a track switch during the brief restore window abandons the position restore.
|
||||
/// </summary>
|
||||
Task ReloadPreservingPositionAsync();
|
||||
}
|
||||
@@ -88,8 +88,15 @@ public sealed class PlayTracker
|
||||
/// nothing is sent (it was a preview/skip, §1d). Idempotent and safe to call when no session is open —
|
||||
/// organic end, track-switch, stop, dispose, and the unload beacon may all race to close, and only the
|
||||
/// first call emits.
|
||||
///
|
||||
/// <para><paramref name="viaUnload"/> selects the transport, not the classification (telemetry
|
||||
/// transport-resilience). The default (false) is the normal close (organic end / track-switch / stop):
|
||||
/// the page is alive, so the event goes over the first-party fetch arm. The unload handler passes true
|
||||
/// so the rare tab-close mid-play uses <c>sendBeacon</c>, the only transport that survives the freeze.
|
||||
/// The fetch arm is fire-and-forget here because the close paths are sync-shaped (a void JS callback,
|
||||
/// or a teardown we must not block on a telemetry POST) — on a live page the task still completes.</para>
|
||||
/// </summary>
|
||||
public void Close()
|
||||
public void Close(bool viaUnload = false)
|
||||
{
|
||||
if (!HasOpenSession)
|
||||
{
|
||||
@@ -112,7 +119,11 @@ public sealed class PlayTracker
|
||||
if (!CrossesFloor(_highWater, duration))
|
||||
return;
|
||||
|
||||
_sink.EmitPlay(key, Classify(fraction));
|
||||
var bucket = Classify(fraction);
|
||||
if (viaUnload)
|
||||
_sink.EmitPlayOnUnload(key, bucket);
|
||||
else
|
||||
_ = _sink.EmitPlayAsync(key, bucket);
|
||||
}
|
||||
|
||||
// The floor is the SMALLER of the absolute-seconds wall and the percentage of duration (§1d / D2).
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
using DeepDrftModels.Enums;
|
||||
using DeepDrftPublic.Client.Clients;
|
||||
using DeepDrftPublic.Client.Common;
|
||||
using Microsoft.Extensions.Logging;
|
||||
|
||||
namespace DeepDrftPublic.Client.Services;
|
||||
|
||||
/// <summary>
|
||||
/// The production player that honours the listener's streaming-quality preference (Phase 18 wave 18.6).
|
||||
/// Extends <see cref="StreamingAudioPlayerService"/> through the single deliberately-overridable seam,
|
||||
/// <see cref="StreamingAudioPlayerService.ResolveStreamFormatAsync"/>, so the rest of the streaming stack
|
||||
/// (seek, telemetry, the seek-beyond-buffer format reuse) is inherited verbatim.
|
||||
/// <para>
|
||||
/// The override is one branch: a <see cref="StreamQuality.Lossless"/> preference returns
|
||||
/// <see cref="AudioFormat.Lossless"/> immediately; anything else falls through to <c>base</c>, which keeps
|
||||
/// the 18.5 invariants intact — the capability gate (AC7: a browser that can't decode Ogg Opus gets lossless)
|
||||
/// and the sidecar-absent → lossless fallback (C2: a legacy / un-backfilled / failed-transcode track gets
|
||||
/// lossless). So a Lossless pick always yields lossless; a Low-data pick yields Opus only when it can
|
||||
/// actually play, and lossless otherwise. No path produces an unplayable stream.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public class PreferenceAwareStreamingPlayerService : StreamingAudioPlayerService
|
||||
{
|
||||
private readonly PublicSiteSettings _settings;
|
||||
|
||||
public PreferenceAwareStreamingPlayerService(
|
||||
AudioInteropService audioInterop,
|
||||
TrackMediaClient trackMediaClient,
|
||||
ILogger<StreamingAudioPlayerService> logger,
|
||||
PublicSiteSettings settings)
|
||||
: base(audioInterop, trackMediaClient, logger)
|
||||
{
|
||||
_settings = settings;
|
||||
}
|
||||
|
||||
protected override async Task<AudioFormat> ResolveStreamFormatAsync(string entryKey, CancellationToken cancellationToken)
|
||||
{
|
||||
// Listener explicitly chose lossless — request it directly, no Opus probe / sidecar fetch needed.
|
||||
if (_settings.StreamQuality == StreamQuality.Lossless)
|
||||
{
|
||||
return AudioFormat.Lossless;
|
||||
}
|
||||
|
||||
// Low-data preference: defer to the base capability-gated resolution, which probes Opus support and
|
||||
// the sidecar's presence and degrades to lossless when either is missing. Both 18.5 invariants are
|
||||
// inherited here, not re-implemented.
|
||||
return await base.ResolveStreamFormatAsync(entryKey, cancellationToken);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
using DeepDrftPublic.Client.Common;
|
||||
using Microsoft.JSInterop;
|
||||
|
||||
namespace DeepDrftPublic.Client.Services;
|
||||
|
||||
/// <summary>
|
||||
/// Client-side runtime writer for public-site settings (Phase 18 wave 18.6), the analogue of
|
||||
/// <see cref="DarkModeCookieService"/>. Reads the current preference off the in-memory
|
||||
/// <see cref="PublicSiteSettings"/> (already seeded at prerender and bridged into WASM), and writes a
|
||||
/// 365-day cookie via <c>document.cookie</c> interop when the listener changes it in the Settings menu —
|
||||
/// the same durable-truth seam dark mode uses, so the choice survives the session and seeds the next visit's
|
||||
/// prerender (no flash).
|
||||
/// </summary>
|
||||
public class SettingsCookieService(PublicSiteSettings settings, IJSRuntime js) : SettingsServiceBase
|
||||
{
|
||||
private const int ExpiryDays = 365;
|
||||
|
||||
public StreamQuality GetStreamQuality() => settings.StreamQuality;
|
||||
|
||||
public async ValueTask SetStreamQualityAsync(StreamQuality quality)
|
||||
{
|
||||
if (settings.StreamQuality == quality) return;
|
||||
|
||||
await WriteCookieAsync(StreamQualityCookieName, FormatStreamQuality(quality));
|
||||
settings.StreamQuality = quality;
|
||||
}
|
||||
|
||||
private async ValueTask WriteCookieAsync(string name, string value)
|
||||
{
|
||||
await js.InvokeVoidAsync("DeepDrftSettings.setCookie", name, value, ExpiryDays);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
using DeepDrftPublic.Client.Common;
|
||||
|
||||
namespace DeepDrftPublic.Client.Services;
|
||||
|
||||
/// <summary>
|
||||
/// Shared cookie contract for the public-site settings seam (Phase 18 wave 18.6), the analogue of
|
||||
/// <see cref="DarkModeServiceBase"/>. Holds the cookie names and the (de)serialization for each preference
|
||||
/// so the server prerender-read service and the client cookie-write service agree on one wire format —
|
||||
/// the load-bearing reason this is shared rather than duplicated. Each new preference adds its cookie name
|
||||
/// and a parse/format pair here, keeping the round-trip in one place.
|
||||
/// </summary>
|
||||
public abstract class SettingsServiceBase
|
||||
{
|
||||
protected const string StreamQualityCookieName = "streamQuality";
|
||||
|
||||
/// <summary>
|
||||
/// Parses the <c>streamQuality</c> cookie value into <see cref="StreamQuality"/>, defaulting to
|
||||
/// <see cref="StreamQuality.LowData"/> (the OQ2 default) for an absent, empty, or unrecognized value so
|
||||
/// a missing/garbled cookie never produces a surprising preference.
|
||||
/// </summary>
|
||||
protected static StreamQuality ParseStreamQuality(string? cookieValue) =>
|
||||
Enum.TryParse<StreamQuality>(cookieValue, ignoreCase: true, out var parsed)
|
||||
? parsed
|
||||
: StreamQuality.LowData;
|
||||
|
||||
/// <summary>Formats a <see cref="StreamQuality"/> for cookie storage (round-trips with <see cref="ParseStreamQuality"/>).</summary>
|
||||
protected static string FormatStreamQuality(StreamQuality quality) => quality.ToString();
|
||||
}
|
||||
@@ -10,13 +10,16 @@ namespace DeepDrftPublic.Client.Services;
|
||||
/// Records share events from <c>SharePopover</c> (Phase 16 §1b / §2.1). After a successful clipboard
|
||||
/// write the popover calls <see cref="RecordShare"/>; this tracker applies the per-(target,channel)
|
||||
/// debounce — at most one event per target+channel per <see cref="DebounceWindow"/> per session — and
|
||||
/// fires the event via <c>navigator.sendBeacon</c> to the proxied <c>api/event/share</c> route.
|
||||
/// fires the event via the first-party <see cref="IEventPoster"/> POST to the proxied <c>api/event/share</c>
|
||||
/// route. A share is always a user-interaction close with the page alive (never a tab-unload), so it uses
|
||||
/// the fetch transport unconditionally — there is no <c>sendBeacon</c> arm here (telemetry
|
||||
/// transport-resilience).
|
||||
///
|
||||
/// <para>
|
||||
/// Scoped (per-session) so the debounce memory lives for the session and resets on a fresh load, matching
|
||||
/// the "feels like one act" intent: copying the same link three times in a row is one share, not three.
|
||||
/// The beacon send is fire-and-forget; the current <c>anonId</c> (wave 16.3) is read synchronously from
|
||||
/// the warmed <see cref="IAnonIdProvider"/> cache and omitted when null.
|
||||
/// The POST is fire-and-forget; the current <c>anonId</c> (wave 16.3) is read synchronously from the
|
||||
/// warmed <see cref="IAnonIdProvider"/> cache and omitted when null.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public sealed class ShareTracker
|
||||
@@ -27,17 +30,17 @@ public sealed class ShareTracker
|
||||
|
||||
// Omit a null anonId from the wire payload (§2.2). Cosmetic — the API tolerates null — and does not
|
||||
// change the integer enum encoding the 16.1 contract relies on.
|
||||
private static readonly JsonSerializerOptions BeaconJson =
|
||||
private static readonly JsonSerializerOptions EventJson =
|
||||
new() { DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull };
|
||||
|
||||
private readonly BeaconInterop _beacon;
|
||||
private readonly IEventPoster _poster;
|
||||
private readonly IAnonIdProvider _anonId;
|
||||
private readonly string _shareUrl;
|
||||
private readonly Dictionary<string, DateTimeOffset> _lastSent = new();
|
||||
|
||||
public ShareTracker(BeaconInterop beacon, IAnonIdProvider anonId, NavigationManager navigation)
|
||||
public ShareTracker(IEventPoster poster, IAnonIdProvider anonId, NavigationManager navigation)
|
||||
{
|
||||
_beacon = beacon;
|
||||
_poster = poster;
|
||||
_anonId = anonId;
|
||||
_shareUrl = $"{navigation.BaseUri}api/event/share";
|
||||
}
|
||||
@@ -71,10 +74,10 @@ public sealed class ShareTracker
|
||||
TargetKey = targetKey,
|
||||
Channel = channel,
|
||||
AnonId = _anonId.Current,
|
||||
}, BeaconJson);
|
||||
}, EventJson);
|
||||
|
||||
// Fire-and-forget — a dropped share telemetry event is acceptable.
|
||||
_ = _beacon.SendAsync(_shareUrl, json);
|
||||
// Fire-and-forget first-party POST — a dropped share telemetry event is acceptable.
|
||||
_ = _poster.PostAsync(_shareUrl, json);
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
using DeepDrftModels.DTOs;
|
||||
using DeepDrftModels.Enums;
|
||||
using DeepDrftPublic.Client.Clients;
|
||||
using System.Buffers;
|
||||
using Microsoft.Extensions.Logging;
|
||||
@@ -15,6 +16,36 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
|
||||
// Adaptive chunk sizing
|
||||
private const int MinBufferSize = 16 * 1024; // 16KB minimum
|
||||
private const int MaxBufferSize = 64 * 1024; // 64KB maximum
|
||||
|
||||
// Phase 21.2a back-pressure poll interval. While the scheduler is over its forward high-water
|
||||
// mark, the segment loop stops fetching the next segment and polls IsProductionPaused at this
|
||||
// cadence until the fill drains below low-water. 100 ms is well under the low-water lookahead
|
||||
// (seconds), so resume is prompt relative to the playhead — no starvation (AC3) — while keeping
|
||||
// the poll cheap. The poll honors the loop's cancellation token, so a track switch/seek during a
|
||||
// pause exits through the same drain discipline as a pause during ReadAsync (C6).
|
||||
private const int BackpressurePollMs = 100;
|
||||
|
||||
// Phase 21 Direction B — forward Range-segment size. The forward stream is fetched as a
|
||||
// sequence of bounded "bytes=cursor-(cursor+SegmentSizeBytes-1)" 206 requests, the next issued
|
||||
// only when the scheduler drains below low-water. Because each request is bounded and fully
|
||||
// consumed before the next is issued, the browser fetch holds AT MOST ~one segment of raw bytes
|
||||
// regardless of file size — this is the network-memory bound the phase exists for (the open-ended
|
||||
// single GET buffered the whole ~970 MB body in the browser even when reads were paused, the
|
||||
// 21.4 finding). 4 MB balances request overhead (a 1 GB mix is ~250 segments) against memory:
|
||||
// at the 30 s high-water mark a fast connection holds well under a segment of unplayed raw bytes,
|
||||
// so the bound is the segment size, not the decoded window. Tunable; not magic.
|
||||
private const long SegmentSizeBytes = 4 * 1024 * 1024;
|
||||
|
||||
// Phase 18 wave 18.6 — position-preserving format switch ("load at timestamp"). When the listener
|
||||
// changes streaming quality mid-track, the new-format stream is started DIRECTLY at the saved
|
||||
// position rather than from byte 0: the load resolves the byte offset for the target time in the
|
||||
// freshly-initialized decoder and streams from there (never audibly playing the start). For WAV the
|
||||
// byte-offset math needs the header, so the byte-0 segment is probed (fed to the decoder WITHOUT
|
||||
// starting playback) until the header parses; this caps that probe so a header that never appears
|
||||
// (corrupt stream) can't read unbounded. The decoder's own header-search ceiling is 256 KB, so this
|
||||
// matches it. Opus needs no probe (its sidecar resolves offsets immediately after init).
|
||||
private const int MaxHeaderProbeBytes = 256 * 1024;
|
||||
|
||||
private int _currentBufferSize = DefaultBufferSize;
|
||||
private int _consecutiveSlowReads = 0;
|
||||
|
||||
@@ -33,6 +64,12 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
|
||||
private readonly ILogger<StreamingAudioPlayerService> _logger;
|
||||
private string? _currentTrackId;
|
||||
|
||||
// The delivery format the active load resolved to (Phase 18). Captured once per LoadTrackStreaming and
|
||||
// reused by the seek-beyond-buffer re-fetch so the Range continuation requests the SAME artifact the
|
||||
// initial stream did — a seek must never switch formats mid-track (the JS decoder, the cached setup
|
||||
// header, and the byte offsets all belong to one artifact). Defaults to Lossless until a load resolves.
|
||||
private AudioFormat _currentFormat = AudioFormat.Lossless;
|
||||
|
||||
// Phase 16 play-session telemetry (§2.1). The tracker observes the playback lifecycle and emits at
|
||||
// most one bucketed play event per session, behind the engagement floor. Attached after construction
|
||||
// by AudioPlayerProvider (the player is not DI-registered), mirroring how QueueService binds — no
|
||||
@@ -84,7 +121,7 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
|
||||
/// freezes. <see cref="PlayTracker.Close"/> is idempotent, so a later organic close is a no-op.
|
||||
/// </summary>
|
||||
[JSInvokable]
|
||||
public void OnPageUnload() => _playTracker?.Close();
|
||||
public void OnPageUnload() => _playTracker?.Close(viaUnload: true);
|
||||
|
||||
// Advance the play-session high-water mark on each progress tick (§2.1). Seeking backward never
|
||||
// lowers it — the tracker takes the max.
|
||||
@@ -129,7 +166,30 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
|
||||
await NotifyStateChanged();
|
||||
}
|
||||
|
||||
private async Task LoadTrackStreaming(TrackDto track)
|
||||
/// <inheritdoc />
|
||||
public async Task ReloadPreservingPositionAsync()
|
||||
{
|
||||
// Nothing playing → nothing to switch. The new preference simply takes effect on the next play.
|
||||
if (CurrentTrack is not { } track || !IsStreamingMode) return;
|
||||
|
||||
// Capture the position to restore before the reload resets streaming state. Near the very start
|
||||
// there is nothing worth preserving — a plain restart in the new format is simpler and avoids a
|
||||
// needless seek-offset resolution.
|
||||
var resumeAt = CurrentTime;
|
||||
|
||||
await EnsureInitializedAsync();
|
||||
await _audioInterop.EnsureAudioContextReady(PlayerId);
|
||||
await NotifyTrackSelected();
|
||||
|
||||
// Reload the same track in the newly-resolved delivery format. A near-start position restarts
|
||||
// from byte 0; otherwise the load begins DIRECTLY at the saved position (no audible playback
|
||||
// from the start). LoadTrackStreaming runs the whole forward segment loop, so this is the last
|
||||
// meaningful await — the caller already fires this fire-and-forget.
|
||||
await LoadTrackStreaming(track, startPosition: resumeAt > 1.0 ? resumeAt : null);
|
||||
await NotifyStateChanged();
|
||||
}
|
||||
|
||||
private async Task LoadTrackStreaming(TrackDto track, double? startPosition = null)
|
||||
{
|
||||
// Always reset to clean state before loading new track. ResetToIdle
|
||||
// both cancels and awaits any in-flight streaming loop, so by the time
|
||||
@@ -174,22 +234,35 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
|
||||
|
||||
await NotifyStateChanged();
|
||||
|
||||
// Pass the streaming token to the HTTP layer so a navigation/track switch
|
||||
// aborts the server connection instead of leaving it draining bytes.
|
||||
var mediaResult = await _trackMediaClient.GetTrackMedia(
|
||||
// Resolve the delivery format for this load BEFORE requesting bytes (Phase 18, default policy
|
||||
// OQ2). When Opus is chosen the sidecar is fetched and injected into the JS player here, ahead of
|
||||
// InitializeStreaming, honouring the 18.4 set-before-init contract. The result is captured so the
|
||||
// seek-beyond-buffer re-fetch reuses the same artifact.
|
||||
_currentFormat = await ResolveStreamFormatAsync(track.EntryKey, loadCts.Token);
|
||||
|
||||
// Direction B: fetch the FIRST bounded segment to learn the total file length and the
|
||||
// content type. The 206 Content-Range carries the total; the segment loop advances its
|
||||
// cursor toward it. The decoder is initialized with the TOTAL length (not the segment
|
||||
// length) so a bounded segment's small Content-Length never trips its byte-count
|
||||
// completion early — segment boundaries are invisible to the decoder, which sees one
|
||||
// continuous in-order byte stream. Passing the streaming token aborts the server
|
||||
// connection on a navigation/track switch instead of leaving it draining bytes.
|
||||
var firstSegment = await _trackMediaClient.GetTrackMedia(
|
||||
track.EntryKey,
|
||||
byteOffset: 0,
|
||||
byteEnd: SegmentSizeBytes - 1,
|
||||
format: _currentFormat,
|
||||
cancellationToken: loadCts.Token);
|
||||
if (!mediaResult.Success)
|
||||
if (!firstSegment.Success)
|
||||
{
|
||||
var technicalError = mediaResult.GetMessage();
|
||||
var technicalError = firstSegment.GetMessage();
|
||||
_logger.LogError("Failed to get track media for {TrackId}: {Error}",
|
||||
track.EntryKey, technicalError);
|
||||
ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(technicalError);
|
||||
return;
|
||||
}
|
||||
|
||||
if (mediaResult.Value == null)
|
||||
if (firstSegment.Value == null)
|
||||
{
|
||||
const string technicalError = "No audio returned from server";
|
||||
_logger.LogError("No audio data returned for track {TrackId}", track.EntryKey);
|
||||
@@ -197,13 +270,22 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
|
||||
return;
|
||||
}
|
||||
|
||||
using var audio = mediaResult.Value;
|
||||
// Ownership of the first segment transfers to the segment loop, which disposes it (and
|
||||
// every subsequent segment). No `using` here — a double dispose is avoided and the socket
|
||||
// is released the moment the loop finishes consuming the segment.
|
||||
var audio = firstSegment.Value;
|
||||
|
||||
// Initialize streaming mode with content length and media type (drives
|
||||
// JS format-decoder selection).
|
||||
var streamingResult = await _audioInterop.InitializeStreaming(PlayerId, audio.ContentLength, audio.ContentType);
|
||||
// The total logical length the decoder must see. On a 206 the Content-Range carries it;
|
||||
// a 200 (server ignored Range / file ≤ one segment) has no Content-Range, so fall back to
|
||||
// the body's own Content-Length — that body IS the whole file in that case.
|
||||
var totalLength = audio.TotalLength ?? audio.ContentLength;
|
||||
|
||||
// Initialize streaming mode with the TOTAL length and media type (drives JS
|
||||
// format-decoder selection). See above: total, not segment, length.
|
||||
var streamingResult = await _audioInterop.InitializeStreaming(PlayerId, totalLength, audio.ContentType);
|
||||
if (!streamingResult.Success)
|
||||
{
|
||||
audio.Dispose();
|
||||
var technicalError = $"Failed to initialize streaming: {streamingResult.Error}";
|
||||
_logger.LogError("Streaming initialization failed for track {TrackId}: {Error}",
|
||||
track.EntryKey, technicalError);
|
||||
@@ -211,8 +293,23 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
|
||||
return;
|
||||
}
|
||||
|
||||
_activeStreamingTask = StreamAudioWithEarlyPlayback(audio, loadCts.Token);
|
||||
await _activeStreamingTask;
|
||||
if (startPosition is { } startAt)
|
||||
{
|
||||
// "Load at timestamp" (Phase 18 wave 18.6 format switch): begin the stream DIRECTLY at
|
||||
// startAt rather than byte 0, so the listener never hears the track restart from the
|
||||
// beginning. The byte-0 segment in hand is used only to parse the header for byte-offset
|
||||
// math (WAV) — Opus resolves the offset from its sidecar with no probe — and then a fresh
|
||||
// segment is fetched from the resolved offset and pumped via the shared seek/refill loop.
|
||||
await StartFromPositionAsync(track.EntryKey, audio, totalLength, startAt, loadCts.Token);
|
||||
}
|
||||
else
|
||||
{
|
||||
// Forward segmentation from byte 0. The first segment is already in hand; the loop pumps
|
||||
// it, then fetches subsequent bounded segments gated on the scheduler fill signal.
|
||||
_activeStreamingTask = RunSegmentedStreamAsync(
|
||||
track.EntryKey, audio, cursor: 0, totalLength, seekPosition: null, loadCts.Token);
|
||||
await _activeStreamingTask;
|
||||
}
|
||||
}
|
||||
catch (OperationCanceledException) when (loadCts.IsCancellationRequested)
|
||||
{
|
||||
@@ -233,10 +330,33 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
|
||||
catch (Exception ex)
|
||||
{
|
||||
StreamingErrorHandler.LogError(_logger, ex, "LoadTrackStreaming", track.EntryKey);
|
||||
ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(ex.Message);
|
||||
LoadProgress = 0;
|
||||
IsLoaded = false;
|
||||
IsStreamingMode = false;
|
||||
var userError = StreamingErrorHandler.GetUserFriendlyMessage(ex.Message);
|
||||
|
||||
// Mid-stream failure (playback was already underway): halt the JS scheduler into a clean
|
||||
// paused-but-loaded state exactly as the seek path does via RecoverFromFailedRefill, rather
|
||||
// than resetting to unloaded and letting the scheduler's buffered tail drain into a silent
|
||||
// false end (AC6). Apply only when this load is still the active operation — a superseding
|
||||
// seek owns state and has already replaced _streamingCancellation with its own CTS.
|
||||
if (_streamingPlaybackStarted && ReferenceEquals(_streamingCancellation, loadCts))
|
||||
{
|
||||
await RecoverFromFailedRefill(CurrentTime, userError);
|
||||
}
|
||||
else if (ReferenceEquals(_streamingCancellation, loadCts))
|
||||
{
|
||||
// First-segment failure (nothing buffered / playing yet), still the active operation:
|
||||
// the normal unload-to-error path is correct — nothing is in the scheduler to halt.
|
||||
ErrorMessage = userError;
|
||||
LoadProgress = 0;
|
||||
IsLoaded = false;
|
||||
IsStreamingMode = false;
|
||||
}
|
||||
else
|
||||
{
|
||||
// Superseded load: a newer seek (or track switch) has already claimed _streamingCancellation
|
||||
// and owns all shared state. Writing IsLoaded/IsStreamingMode here would corrupt the live
|
||||
// operation — mirror the OCE catch's identity guard and do nothing to shared state.
|
||||
_logger.LogDebug("Generic throw on superseded load for track {TrackId} — newer operation owns state, skipping unload", track.EntryKey);
|
||||
}
|
||||
}
|
||||
finally
|
||||
{
|
||||
@@ -250,6 +370,178 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Begin streaming the freshly-initialized track DIRECTLY at <paramref name="startPosition"/> instead
|
||||
/// of byte 0 (Phase 18 wave 18.6 — the position-preserving format switch). The decoder has already been
|
||||
/// built by <c>InitializeStreaming</c>; this resolves the file-absolute byte offset for the target time
|
||||
/// and then converges onto the shared seek/refill loop (<see cref="RunSegmentedStreamAsync"/> with a
|
||||
/// non-null seekPosition), which reinitializes the decoder for a header-less Range continuation and
|
||||
/// starts playback at the target — so nothing is ever audibly played from the start.
|
||||
/// <para>
|
||||
/// Opus resolves the offset from its sidecar immediately; WAV needs its header, so the byte-0 segment
|
||||
/// already in hand is fed to the decoder (WITHOUT starting playback) until the header parses, then the
|
||||
/// offset resolves. The probe is bounded by <see cref="MaxHeaderProbeBytes"/>. The byte-0 segment is
|
||||
/// disposed once the header is in hand; the continuation is a fresh fetch from the resolved offset.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
private async Task StartFromPositionAsync(
|
||||
string trackId,
|
||||
TrackMediaResponse headerSegment,
|
||||
long totalLength,
|
||||
double startPosition,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
// Resolve the byte offset for the target time. Opus answers immediately from its sidecar; WAV
|
||||
// returns failure until its header is parsed, so we probe the byte-0 segment and retry. The
|
||||
// byte-0 segment is disposed once it has served its purpose (header probe / nothing for Opus),
|
||||
// even if the probe throws, so its socket never leaks before the continuation fetch.
|
||||
SeekResult resolved;
|
||||
try
|
||||
{
|
||||
resolved = await _audioInterop.ResolveStreamOffsetAsync(PlayerId, startPosition);
|
||||
if (!resolved.Success || !resolved.SeekBeyondBuffer)
|
||||
{
|
||||
await ProbeHeaderAsync(headerSegment, cancellationToken);
|
||||
resolved = await _audioInterop.ResolveStreamOffsetAsync(PlayerId, startPosition);
|
||||
}
|
||||
}
|
||||
finally
|
||||
{
|
||||
headerSegment.Dispose();
|
||||
}
|
||||
|
||||
if (!resolved.Success || !resolved.SeekBeyondBuffer)
|
||||
{
|
||||
// Could not resolve an offset even after probing — the stream is unusable for a positioned
|
||||
// start. Surface as an error rather than silently restarting from 0 (which would contradict
|
||||
// the "preserve position" contract the listener invoked). The catch in LoadTrackStreaming
|
||||
// settles the error state.
|
||||
throw new Exception(resolved.Error ?? "Could not resolve a stream offset for the requested position");
|
||||
}
|
||||
|
||||
// Fetch the FIRST bounded segment from the resolved offset and pump it through the shared loop
|
||||
// exactly as a seek-beyond-buffer does (reinit for the header-less continuation happens inside,
|
||||
// and playback starts at startPosition). Reuse the format the load already resolved to.
|
||||
var byteOffset = resolved.ByteOffset;
|
||||
var firstSegment = await _trackMediaClient.GetTrackMedia(
|
||||
trackId,
|
||||
byteOffset,
|
||||
byteEnd: byteOffset + SegmentSizeBytes - 1,
|
||||
format: _currentFormat,
|
||||
cancellationToken: cancellationToken);
|
||||
if (!firstSegment.Success || firstSegment.Value == null)
|
||||
{
|
||||
var technicalError = firstSegment.GetMessage() ?? "Failed to load audio from position";
|
||||
_logger.LogError("Failed to get track media from offset {Offset} for {TrackId}: {Error}",
|
||||
byteOffset, trackId, technicalError);
|
||||
throw new Exception(technicalError);
|
||||
}
|
||||
|
||||
var audio = firstSegment.Value;
|
||||
// The absolute EOF boundary the segment loop targets. On a 206 the Content-Range carries the file
|
||||
// total; on a 200 (single-segment tail) fall back to the offset plus this body's length.
|
||||
var continuationTotal = audio.TotalLength ?? (byteOffset + audio.ContentLength);
|
||||
|
||||
// Fresh playback-start transition for the positioned stream (it has not started yet).
|
||||
_streamingPlaybackStarted = false;
|
||||
CanStartStreaming = false;
|
||||
BufferedChunks = 0;
|
||||
// Reflect the landing position immediately so the UI seek bar shows the right spot while the
|
||||
// first post-offset buffers decode.
|
||||
CurrentTime = startPosition;
|
||||
|
||||
_activeStreamingTask = RunSegmentedStreamAsync(
|
||||
trackId, audio, cursor: byteOffset, continuationTotal, seekPosition: startPosition, cancellationToken);
|
||||
await _activeStreamingTask;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Feed bytes from the byte-0 <paramref name="segment"/> into the decoder until its header parses
|
||||
/// (<see cref="HeaderParsed"/>), WITHOUT starting playback — the WAV byte-offset math needs the header
|
||||
/// before <see cref="AudioInteropService.ResolveStreamOffsetAsync"/> can answer. Bounded by
|
||||
/// <see cref="MaxHeaderProbeBytes"/> so a stream that never yields a header cannot read unbounded. The
|
||||
/// decoded buffers this queues are dropped by the subsequent Range-continuation reinit (clearForSeek),
|
||||
/// so nothing is audible and nothing leaks. Opus never reaches here (its offset resolves pre-probe).
|
||||
/// </summary>
|
||||
private async Task ProbeHeaderAsync(TrackMediaResponse segment, CancellationToken cancellationToken)
|
||||
{
|
||||
var buffer = ArrayPool<byte>.Shared.Rent(MaxBufferSize);
|
||||
try
|
||||
{
|
||||
var probed = 0;
|
||||
while (!HeaderParsed && probed < MaxHeaderProbeBytes)
|
||||
{
|
||||
var read = await segment.Stream.ReadAsync(buffer, 0, _currentBufferSize, cancellationToken);
|
||||
if (read <= 0) break; // segment exhausted before a header — let the caller surface the failure.
|
||||
probed += read;
|
||||
|
||||
// Slice to the exact bytes read (the pooled buffer may carry stale tail bytes).
|
||||
var chunk = buffer.AsSpan(0, read).ToArray();
|
||||
var result = await _audioInterop.ProcessStreamingChunk(PlayerId, chunk);
|
||||
if (!result.Success)
|
||||
{
|
||||
throw new Exception($"Failed to process header probe chunk: {result.Error}");
|
||||
}
|
||||
|
||||
HeaderParsed = result.HeaderParsed;
|
||||
// Capture the once-only duration the header yields so the UI and play session have it.
|
||||
if (result.Duration.HasValue && Duration == null)
|
||||
{
|
||||
Duration = result.Duration.Value;
|
||||
_playTracker?.SetDuration(result.Duration.Value);
|
||||
}
|
||||
}
|
||||
}
|
||||
finally
|
||||
{
|
||||
ArrayPool<byte>.Shared.Return(buffer);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Resolves which delivery format this load should request (Phase 18 default policy, OQ2): Opus when the
|
||||
/// browser can decode Ogg Opus AND a sidecar exists for the track, otherwise lossless. When Opus is
|
||||
/// chosen the sidecar is injected into the JS player here (set-before-init, the 18.4 contract) so the
|
||||
/// decoder has its setup header + seek index before <c>InitializeStreaming</c> builds it.
|
||||
/// <para>
|
||||
/// This is the single, deliberately-overridable seam for the listener quality preference (wave 18.6).
|
||||
/// 18.6 overrides this to honour the user's "streaming quality" toggle — returning lossless when the
|
||||
/// listener picked it, and otherwise falling through to this capability-gated default. The capability
|
||||
/// gate (AC7) and the sidecar-absent → lossless fallback (C2) stay here so any override inherits both:
|
||||
/// a browser that cannot decode Opus, or a track with no sidecar, always lands on lossless and plays.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
protected virtual async Task<AudioFormat> ResolveStreamFormatAsync(string entryKey, CancellationToken cancellationToken)
|
||||
{
|
||||
// Capability gate first (AC7): never hand Ogg Opus to a browser that cannot decode it.
|
||||
if (!await _audioInterop.CanDecodeOggOpus())
|
||||
{
|
||||
return AudioFormat.Lossless;
|
||||
}
|
||||
|
||||
// The sidecar must be present (and parseable by the JS decoder) to seek an Opus stream. Its absence
|
||||
// means the track has no Opus artifact yet (legacy / not backfilled / transcode failed) — request
|
||||
// lossless rather than Opus-without-a-sidecar (the server would C2-fall-back anyway, but asking for
|
||||
// lossless keeps the request honest and avoids a wasted Opus-then-fallback round-trip).
|
||||
var sidecar = await _trackMediaClient.GetOpusSidecarAsync(entryKey, cancellationToken);
|
||||
if (!sidecar.Success || sidecar.Value is not { Length: > 0 } sidecarBytes)
|
||||
{
|
||||
return AudioFormat.Lossless;
|
||||
}
|
||||
|
||||
// Inject BEFORE InitializeStreaming (the set-before-init contract). A parse failure here means the
|
||||
// bytes are not a usable sidecar — fall back to lossless so a malformed sidecar never breaks playback.
|
||||
var injected = await _audioInterop.SetOpusSidecar(PlayerId, sidecarBytes);
|
||||
if (!injected.Success)
|
||||
{
|
||||
_logger.LogWarning("Opus sidecar for {EntryKey} failed to parse ({Error}); falling back to lossless.",
|
||||
entryKey, injected.Error);
|
||||
return AudioFormat.Lossless;
|
||||
}
|
||||
|
||||
return AudioFormat.Opus;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Fetches and decodes the track's waveform loudness profile, then notifies state so the
|
||||
/// seek zone re-renders with real bars. Best-effort: a 404 (no stored profile) or any other
|
||||
@@ -311,112 +603,223 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
|
||||
return profile;
|
||||
}
|
||||
|
||||
private async Task StreamAudioWithEarlyPlayback(TrackMediaResponse audio, CancellationToken cancellationToken)
|
||||
/// <summary>
|
||||
/// Phase 21 Direction B — the single segmented forward read loop, shared by the initial load and
|
||||
/// the seek/refill path (the convergence C1/C5 require: one cursor, one fetch mechanism, no forked
|
||||
/// path). It pumps the FIRST segment (already fetched by the caller), then fetches subsequent
|
||||
/// bounded <c>bytes=cursor-(cursor+SegmentSizeBytes-1)</c> 206 segments — each only AFTER the
|
||||
/// scheduler drains below low-water — until the cursor reaches <paramref name="totalLength"/>.
|
||||
/// Because each segment is bounded and fully consumed before the next is requested, the browser
|
||||
/// holds at most ~one segment of raw bytes (the network-memory bound), while the decoder sees one
|
||||
/// continuous in-order byte stream across segment boundaries (the demuxer/decoder buffer partial
|
||||
/// frames/pages across the boundary exactly as for arbitrary chunks today — no per-segment reinit).
|
||||
/// </summary>
|
||||
/// <param name="firstSegment">The already-fetched first segment (byte <paramref name="cursor"/>).
|
||||
/// Owned by this method, which disposes it; subsequent segments are fetched and disposed inline.</param>
|
||||
/// <param name="cursor">File-absolute byte offset the first segment starts at (0 for a fresh load,
|
||||
/// the resolved seek offset for a refill).</param>
|
||||
/// <param name="totalLength">Total file length in bytes — the EOF boundary the cursor advances
|
||||
/// toward. The decoder is initialized/reinitialized against this, not the per-segment length.</param>
|
||||
/// <param name="seekPosition">Non-null for a seek/refill: the decoder is reinitialized for the
|
||||
/// header-less Range continuation at this time before the first segment's bytes are fed (WAV
|
||||
/// retains its header, Opus re-applies the cached setup + lead-trim). Null for a forward load from
|
||||
/// byte 0, where the first segment carries the header and no reinit is needed.</param>
|
||||
private async Task RunSegmentedStreamAsync(
|
||||
string trackId,
|
||||
TrackMediaResponse firstSegment,
|
||||
long cursor,
|
||||
long totalLength,
|
||||
double? seekPosition,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
byte[]? buffer = null;
|
||||
var segment = firstSegment;
|
||||
try
|
||||
{
|
||||
long totalBytesRead = 0;
|
||||
buffer = ArrayPool<byte>.Shared.Rent(MaxBufferSize); // Rent larger buffer to accommodate adaptive sizing
|
||||
int currentBytes;
|
||||
var readTimer = System.Diagnostics.Stopwatch.StartNew();
|
||||
|
||||
do
|
||||
// Seek/refill: reinitialize the active decoder for the header-less continuation ONCE,
|
||||
// before any continuation bytes are fed. Forward-from-zero (seekPosition null) skips this
|
||||
// — its first segment carries the real header the decoder parses. Done here, inside the
|
||||
// single loop, so seek and forward share the same fetch+pump mechanism (no forked path).
|
||||
if (seekPosition is { } resumeAt)
|
||||
{
|
||||
readTimer.Restart();
|
||||
currentBytes = await audio.Stream.ReadAsync(buffer, 0, _currentBufferSize, cancellationToken);
|
||||
readTimer.Stop();
|
||||
|
||||
// Adapt buffer size based on read performance
|
||||
AdaptBufferSize(currentBytes, readTimer.ElapsedMilliseconds);
|
||||
|
||||
if (currentBytes > 0)
|
||||
// The decoder byte-counts the header-less continuation against the bytes REMAINING
|
||||
// from the range start to EOF (total − cursor), not the absolute total — that is what
|
||||
// reinitializeForRangeContinuation expects (StreamDecoder.remainingByteLength). The
|
||||
// loop's own cursor still targets the absolute totalLength for EOF.
|
||||
var remainingBytes = Math.Max(0, totalLength - cursor);
|
||||
var reinitResult = await _audioInterop.ReinitializeFromOffset(PlayerId, remainingBytes, resumeAt);
|
||||
if (!reinitResult.Success)
|
||||
{
|
||||
totalBytesRead += currentBytes;
|
||||
|
||||
// Always slice to the exact number of bytes read. The pooled buffer
|
||||
// is rented at MaxBufferSize and may carry stale bytes past
|
||||
// currentBytes from a prior rental — handing the full array to JS
|
||||
// interop would serialise that garbage into the audio stream.
|
||||
var actualBuffer = buffer.AsSpan(0, currentBytes).ToArray();
|
||||
|
||||
// Process chunk for streaming
|
||||
var chunkResult = await _audioInterop.ProcessStreamingChunk(PlayerId, actualBuffer);
|
||||
if (!chunkResult.Success)
|
||||
{
|
||||
var error = $"Failed to process streaming chunk: {chunkResult.Error}";
|
||||
_logger.LogWarning("Chunk processing failed: {Error}", error);
|
||||
throw new Exception(error);
|
||||
}
|
||||
|
||||
// Update streaming state
|
||||
CanStartStreaming = chunkResult.CanStartStreaming;
|
||||
HeaderParsed = chunkResult.HeaderParsed;
|
||||
BufferedChunks = chunkResult.BufferCount;
|
||||
|
||||
// Set duration from WAV header when available (only set once)
|
||||
if (chunkResult.Duration.HasValue && Duration == null)
|
||||
{
|
||||
Duration = chunkResult.Duration.Value;
|
||||
_logger.LogInformation("Duration set from WAV header: {Duration:F2} seconds", Duration);
|
||||
// Feed the same once-only duration to the play session so it can compute the
|
||||
// completion fraction at close. Safe before/after session open — SetDuration
|
||||
// is a no-op when no session is open and idempotent otherwise.
|
||||
_playTracker?.SetDuration(chunkResult.Duration.Value);
|
||||
}
|
||||
|
||||
// Start playback as soon as we can
|
||||
if (!_streamingPlaybackStarted && CanStartStreaming)
|
||||
{
|
||||
var playbackResult = await _audioInterop.StartStreamingPlayback(PlayerId);
|
||||
if (playbackResult.Success)
|
||||
{
|
||||
_streamingPlaybackStarted = true;
|
||||
IsPlaying = true;
|
||||
IsPaused = false;
|
||||
IsLoaded = true; // Track is loaded and ready to play (even if still downloading)
|
||||
ErrorMessage = null;
|
||||
|
||||
// Open the play session exactly once per load, at the moment playback truly
|
||||
// begins (§2.1). The _sessionOpened guard keeps the SeekBeyondBuffer re-stream
|
||||
// — which re-enters this transition with _streamingPlaybackStarted reset —
|
||||
// from opening a second session for the same play. Duration may already be
|
||||
// known from a prior chunk, so re-feed it after opening.
|
||||
if (!_sessionOpened && _currentTrackId is { } trackKey)
|
||||
{
|
||||
_sessionOpened = true;
|
||||
_playTracker?.OnPlaybackStarted(trackKey);
|
||||
if (Duration is { } d)
|
||||
_playTracker?.SetDuration(d);
|
||||
}
|
||||
|
||||
await NotifyStateChanged(); // Immediate notification for critical state change
|
||||
}
|
||||
else
|
||||
{
|
||||
var technicalError = $"Failed to start streaming playback: {playbackResult.Error}";
|
||||
_logger.LogError("Failed to start playback: {Error}", technicalError);
|
||||
ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(technicalError);
|
||||
}
|
||||
}
|
||||
|
||||
// Update progress
|
||||
if (audio.ContentLength > 0)
|
||||
{
|
||||
LoadProgress = Math.Min(1.0, (double)totalBytesRead / audio.ContentLength);
|
||||
}
|
||||
|
||||
await ThrottledNotifyStateChanged();
|
||||
throw new Exception($"Failed to reinitialize for offset streaming: {reinitResult.Error}");
|
||||
}
|
||||
} while (currentBytes > 0);
|
||||
}
|
||||
|
||||
// Notify the JS decoder that the stream is finished. When the server omits
|
||||
// Content-Length the StreamDecoder cannot determine completion via byte counting
|
||||
// alone; this explicit signal ensures the tail-decoding path (streamComplete=true)
|
||||
// fires regardless of whether Content-Length was present.
|
||||
buffer = ArrayPool<byte>.Shared.Rent(MaxBufferSize); // larger rental to fit adaptive sizing
|
||||
var readTimer = System.Diagnostics.Stopwatch.StartNew();
|
||||
|
||||
// Segment loop. Each iteration fully consumes one bounded 206 body, advancing the cursor by
|
||||
// the bytes received. The next segment is fetched only when the scheduler is below
|
||||
// high-water (the inter-segment gate). EOF is the cursor reaching totalLength, or a short
|
||||
// segment (server returned fewer bytes than requested — the final slice).
|
||||
while (true)
|
||||
{
|
||||
long segmentBytesRead = 0;
|
||||
int currentBytes;
|
||||
do
|
||||
{
|
||||
readTimer.Restart();
|
||||
currentBytes = await segment.Stream.ReadAsync(buffer, 0, _currentBufferSize, cancellationToken);
|
||||
readTimer.Stop();
|
||||
|
||||
AdaptBufferSize(currentBytes, readTimer.ElapsedMilliseconds);
|
||||
|
||||
if (currentBytes > 0)
|
||||
{
|
||||
segmentBytesRead += currentBytes;
|
||||
|
||||
// Slice to the exact bytes read: the pooled buffer is rented at MaxBufferSize
|
||||
// and may carry stale bytes past currentBytes from a prior rental — handing the
|
||||
// full array to JS would serialise that garbage into the audio stream.
|
||||
var actualBuffer = buffer.AsSpan(0, currentBytes).ToArray();
|
||||
|
||||
var chunkResult = await _audioInterop.ProcessStreamingChunk(PlayerId, actualBuffer);
|
||||
if (!chunkResult.Success)
|
||||
{
|
||||
var error = $"Failed to process streaming chunk: {chunkResult.Error}";
|
||||
_logger.LogWarning("Chunk processing failed: {Error}", error);
|
||||
throw new Exception(error);
|
||||
}
|
||||
|
||||
CanStartStreaming = chunkResult.CanStartStreaming;
|
||||
HeaderParsed = chunkResult.HeaderParsed;
|
||||
BufferedChunks = chunkResult.BufferCount;
|
||||
|
||||
// Set duration from header when available (only set once)
|
||||
if (chunkResult.Duration.HasValue && Duration == null)
|
||||
{
|
||||
Duration = chunkResult.Duration.Value;
|
||||
_logger.LogInformation("Duration set from header: {Duration:F2} seconds", Duration);
|
||||
// Feed the once-only duration to the play session for the completion
|
||||
// fraction. No-op when no session is open; idempotent otherwise.
|
||||
_playTracker?.SetDuration(chunkResult.Duration.Value);
|
||||
}
|
||||
|
||||
// Start playback as soon as we can — at the min-buffer threshold, exactly as
|
||||
// before (C2: first audio is not gated on the segment boundary; the first
|
||||
// segment alone clears the threshold).
|
||||
if (!_streamingPlaybackStarted && CanStartStreaming)
|
||||
{
|
||||
await TryStartPlaybackAsync();
|
||||
}
|
||||
|
||||
// Progress against the total file length (cursor + bytes consumed so far).
|
||||
if (totalLength > 0)
|
||||
{
|
||||
LoadProgress = Math.Min(1.0, (double)(cursor + segmentBytesRead) / totalLength);
|
||||
}
|
||||
|
||||
await ThrottledNotifyStateChanged();
|
||||
|
||||
// Per-chunk back-pressure — the bound that actually holds for high-density codecs.
|
||||
// The inter-segment gate alone is matched to WAV's byte density (~24 s of audio per
|
||||
// 4 MB segment) but NOT to Opus: at 320 kbps a 4 MB segment is ~100 s of decodable
|
||||
// audio. The inner loop has the whole segment's bytes already in hand, so with no
|
||||
// network wait to pace it, it would decode the ENTIRE segment eagerly — piling tens
|
||||
// of MB of decoded f32 PCM AHEAD of a playhead that has barely moved, before the
|
||||
// inter-segment gate ever runs. With HW accel off that lookahead lives in main-
|
||||
// process RAM, and the byte ceiling cannot save us because nothing on this path
|
||||
// polls it. So drain to low-water per chunk once the scheduler is over high-water.
|
||||
//
|
||||
// Gated on _streamingPlaybackStarted so this can NEVER block first audio (C2): until
|
||||
// playback starts the playhead does not advance, so the forward fill would never
|
||||
// drain and the loop would deadlock. The 30 s high-water sits far above the
|
||||
// 6-buffer playback-start minimum, so in practice the gate is not even reached
|
||||
// before playback begins — the guard is the correctness backstop, not the common
|
||||
// case. Reads the piggybacked flag (no extra interop hop) to DECIDE to drain; the
|
||||
// drain helper then polls IsProductionPaused — the same steady-state-reads-flag /
|
||||
// throttled-state-polls split the inter-segment gate uses.
|
||||
if (_streamingPlaybackStarted && chunkResult.ProductionPaused)
|
||||
{
|
||||
await DrainBackpressureAsync(cancellationToken);
|
||||
}
|
||||
}
|
||||
} while (currentBytes > 0);
|
||||
|
||||
// Segment fully consumed; advance the cursor and release this segment's stream/socket
|
||||
// before deciding whether to fetch the next. Disposing here keeps exactly one segment's
|
||||
// raw bytes resident at a time.
|
||||
cursor += segmentBytesRead;
|
||||
segment.Dispose();
|
||||
segment = null!;
|
||||
|
||||
// EOF: cursor reached the total file length. This is the sole forward-EOF condition.
|
||||
// A short segment body (segmentBytesRead < SegmentSizeBytes) is NOT an EOF signal —
|
||||
// the inner read loop fully drains the HTTP body, so a short body means the server
|
||||
// sent fewer bytes than the bounded range we requested. While cursor < totalLength that
|
||||
// can only be a connection drop / truncated stream, NOT the file tail — route it to
|
||||
// the same clean-failure recovery as a fetch error rather than silently completing.
|
||||
var reachedTotal = totalLength > 0 && cursor >= totalLength;
|
||||
if (reachedTotal)
|
||||
{
|
||||
break;
|
||||
}
|
||||
|
||||
// Guard: if the body was short but we haven't reached totalLength, the stream was
|
||||
// truncated mid-segment (connection drop / premature close). Surface as an error so
|
||||
// the scheduler is halted rather than left to drain its buffered tail into a false end.
|
||||
if (segmentBytesRead < SegmentSizeBytes)
|
||||
{
|
||||
throw new Exception(
|
||||
$"Stream truncated at byte {cursor} of {totalLength}: received {segmentBytesRead} bytes " +
|
||||
$"but expected up to {SegmentSizeBytes} and have not reached EOF");
|
||||
}
|
||||
|
||||
// Inter-segment back-pressure gate (Phase 21.2 fill signal, gating SEGMENT FETCH). Do not
|
||||
// fetch the next segment while the scheduler is over high-water; wait until it drains
|
||||
// below low-water. Because the browser only buffers bounded segments and we hold off
|
||||
// requesting the next one, raw network memory stays at ~one segment. Shares the same
|
||||
// drain helper as the per-chunk gate above. No _streamingPlaybackStarted guard is needed
|
||||
// here (unlike the per-chunk gate): reaching this point means a full segment was consumed,
|
||||
// which is ~24 s (WAV) / ~100 s (Opus) of audio — far past the 6-buffer playback-start
|
||||
// minimum — so playback is always running by now and the fill can drain. A file that fits
|
||||
// in one segment hits EOF and breaks above, never reaching this gate.
|
||||
await DrainBackpressureAsync(cancellationToken);
|
||||
|
||||
// Fetch the next bounded segment. The end offset is clamped implicitly by the server
|
||||
// (a request past EOF yields the available tail as a short slice, caught above).
|
||||
var nextEnd = cursor + SegmentSizeBytes - 1;
|
||||
var nextResult = await _trackMediaClient.GetTrackMedia(
|
||||
trackId,
|
||||
byteOffset: cursor,
|
||||
byteEnd: nextEnd,
|
||||
format: _currentFormat,
|
||||
cancellationToken: cancellationToken);
|
||||
if (!nextResult.Success || nextResult.Value == null)
|
||||
{
|
||||
var technicalError = nextResult.GetMessage() ?? "Failed to fetch next stream segment";
|
||||
_logger.LogError("Failed to fetch segment at offset {Offset} for {TrackId}: {Error}",
|
||||
cursor, trackId, technicalError);
|
||||
throw new Exception(technicalError);
|
||||
}
|
||||
segment = nextResult.Value;
|
||||
}
|
||||
|
||||
// Notify the JS decoder that the stream is finished. The decoder marks completion by byte
|
||||
// count against the total it was initialized with; this explicit signal flushes the
|
||||
// residual tail and covers the (rare) case where the total was unknown.
|
||||
await _audioInterop.MarkStreamCompleteAsync(PlayerId);
|
||||
|
||||
// Mark as fully loaded
|
||||
// Complete-without-start fallback: if the track's total decodable audio never crossed the
|
||||
// start threshold (e.g. total Opus audio < 1s lead, or WAV < 6 buffers), the in-loop
|
||||
// CanStartStreaming check never fired and _streamingPlaybackStarted is still false. Now that
|
||||
// streamComplete is set on the JS scheduler, calling StartStreamingPlayback lets it drain
|
||||
// the accumulated buffers and fires onPlaybackEnded exactly once — same transition the
|
||||
// normal path uses, so session/_sessionOpened/Duration handling is identical.
|
||||
if (!_streamingPlaybackStarted)
|
||||
{
|
||||
await TryStartPlaybackAsync();
|
||||
}
|
||||
|
||||
LoadProgress = 1.0;
|
||||
await NotifyStateChanged();
|
||||
}
|
||||
@@ -427,7 +830,7 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
StreamingErrorHandler.LogError(_logger, ex, "StreamAudioWithEarlyPlayback");
|
||||
StreamingErrorHandler.LogError(_logger, ex, "RunSegmentedStreamAsync");
|
||||
ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(ex.Message);
|
||||
LoadProgress = 0;
|
||||
IsLoaded = false;
|
||||
@@ -437,6 +840,8 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
|
||||
}
|
||||
finally
|
||||
{
|
||||
// Release the last segment (if a fetch failed mid-loop it may still be held) and the buffer.
|
||||
segment?.Dispose();
|
||||
if (buffer != null)
|
||||
{
|
||||
ArrayPool<byte>.Shared.Return(buffer);
|
||||
@@ -444,6 +849,46 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Call <c>StartStreamingPlayback</c> on the JS player and apply the resulting state transitions.
|
||||
/// This is the single playback-start transition shared by the in-loop threshold path and the
|
||||
/// completion-path fallback — both callers set the guard and apply session/Duration handling
|
||||
/// identically so neither path diverges.
|
||||
/// </summary>
|
||||
private async Task TryStartPlaybackAsync()
|
||||
{
|
||||
var playbackResult = await _audioInterop.StartStreamingPlayback(PlayerId);
|
||||
if (playbackResult.Success)
|
||||
{
|
||||
_streamingPlaybackStarted = true;
|
||||
IsPlaying = true;
|
||||
IsPaused = false;
|
||||
IsLoaded = true; // loaded and ready, even while still downloading
|
||||
ErrorMessage = null;
|
||||
|
||||
// Open the play session exactly once per load, at the moment playback
|
||||
// truly begins (§2.1). The _sessionOpened guard keeps a seek/refill
|
||||
// re-stream — which re-enters this transition with
|
||||
// _streamingPlaybackStarted reset — from opening a second session for
|
||||
// the same play. Duration may already be known, so re-feed it.
|
||||
if (!_sessionOpened && _currentTrackId is { } trackKey)
|
||||
{
|
||||
_sessionOpened = true;
|
||||
_playTracker?.OnPlaybackStarted(trackKey);
|
||||
if (Duration is { } d)
|
||||
_playTracker?.SetDuration(d);
|
||||
}
|
||||
|
||||
await NotifyStateChanged(); // immediate — critical state change
|
||||
}
|
||||
else
|
||||
{
|
||||
var technicalError = $"Failed to start streaming playback: {playbackResult.Error}";
|
||||
_logger.LogError("Failed to start playback: {Error}", technicalError);
|
||||
ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(technicalError);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// In streaming mode, Stop fully resets to Idle state since audio data is consumed.
|
||||
/// This is equivalent to Unload for streaming playback.
|
||||
@@ -515,6 +960,10 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
|
||||
return;
|
||||
}
|
||||
|
||||
// Capture into a non-null local: _currentTrackId is the field a track-switch could clear, but
|
||||
// this seek operates against the track loaded NOW; the segment loop needs a stable id.
|
||||
var trackId = _currentTrackId;
|
||||
|
||||
IsSeekingBeyondBuffer = true;
|
||||
|
||||
// Cancel the current streaming loop AND wait for it to fully exit before
|
||||
@@ -538,46 +987,68 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
|
||||
await DrainActiveStreamingTaskAsync();
|
||||
oldCts?.Dispose();
|
||||
|
||||
// Single-writer discipline (C6/AC8): all three failure exits must share the same guard.
|
||||
// TrackMediaClient.GetTrackMedia swallows OperationCanceledException and returns
|
||||
// Success==false, so a superseded seek lands in the media-fetch-fail branch below
|
||||
// rather than in the OCE catch. Without the guard those branches would call
|
||||
// RecoverFromFailedRefill — running clearForSeek + setPlaybackOffset against the player
|
||||
// state the NEWER seek now owns. A local predicate keeps all three exits symmetric so a
|
||||
// future exit cannot forget the check.
|
||||
bool IsStillActiveSeek() => ReferenceEquals(_streamingCancellation, seekCts);
|
||||
|
||||
try
|
||||
{
|
||||
// Update UI immediately
|
||||
CurrentTime = seekPosition;
|
||||
await NotifyStateChanged();
|
||||
|
||||
// Request new stream from offset
|
||||
var mediaResult = await _trackMediaClient.GetTrackMedia(
|
||||
_currentTrackId,
|
||||
// Request the FIRST bounded segment from the resolved offset (Direction B — converged with
|
||||
// the forward path). Reuse the format the initial load resolved to (_currentFormat): an
|
||||
// Opus seek must come back as Opus bytes so the cached setup header + page-aligned
|
||||
// byteOffset (resolved JS-side from the Opus seek index) match the continuation; WAV resolves
|
||||
// its offset from the header — one seam, format-appropriate math (AC9 / §3.4a C). The
|
||||
// segment loop then continues forward segmentation from this offset exactly as a fresh load
|
||||
// does from 0 — no forked fetch path (C1/C5).
|
||||
var firstSegment = await _trackMediaClient.GetTrackMedia(
|
||||
trackId,
|
||||
byteOffset,
|
||||
byteEnd: byteOffset + SegmentSizeBytes - 1,
|
||||
format: _currentFormat,
|
||||
cancellationToken: seekCts.Token);
|
||||
if (!mediaResult.Success || mediaResult.Value == null)
|
||||
if (!firstSegment.Success || firstSegment.Value == null)
|
||||
{
|
||||
var technicalError = mediaResult.GetMessage() ?? "Failed to load audio from position";
|
||||
var technicalError = firstSegment.GetMessage() ?? "Failed to load audio from position";
|
||||
_logger.LogError("Failed to get track media from offset {Offset}: {Error}", byteOffset, technicalError);
|
||||
ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(technicalError);
|
||||
IsSeekingBeyondBuffer = false;
|
||||
// Guard: a superseded seek must NOT touch shared state. The newer seek owns teardown.
|
||||
if (IsStillActiveSeek())
|
||||
{
|
||||
await RecoverFromFailedRefill(seekPosition, StreamingErrorHandler.GetUserFriendlyMessage(technicalError));
|
||||
}
|
||||
else
|
||||
{
|
||||
_logger.LogDebug("Media-fetch failed on superseded seek to {Position} — newer seek owns state, skipping recovery", seekPosition);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
using var audio = mediaResult.Value;
|
||||
var audio = firstSegment.Value;
|
||||
// The absolute EOF boundary the segment loop's cursor targets. On a 206 the Content-Range
|
||||
// carries the file total; on a 200 (single-segment file) fall back to cursor + body length.
|
||||
var totalLength = audio.TotalLength ?? (byteOffset + audio.ContentLength);
|
||||
|
||||
// Reinitialize JS player for offset streaming
|
||||
var reinitResult = await _audioInterop.ReinitializeFromOffset(PlayerId, audio.ContentLength, seekPosition);
|
||||
if (!reinitResult.Success)
|
||||
{
|
||||
_logger.LogError("Failed to reinitialize for offset streaming: {Error}", reinitResult.Error);
|
||||
ErrorMessage = "Failed to seek to position";
|
||||
IsSeekingBeyondBuffer = false;
|
||||
return;
|
||||
}
|
||||
|
||||
// Reset streaming state for new stream
|
||||
// Reset streaming state for the new stream. The decoder reinit for the header-less
|
||||
// continuation happens INSIDE RunSegmentedStreamAsync (seekPosition non-null), so seek and
|
||||
// forward share one fetch+pump+reinit mechanism. A reinit failure there throws and lands in
|
||||
// the catch below, which recovers when still the active seek — the same clean-failure path
|
||||
// (AC6) the old explicit reinit branch had, now unified with the fetch-failure path.
|
||||
_streamingPlaybackStarted = false;
|
||||
CanStartStreaming = false;
|
||||
HeaderParsed = false;
|
||||
BufferedChunks = 0;
|
||||
|
||||
// Stream audio from offset
|
||||
_activeStreamingTask = StreamAudioWithEarlyPlayback(audio, seekCts.Token);
|
||||
// Stream from offset via the shared segment loop. Ownership of `audio` transfers to it.
|
||||
_activeStreamingTask = RunSegmentedStreamAsync(
|
||||
trackId, audio, cursor: byteOffset, totalLength, seekPosition, seekCts.Token);
|
||||
await _activeStreamingTask;
|
||||
|
||||
IsSeekingBeyondBuffer = false;
|
||||
@@ -588,22 +1059,65 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
|
||||
// still the active seek — if _streamingCancellation has been replaced, a
|
||||
// newer seek is in progress and owns the flag.
|
||||
_logger.LogDebug("Seek beyond buffer cancelled");
|
||||
if (ReferenceEquals(_streamingCancellation, seekCts))
|
||||
if (IsStillActiveSeek())
|
||||
{
|
||||
IsSeekingBeyondBuffer = false;
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
// A refill fetch can fail deep into a long mix (the listener didn't initiate it). Recover
|
||||
// into a clean paused-but-loaded state (AC6) rather than leaving the starved scheduler to
|
||||
// fire a silent false end. Only when we are still the active seek — a superseding seek owns
|
||||
// the state and the OCE catch above handles its own teardown.
|
||||
_logger.LogError(ex, "Error during seek beyond buffer to position {Position}", seekPosition);
|
||||
ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(ex.Message);
|
||||
IsSeekingBeyondBuffer = false;
|
||||
await NotifyStateChanged();
|
||||
if (IsStillActiveSeek())
|
||||
{
|
||||
await RecoverFromFailedRefill(seekPosition, StreamingErrorHandler.GetUserFriendlyMessage(ex.Message));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Single method to reset all state - called by both Stop and Unload.
|
||||
/// Clean-failure recovery for a window-miss refill (Phase 21.3 / AC6). A backward seek past the
|
||||
/// retained tail re-fetches via the existing Range path; that mid-stream fetch the listener did not
|
||||
/// initiate can fail deep into a long mix. When it does, the pre-seek loop has already been
|
||||
/// cancelled and drained, but the JS scheduler is still holding stale pre-seek buffers and still
|
||||
/// "playing" — left alone it drains them and fires a silent false end (the wedged/starved state AC6
|
||||
/// forbids). This halts the scheduler into a paused-but-loaded state at <paramref name="seekPosition"/>,
|
||||
/// surfaces a clear error, and leaves the track loaded so the listener can retry the seek or pick
|
||||
/// another track. Mirrors <c>PlaybackScheduler.playFromPosition</c>'s end-of-buffer recovery: stop
|
||||
/// pretending to play.
|
||||
/// </summary>
|
||||
private async Task RecoverFromFailedRefill(double seekPosition, string userFacingError)
|
||||
{
|
||||
// Halt the starved scheduler JS-side (stop sources, drop stale buffers, anchor at the target).
|
||||
// Best-effort: if even this interop fails the player is no worse off, and we still surface the
|
||||
// error and settle C# state below.
|
||||
var recovered = await _audioInterop.RecoverFromFailedRefill(PlayerId, seekPosition);
|
||||
if (!recovered.Success)
|
||||
{
|
||||
_logger.LogWarning("Refill-failure recovery interop did not succeed: {Error}", recovered.Error);
|
||||
}
|
||||
|
||||
// Settle C# into the matching recoverable state: not playing, paused at the target, still loaded
|
||||
// and still in streaming mode. IsLoaded = true and IsStreamingMode = true are both load-bearing —
|
||||
// the "paused-but-loaded" contract lets the listener retry the seek (Seek early-returns when
|
||||
// !IsLoaded || !IsStreamingMode), resume via TogglePlayPause, or pick another track. Resetting
|
||||
// either to false would wedge at least one of the three retry routes (AC6 / Phase 21.3).
|
||||
ErrorMessage = userFacingError;
|
||||
IsPlaying = false;
|
||||
IsPaused = true;
|
||||
IsLoaded = true;
|
||||
IsStreamingMode = true;
|
||||
CurrentTime = seekPosition;
|
||||
IsSeekingBeyondBuffer = false;
|
||||
await NotifyStateChanged();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Single method to reset all state - called by both Stop and Unload, and as the prologue of a new
|
||||
/// load.
|
||||
/// </summary>
|
||||
private async Task ResetToIdle()
|
||||
{
|
||||
@@ -653,6 +1167,7 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
|
||||
_streamingPlaybackStarted = false;
|
||||
IsSeekingBeyondBuffer = false;
|
||||
_currentTrackId = null;
|
||||
_currentFormat = AudioFormat.Lossless;
|
||||
|
||||
await NotifyStateChanged();
|
||||
}
|
||||
@@ -691,6 +1206,27 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Block the segment loop while the scheduler's decoded forward fill is over high-water, resuming
|
||||
/// once it drains below low-water (Phase 21.2 hysteresis). Shared by the per-chunk gate (inside a
|
||||
/// segment) and the inter-segment gate so both honor identical drain discipline — a guard present on
|
||||
/// one path and absent on the other would let one path overshoot the memory bound.
|
||||
/// <para>
|
||||
/// The poll awaits on <paramref name="cancellationToken"/>, so a track switch/seek mid-wait throws
|
||||
/// OCE and unwinds through the existing drain discipline (C6). UC5: a user pause freezes the playhead
|
||||
/// so the fill never drains on its own — hold here until playback resumes (IsPaused clears) OR the
|
||||
/// fill drains. Returns immediately when nothing is throttled (the steady-state common case).
|
||||
/// </para>
|
||||
/// </summary>
|
||||
private async Task DrainBackpressureAsync(CancellationToken cancellationToken)
|
||||
{
|
||||
while (IsPaused || await _audioInterop.IsProductionPaused(PlayerId))
|
||||
{
|
||||
cancellationToken.ThrowIfCancellationRequested();
|
||||
await Task.Delay(BackpressurePollMs, cancellationToken);
|
||||
}
|
||||
}
|
||||
|
||||
private async Task ThrottledNotifyStateChanged()
|
||||
{
|
||||
var now = DateTime.UtcNow;
|
||||
|
||||
@@ -160,6 +160,47 @@ public sealed class WaveformVisualizerControlState
|
||||
/// </summary>
|
||||
public event Action? Changed;
|
||||
|
||||
// Whether the one-time, capability-driven default has been applied this session. The default-set
|
||||
// (lava off when the browser has no WebGL hardware acceleration) must run exactly once — on the
|
||||
// first interactive render, before the listener has touched a toggle — so it sets the *initial
|
||||
// default* and never clobbers a later explicit in-session toggle. Scoped with the rest of this
|
||||
// state, so it survives SPA navigation (a remounted visualizer does not re-apply) and resets on a
|
||||
// fresh page load (F5 re-probes).
|
||||
private bool _capabilityDefaultApplied;
|
||||
|
||||
/// <summary>
|
||||
/// Applies the hardware-capability default exactly once per session: when the browser reports no
|
||||
/// WebGL hardware acceleration, the lava subsystem (the expensive, main-thread software-rendered
|
||||
/// part that starves audio decode) defaults <c>off</c> while the waveform stays <c>on</c>. With
|
||||
/// acceleration present this is a no-op — lava keeps its <see cref="DefaultLavaEnabled"/> on-state.
|
||||
///
|
||||
/// <para>
|
||||
/// Idempotent and guarded: only the FIRST call this session has any effect, so it sets the initial
|
||||
/// default and never overrides a listener's explicit toggle (the control remains fully functional —
|
||||
/// a user on a software renderer may re-enable lava at their own risk). Mutates, coerces
|
||||
/// Theater Mode, then raises <see cref="Changed"/> once so the controls UI, the visualizer bridge,
|
||||
/// and the Theater observers all reflect the default in a single cycle. Called by the visualizer
|
||||
/// bridge on first interactive render, once JS interop (the probe) is available.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
/// <param name="hardwareAccelerated">
|
||||
/// The probe result — <c>true</c> when WebGL hardware acceleration is present (or the renderer is
|
||||
/// unknown/masked, favoring the common case), <c>false</c> only on a positive software-renderer
|
||||
/// match or total WebGL failure.
|
||||
/// </param>
|
||||
public void ApplyCapabilityDefault(bool hardwareAccelerated)
|
||||
{
|
||||
if (_capabilityDefaultApplied) return;
|
||||
_capabilityDefaultApplied = true;
|
||||
|
||||
// Accelerated (or unknown): keep the as-shipped defaults — no observer churn.
|
||||
if (hardwareAccelerated) return;
|
||||
|
||||
LavaEnabled = false; // the expensive subsystem off; WaveformEnabled stays at its default (true).
|
||||
CoerceTheaterMode();
|
||||
NotifyChanged();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Enforces the Theater-Mode invariant: Theater Mode cannot remain on when both visualizer
|
||||
/// subsystems are off (there is nothing to go to theater FOR). Call this after mutating
|
||||
|
||||
@@ -14,6 +14,13 @@ public static class Startup
|
||||
services.AddScoped<DarkModeSettings>();
|
||||
services.AddScoped<DarkModeCookieService>();
|
||||
|
||||
// Public-site listener settings (Phase 18 wave 18.6). PublicSiteSettings is the generalized,
|
||||
// prerender-seeded preference object (today: streaming quality); SettingsCookieService writes the
|
||||
// 365-day cookie at runtime. Same scoped lifetime + cookie seam as the dark-mode pair above, so the
|
||||
// preference survives SPA nav within a session and seeds the next visit's prerender.
|
||||
services.AddScoped<PublicSiteSettings>();
|
||||
services.AddScoped<SettingsCookieService>();
|
||||
|
||||
// Track Client. The HTTP-backed ITrackDataService is used by both WASM and SSR
|
||||
// prerender — both call DeepDrftAPI over the "DeepDrft.API" client.
|
||||
services.AddScoped<TrackClient>();
|
||||
@@ -35,12 +42,16 @@ public static class Startup
|
||||
// within a session and reset on a fresh page load (see WaveformVisualizerControlState).
|
||||
services.AddScoped<WaveformVisualizerControlState>();
|
||||
|
||||
// Phase 16 anonymous telemetry (client side). BeaconInterop wraps sendBeacon; the play sink and
|
||||
// share tracker fire events through it. The play tracker itself is NOT registered — the player
|
||||
// is not DI-registered, so AudioPlayerProvider constructs the tracker and attaches it. ShareTracker
|
||||
// is scoped so its per-(target,channel) debounce memory lives for the session. AnonIdProvider
|
||||
// (wave 16.3) caches the first-party localStorage listener id; scoped so the cache lives for the
|
||||
// session, warmed when a surface goes interactive (the player provider, the share popover).
|
||||
// Phase 16 anonymous telemetry (client side), transport-resilience split. IEventPoster is the
|
||||
// first-party HttpClient POST used for normal play closes (end/switch/stop) and every share — a
|
||||
// same-origin fetch that privacy/tracking heuristics don't name-match. BeaconInterop wraps
|
||||
// sendBeacon and is retained only for the tab-unload edge. The play sink picks the arm; the share
|
||||
// tracker is fetch-only. The play tracker itself is NOT registered — the player is not
|
||||
// DI-registered, so AudioPlayerProvider constructs the tracker and attaches it. ShareTracker is
|
||||
// scoped so its per-(target,channel) debounce memory lives for the session. AnonIdProvider (wave
|
||||
// 16.3) caches the first-party localStorage listener id; scoped so the cache lives for the session,
|
||||
// warmed when a surface goes interactive (the player provider, the share popover).
|
||||
services.AddScoped<IEventPoster, HttpEventPoster>();
|
||||
services.AddScoped<BeaconInterop>();
|
||||
services.AddScoped<IAnonIdProvider, AnonIdProvider>();
|
||||
services.AddScoped<IPlayEventSink, BeaconPlayEventSink>();
|
||||
|
||||
@@ -25,9 +25,10 @@
|
||||
<script src="_framework/blazor.web.js"></script>
|
||||
<script src=@Assets["_content/MudBlazor/MudBlazor.min.js"]></script>
|
||||
<script type="module">
|
||||
import('./js/settings/settings.js');
|
||||
import('./js/audio/index.js');
|
||||
import('./js/telemetry/beacon.js');
|
||||
import('./js/telemetry/anonid.js');
|
||||
import('./js/session/lifecycle.js');
|
||||
import('./js/session/anonid.js');
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -37,6 +38,7 @@
|
||||
[Inject] public required DarkModeService DarkModeService { get; set; }
|
||||
[Inject] public required SeoEnvironment SeoEnvironment { get; set; }
|
||||
[Inject] public required IWebHostEnvironment HostEnvironment { get; set; }
|
||||
[Inject] public required SettingsService SettingsService { get; set; }
|
||||
|
||||
protected override void OnInitialized()
|
||||
{
|
||||
@@ -45,6 +47,7 @@
|
||||
// Seed the environment-gated robots bridge during prerender; [PersistentState] rounds it to WASM
|
||||
// so both render passes resolve the same default robots (Production → index, else noindex).
|
||||
SeoEnvironment.IsProduction = HostEnvironment.IsProduction();
|
||||
SettingsService.CheckSettings();
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@@ -205,21 +205,26 @@ public class TrackProxyController : ControllerBase
|
||||
|
||||
/// <summary>
|
||||
/// Proxies audio streaming from DeepDrftAPI as a transparent HTTP Range relay.
|
||||
/// Forwards the incoming Range header upstream and relays the upstream status
|
||||
/// (200 full, 206 partial, 416 unsatisfiable) and range-related response headers
|
||||
/// back to the browser verbatim. The proxy does not slice — the upstream already did.
|
||||
/// Forwards the incoming Range header upstream and the optional <c>format</c> selector
|
||||
/// (Phase 18.3 — <c>opus|lossless</c>, threaded the same way the listing params are),
|
||||
/// and relays the upstream status (200 full, 206 partial, 416 unsatisfiable) and
|
||||
/// range-related response headers back to the browser verbatim. The proxy does not
|
||||
/// slice — the upstream already did.
|
||||
/// </summary>
|
||||
[HttpGet("{trackId}")]
|
||||
public async Task<ActionResult> GetTrack(
|
||||
string trackId,
|
||||
[FromQuery] string? format = null,
|
||||
CancellationToken ct = default)
|
||||
{
|
||||
var rangeHeader = Request.Headers.Range.ToString();
|
||||
_logger.LogInformation("Proxying track {TrackId} range '{Range}'", trackId, rangeHeader);
|
||||
_logger.LogInformation("Proxying track {TrackId} range '{Range}' format '{Format}'", trackId, rangeHeader, format);
|
||||
|
||||
var request = new HttpRequestMessage(
|
||||
HttpMethod.Get,
|
||||
$"api/track/{Uri.EscapeDataString(trackId)}");
|
||||
var path = $"api/track/{Uri.EscapeDataString(trackId)}";
|
||||
if (!string.IsNullOrWhiteSpace(format))
|
||||
path += $"?format={Uri.EscapeDataString(format)}";
|
||||
|
||||
var request = new HttpRequestMessage(HttpMethod.Get, path);
|
||||
|
||||
// Forward the browser's Range header upstream so DeepDrftAPI slices the file.
|
||||
// TryAddWithoutValidation avoids RangeHeaderValue reparsing — we relay the raw
|
||||
@@ -355,4 +360,40 @@ public class TrackProxyController : ControllerBase
|
||||
return Content(json, "application/json");
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Proxies a track's Opus seek/setup sidecar (raw bytes) from DeepDrftAPI (Phase 18.3). Unauthenticated,
|
||||
/// same posture as the audio stream forward. The sidecar is a small one-time fetch (≤ ~115 KB), so it is
|
||||
/// buffered and relayed; a 404 (no Opus artifact / no sidecar stored) passes through so the client
|
||||
/// degrades to lossless rather than treating it as an error. The "opus/seekdata" 3-segment route makes a
|
||||
/// collision with the parameterized "{trackId}" audio route impossible.
|
||||
/// </summary>
|
||||
[HttpGet("{trackId}/opus/seekdata")]
|
||||
public async Task<ActionResult> GetOpusSeekData(string trackId, CancellationToken ct = default)
|
||||
{
|
||||
var path = $"api/track/{Uri.EscapeDataString(trackId)}/opus/seekdata";
|
||||
|
||||
HttpResponseMessage upstream;
|
||||
try
|
||||
{
|
||||
upstream = await _upstream.GetAsync(path, HttpCompletionOption.ResponseHeadersRead, ct);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Upstream call to DeepDrftAPI track/{TrackId}/opus/seekdata failed", trackId);
|
||||
return StatusCode(502, "Upstream unavailable");
|
||||
}
|
||||
|
||||
using (upstream)
|
||||
{
|
||||
if (!upstream.IsSuccessStatusCode)
|
||||
{
|
||||
_logger.LogWarning("DeepDrftAPI track/{TrackId}/opus/seekdata returned {Status}", trackId, (int)upstream.StatusCode);
|
||||
return StatusCode((int)upstream.StatusCode);
|
||||
}
|
||||
|
||||
var bytes = await upstream.Content.ReadAsByteArrayAsync(ct);
|
||||
return File(bytes, "application/octet-stream");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,298 @@
|
||||
/**
|
||||
* AudioPlayer window-miss refill tests (Phase 21.3) — the seek-dispatch TRIGGER and the AC6
|
||||
* clean-failure recovery.
|
||||
*
|
||||
* What this pins (the genuinely-new 21.3 work):
|
||||
* - The window-miss TRIGGER. AudioPlayer.seek() routes by whether the target falls inside the
|
||||
* retained window [playbackOffset, playbackOffset + totalDuration]. After 21.1 partial eviction
|
||||
* playbackOffset is the absolute start of the retained back-window tail, so:
|
||||
* * seek back WITHIN the tail -> seekWithinBuffer, NO refetch (UC3 / AC4),
|
||||
* * seek back PAST the tail -> seekBeyondBuffer with the EARLIER resolved offset (UC4 / AC5),
|
||||
* using whichever resolver the active path ships (WAV calculateByteOffset; Opus
|
||||
* resolveOpusByteOffset over the sidecar index),
|
||||
* * seek forward past the decoded end -> seekBeyondBuffer forward, unchanged (UC2/UC5).
|
||||
* - The AC6 recovery. recoverFromFailedRefill() halts the scheduler (clearForSeek), anchors the
|
||||
* offset at the seek target, and leaves the player paused-but-loaded so no silent false end fires.
|
||||
*
|
||||
* The seek dispatch and recovery are pure given the scheduler + active decoder, so they are testable
|
||||
* in Node by white-box-injecting fakes for `scheduler`, `streamDecoder`, and `opusDecoder` (the same
|
||||
* private-field injection idiom the scheduler/Opus tests use). The AudioPlayer constructor itself is
|
||||
* Node-safe: it builds AudioContextManager/StreamDecoder/PlaybackScheduler, none of which touch Web
|
||||
* Audio until initialize(). No AudioContext, no WebCodecs.
|
||||
*
|
||||
* Same harness convention as the sibling tests (no runner in this repo); run a copy from the COMPILED
|
||||
* output so the `.js` import specifiers resolve:
|
||||
*
|
||||
* dotnet build DeepDrftPublic/DeepDrftPublic.csproj
|
||||
* cp DeepDrftPublic/Interop/audio/AudioPlayer.test.ts DeepDrftPublic/wwwroot/js/audio/
|
||||
* node DeepDrftPublic/wwwroot/js/audio/AudioPlayer.test.ts
|
||||
*
|
||||
* A thrown error / non-zero exit signals failure; "ALL <n> TESTS PASSED" signals success.
|
||||
* Excluded from the production tsc build via tsconfig `exclude: Interop/ ** /*.test.ts`.
|
||||
*/
|
||||
|
||||
import { AudioPlayer } from './AudioPlayer.js';
|
||||
import { parseSidecar } from './OpusSidecar.js';
|
||||
import type { OpusSeekData } from './OpusSidecar.js';
|
||||
|
||||
// --- tiny inline harness (no dependencies) ---------------------------------------------------
|
||||
let passed = 0;
|
||||
const failures: string[] = [];
|
||||
function test(name: string, fn: () => void): void {
|
||||
try {
|
||||
fn();
|
||||
passed++;
|
||||
} catch (e) {
|
||||
failures.push(`FAIL: ${name}\n ${(e as Error).message}`);
|
||||
}
|
||||
}
|
||||
function assertEqual(actual: unknown, expected: unknown, msg?: string): void {
|
||||
if (actual !== expected) {
|
||||
throw new Error(`${msg ?? 'assertEqual'}: expected ${String(expected)}, got ${String(actual)}`);
|
||||
}
|
||||
}
|
||||
function assertTrue(cond: boolean, msg?: string): void {
|
||||
if (!cond) throw new Error(msg ?? 'assertTrue failed');
|
||||
}
|
||||
|
||||
// --- fakes -----------------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* A scheduler stand-in exposing only what AudioPlayer.seek / seekWithinBuffer / seekBeyondBuffer /
|
||||
* recoverFromFailedRefill read or call. The retained window is [offset, offset + total]. Records the
|
||||
* methods that mutate so the recovery test can assert the cleanup happened.
|
||||
*/
|
||||
class FakeScheduler {
|
||||
private offset: number;
|
||||
private total: number;
|
||||
// hasBuffers reflects whether the scheduler holds decoded audio. Starts true when total > 0
|
||||
// (a populated window), set to false by clearForSeek() (recovery drains the buffers).
|
||||
private _hasBuffers: boolean;
|
||||
public clearedForSeek = false;
|
||||
public stoppedAllSources = false;
|
||||
public offsetSetTo: number | null = null;
|
||||
constructor(offset: number, total: number) {
|
||||
this.offset = offset;
|
||||
this.total = total;
|
||||
this._hasBuffers = total > 0;
|
||||
}
|
||||
|
||||
getPlaybackOffset(): number { return this.offset; }
|
||||
getTotalDuration(): number { return this.total; }
|
||||
hasBuffers(): boolean { return this._hasBuffers; }
|
||||
stopAllSources(): void { this.stoppedAllSources = true; }
|
||||
// seekWithinBuffer calls playFromPosition only when wasPlaying; isPlaying is false in these
|
||||
// unit constructions, so it is never invoked — present for completeness.
|
||||
playFromPosition(_position: number): void { /* no-op */ }
|
||||
clearForSeek(): void { this.clearedForSeek = true; this._hasBuffers = false; }
|
||||
setPlaybackOffset(o: number): void { this.offset = o; this.offsetSetTo = o; }
|
||||
}
|
||||
|
||||
/** A StreamDecoder stand-in for the WAV path: a format is parsed and byte math is identity-scaled. */
|
||||
class FakeStreamDecoder {
|
||||
private hasFormat: boolean;
|
||||
private bytesPerSecond: number;
|
||||
public requestedOffsetFor: number | null = null;
|
||||
constructor(hasFormat: boolean, bytesPerSecond: number) { this.hasFormat = hasFormat; this.bytesPerSecond = bytesPerSecond; }
|
||||
getFormatInfo(): unknown { return this.hasFormat ? { ok: true } : null; }
|
||||
calculateByteOffset(position: number): number {
|
||||
this.requestedOffsetFor = position;
|
||||
return Math.round(position * this.bytesPerSecond);
|
||||
}
|
||||
}
|
||||
|
||||
function makePlayer(): AudioPlayer {
|
||||
// Constructor is Node-safe (no Web Audio until initialize()).
|
||||
return new AudioPlayer();
|
||||
}
|
||||
|
||||
/** Inject the seek-relevant private fields and put the player in a loaded/streaming/playing state. */
|
||||
function arm(
|
||||
player: AudioPlayer,
|
||||
opts: {
|
||||
scheduler: FakeScheduler;
|
||||
duration: number;
|
||||
streamDecoder?: FakeStreamDecoder;
|
||||
opusDecoder?: object | null;
|
||||
sidecar?: OpusSeekData | null;
|
||||
}
|
||||
): void {
|
||||
const priv = player as unknown as Record<string, unknown>;
|
||||
priv.scheduler = opts.scheduler;
|
||||
priv.duration = opts.duration;
|
||||
priv.isStreamingMode = true;
|
||||
priv.isPlaying = false; // keep dispatch pure (no real playFromPosition needed)
|
||||
if (opts.streamDecoder) priv.streamDecoder = opts.streamDecoder;
|
||||
priv.opusDecoder = opts.opusDecoder ?? null;
|
||||
priv.activeOpusSidecar = opts.sidecar ?? null;
|
||||
}
|
||||
|
||||
/** Read back private fields the recovery sets. */
|
||||
function priv(player: AudioPlayer): Record<string, unknown> {
|
||||
return player as unknown as Record<string, unknown>;
|
||||
}
|
||||
|
||||
// A minimal real sidecar (parsed) so the Opus resolver returns a deterministic page offset.
|
||||
// Index: t=0 -> byte 4096, t=1s -> byte 9000 (granule uses 48 kHz + preSkip).
|
||||
function makeOpusSidecar(): OpusSeekData {
|
||||
const setup = [0x4f, 0x70, 0x75, 0x73, 0x48, 0x65, 0x61, 0x64];
|
||||
const SEEK_INDEX_HEADER_SIZE = 24;
|
||||
const SEEK_POINT_SIZE = 16;
|
||||
const preSkip = 312;
|
||||
const points = [
|
||||
{ granule: preSkip, byteOffset: 4096 }, // t = 0
|
||||
{ granule: preSkip + 48000, byteOffset: 9000 }, // t = 1 s
|
||||
];
|
||||
const total = 4 + setup.length + SEEK_INDEX_HEADER_SIZE + points.length * SEEK_POINT_SIZE;
|
||||
const bytes = new Uint8Array(total);
|
||||
const view = new DataView(bytes.buffer);
|
||||
view.setUint32(0, setup.length, true);
|
||||
bytes.set(setup, 4);
|
||||
let p = 4 + setup.length;
|
||||
const writeU64 = (off: number, v: number) => {
|
||||
view.setUint32(off, v >>> 0, true);
|
||||
view.setUint32(off + 4, Math.floor(v / 0x100000000), true);
|
||||
};
|
||||
writeU64(p, 500_000);
|
||||
view.setFloat64(p + 8, 100, true);
|
||||
view.setUint32(p + 16, points.length, true);
|
||||
view.setUint16(p + 20, preSkip, true);
|
||||
p += SEEK_INDEX_HEADER_SIZE;
|
||||
for (const pt of points) { writeU64(p, pt.granule); writeU64(p + 8, pt.byteOffset); p += SEEK_POINT_SIZE; }
|
||||
const parsed = parseSidecar(bytes);
|
||||
if (!parsed) throw new Error('test setup: sidecar failed to parse');
|
||||
return parsed;
|
||||
}
|
||||
|
||||
// --- TRIGGER: within-window vs past-tail vs forward ------------------------------------------
|
||||
|
||||
// UC3 / AC4: a backward seek INTO the retained tail resolves from buffer — NO seekBeyondBuffer,
|
||||
// NO refetch signal. Window is [30, 60); target 40 is inside.
|
||||
test('seek back within retained tail resolves in-buffer (no refetch) — AC4', () => {
|
||||
const player = makePlayer();
|
||||
const scheduler = new FakeScheduler(30, 30); // retained window [30, 60)
|
||||
arm(player, { scheduler, duration: 120, streamDecoder: new FakeStreamDecoder(true, 1000) });
|
||||
|
||||
const result = player.seek(40);
|
||||
assertEqual(result.success, true, 'seek succeeds');
|
||||
assertEqual(result.seekBeyondBuffer ?? false, false, 'within-window seek does NOT signal a refetch');
|
||||
// No clearForSeek / no offset request — the retained window served it.
|
||||
assertEqual(scheduler.clearedForSeek, false, 'no clearForSeek for an in-buffer seek');
|
||||
});
|
||||
|
||||
// UC4 / AC5 (WAV): a backward seek PAST the retained tail signals a refill at the EARLIER resolved
|
||||
// offset, using the WAV resolver. Window [30, 60); target 10 is before the tail.
|
||||
test('seek back past retained tail refetches at the WAV-resolved earlier offset — AC5', () => {
|
||||
const player = makePlayer();
|
||||
const scheduler = new FakeScheduler(30, 30); // retained window [30, 60)
|
||||
const wav = new FakeStreamDecoder(true, 2000); // 2000 bytes/sec
|
||||
arm(player, { scheduler, duration: 120, streamDecoder: wav });
|
||||
|
||||
const result = player.seek(10); // earlier than the retained tail start (30)
|
||||
assertEqual(result.success, true, 'seek succeeds');
|
||||
assertEqual(result.seekBeyondBuffer, true, 'past-tail back seek signals a refill (window miss)');
|
||||
assertEqual(wav.requestedOffsetFor, 10, 'WAV resolver consulted for the EARLIER target');
|
||||
assertEqual(result.byteOffset, 20000, 'refill offset is the WAV-resolved earlier byte offset');
|
||||
});
|
||||
|
||||
// UC4 / AC5 (Opus): the same window miss on the Opus path uses resolveOpusByteOffset over the
|
||||
// sidecar index (the live seek), not WAV byte math. Target 0.3 s resolves to the t=0 page (4096).
|
||||
test('seek back past retained tail refetches at the Opus index-resolved offset — AC5', () => {
|
||||
const player = makePlayer();
|
||||
const scheduler = new FakeScheduler(30, 30); // retained window [30, 60)
|
||||
arm(player, {
|
||||
scheduler,
|
||||
duration: 100,
|
||||
opusDecoder: {}, // presence routes seekBeyondBuffer down the Opus branch
|
||||
sidecar: makeOpusSidecar(),
|
||||
});
|
||||
|
||||
const result = player.seek(0.3); // earlier than the retained tail (30) -> window miss
|
||||
assertEqual(result.success, true, 'seek succeeds');
|
||||
assertEqual(result.seekBeyondBuffer, true, 'past-tail back seek signals a refill on Opus too');
|
||||
assertEqual(result.byteOffset, 4096, 'Opus index resolved the t=0 page start for the earlier target');
|
||||
// The landing time of the resolved page is captured for the decoder lead-trim (AC9 reuse).
|
||||
assertEqual(priv(player)._seekLandingTime, 0, 'landing time of the resolved page captured for lead-trim');
|
||||
});
|
||||
|
||||
// UC2/UC5: a forward seek past the decoded end still routes to seekBeyondBuffer forward, unchanged.
|
||||
test('forward seek past decoded end still routes to seekBeyondBuffer (unchanged)', () => {
|
||||
const player = makePlayer();
|
||||
const scheduler = new FakeScheduler(30, 30); // decoded [30, 60)
|
||||
const wav = new FakeStreamDecoder(true, 1500);
|
||||
arm(player, { scheduler, duration: 120, streamDecoder: wav });
|
||||
|
||||
const result = player.seek(90); // past the decoded end (60)
|
||||
assertEqual(result.seekBeyondBuffer, true, 'forward-beyond-buffer still signals a fetch');
|
||||
assertEqual(wav.requestedOffsetFor, 90, 'forward target resolved through the same WAV resolver');
|
||||
assertEqual(result.byteOffset, 135000, 'forward offset is the resolved later byte offset');
|
||||
});
|
||||
|
||||
// --- AC6: clean-failure recovery -------------------------------------------------------------
|
||||
|
||||
// A failed refill must leave the player recoverable: scheduler halted (clearForSeek), offset anchored
|
||||
// at the seek target, paused-but-loaded — never a starved "playing" scheduler that fires a false end.
|
||||
test('recoverFromFailedRefill halts the scheduler and leaves a paused-but-loaded state — AC6', () => {
|
||||
const player = makePlayer();
|
||||
const scheduler = new FakeScheduler(30, 30);
|
||||
arm(player, { scheduler, duration: 120, streamDecoder: new FakeStreamDecoder(true, 1000) });
|
||||
// Simulate the pre-failure "playing" state the drained pre-seek loop leaves behind.
|
||||
priv(player).isPlaying = true;
|
||||
priv(player).isPaused = false;
|
||||
priv(player).streamingStarted = true;
|
||||
|
||||
const result = player.recoverFromFailedRefill(15);
|
||||
assertEqual(result.success, true, 'recovery succeeds');
|
||||
assertTrue(scheduler.clearedForSeek, 'stale buffers dropped (no false end can fire)');
|
||||
assertEqual(scheduler.offsetSetTo, 15, 'offset anchored at the seek target for a retry');
|
||||
assertEqual(priv(player).isPlaying, false, 'not playing after recovery');
|
||||
assertEqual(priv(player).isPaused, true, 'paused after recovery');
|
||||
assertEqual(priv(player).pausePosition, 15, 'pause anchor is the seek target');
|
||||
assertEqual(priv(player).streamingStarted, false, 'streaming flagged not-started for a clean retry');
|
||||
});
|
||||
|
||||
// --- AC6 retry contract: same-target seek after recovery refetches -------------------------
|
||||
|
||||
// After recoverFromFailedRefill the scheduler is empty (clearForSeek was called). A seek to
|
||||
// the SAME position (seekPosition == playbackOffset) must route to seekBeyondBuffer — not
|
||||
// seekWithinBuffer, which would be a silent no-op against the degenerate [P,P] empty window.
|
||||
test('same-target seek after recovery routes to seekBeyondBuffer (AC6 retry)', () => {
|
||||
const player = makePlayer();
|
||||
const wav = new FakeStreamDecoder(true, 1000);
|
||||
// Start with a populated window [30, 60), then simulate recovery at position 15:
|
||||
// clearForSeek empties the scheduler; setPlaybackOffset anchors it to 15.
|
||||
const scheduler = new FakeScheduler(30, 30);
|
||||
arm(player, { scheduler, duration: 120, streamDecoder: wav });
|
||||
// Drive recovery state manually (the same state recoverFromFailedRefill leaves).
|
||||
player.recoverFromFailedRefill(15);
|
||||
// At this point: scheduler.hasBuffers() == false, playbackOffset == 15, totalDuration == 0.
|
||||
// A seek to 15 (the recovery anchor) must refetch, not silently resolve from the empty window.
|
||||
const result = player.seek(15);
|
||||
assertEqual(result.success, true, 'seek succeeds after recovery');
|
||||
assertEqual(result.seekBeyondBuffer, true, 'same-target seek after recovery signals a refetch (AC6 retry)');
|
||||
assertEqual(wav.requestedOffsetFor, 15, 'WAV resolver used for the retry offset');
|
||||
});
|
||||
|
||||
// AC4 not regressed: a seek within a POPULATED retained window still resolves from buffer.
|
||||
// This is the same test as the existing AC4 test but named explicitly to confirm the
|
||||
// hasBuffers() guard does not affect the populated case.
|
||||
test('seek within populated retained window still resolves in-buffer — AC4 not regressed', () => {
|
||||
const player = makePlayer();
|
||||
// Populated window [30, 60) — hasBuffers() starts true (total=30 > 0).
|
||||
const scheduler = new FakeScheduler(30, 30);
|
||||
arm(player, { scheduler, duration: 120, streamDecoder: new FakeStreamDecoder(true, 1000) });
|
||||
|
||||
const result = player.seek(45); // inside [30, 60)
|
||||
assertEqual(result.success, true, 'seek succeeds');
|
||||
assertEqual(result.seekBeyondBuffer ?? false, false, 'populated in-window seek does NOT signal a refetch');
|
||||
assertEqual(scheduler.clearedForSeek, false, 'scheduler not cleared for an in-buffer seek (no refetch)');
|
||||
});
|
||||
|
||||
// --- run -------------------------------------------------------------------------------------
|
||||
if (failures.length > 0) {
|
||||
console.error(failures.join('\n'));
|
||||
console.error(`\n${failures.length} FAILED, ${passed} passed`);
|
||||
process.exit(1);
|
||||
} else {
|
||||
console.log(`ALL ${passed} TESTS PASSED`);
|
||||
}
|
||||
@@ -11,9 +11,12 @@ import { AudioContextManager } from './AudioContextManager.js';
|
||||
import { StreamDecoder } from './StreamDecoder.js';
|
||||
import { PlaybackScheduler } from './PlaybackScheduler.js';
|
||||
import { IFormatDecoder } from './IFormatDecoder.js';
|
||||
import { IStreamingDecoder } from './IStreamingDecoder.js';
|
||||
import { WavFormatDecoder } from './WavFormatDecoder.js';
|
||||
import { Mp3FormatDecoder } from './Mp3FormatDecoder.js';
|
||||
import { FlacFormatDecoder } from './FlacFormatDecoder.js';
|
||||
import { OpusStreamDecoder } from './OpusStreamDecoder.js';
|
||||
import { OpusSeekData, parseSidecar, resolveOpusByteOffset, OpusSeekResolution, OPUS_SAMPLE_RATE } from './OpusSidecar.js';
|
||||
|
||||
export interface AudioResult {
|
||||
success: boolean;
|
||||
@@ -27,6 +30,10 @@ export interface StreamingResult extends AudioResult {
|
||||
headerParsed?: boolean;
|
||||
bufferCount?: number;
|
||||
duration?: number;
|
||||
// Phase 21.2a back-pressure signal piggybacked on the chunk result the C# read loop already
|
||||
// awaits — true means the scheduler's forward fill is over the high-water mark and the loop
|
||||
// should stop calling ReadAsync until it drains (no extra interop hop in the common case).
|
||||
productionPaused?: boolean;
|
||||
}
|
||||
|
||||
export interface AudioState {
|
||||
@@ -45,6 +52,19 @@ export class AudioPlayer {
|
||||
private streamDecoder: StreamDecoder;
|
||||
private scheduler: PlaybackScheduler;
|
||||
|
||||
// The Opus WebCodecs decode path (IStreamingDecoder seam), used INSTEAD of streamDecoder when the
|
||||
// active stream is Ogg Opus. Null for WAV/MP3/FLAC, which keep the streamDecoder path unchanged.
|
||||
// Holding both is deliberate: the change is the decode stage only; the same scheduler/Web Audio
|
||||
// graph feeds from whichever decoder is active for the current stream.
|
||||
private opusDecoder: IStreamingDecoder | null = null;
|
||||
// The sidecar in effect for the active Opus stream (its seek index resolves byte offsets). Distinct
|
||||
// from pendingOpusSidecar, which is the one set for the NEXT stream init.
|
||||
private activeOpusSidecar: OpusSeekData | null = null;
|
||||
// The landing time of the most recent seek-beyond-buffer page resolution (seconds). Set by
|
||||
// seekBeyondBuffer, consumed by reinitializeFromOffset to trim leading decoded frames so the
|
||||
// audible position matches the requested seek target (AC9 fine re-sync, §3.4a step 4).
|
||||
private _seekLandingTime: number = 0;
|
||||
|
||||
// Playback state
|
||||
private isPlaying: boolean = false;
|
||||
private isPaused: boolean = false;
|
||||
@@ -62,6 +82,11 @@ export class AudioPlayer {
|
||||
private onEndCallback: EndCallback | null = null;
|
||||
private progressInterval: number | null = null;
|
||||
|
||||
// Pending Opus sidecar (setup header + seek index), parsed from the one-time sidecar fetch and
|
||||
// applied to the OpusFormatDecoder when the next Opus stream initializes. Wave 18.5 sets this
|
||||
// (via setOpusSidecar) before initializeStreaming; this class never fetches it.
|
||||
private pendingOpusSidecar: OpusSeekData | null = null;
|
||||
|
||||
constructor() {
|
||||
this.contextManager = new AudioContextManager();
|
||||
this.streamDecoder = new StreamDecoder(this.contextManager);
|
||||
@@ -93,17 +118,53 @@ export class AudioPlayer {
|
||||
|
||||
// ==================== Streaming ====================
|
||||
|
||||
initializeStreaming(totalStreamLength: number, contentType: string): AudioResult {
|
||||
async initializeStreaming(totalStreamLength: number, contentType: string): Promise<AudioResult> {
|
||||
try {
|
||||
// Full cleanup before starting new stream
|
||||
this.stopProgressTracking();
|
||||
this.scheduler.clear();
|
||||
this.streamDecoder.reset();
|
||||
this.disposeOpusDecoder();
|
||||
this.resetState();
|
||||
|
||||
// Initialize new stream with the format decoder selected from Content-Type.
|
||||
this.isStreamingMode = true;
|
||||
const formatDecoder = AudioPlayer.createFormatDecoder(contentType);
|
||||
|
||||
// Opus routes to the WebCodecs streaming seam (IStreamingDecoder); WAV/MP3/FLAC keep the
|
||||
// StreamDecoder wrap-and-decode path byte-for-byte. The sidecar (setup header + seek index)
|
||||
// must already be set (setOpusSidecar, before init) — without it Opus cannot be decoded or
|
||||
// seeked, so we fall back by leaving opusDecoder null and using the StreamDecoder path,
|
||||
// which the server's C2 fallback (lossless bytes) matches. In practice the C# resolver only
|
||||
// selects Opus when the sidecar parsed, so the null branch is defensive.
|
||||
if (this.isOpusContentType(contentType) && this.pendingOpusSidecar) {
|
||||
this.activeOpusSidecar = this.pendingOpusSidecar;
|
||||
|
||||
// Align the AudioContext to 48 kHz NOW, before any Opus bytes flow — the format is
|
||||
// already resolved (C# resolves Opus + injects the sidecar before this call), so the
|
||||
// target rate is known up front. Done here, the decoder's own lazy
|
||||
// recreateWithSampleRate(48000) in ensureConfigured hits its sampleRate-equal early
|
||||
// return and is a no-op; the live graph is never close()'d and rebuilt mid-decode (the
|
||||
// teardown that double-decoded the stream and OOM'd the tab with HW accel off). The
|
||||
// recreate seam itself stays — it is the WAV path's mechanism for non-44.1 sources and
|
||||
// remains the defensive backstop here.
|
||||
if (this.contextManager.sampleRate !== OPUS_SAMPLE_RATE) {
|
||||
await this.contextManager.recreateWithSampleRate(OPUS_SAMPLE_RATE);
|
||||
}
|
||||
|
||||
// Pass the shared back-pressure signal (21.2b): the Opus decoder stops demuxing/
|
||||
// decoding new packets while the scheduler is full, so the WebCodecs decode queue
|
||||
// and decodedQueue do not balloon behind a throttled socket (OQ7). Same signal the
|
||||
// C# read loop honors — one policy, two thin hooks.
|
||||
this.opusDecoder = new OpusStreamDecoder(
|
||||
this.contextManager,
|
||||
this.pendingOpusSidecar,
|
||||
() => this.scheduler.evaluateProductionPause());
|
||||
return { success: true };
|
||||
}
|
||||
|
||||
// Non-Opus (or Opus-without-sidecar): the existing StreamDecoder path, unchanged. The
|
||||
// context sample rate is untouched here, so the WAV/lossless path is byte-for-byte
|
||||
// unaffected by the Opus up-front alignment above.
|
||||
const formatDecoder = this.createFormatDecoder(contentType);
|
||||
this.streamDecoder.initialize(totalStreamLength, formatDecoder);
|
||||
return { success: true };
|
||||
} catch (error) {
|
||||
@@ -111,10 +172,41 @@ export class AudioPlayer {
|
||||
}
|
||||
}
|
||||
|
||||
private isOpusContentType(contentType: string): boolean {
|
||||
return contentType.includes('audio/ogg') || contentType.includes('audio/opus');
|
||||
}
|
||||
|
||||
private disposeOpusDecoder(): void {
|
||||
if (this.opusDecoder) {
|
||||
this.opusDecoder.dispose();
|
||||
this.opusDecoder = null;
|
||||
}
|
||||
this.activeOpusSidecar = null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Select a format decoder from the response Content-Type.
|
||||
* Inject the Opus sidecar (setup header + seek index) for the next Opus stream. Wave 18.5 calls
|
||||
* this with the raw sidecar bytes (from its one-time HTTP fetch) BEFORE initializeStreaming; the
|
||||
* parsed result is applied to the OpusFormatDecoder when the stream initializes. This is the
|
||||
* injection seam — the player owns no transport, only the parse + hand-off.
|
||||
*
|
||||
* @returns success:false with an error if the bytes are not a valid sidecar blob.
|
||||
*/
|
||||
private static createFormatDecoder(contentType: string): IFormatDecoder {
|
||||
setOpusSidecar(sidecarBytes: Uint8Array): AudioResult {
|
||||
const parsed = parseSidecar(sidecarBytes);
|
||||
if (!parsed) {
|
||||
return { success: false, error: 'Invalid Opus sidecar blob' };
|
||||
}
|
||||
this.pendingOpusSidecar = parsed;
|
||||
return { success: true };
|
||||
}
|
||||
|
||||
/**
|
||||
* Select a format decoder from the response Content-Type for the StreamDecoder (wrap-and-decode)
|
||||
* path. Opus is NOT handled here — it routes to the WebCodecs IStreamingDecoder seam in
|
||||
* initializeStreaming. This factory serves WAV/MP3/FLAC only.
|
||||
*/
|
||||
private createFormatDecoder(contentType: string): IFormatDecoder {
|
||||
if (contentType.includes('audio/mpeg') || contentType.includes('audio/mp3')) {
|
||||
return new Mp3FormatDecoder();
|
||||
}
|
||||
@@ -133,16 +225,24 @@ export class AudioPlayer {
|
||||
*/
|
||||
async markStreamComplete(): Promise<StreamingResult> {
|
||||
try {
|
||||
const results = await this.streamDecoder.markStreamComplete();
|
||||
const results = this.opusDecoder
|
||||
? await this.opusDecoder.complete()
|
||||
: (await this.streamDecoder.markStreamComplete()).map(r => r.buffer);
|
||||
if (results.length > 0) {
|
||||
for (const result of results) {
|
||||
this.scheduler.addBuffer(result.buffer);
|
||||
for (const buffer of results) {
|
||||
this.scheduler.addBuffer(buffer);
|
||||
}
|
||||
if (this.streamingStarted && this.isPlaying) {
|
||||
this.scheduler.scheduleNewBuffers();
|
||||
}
|
||||
}
|
||||
this.streamingCompleted = true;
|
||||
// Hand the genuine-end signal to the scheduler AFTER the tail buffers are added and
|
||||
// scheduled: now an empty scheduled queue is a real end-of-track, not a startup gap, so
|
||||
// the scheduler may fire onPlaybackEnded when its queue drains. If the queue was already
|
||||
// empty at this point (the tail produced no buffers, or they were already played),
|
||||
// setStreamComplete finalises immediately.
|
||||
this.scheduler.setStreamComplete(true);
|
||||
return { success: true, bufferCount: this.scheduler.getBufferCount() };
|
||||
} catch (error) {
|
||||
return { success: false, error: (error as Error).message };
|
||||
@@ -150,6 +250,66 @@ export class AudioPlayer {
|
||||
}
|
||||
|
||||
async processStreamingChunk(chunk: Uint8Array): Promise<StreamingResult> {
|
||||
return this.opusDecoder
|
||||
? this.processOpusChunk(chunk)
|
||||
: this.processFormatChunk(chunk);
|
||||
}
|
||||
|
||||
/** Opus (WebCodecs) chunk path. Mirrors processFormatChunk's add->schedule->report shape. */
|
||||
private async processOpusChunk(chunk: Uint8Array): Promise<StreamingResult> {
|
||||
try {
|
||||
const decoder = this.opusDecoder!;
|
||||
const buffers = await decoder.push(chunk);
|
||||
|
||||
// Duration is known up front from the sidecar — surface it as soon as the decoder reports it,
|
||||
// NOT gated on the first decoded buffers. The C# layer locks Duration on the first chunk whose
|
||||
// result carries a value (the `Duration == null` guard), and WebCodecs decode is async, so the
|
||||
// earliest chunks can return zero buffers; gating duration on buffers means C# captures the
|
||||
// initial 0 and never overwrites it — the WAV header path sets duration on chunk 1 because its
|
||||
// header parses synchronously, which is the asymmetry this closes. Set once so a seek (which
|
||||
// reinitialises the decoder) cannot overwrite it.
|
||||
if (this.duration === 0 && decoder.totalDuration) {
|
||||
this.duration = decoder.totalDuration;
|
||||
}
|
||||
|
||||
if (buffers.length > 0) {
|
||||
for (const buffer of buffers) {
|
||||
this.scheduler.addBuffer(buffer);
|
||||
}
|
||||
if (this.streamingStarted && this.isPlaying) {
|
||||
this.scheduler.scheduleNewBuffers();
|
||||
}
|
||||
}
|
||||
|
||||
if (decoder.hasFatalError) {
|
||||
return { success: false, error: 'Opus decode failed' };
|
||||
}
|
||||
|
||||
// "headerParsed" maps to the decoder being configured (codec ready). canStart needs a
|
||||
// healthy decoded lead before first playback — measured in SECONDS, not a buffer count.
|
||||
// An Opus WebCodecs packet is ~20 ms, so the WAV-tuned 6-BUFFER minimum is only ~0.12 s of
|
||||
// lead: playback would start, drain it before the async decode ramps, and underrun
|
||||
// immediately. The seconds-based lead gate (same threshold the scheduler's underrun-resume
|
||||
// hysteresis uses) gives Opus the cushion its decode ramp needs. WAV keeps the buffer-count
|
||||
// gate below — its large synchronous segments rarely underrun and its start must not change.
|
||||
const headerParsed = decoder.ready;
|
||||
const canStart = headerParsed && this.scheduler.hasMinimumPlaybackLead();
|
||||
|
||||
return {
|
||||
success: true,
|
||||
canStartStreaming: canStart,
|
||||
headerParsed,
|
||||
bufferCount: this.scheduler.getBufferCount(),
|
||||
duration: this.duration,
|
||||
productionPaused: this.scheduler.evaluateProductionPause()
|
||||
};
|
||||
} catch (error) {
|
||||
return { success: false, error: (error as Error).message };
|
||||
}
|
||||
}
|
||||
|
||||
/** WAV/MP3/FLAC (StreamDecoder) chunk path — unchanged from before the Opus seam split. */
|
||||
private async processFormatChunk(chunk: Uint8Array): Promise<StreamingResult> {
|
||||
try {
|
||||
const results = await this.streamDecoder.processChunk(chunk);
|
||||
|
||||
@@ -172,9 +332,13 @@ export class AudioPlayer {
|
||||
}
|
||||
}
|
||||
|
||||
// Check if streaming is complete
|
||||
// Check if streaming is complete. The StreamDecoder self-detects completion by byte
|
||||
// count (WAV/MP3/FLAC); propagate that to the scheduler so a drained queue past this
|
||||
// point is treated as a genuine end. Buffers from this chunk were already added above,
|
||||
// so any final end fires through handleSourceEnded when they drain.
|
||||
if (this.streamDecoder.isComplete) {
|
||||
this.streamingCompleted = true;
|
||||
this.scheduler.setStreamComplete(true);
|
||||
}
|
||||
|
||||
const canStart = this.streamDecoder.headerParsed &&
|
||||
@@ -185,7 +349,8 @@ export class AudioPlayer {
|
||||
canStartStreaming: canStart,
|
||||
headerParsed: this.streamDecoder.headerParsed,
|
||||
bufferCount: this.scheduler.getBufferCount(),
|
||||
duration: this.duration
|
||||
duration: this.duration,
|
||||
productionPaused: this.scheduler.evaluateProductionPause()
|
||||
};
|
||||
} catch (error) {
|
||||
return { success: false, error: (error as Error).message };
|
||||
@@ -278,6 +443,7 @@ export class AudioPlayer {
|
||||
try {
|
||||
this.scheduler.clear();
|
||||
this.streamDecoder.reset();
|
||||
this.disposeOpusDecoder();
|
||||
this.resetState();
|
||||
this.stopProgressTracking();
|
||||
|
||||
@@ -296,18 +462,36 @@ export class AudioPlayer {
|
||||
return { success: false, error: 'Invalid seek position' };
|
||||
}
|
||||
|
||||
// bufferStart is the absolute track time at which buffers[0] begins. Under Phase 21.1
|
||||
// partial eviction this is the start of the RETAINED BACK-WINDOW TAIL — eviction advances
|
||||
// playbackOffset as it drops played buffers off the front — so [bufferStart, bufferEnd] is
|
||||
// exactly the window currently held in memory.
|
||||
const bufferStart = this.scheduler.getPlaybackOffset();
|
||||
const bufferEnd = this.scheduler.getTotalDuration() + bufferStart;
|
||||
|
||||
// Position must be within [bufferStart, bufferEnd] to use buffered content.
|
||||
// A lower-bound check is required: after a seek-beyond-buffer, bufferStart is
|
||||
// set to the prior seek position. Seeking to a position below bufferStart would
|
||||
// produce a negative bufferRelativePosition in seekWithinBuffer, silently
|
||||
// clamping to position 0 of the offset buffer instead of the requested time.
|
||||
if (position >= bufferStart && position <= bufferEnd) {
|
||||
// The window-miss test for BOTH directions, and the 21.3 refill trigger for backward seeks.
|
||||
// Position must be within [bufferStart, bufferEnd] AND the scheduler must hold buffers to
|
||||
// resolve from the retained window:
|
||||
// - position >= bufferStart AND hasBuffers : UC3 — seek back within the retained back-window.
|
||||
// Served from buffer with NO network refetch. (The lower bound is load-bearing: after
|
||||
// eviction or a prior seek-beyond-buffer, bufferStart > 0, and a target below it would
|
||||
// otherwise produce a negative bufferRelativePosition in seekWithinBuffer, silently clamping
|
||||
// to position 0.)
|
||||
// - position < bufferStart : UC4 — seek back PAST the retained tail (the window was evicted).
|
||||
// Falls through to seekBeyondBuffer, which is the existing Range path run toward an EARLIER
|
||||
// offset. This is the 21.3 window-miss refill: "a seek the listener didn't initiate" reuses
|
||||
// the same per-path resolver + reinit a forward seek-beyond-buffer uses, no new mechanism.
|
||||
// - position > bufferEnd : UC2/UC5 — forward seek beyond buffer, unchanged.
|
||||
// - !hasBuffers (degenerate [P,P] window post-recovery): the window check above would
|
||||
// spuriously route ANY target to seekWithinBuffer (bufferStart==bufferEnd==seekPosition
|
||||
// after recoverFromFailedRefill). Force seekBeyondBuffer so a same-target retry actually
|
||||
// refetches (AC6 retry contract). The !hasBuffers guard only fires in the degenerate case —
|
||||
// a populated retained window has buffers and is unaffected (AC4 not regressed).
|
||||
if (position >= bufferStart && position <= bufferEnd && this.scheduler.hasBuffers()) {
|
||||
return this.seekWithinBuffer(position);
|
||||
} else {
|
||||
// Seeking outside buffered window - signal C# to fetch new stream
|
||||
// Seeking outside the retained window, or to any position in an empty scheduler —
|
||||
// signal C# to fetch a new stream from the resolved offset.
|
||||
return this.seekBeyondBuffer(position);
|
||||
}
|
||||
}
|
||||
@@ -339,8 +523,25 @@ export class AudioPlayer {
|
||||
*/
|
||||
private seekBeyondBuffer(position: number): AudioResult {
|
||||
try {
|
||||
// The header must be parsed for byte-offset math; without it we cannot
|
||||
// build a valid Range request.
|
||||
// Opus: resolve the offset from the precomputed seek index (the accurate VBR-safe transfer
|
||||
// function). The returned offset is a real page start, so the Range continuation lands the
|
||||
// demuxer/decoder Ogg-sync-aligned. Also capture the landing time (t_page ≤ position) so
|
||||
// reinitializeFromOffset can trim the leading decoded frames and land precisely at `position`
|
||||
// (AC9 fine re-sync, §3.4a step 4).
|
||||
if (this.opusDecoder) {
|
||||
if (!this.activeOpusSidecar) {
|
||||
return { success: false, error: 'Cannot calculate byte offset' };
|
||||
}
|
||||
const resolution: OpusSeekResolution = resolveOpusByteOffset(this.activeOpusSidecar, position);
|
||||
this._seekLandingTime = resolution.landingTimeSeconds;
|
||||
return {
|
||||
success: true,
|
||||
seekBeyondBuffer: true,
|
||||
byteOffset: resolution.byteOffset
|
||||
};
|
||||
}
|
||||
|
||||
// WAV/MP3/FLAC: the header must be parsed for byte-offset math.
|
||||
if (!this.streamDecoder.getFormatInfo()) {
|
||||
return { success: false, error: 'Cannot calculate byte offset' };
|
||||
}
|
||||
@@ -361,6 +562,22 @@ export class AudioPlayer {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the file-absolute byte offset to begin a stream at `position`, WITHOUT requiring active
|
||||
* playback or buffered audio (the "load at timestamp" entry point — Phase 18 wave 18.6 format switch).
|
||||
* Unlike seek(), it has no duration guard and never routes to the within-buffer path: a fresh load has
|
||||
* no scheduler window, so the answer is always "start the byte stream here". For Opus the sidecar
|
||||
* resolves the offset (and captures the page landing time for the lead-trim) immediately after init; for
|
||||
* WAV the header must already be parsed (feed the byte-0 segment first). Returns success:false when the
|
||||
* decoder cannot yet resolve an offset (no header / no sidecar), so the caller can probe and retry.
|
||||
*/
|
||||
resolveStreamOffset(position: number): AudioResult {
|
||||
if (!this.isStreamingMode) {
|
||||
return { success: false, error: 'Not in streaming mode' };
|
||||
}
|
||||
return this.seekBeyondBuffer(position);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the total buffered duration (for C# to check if seek is within buffer)
|
||||
*/
|
||||
@@ -368,10 +585,26 @@ export class AudioPlayer {
|
||||
return this.scheduler.getTotalDuration() + this.scheduler.getPlaybackOffset();
|
||||
}
|
||||
|
||||
/**
|
||||
* The shared back-pressure signal (Phase 21.2a), polled by the C# read loop WHILE it is
|
||||
* already throttled to learn when the forward fill has drained below the low-water mark and it
|
||||
* may resume reading. The steady-state (unthrottled) loop never calls this — it reads the
|
||||
* piggybacked productionPaused flag off each chunk result instead, so there is no extra
|
||||
* interop hop until back-pressure actually engages.
|
||||
*/
|
||||
isProductionPaused(): boolean {
|
||||
return this.scheduler.evaluateProductionPause();
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculate byte offset for a time position (for C# layer)
|
||||
*/
|
||||
calculateByteOffset(positionSeconds: number): number {
|
||||
if (this.opusDecoder) {
|
||||
return this.activeOpusSidecar
|
||||
? resolveOpusByteOffset(this.activeOpusSidecar, positionSeconds).byteOffset
|
||||
: 0;
|
||||
}
|
||||
if (!this.streamDecoder.getFormatInfo()) return 0;
|
||||
return this.streamDecoder.calculateByteOffset(positionSeconds);
|
||||
}
|
||||
@@ -384,17 +617,21 @@ export class AudioPlayer {
|
||||
try {
|
||||
// Stop current playback
|
||||
this.stopProgressTracking();
|
||||
const wasPlaying = this.isPlaying;
|
||||
this.isPlaying = false;
|
||||
|
||||
// Clear buffers and set new offset
|
||||
this.scheduler.clearForSeek();
|
||||
this.scheduler.setPlaybackOffset(seekPosition);
|
||||
|
||||
// Reinitialize decoder for the Range-continuation stream. totalStreamLength
|
||||
// here is the 206 Content-Length (range start → EOF), not the full file size —
|
||||
// the decoder uses it to detect stream-complete against raw audio bytes.
|
||||
this.streamDecoder.reinitializeForRangeContinuation(totalStreamLength);
|
||||
// Reinitialize the active decoder for the Range-continuation stream (206 body, no header/
|
||||
// setup pages). Opus resets demux + codec state (keeping the cached config) and arms the
|
||||
// lead-trim so decoded audio starts at `seekPosition`, not at the page boundary (AC9). The
|
||||
// StreamDecoder path uses totalStreamLength (the 206 Content-Length) to detect completion.
|
||||
if (this.opusDecoder) {
|
||||
this.opusDecoder.reinitializeForRangeContinuation(this._seekLandingTime, seekPosition);
|
||||
} else {
|
||||
this.streamDecoder.reinitializeForRangeContinuation(totalStreamLength);
|
||||
}
|
||||
|
||||
// Update state
|
||||
this.pausePosition = seekPosition;
|
||||
@@ -407,6 +644,44 @@ export class AudioPlayer {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Recover the player into a clean, paused-but-loaded state after a window-miss REFILL failed
|
||||
* (Phase 21.3 / AC6). A refill is "a seek the listener didn't initiate"; when its Range fetch or
|
||||
* reinit fails mid-stream, the pre-seek loop has already been cancelled and drained, but the
|
||||
* scheduler is still holding stale pre-seek buffers and is still `isActive_`. Left alone it would
|
||||
* play the retained tail to exhaustion and fire `onPlaybackEnded` — a SILENT FALSE END (the
|
||||
* "wedged playing with a starved scheduler" AC6 forbids).
|
||||
*
|
||||
* The recovery mirrors `PlaybackScheduler.playFromPosition`'s end-of-buffer recovery in spirit:
|
||||
* stop pretending to play. We stop all sources and clear the buffers for a seek (clearForSeek
|
||||
* keeps no stale audio but is ready to accept a fresh continuation), set the offset to the
|
||||
* requested seek position, and leave the player paused there. The track stays loaded so the
|
||||
* listener can retry the seek or pick another track — no new transport control, only a recoverable
|
||||
* stop (C4). A subsequent seek to the same target re-enters seekBeyondBuffer cleanly because the
|
||||
* offset names the seek position and the scheduler is empty (so it routes to a fresh fetch).
|
||||
*
|
||||
* @param seekPosition The seek target the failed refill was aiming for; becomes the resume anchor.
|
||||
*/
|
||||
recoverFromFailedRefill(seekPosition: number): AudioResult {
|
||||
try {
|
||||
this.stopProgressTracking();
|
||||
// Halt the starved scheduler and drop the stale pre-seek buffers so no false end can fire.
|
||||
this.scheduler.clearForSeek();
|
||||
this.scheduler.setPlaybackOffset(seekPosition);
|
||||
|
||||
// Paused-but-loaded: not playing, not mid-seek-stream. pausePosition anchors a retry.
|
||||
this.isPlaying = false;
|
||||
this.isPaused = true;
|
||||
this.pausePosition = seekPosition;
|
||||
this.streamingStarted = false;
|
||||
this.streamingCompleted = false;
|
||||
|
||||
return { success: true };
|
||||
} catch (error) {
|
||||
return { success: false, error: (error as Error).message };
|
||||
}
|
||||
}
|
||||
|
||||
// ==================== Volume ====================
|
||||
|
||||
setVolume(volume: number): AudioResult {
|
||||
@@ -510,6 +785,7 @@ export class AudioPlayer {
|
||||
this.isStreamingMode = false;
|
||||
this.streamingStarted = false;
|
||||
this.streamingCompleted = false;
|
||||
this._seekLandingTime = 0;
|
||||
}
|
||||
|
||||
private handlePlaybackEnded(): void {
|
||||
|
||||
@@ -36,6 +36,8 @@ export interface FormatInfo {
|
||||
* MP3 VBR: Xing/VBRI TOC (100-entry Uint8Array, values are file-percentage * 255).
|
||||
* FLAC: SeekTable (array of {sampleNumber: number, streamOffset: number} — stream_offset
|
||||
* is bytes from the start of audio frames, i.e. after all metadata blocks).
|
||||
* Opus does NOT flow through this seam — it uses the WebCodecs IStreamingDecoder path and resolves
|
||||
* seek offsets via OpusSidecar.resolveOpusByteOffset, not FormatInfo.seekData.
|
||||
*/
|
||||
seekData?: Mp3VbrSeekData | FlacSeekData | null;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
/**
|
||||
* IStreamingDecoder - the stateful streaming-decode seam, parallel to IFormatDecoder.
|
||||
*
|
||||
* Why a second seam. `IFormatDecoder` (WAV/MP3/FLAC) is a *wrap-and-decode-each-segment* strategy:
|
||||
* `StreamDecoder` cuts the stream into independently-decodable segments, `wrapSegment` makes each a
|
||||
* standalone file, and `decodeAudioData` decodes each in isolation. That model is correct for raw PCM
|
||||
* (WAV) and independently-decodable frames (FLAC), but it is fundamentally wrong for Opus: Opus has
|
||||
* pre-skip (encoder delay) and inter-frame state (MDCT overlap-add, SILK/CELT continuity), so decoding
|
||||
* page-runs independently re-applies the pre-skip and starts from cold codec state at every boundary —
|
||||
* audible glitching and a broken timeline.
|
||||
*
|
||||
* A WebCodecs `AudioDecoder` is the right tool: one stateful decoder fed packets sequentially, decoding
|
||||
* continuously with correct pre-skip-once handling and full inter-frame continuity. But it does NOT fit
|
||||
* `IFormatDecoder` — it is async/callback-driven and owns its own buffering. So Opus gets this seam
|
||||
* instead. `AudioPlayer` dispatches by content-type: WAV/MP3/FLAC keep the `StreamDecoder` path
|
||||
* byte-for-byte; Opus routes here. Both feed the SAME `PlaybackScheduler` — the change is the decode
|
||||
* stage only, never the schedule/playback stage.
|
||||
*
|
||||
* The seam is intentionally minimal and mirrors the lifecycle `StreamDecoder` already exposes so
|
||||
* `AudioPlayer` can treat the two uniformly: initialize -> push chunks -> mark complete, plus a
|
||||
* range-continuation reinit for seek-beyond-buffer.
|
||||
*/
|
||||
|
||||
export interface IStreamingDecoder {
|
||||
/**
|
||||
* Decoded buffers ready to schedule, drained by AudioPlayer after each push/flush. Each entry is a
|
||||
* standard AudioBuffer at the AudioContext's sample rate, ready for PlaybackScheduler.addBuffer.
|
||||
*/
|
||||
readonly hasFatalError: boolean;
|
||||
|
||||
/** True once the decoder has enough to begin playback (header/config established). */
|
||||
readonly ready: boolean;
|
||||
|
||||
/** Total stream duration in seconds if known up front (Opus knows it from the sidecar), else null. */
|
||||
readonly totalDuration: number | null;
|
||||
|
||||
/**
|
||||
* Push raw stream bytes. Returns decoded AudioBuffers that became ready (possibly empty — WebCodecs
|
||||
* decode is async, so a push may return nothing and a later push returns several).
|
||||
*/
|
||||
push(chunk: Uint8Array): Promise<AudioBuffer[]>;
|
||||
|
||||
/**
|
||||
* Signal end-of-stream. Flushes the decoder and returns any residual decoded buffers (including the
|
||||
* end-trimmed final buffer).
|
||||
*/
|
||||
complete(): Promise<AudioBuffer[]>;
|
||||
|
||||
/**
|
||||
* Reinitialize for a Range-continuation after seek-beyond-buffer. The 206 body begins on an Ogg page
|
||||
* boundary and carries no setup pages — the decoder reuses the cached config and resets demux/codec
|
||||
* state so inter-frame continuity restarts cleanly from the new offset.
|
||||
*
|
||||
* @param landingTimeSeconds The actual presentation time of the resolved seek page (t_page ≤ target).
|
||||
* @param targetTimeSeconds The user-requested seek position. The decoder trims the leading
|
||||
* `(target - landing) * sampleRate` frames so playback lands at target
|
||||
* (AC9 fine re-sync, §3.4a step 4).
|
||||
*/
|
||||
reinitializeForRangeContinuation(landingTimeSeconds: number, targetTimeSeconds: number): void;
|
||||
|
||||
/** Tear down the underlying WebCodecs decoder and release resources. */
|
||||
dispose(): void;
|
||||
}
|
||||
@@ -0,0 +1,298 @@
|
||||
/**
|
||||
* OggDemuxer - streaming Ogg-page -> Opus-packet demuxer for the WebCodecs decode path.
|
||||
*
|
||||
* Ogg Opus is a containerized, paged format. To feed a WebCodecs `AudioDecoder` we must extract the
|
||||
* individual Opus *packets* from the Ogg container — the decoder takes packets (as `EncodedAudioChunk`s),
|
||||
* not raw container bytes. This module is the client-side analogue of the C# `OggOpusParser`: it reads
|
||||
* the page structure directly (the "OggS" capture pattern + the 27-byte page header + segment table) and
|
||||
* reassembles packets across the lacing, tracking the granule position that gives each packet its time.
|
||||
*
|
||||
* It is deliberately *streaming*: `push(bytes)` accepts arbitrary network chunks (a packet, a page, or a
|
||||
* fraction of either) and returns whatever WHOLE packets have become available, holding partial state
|
||||
* across calls. This matches how `StreamAudioWithEarlyPlayback` feeds bytes in adaptive 16–64 KB chunks.
|
||||
*
|
||||
* Lacing rules (RFC 3533 §6): a page's segment table lists N segment lengths (0–255). A packet is the
|
||||
* concatenation of consecutive segments up to and including the first segment whose length is < 255. A
|
||||
* segment of exactly 255 means "this packet continues into the next segment" — and if it is the page's
|
||||
* LAST segment, the packet continues into the next page (the next page's header-type has the
|
||||
* continuation flag set). The granule position on a page is the end-granule of the LAST packet that
|
||||
* *completes* on that page.
|
||||
*
|
||||
* The two leading setup packets (OpusHead, OpusTags) are NOT audio and are skipped — they configure the
|
||||
* decoder (the sidecar carries them as the codec description), they are never decoded as audio packets.
|
||||
*/
|
||||
|
||||
const OGG_CAPTURE = [0x4f, 0x67, 0x67, 0x53]; // "OggS"
|
||||
const OGG_PAGE_HEADER_SIZE = 27;
|
||||
const GRANULE_OFFSET = 6; // 64-bit granule position within the page header
|
||||
const HEADER_TYPE_OFFSET = 5; // bit 0x01 = continued packet, 0x02 = BOS, 0x04 = EOS
|
||||
const SEGMENT_COUNT_OFFSET = 26; // number of segment-table entries
|
||||
const CONTINUATION_FLAG = 0x01;
|
||||
|
||||
const OPUS_HEAD_SIG = [0x4f, 0x70, 0x75, 0x73, 0x48, 0x65, 0x61, 0x64]; // "OpusHead"
|
||||
const OPUS_TAGS_SIG = [0x4f, 0x70, 0x75, 0x73, 0x54, 0x61, 0x67, 0x73]; // "OpusTags"
|
||||
|
||||
/** A demuxed Opus audio packet plus the timing context needed to schedule and trim it. */
|
||||
export interface OpusPacket {
|
||||
/** Raw Opus packet bytes (one Opus frame's worth — fed straight to the AudioDecoder). */
|
||||
data: Uint8Array;
|
||||
/**
|
||||
* The end-granule of the page this packet completed on, or null if the page carried no usable
|
||||
* granule (mid-stream pages between completion points share the next completing page's granule —
|
||||
* we attach the granule only to the packet that completes on a granule-bearing page). A 48 kHz
|
||||
* sample count; presentation time = (granule - preSkip) / 48000.
|
||||
*/
|
||||
pageGranule: number | null;
|
||||
/** True when this packet completed on the stream's final (EOS) page — drives end-trim. */
|
||||
isLastPage: boolean;
|
||||
}
|
||||
|
||||
/** Read a little-endian uint64 as a JS number (exact to 2^53 — far beyond any real granule). */
|
||||
function readUint64LE(buf: Uint8Array, offset: number): number {
|
||||
let lo = 0;
|
||||
let hi = 0;
|
||||
for (let i = 0; i < 4; i++) lo += buf[offset + i] * 2 ** (8 * i);
|
||||
for (let i = 0; i < 4; i++) hi += buf[offset + 4 + i] * 2 ** (8 * i);
|
||||
return hi * 0x100000000 + lo;
|
||||
}
|
||||
|
||||
function startsWith(buf: Uint8Array, sig: number[]): boolean {
|
||||
if (buf.length < sig.length) return false;
|
||||
for (let i = 0; i < sig.length; i++) if (buf[i] !== sig[i]) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
export class OggDemuxer {
|
||||
// Unconsumed raw bytes carried across push() calls (a page may straddle a network-chunk boundary).
|
||||
private pending: Uint8Array = new Uint8Array(0);
|
||||
// Bytes of a packet that spans pages (255-length last segment + continuation flag next page).
|
||||
private partialPacket: Uint8Array[] = [];
|
||||
// Once both setup packets are seen, every subsequent packet is audio.
|
||||
private setupPacketsSeen = 0;
|
||||
|
||||
/**
|
||||
* Feed raw stream bytes (any size). Returns all WHOLE Opus AUDIO packets that became decodable,
|
||||
* in order. Setup packets (OpusHead/OpusTags) are consumed and skipped. Incomplete trailing bytes
|
||||
* are retained for the next push.
|
||||
*/
|
||||
push(bytes: Uint8Array): OpusPacket[] {
|
||||
this.pending = this.concat(this.pending, bytes);
|
||||
return this.drainPages();
|
||||
}
|
||||
|
||||
/**
|
||||
* Reset to a fresh stream. Used on seek/range-continuation: the new 206 body begins on a page
|
||||
* boundary, so all partial-packet and pending state must be dropped. setupPacketsSeen is reset to
|
||||
* 2 (already configured) for a continuation — a mid-stream slice carries no setup pages, only audio
|
||||
* pages — so the demuxer treats the first page's packets as audio immediately.
|
||||
*/
|
||||
reset(isContinuation: boolean): void {
|
||||
this.pending = new Uint8Array(0);
|
||||
this.partialPacket = [];
|
||||
this.setupPacketsSeen = isContinuation ? 2 : 0;
|
||||
}
|
||||
|
||||
private drainPages(): OpusPacket[] {
|
||||
const packets: OpusPacket[] = [];
|
||||
|
||||
for (;;) {
|
||||
const page = this.tryReadPage();
|
||||
if (!page) break;
|
||||
this.parsePage(page, packets);
|
||||
}
|
||||
|
||||
return packets;
|
||||
}
|
||||
|
||||
/**
|
||||
* Try to slice one complete Ogg page off the front of `pending`. Returns null (and leaves `pending`
|
||||
* intact) when a whole page is not yet buffered. Resynchronises by scanning for "OggS" if `pending`
|
||||
* does not start on a page boundary (defensive — the encoder writes contiguous pages, but a
|
||||
* continuation stream could in theory begin mid-garbage; the seek offset is always a page start).
|
||||
*/
|
||||
private tryReadPage(): { header: Uint8Array; segTable: Uint8Array; payload: Uint8Array; total: number } | null {
|
||||
const buf = this.pending;
|
||||
if (buf.length < OGG_PAGE_HEADER_SIZE) return null;
|
||||
|
||||
// Resync: ensure we are positioned at a capture pattern.
|
||||
if (!startsWith(buf, OGG_CAPTURE)) {
|
||||
const sync = this.findCapture(buf, 0);
|
||||
if (sync < 0) {
|
||||
// No capture pattern at all — keep only the last 3 bytes (a capture could straddle).
|
||||
this.pending = buf.subarray(Math.max(0, buf.length - 3));
|
||||
return null;
|
||||
}
|
||||
this.pending = buf.subarray(sync);
|
||||
return this.tryReadPage();
|
||||
}
|
||||
|
||||
const segCount = buf[SEGMENT_COUNT_OFFSET];
|
||||
const segTableEnd = OGG_PAGE_HEADER_SIZE + segCount;
|
||||
if (buf.length < segTableEnd) return null; // segment table not fully buffered yet
|
||||
|
||||
const segTable = buf.subarray(OGG_PAGE_HEADER_SIZE, segTableEnd);
|
||||
let payloadSize = 0;
|
||||
for (let i = 0; i < segCount; i++) payloadSize += segTable[i];
|
||||
|
||||
const total = segTableEnd + payloadSize;
|
||||
if (buf.length < total) return null; // payload not fully buffered yet
|
||||
|
||||
const header = buf.subarray(0, OGG_PAGE_HEADER_SIZE);
|
||||
const payload = buf.subarray(segTableEnd, total);
|
||||
|
||||
// Advance past this page.
|
||||
this.pending = buf.subarray(total);
|
||||
return { header, segTable, payload, total };
|
||||
}
|
||||
|
||||
private parsePage(
|
||||
page: { header: Uint8Array; segTable: Uint8Array; payload: Uint8Array; total: number },
|
||||
out: OpusPacket[]
|
||||
): void {
|
||||
const { header, segTable, payload } = page;
|
||||
const headerType = header[HEADER_TYPE_OFFSET];
|
||||
const continued = (headerType & CONTINUATION_FLAG) !== 0;
|
||||
const isEos = (headerType & 0x04) !== 0;
|
||||
const granule = readUint64LE(header, GRANULE_OFFSET);
|
||||
// 0xFFFFFFFFFFFFFFFF (-1) means "no packet completed on this page" — no usable timestamp.
|
||||
// We check the raw bytes rather than comparing `granule === -1` (or the equivalent JS number):
|
||||
// the full 64-bit sentinel exceeds 2^53 and cannot be represented exactly as an IEEE-754 double,
|
||||
// so the parsed value from readUint64LE would not equal the sentinel. The byte check is exact.
|
||||
const hasGranule = !(header[GRANULE_OFFSET] === 0xff && header[GRANULE_OFFSET + 1] === 0xff &&
|
||||
header[GRANULE_OFFSET + 2] === 0xff && header[GRANULE_OFFSET + 3] === 0xff &&
|
||||
header[GRANULE_OFFSET + 4] === 0xff && header[GRANULE_OFFSET + 5] === 0xff &&
|
||||
header[GRANULE_OFFSET + 6] === 0xff && header[GRANULE_OFFSET + 7] === 0xff);
|
||||
|
||||
// If this page does NOT begin with a continuation, any half-built packet from a prior page is
|
||||
// orphaned (should not happen in a well-formed stream, but never carry garbage forward).
|
||||
if (!continued) this.partialPacket = [];
|
||||
|
||||
// Walk the segment table, reassembling packets. A packet ends at the first segment < 255.
|
||||
const completedPackets: Uint8Array[] = [];
|
||||
let segStart = 0;
|
||||
let cursor = 0;
|
||||
for (let i = 0; i < segTable.length; i++) {
|
||||
const len = segTable[i];
|
||||
cursor += len;
|
||||
if (len < 255) {
|
||||
// Packet boundary: segments [segStart, cursor) form (the tail of) a packet.
|
||||
const slice = payload.subarray(segStart, cursor);
|
||||
if (this.partialPacket.length > 0) {
|
||||
this.partialPacket.push(slice);
|
||||
completedPackets.push(this.flattenPartial());
|
||||
this.partialPacket = [];
|
||||
} else {
|
||||
completedPackets.push(slice);
|
||||
}
|
||||
segStart = cursor;
|
||||
}
|
||||
// len === 255 with i === last segment -> packet spans into the next page (handled below).
|
||||
}
|
||||
|
||||
// Any trailing 255-run that did not terminate is a packet continuing into the next page.
|
||||
if (segStart < cursor) {
|
||||
this.partialPacket.push(payload.subarray(segStart, cursor));
|
||||
}
|
||||
|
||||
// Classify completed packets: the first two whole packets in the whole stream are the setup
|
||||
// packets (OpusHead, OpusTags) and are skipped. Everything after is audio. The page granule is
|
||||
// attached to the LAST completing audio packet on a granule-bearing page (the granule is that
|
||||
// page's end-granule per RFC 7845).
|
||||
for (let p = 0; p < completedPackets.length; p++) {
|
||||
const pkt = completedPackets[p];
|
||||
if (this.setupPacketsSeen < 2) {
|
||||
// Only count packets that are actually the Opus setup headers; guard against a stray
|
||||
// first audio packet being mistaken for setup on a continuation (reset handles that).
|
||||
if (this.setupPacketsSeen === 0 && startsWith(pkt, OPUS_HEAD_SIG)) {
|
||||
this.setupPacketsSeen = 1;
|
||||
continue;
|
||||
}
|
||||
if (this.setupPacketsSeen === 1 && startsWith(pkt, OPUS_TAGS_SIG)) {
|
||||
this.setupPacketsSeen = 2;
|
||||
continue;
|
||||
}
|
||||
// Not a recognised setup packet while we still expected one — treat as audio (a
|
||||
// continuation slice that began mid-stream). Fall through.
|
||||
}
|
||||
|
||||
const isLastCompleting = p === completedPackets.length - 1;
|
||||
out.push({
|
||||
data: pkt,
|
||||
pageGranule: hasGranule && isLastCompleting ? granule : null,
|
||||
isLastPage: isEos
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
private flattenPartial(): Uint8Array {
|
||||
if (this.partialPacket.length === 1) return this.partialPacket[0];
|
||||
let len = 0;
|
||||
for (const s of this.partialPacket) len += s.length;
|
||||
const out = new Uint8Array(len);
|
||||
let o = 0;
|
||||
for (const s of this.partialPacket) {
|
||||
out.set(s, o);
|
||||
o += s.length;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
private findCapture(buf: Uint8Array, from: number): number {
|
||||
for (let i = from; i + 4 <= buf.length; i++) {
|
||||
if (buf[i] === OGG_CAPTURE[0] && buf[i + 1] === OGG_CAPTURE[1] &&
|
||||
buf[i + 2] === OGG_CAPTURE[2] && buf[i + 3] === OGG_CAPTURE[3]) {
|
||||
return i;
|
||||
}
|
||||
}
|
||||
return -1;
|
||||
}
|
||||
|
||||
private concat(a: Uint8Array, b: Uint8Array): Uint8Array {
|
||||
if (a.length === 0) return b;
|
||||
if (b.length === 0) return a;
|
||||
const out = new Uint8Array(a.length + b.length);
|
||||
out.set(a, 0);
|
||||
out.set(b, a.length);
|
||||
return out;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract the raw OpusHead identification-header *packet* from the sidecar's setup-header bytes (which
|
||||
* are the verbatim Ogg PAGES wrapping OpusHead + OpusTags). WebCodecs' `AudioDecoderConfig.description`
|
||||
* for Opus is the OpusHead packet (RFC 7845 §5.1), not the Ogg page — so we demux the setup pages and
|
||||
* return the first packet's bytes. Returns null if no OpusHead packet is found.
|
||||
*/
|
||||
export function extractOpusHead(setupHeaderBytes: Uint8Array): Uint8Array | null {
|
||||
// Walk pages manually (the setup region is small — at most two pages) and return the first packet
|
||||
// that starts with the OpusHead signature.
|
||||
let offset = 0;
|
||||
while (offset + OGG_PAGE_HEADER_SIZE <= setupHeaderBytes.length) {
|
||||
if (!(setupHeaderBytes[offset] === OGG_CAPTURE[0] && setupHeaderBytes[offset + 1] === OGG_CAPTURE[1] &&
|
||||
setupHeaderBytes[offset + 2] === OGG_CAPTURE[2] && setupHeaderBytes[offset + 3] === OGG_CAPTURE[3])) {
|
||||
return null;
|
||||
}
|
||||
const segCount = setupHeaderBytes[offset + SEGMENT_COUNT_OFFSET];
|
||||
const segTableEnd = offset + OGG_PAGE_HEADER_SIZE + segCount;
|
||||
if (segTableEnd > setupHeaderBytes.length) return null;
|
||||
let payloadSize = 0;
|
||||
for (let i = 0; i < segCount; i++) payloadSize += setupHeaderBytes[segTableEnd - segCount + i];
|
||||
const payloadStart = segTableEnd;
|
||||
const payloadEnd = payloadStart + payloadSize;
|
||||
if (payloadEnd > setupHeaderBytes.length) return null;
|
||||
|
||||
const payload = setupHeaderBytes.subarray(payloadStart, payloadEnd);
|
||||
if (startsWith(payload, OPUS_HEAD_SIG)) {
|
||||
// The OpusHead packet is the whole first-page payload (it always fits one segment / page).
|
||||
return payload;
|
||||
}
|
||||
offset = payloadEnd;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Channel count from an OpusHead packet (RFC 7845 §5.1: byte 9, after the 8-byte magic + version). */
|
||||
export function opusHeadChannelCount(opusHead: Uint8Array): number {
|
||||
if (opusHead.length < 10) return 2; // safe nominal
|
||||
return opusHead[9];
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
/**
|
||||
* OpusCapability - runtime detection of WebCodecs Ogg-Opus decode support.
|
||||
*
|
||||
* The Opus decode path is a WebCodecs `AudioDecoder` streaming pipeline (OpusStreamDecoder), NOT
|
||||
* `decodeAudioData`. So the capability gate must test the path actually used: whether the browser has
|
||||
* `AudioDecoder` AND supports the `codec: 'opus'` config. `AudioDecoder` is available on Chrome/Edge,
|
||||
* Firefox 130+, and Safari 16.4+; older Safari and older Firefox lack it, and those listeners fall back
|
||||
* to the universal lossless WAV path (§3.4 / OQ2 / AC7 — no listener ever gets silence over a codec gap).
|
||||
*
|
||||
* This module is the detection *seam* only — it answers "can this browser stream-decode Opus via
|
||||
* WebCodecs?". The player (StreamingAudioPlayerService.ResolveStreamFormatAsync) consumes the answer to
|
||||
* choose the delivery format; this module never touches the player or the stream request. The result is
|
||||
* cached after the first probe (capability does not change within a session).
|
||||
*/
|
||||
|
||||
let cachedSupport: Promise<boolean> | null = null;
|
||||
|
||||
/**
|
||||
* Resolve whether this browser can stream-decode Ogg Opus via WebCodecs. Cached after the first call.
|
||||
* Never rejects — any failure (no AudioDecoder, unsupported config, thrown probe) resolves to `false`
|
||||
* (treat as unsupported, fall back to lossless) so an interop error can never silence playback.
|
||||
*/
|
||||
export function canDecodeOggOpus(): Promise<boolean> {
|
||||
if (cachedSupport === null) {
|
||||
cachedSupport = probe();
|
||||
}
|
||||
return cachedSupport;
|
||||
}
|
||||
|
||||
async function probe(): Promise<boolean> {
|
||||
try {
|
||||
if (typeof AudioDecoder === 'undefined' || typeof AudioDecoder.isConfigSupported !== 'function') {
|
||||
return false;
|
||||
}
|
||||
// 48 kHz stereo is the canonical fullband Opus shape this site produces. isConfigSupported does
|
||||
// not need the OpusHead `description` to report codec support, so we probe without it.
|
||||
const result = await AudioDecoder.isConfigSupported({
|
||||
codec: 'opus',
|
||||
sampleRate: 48000,
|
||||
numberOfChannels: 2
|
||||
});
|
||||
return result.supported === true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,196 @@
|
||||
/**
|
||||
* OpusSidecar - parser for the per-track Opus seek/setup sidecar artifact.
|
||||
*
|
||||
* The sidecar is built once at transcode time (wave 18.1, C# `OpusSidecar` /
|
||||
* `OggOpusSeekIndex`) and fetched once on track load (wired by wave 18.5). It carries
|
||||
* everything the client needs to seek a VBR Opus stream accurately and to decode any
|
||||
* mid-stream slice:
|
||||
* - the verbatim OpusHead + OpusTags setup pages (prepended to every post-seek slice),
|
||||
* - the precomputed granule->byte seek index (the exact time->byte transfer function),
|
||||
* - the pre_skip and totals needed for presentation-time math and seek clamping.
|
||||
*
|
||||
* This module is the byte-for-byte counterpart to the C# serializer. It is pure: it parses
|
||||
* a blob into an `OpusSeekData` accelerator with no I/O. Wave 18.5 owns the HTTP fetch and
|
||||
* injects the parsed result into `OpusFormatDecoder.setSidecar`.
|
||||
*
|
||||
* Binary layout (all little-endian), matching DeepDrftContent.Processors.Opus:
|
||||
* [uint32 setupHeaderLength]
|
||||
* [setupHeaderLength bytes -> OpusHead + OpusTags pages]
|
||||
* [seek-index blob]:
|
||||
* header (24 bytes):
|
||||
* uint64 totalByteLength
|
||||
* double totalDurationSeconds (pre-skip-corrected)
|
||||
* uint32 pointCount
|
||||
* uint16 preSkip
|
||||
* uint16 reserved
|
||||
* pointCount x 16-byte points:
|
||||
* uint64 granulePosition (48 kHz sample count)
|
||||
* uint64 byteOffset (page-start offset in the Opus file)
|
||||
*/
|
||||
|
||||
/** Opus granule positions are always 48 kHz sample counts, regardless of input rate. */
|
||||
export const OPUS_SAMPLE_RATE = 48000;
|
||||
|
||||
/** Size of the seek-index blob header: totalBytes(8) + duration(8) + count(4) + preSkip(2) + reserved(2). */
|
||||
const SEEK_INDEX_HEADER_SIZE = 24;
|
||||
/** Size of one serialized seek point: granulepos(8) + byteOffset(8). */
|
||||
const SEEK_POINT_SIZE = 16;
|
||||
|
||||
/** One (granule, byteOffset) seek-index entry. Both are page-start-accurate. */
|
||||
export interface OpusSeekPoint {
|
||||
/** Page end granule position — a 48 kHz sample count. */
|
||||
granulePosition: number;
|
||||
/** Byte offset of the page start in the Opus file. */
|
||||
byteOffset: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parsed sidecar: the `seekData` accelerator the `OpusFormatDecoder` holds for the stream's
|
||||
* lifetime. Holds the setup bytes (for `wrapSegment` carry) and the index (for `calculateByteOffset`).
|
||||
*/
|
||||
export interface OpusSeekData {
|
||||
kind: 'opus-sidecar';
|
||||
/** Verbatim OpusHead + OpusTags pages, prepended to every decodable segment. */
|
||||
setupHeaderBytes: Uint8Array;
|
||||
/** Ordered (granule, byteOffset) entries, ascending by granule. */
|
||||
points: OpusSeekPoint[];
|
||||
/** Pre-skip-corrected total stream duration in seconds. */
|
||||
totalDurationSeconds: number;
|
||||
/** Total Opus file byte length, for clamping a seek past the end. */
|
||||
totalByteLength: number;
|
||||
/** pre_skip from OpusHead (RFC 7845 §5.1); samples to discard before presentation. */
|
||||
preSkip: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a sidecar blob produced by the C# `OpusSidecar.ToBytes`. Returns null on any structural
|
||||
* inconsistency (short blob, length prefix overrun, declared point count that does not fit) —
|
||||
* the format is exact, so a malformed blob is corruption, not a recoverable shape.
|
||||
*
|
||||
* Accepts a `Uint8Array`, an `ArrayBuffer`, or a typed-array view; copies nothing it can borrow.
|
||||
*/
|
||||
export function parseSidecar(input: Uint8Array | ArrayBuffer | ArrayBufferView): OpusSeekData | null {
|
||||
const bytes = toUint8Array(input);
|
||||
// DataView over the same backing buffer; honour the view's byteOffset so a borrowed view parses.
|
||||
const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
|
||||
|
||||
if (bytes.byteLength < 4) return null;
|
||||
|
||||
const setupLength = view.getUint32(0, true);
|
||||
const indexStart = 4 + setupLength;
|
||||
// Need the setup region plus at least the index header.
|
||||
if (bytes.byteLength < indexStart + SEEK_INDEX_HEADER_SIZE) return null;
|
||||
|
||||
// subarray is zero-copy; setup bytes are retained for wrapSegment for the stream's lifetime.
|
||||
const setupHeaderBytes = bytes.subarray(4, indexStart);
|
||||
|
||||
// Seek-index blob header (relative to the DataView, which is bytes-relative).
|
||||
const totalByteLength = readUint64(view, indexStart);
|
||||
const totalDurationSeconds = view.getFloat64(indexStart + 8, true);
|
||||
const pointCount = view.getUint32(indexStart + 16, true);
|
||||
const preSkip = view.getUint16(indexStart + 20, true);
|
||||
// bytes 22-23: reserved — ignored on read, for forward-compatibility (matches C#).
|
||||
|
||||
const pointsStart = indexStart + SEEK_INDEX_HEADER_SIZE;
|
||||
const expectedEnd = pointsStart + pointCount * SEEK_POINT_SIZE;
|
||||
if (bytes.byteLength < expectedEnd) return null;
|
||||
|
||||
const points: OpusSeekPoint[] = new Array(pointCount);
|
||||
let cursor = pointsStart;
|
||||
for (let i = 0; i < pointCount; i++) {
|
||||
const granulePosition = readUint64(view, cursor);
|
||||
const byteOffset = readUint64(view, cursor + 8);
|
||||
points[i] = { granulePosition, byteOffset };
|
||||
cursor += SEEK_POINT_SIZE;
|
||||
}
|
||||
|
||||
return {
|
||||
kind: 'opus-sidecar',
|
||||
setupHeaderBytes,
|
||||
points,
|
||||
totalDurationSeconds,
|
||||
totalByteLength,
|
||||
preSkip
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Pre-skip-corrected presentation time for a granule position: max(0, (granule - preSkip) / 48000).
|
||||
* Matches the C# `OggOpusSeekIndex.PresentationTimeSeconds` so client and server agree on the
|
||||
* seek transfer function.
|
||||
*/
|
||||
export function presentationTimeSeconds(granulePosition: number, preSkip: number): number {
|
||||
return Math.max(0, (granulePosition - preSkip) / OPUS_SAMPLE_RATE);
|
||||
}
|
||||
|
||||
/**
|
||||
* Result of resolving a seek time to a page-start byte offset.
|
||||
* `byteOffset` is the Range request origin; `landingTimeSeconds` is the actual presentation time of that
|
||||
* page (t_page ≤ positionSeconds). The caller uses the delta `positionSeconds - landingTimeSeconds` to
|
||||
* trim the decoded leading frames so playback lands at the requested position, not at t_page (AC9).
|
||||
*/
|
||||
export interface OpusSeekResolution {
|
||||
/** Page-start byte offset to use as the Range request origin (Ogg-sync-aligned). */
|
||||
byteOffset: number;
|
||||
/**
|
||||
* Presentation time of the resolved index page (seconds). Always ≤ positionSeconds. The decoder
|
||||
* must trim `(positionSeconds - landingTimeSeconds) * OPUS_SAMPLE_RATE` leading frames so the
|
||||
* audible start and the reported clock both land at positionSeconds, not at landingTimeSeconds.
|
||||
*/
|
||||
landingTimeSeconds: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve a seek time (seconds) to a file-absolute, page-start byte offset via the precomputed index —
|
||||
* the accurate VBR-safe transfer function (§3.4a A/C). Binary-searches for the largest entry whose
|
||||
* presentation time is <= `positionSeconds`. Returns both the page-start byte offset AND the actual
|
||||
* landing time of that page, so callers can trim leading decoded frames to land precisely at
|
||||
* `positionSeconds` (AC9 fine re-sync). NOT interpolation, NOT byteRate math.
|
||||
*
|
||||
* With an empty index it degrades to the start of audio (offset == setup-header length, landing == 0).
|
||||
*
|
||||
* This is the single source of truth for Opus seek-offset math, shared by the seek-beyond-buffer path
|
||||
* (AudioPlayer) and any byte-offset resolver. The Range fetch from this offset lands the decoder
|
||||
* Ogg-sync-aligned because every indexed offset is a real page start.
|
||||
*/
|
||||
export function resolveOpusByteOffset(sidecar: OpusSeekData, positionSeconds: number): OpusSeekResolution {
|
||||
const points = sidecar.points;
|
||||
if (points.length === 0) {
|
||||
return { byteOffset: sidecar.setupHeaderBytes.length, landingTimeSeconds: 0 };
|
||||
}
|
||||
|
||||
let lo = 0;
|
||||
let hi = points.length - 1;
|
||||
let best = 0;
|
||||
while (lo <= hi) {
|
||||
const mid = (lo + hi) >> 1;
|
||||
const t = presentationTimeSeconds(points[mid].granulePosition, sidecar.preSkip);
|
||||
if (t <= positionSeconds) {
|
||||
best = mid;
|
||||
lo = mid + 1;
|
||||
} else {
|
||||
hi = mid - 1;
|
||||
}
|
||||
}
|
||||
return {
|
||||
byteOffset: points[best].byteOffset,
|
||||
landingTimeSeconds: presentationTimeSeconds(points[best].granulePosition, sidecar.preSkip)
|
||||
};
|
||||
}
|
||||
|
||||
function toUint8Array(input: Uint8Array | ArrayBuffer | ArrayBufferView): Uint8Array {
|
||||
if (input instanceof Uint8Array) return input;
|
||||
if (input instanceof ArrayBuffer) return new Uint8Array(input);
|
||||
return new Uint8Array(input.buffer, input.byteOffset, input.byteLength);
|
||||
}
|
||||
|
||||
/**
|
||||
* Read a little-endian uint64 as a JS number. Opus byte offsets and granule positions are exact
|
||||
* to 2^53 (~8 PB / ~5,700 years of audio at 48 kHz), far beyond any real file — no BigInt needed,
|
||||
* matching the FLAC seektable's same 2^53 assumption.
|
||||
*/
|
||||
function readUint64(view: DataView, offset: number): number {
|
||||
const lo = view.getUint32(offset, true);
|
||||
const hi = view.getUint32(offset + 4, true);
|
||||
return hi * 0x100000000 + lo;
|
||||
}
|
||||
@@ -0,0 +1,524 @@
|
||||
/**
|
||||
* Opus WebCodecs decode-path tests — the browser-independent pieces.
|
||||
*
|
||||
* The WebCodecs decode/playback/seek itself can only run in a real browser (verified by Daniel), so
|
||||
* these tests cover the pure logic that surrounds it and that determines correctness:
|
||||
* - OggSidecar parse: byte-for-byte round-trip against the C# wire format.
|
||||
* - resolveOpusByteOffset: the seek transfer function (binary search over the precomputed index).
|
||||
* - OggDemuxer: Ogg page -> Opus packet extraction (segment-table lacing, packets spanning pages,
|
||||
* granule tracking, OpusHead/OpusTags setup-packet skipping, continuation reset).
|
||||
* - extractOpusHead / opusHeadChannelCount: pulling the WebCodecs `description` out of the sidecar.
|
||||
*
|
||||
* There is no TS test runner configured in this repo (no package.json, no jest/vitest). This is a
|
||||
* self-contained, zero-dependency test: a tiny inline assert harness, no `node:` imports beyond Buffer
|
||||
* (Node global). It is EXCLUDED from the production tsc build (tsconfig `exclude: Interop/**\/*.test.ts`)
|
||||
* so it never ships in wwwroot/js. To run it (Node 22+ strips TS types natively — no tsc, no deps), the
|
||||
* `.js` import specifiers must resolve to the COMPILED modules, so run a copy from the compiled output:
|
||||
*
|
||||
* # 1. produce the compiled modules (the normal build already does this):
|
||||
* dotnet build DeepDrftPublic/DeepDrftPublic.csproj
|
||||
* # 2. run this test next to the compiled .js siblings (Node strips the types at load):
|
||||
* cp DeepDrftPublic/Interop/audio/OpusStreamDecoder.test.ts DeepDrftPublic/wwwroot/js/audio/
|
||||
* node DeepDrftPublic/wwwroot/js/audio/OpusStreamDecoder.test.ts
|
||||
*
|
||||
* A thrown error / non-zero exit signals failure; "ALL <n> TESTS PASSED" signals success.
|
||||
*
|
||||
* The sidecar bytes built in `makeSidecar` reproduce the C# wire format byte-for-byte
|
||||
* (DeepDrftContent.Processors.Opus.OpusSidecar.ToBytes / OggOpusSeekIndex.ToBytes):
|
||||
* [uint32 setupHeaderLength][setup bytes]
|
||||
* [uint64 totalByteLength][double totalDuration][uint32 count][uint16 preSkip][uint16 reserved]
|
||||
* count x [uint64 granulePosition][uint64 byteOffset] — all little-endian.
|
||||
*/
|
||||
|
||||
import { parseSidecar, presentationTimeSeconds, resolveOpusByteOffset, OPUS_SAMPLE_RATE } from './OpusSidecar.js';
|
||||
import type { OpusSeekData, OpusSeekResolution } from './OpusSidecar.js';
|
||||
import { OggDemuxer, extractOpusHead, opusHeadChannelCount } from './OggDemuxer.js';
|
||||
import { OpusStreamDecoder } from './OpusStreamDecoder.js';
|
||||
|
||||
// --- tiny inline harness (no dependencies) ---------------------------------------------------
|
||||
let passed = 0;
|
||||
const failures: string[] = [];
|
||||
function test(name: string, fn: () => void): void {
|
||||
try {
|
||||
fn();
|
||||
passed++;
|
||||
} catch (e) {
|
||||
failures.push(`FAIL: ${name}\n ${(e as Error).message}`);
|
||||
}
|
||||
}
|
||||
function assertEqual(actual: unknown, expected: unknown, msg?: string): void {
|
||||
if (actual !== expected) {
|
||||
throw new Error(`${msg ?? 'assertEqual'}: expected ${String(expected)}, got ${String(actual)}`);
|
||||
}
|
||||
}
|
||||
function assertArray(actual: ArrayLike<number>, expected: number[], msg?: string): void {
|
||||
const a = Array.from(actual);
|
||||
if (a.length !== expected.length || a.some((v, i) => v !== expected[i])) {
|
||||
throw new Error(`${msg ?? 'assertArray'}: expected [${expected}], got [${a}]`);
|
||||
}
|
||||
}
|
||||
function assertNull(actual: unknown, msg?: string): void {
|
||||
if (actual !== null) throw new Error(`${msg ?? 'assertNull'}: expected null, got ${String(actual)}`);
|
||||
}
|
||||
function assertNotNull<T>(actual: T | null, msg?: string): T {
|
||||
if (actual === null) throw new Error(`${msg ?? 'assertNotNull'}: got null`);
|
||||
return actual;
|
||||
}
|
||||
|
||||
interface SidecarSpec {
|
||||
setupHeader: number[];
|
||||
totalByteLength: number;
|
||||
totalDuration: number;
|
||||
preSkip: number;
|
||||
points: Array<{ granule: number; byteOffset: number }>;
|
||||
}
|
||||
|
||||
/** Serialize a sidecar blob exactly as the C# OpusSidecar/OggOpusSeekIndex writers do. */
|
||||
function makeSidecar(spec: SidecarSpec): Uint8Array {
|
||||
const SEEK_INDEX_HEADER_SIZE = 24;
|
||||
const SEEK_POINT_SIZE = 16;
|
||||
const setupLen = spec.setupHeader.length;
|
||||
const total = 4 + setupLen + SEEK_INDEX_HEADER_SIZE + spec.points.length * SEEK_POINT_SIZE;
|
||||
|
||||
const bytes = new Uint8Array(total);
|
||||
const view = new DataView(bytes.buffer);
|
||||
|
||||
view.setUint32(0, setupLen, true);
|
||||
bytes.set(spec.setupHeader, 4);
|
||||
|
||||
let p = 4 + setupLen;
|
||||
writeUint64(view, p, spec.totalByteLength);
|
||||
view.setFloat64(p + 8, spec.totalDuration, true);
|
||||
view.setUint32(p + 16, spec.points.length, true);
|
||||
view.setUint16(p + 20, spec.preSkip, true);
|
||||
// bytes 22-23 reserved (zero)
|
||||
|
||||
p += SEEK_INDEX_HEADER_SIZE;
|
||||
for (const pt of spec.points) {
|
||||
writeUint64(view, p, pt.granule);
|
||||
writeUint64(view, p + 8, pt.byteOffset);
|
||||
p += SEEK_POINT_SIZE;
|
||||
}
|
||||
return bytes;
|
||||
}
|
||||
|
||||
function writeUint64(view: DataView, offset: number, value: number): void {
|
||||
view.setUint32(offset, value >>> 0, true);
|
||||
view.setUint32(offset + 4, Math.floor(value / 0x100000000), true);
|
||||
}
|
||||
|
||||
// --- parseSidecar: byte-for-byte round-trip against the C# layout -----------------------------
|
||||
|
||||
test('parseSidecar round-trips the C# binary layout exactly', () => {
|
||||
const setup = [0x4f, 0x70, 0x75, 0x73, 0x48, 0x65, 0x61, 0x64]; // "OpusHead" stand-in
|
||||
const spec: SidecarSpec = {
|
||||
setupHeader: setup,
|
||||
totalByteLength: 1_234_567,
|
||||
totalDuration: 212.5,
|
||||
preSkip: 312,
|
||||
points: [
|
||||
{ granule: 312, byteOffset: 4096 }, // first point: granule == preSkip -> t=0
|
||||
{ granule: 312 + 24000, byteOffset: 9000 }, // +0.5 s
|
||||
{ granule: 312 + 48000, byteOffset: 14000 }, // +1.0 s
|
||||
],
|
||||
};
|
||||
|
||||
const parsed: OpusSeekData = assertNotNull(parseSidecar(makeSidecar(spec)));
|
||||
assertEqual(parsed.kind, 'opus-sidecar', 'kind');
|
||||
assertArray(parsed.setupHeaderBytes, setup, 'setup header bytes');
|
||||
assertEqual(parsed.totalByteLength, spec.totalByteLength, 'totalByteLength');
|
||||
assertEqual(parsed.totalDurationSeconds, spec.totalDuration, 'totalDuration');
|
||||
assertEqual(parsed.preSkip, spec.preSkip, 'preSkip');
|
||||
assertEqual(parsed.points.length, 3, 'point count');
|
||||
assertEqual(parsed.points[1].granulePosition, 312 + 24000, 'point[1].granule');
|
||||
assertEqual(parsed.points[1].byteOffset, 9000, 'point[1].byteOffset');
|
||||
});
|
||||
|
||||
test('parseSidecar honours a borrowed view byteOffset (sidecar not at buffer start)', () => {
|
||||
const blob = makeSidecar({
|
||||
setupHeader: [1, 2, 3, 4],
|
||||
totalByteLength: 100,
|
||||
totalDuration: 1.0,
|
||||
preSkip: 0,
|
||||
points: [{ granule: 0, byteOffset: 8 }],
|
||||
});
|
||||
const padded = new Uint8Array(blob.length + 7);
|
||||
padded.set(blob, 7);
|
||||
const parsed = assertNotNull(parseSidecar(padded.subarray(7)));
|
||||
assertArray(parsed.setupHeaderBytes, [1, 2, 3, 4], 'borrowed setup bytes');
|
||||
assertEqual(parsed.points[0].byteOffset, 8, 'borrowed point offset');
|
||||
});
|
||||
|
||||
test('parseSidecar returns null on a truncated blob', () => {
|
||||
const blob = makeSidecar({
|
||||
setupHeader: [0],
|
||||
totalByteLength: 1,
|
||||
totalDuration: 0,
|
||||
preSkip: 0,
|
||||
points: [{ granule: 0, byteOffset: 0 }],
|
||||
});
|
||||
assertNull(parseSidecar(blob.subarray(0, 3)), 'short of length prefix');
|
||||
assertNull(parseSidecar(blob.subarray(0, blob.length - 4)), 'declared count overruns');
|
||||
});
|
||||
|
||||
test('presentationTimeSeconds applies preSkip and clamps at zero (RFC 7845)', () => {
|
||||
assertEqual(presentationTimeSeconds(312, 312), 0, 'granule == preSkip');
|
||||
assertEqual(presentationTimeSeconds(0, 312), 0, 'below preSkip clamps');
|
||||
assertEqual(presentationTimeSeconds(312 + OPUS_SAMPLE_RATE, 312), 1.0, '+48000 -> 1 s');
|
||||
});
|
||||
|
||||
// --- resolveOpusByteOffset: binary search over the precomputed index (exact, not interpolation) -
|
||||
|
||||
function sidecarFrom(spec: SidecarSpec): OpusSeekData {
|
||||
return assertNotNull(parseSidecar(makeSidecar(spec)), 'sidecar should parse');
|
||||
}
|
||||
|
||||
test('resolveOpusByteOffset returns the page-start of the largest entry with time <= t', () => {
|
||||
const points = [0, 1, 2, 3].map(i => ({
|
||||
granule: 1000 + i * (OPUS_SAMPLE_RATE / 2),
|
||||
byteOffset: 4096 + i * 5000,
|
||||
}));
|
||||
const sc = sidecarFrom({
|
||||
setupHeader: [9, 9, 9, 9], totalByteLength: 999_999, totalDuration: 1.5, preSkip: 1000, points,
|
||||
});
|
||||
assertEqual(resolveOpusByteOffset(sc, 0.0).byteOffset, 4096, 't=0 -> first point');
|
||||
assertEqual(resolveOpusByteOffset(sc, 0.4).byteOffset, 4096, 'just before bucket 1');
|
||||
assertEqual(resolveOpusByteOffset(sc, 0.5).byteOffset, 9096, 'exactly bucket 1');
|
||||
assertEqual(resolveOpusByteOffset(sc, 0.9).byteOffset, 9096, 'within bucket 1');
|
||||
assertEqual(resolveOpusByteOffset(sc, 1.0).byteOffset, 14096, 'exactly bucket 2');
|
||||
assertEqual(resolveOpusByteOffset(sc, 99).byteOffset, 19096, 'past end -> last point');
|
||||
});
|
||||
|
||||
test('resolveOpusByteOffset never interpolates between points', () => {
|
||||
const sc = sidecarFrom({
|
||||
setupHeader: [0], totalByteLength: 10_000, totalDuration: 1.0, preSkip: 0,
|
||||
points: [{ granule: 0, byteOffset: 100 }, { granule: OPUS_SAMPLE_RATE, byteOffset: 9000 }],
|
||||
});
|
||||
assertEqual(resolveOpusByteOffset(sc, 0.5).byteOffset, 100, 'midpoint snaps to lower page start');
|
||||
});
|
||||
|
||||
test('resolveOpusByteOffset degrades to start of audio with an empty index', () => {
|
||||
const sc = sidecarFrom({
|
||||
setupHeader: [1, 2, 3, 4, 5], totalByteLength: 0, totalDuration: 0, preSkip: 0, points: [],
|
||||
});
|
||||
// start of audio == setup header length (server emits [setup pages][audio pages]).
|
||||
assertEqual(resolveOpusByteOffset(sc, 10).byteOffset, 5, 'empty index degrades to audio start');
|
||||
});
|
||||
|
||||
// --- resolveOpusByteOffset: landingTimeSeconds (AC9 fine re-sync, §3.4a step 4) -----------------
|
||||
|
||||
test('resolveOpusByteOffset landingTimeSeconds equals the resolved page time, not the requested time', () => {
|
||||
// Index: two points at t=0 s and t=0.5 s.
|
||||
const preSkip = 312;
|
||||
const sc = sidecarFrom({
|
||||
setupHeader: [0], totalByteLength: 50_000, totalDuration: 1.5, preSkip,
|
||||
points: [
|
||||
{ granule: preSkip, byteOffset: 4096 }, // t=0
|
||||
{ granule: preSkip + OPUS_SAMPLE_RATE / 2, byteOffset: 9000 }, // t=0.5 s
|
||||
],
|
||||
});
|
||||
// Seeking to 0.3 s lands on the t=0 page; landing should be 0, not 0.3.
|
||||
const r03: OpusSeekResolution = resolveOpusByteOffset(sc, 0.3);
|
||||
assertEqual(r03.byteOffset, 4096, 'seek 0.3 -> first page offset');
|
||||
assertEqual(r03.landingTimeSeconds, 0, 'landing at t=0 (page time, not target)');
|
||||
|
||||
// Seeking to exactly 0.5 s lands on the second page; landing == requested time.
|
||||
const r05: OpusSeekResolution = resolveOpusByteOffset(sc, 0.5);
|
||||
assertEqual(r05.byteOffset, 9000, 'seek 0.5 -> second page offset');
|
||||
assertEqual(r05.landingTimeSeconds, 0.5, 'landing == requested when exact page boundary');
|
||||
});
|
||||
|
||||
test('resolveOpusByteOffset empty index returns landingTimeSeconds = 0', () => {
|
||||
const sc = sidecarFrom({
|
||||
setupHeader: [0, 1, 2], totalByteLength: 1000, totalDuration: 1.0, preSkip: 0, points: [],
|
||||
});
|
||||
const r = resolveOpusByteOffset(sc, 5.0);
|
||||
assertEqual(r.landingTimeSeconds, 0, 'empty index: landing is stream start (0 s)');
|
||||
});
|
||||
|
||||
// --- Lead-trim frame math (AC9 fine re-sync) ---------------------------------------------------
|
||||
// The trim frame count is purely arithmetic: (target - landing) * 48000, rounded, clamped to ≥0.
|
||||
// This is the exact formula in OpusStreamDecoder.reinitializeForRangeContinuation so we test it
|
||||
// independently of the browser-bound WebCodecs decode.
|
||||
|
||||
function leadTrimFrames(landingTimeSeconds: number, targetTimeSeconds: number): number {
|
||||
return Math.max(0, Math.round((targetTimeSeconds - landingTimeSeconds) * OPUS_SAMPLE_RATE));
|
||||
}
|
||||
|
||||
test('lead-trim frame count is (target - landing) * 48000, rounded', () => {
|
||||
// Page at t=0, seek to 0.3 s: trim 0.3 * 48000 = 14400 frames.
|
||||
assertEqual(leadTrimFrames(0, 0.3), 14400, 'trim for 0.3 s offset');
|
||||
// Page at t=0.5 s, seek to 0.7 s: trim 0.2 * 48000 = 9600 frames.
|
||||
assertEqual(leadTrimFrames(0.5, 0.7), 9600, 'trim for 0.2 s offset');
|
||||
// Exact page boundary: no trim needed.
|
||||
assertEqual(leadTrimFrames(0.5, 0.5), 0, 'no trim when target == landing');
|
||||
// Guard against floating-point rounding producing a tiny negative: clamp to 0.
|
||||
assertEqual(leadTrimFrames(0.5000001, 0.5), 0, 'negative rounds to zero (guard)');
|
||||
});
|
||||
|
||||
// --- OggDemuxer: page -> packet extraction ----------------------------------------------------
|
||||
//
|
||||
// Builds minimal Ogg pages by hand (no codec) so the lacing logic is exercised deterministically.
|
||||
|
||||
interface PageSpec {
|
||||
granule: number; // -1 (0xFFFF...) means "no granule"
|
||||
continued?: boolean; // header-type bit 0x01
|
||||
eos?: boolean; // header-type bit 0x04
|
||||
/** Packet payloads to lace into this page (each split into 255-byte segments per Ogg rules). */
|
||||
packets?: Uint8Array[];
|
||||
/** Raw segment lengths + payload, for hand-crafting page-spanning packets. */
|
||||
rawSegments?: number[];
|
||||
rawPayload?: Uint8Array;
|
||||
}
|
||||
|
||||
function buildPage(spec: PageSpec): Uint8Array {
|
||||
let segTable: number[];
|
||||
let payload: Uint8Array;
|
||||
|
||||
if (spec.rawSegments && spec.rawPayload) {
|
||||
segTable = spec.rawSegments;
|
||||
payload = spec.rawPayload;
|
||||
} else {
|
||||
segTable = [];
|
||||
const chunks: number[] = [];
|
||||
for (const pkt of spec.packets ?? []) {
|
||||
let remaining = pkt.length;
|
||||
let o = 0;
|
||||
// Lace: emit 255-byte segments until the final (< 255) segment terminates the packet.
|
||||
for (;;) {
|
||||
const seg = Math.min(255, remaining);
|
||||
segTable.push(seg);
|
||||
for (let i = 0; i < seg; i++) chunks.push(pkt[o + i]);
|
||||
o += seg;
|
||||
remaining -= seg;
|
||||
if (seg < 255) break; // terminating segment
|
||||
}
|
||||
}
|
||||
payload = new Uint8Array(chunks);
|
||||
}
|
||||
|
||||
const header = new Uint8Array(OGG_HDR + segTable.length + payload.length);
|
||||
header.set([0x4f, 0x67, 0x67, 0x53], 0); // "OggS"
|
||||
header[4] = 0; // version
|
||||
header[5] = (spec.continued ? 0x01 : 0) | (spec.eos ? 0x04 : 0);
|
||||
// granule (LE uint64)
|
||||
if (spec.granule < 0) {
|
||||
for (let i = 0; i < 8; i++) header[6 + i] = 0xff;
|
||||
} else {
|
||||
let g = spec.granule;
|
||||
for (let i = 0; i < 8; i++) { header[6 + i] = g & 0xff; g = Math.floor(g / 256); }
|
||||
}
|
||||
header[26] = segTable.length;
|
||||
header.set(segTable, OGG_HDR);
|
||||
header.set(payload, OGG_HDR + segTable.length);
|
||||
return header;
|
||||
}
|
||||
const OGG_HDR = 27;
|
||||
|
||||
function opusHeadPacket(channels: number, preSkip: number): Uint8Array {
|
||||
// "OpusHead"(8) version(1) channels(1) preSkip(2 LE) inputRate(4) gain(2) mapping(1) = 19 bytes
|
||||
const p = new Uint8Array(19);
|
||||
p.set([0x4f, 0x70, 0x75, 0x73, 0x48, 0x65, 0x61, 0x64], 0);
|
||||
p[8] = 1;
|
||||
p[9] = channels;
|
||||
p[10] = preSkip & 0xff;
|
||||
p[11] = (preSkip >> 8) & 0xff;
|
||||
return p;
|
||||
}
|
||||
function opusTagsPacket(): Uint8Array {
|
||||
const p = new Uint8Array(16);
|
||||
p.set([0x4f, 0x70, 0x75, 0x73, 0x54, 0x61, 0x67, 0x73], 0); // "OpusTags"
|
||||
return p;
|
||||
}
|
||||
|
||||
test('OggDemuxer skips OpusHead/OpusTags and returns audio packets with the page granule', () => {
|
||||
const head = buildPage({ granule: 0, packets: [opusHeadPacket(2, 312)] });
|
||||
const tags = buildPage({ granule: 0, packets: [opusTagsPacket()] });
|
||||
const audio = buildPage({ granule: 24000, packets: [new Uint8Array([0xaa, 0xbb]), new Uint8Array([0xcc])] });
|
||||
|
||||
const d = new OggDemuxer();
|
||||
const packets = d.push(concat([head, tags, audio]));
|
||||
assertEqual(packets.length, 2, 'two audio packets, setup skipped');
|
||||
assertArray(packets[0].data, [0xaa, 0xbb], 'first audio packet bytes');
|
||||
assertEqual(packets[0].pageGranule, null, 'non-final packet carries no granule');
|
||||
assertArray(packets[1].data, [0xcc], 'second audio packet bytes');
|
||||
assertEqual(packets[1].pageGranule, 24000, 'final completing packet carries the page granule');
|
||||
assertEqual(packets[1].isLastPage, false, 'not EOS');
|
||||
});
|
||||
|
||||
test('OggDemuxer flags the EOS page', () => {
|
||||
const head = buildPage({ granule: 0, packets: [opusHeadPacket(1, 100)] });
|
||||
const tags = buildPage({ granule: 0, packets: [opusTagsPacket()] });
|
||||
const audio = buildPage({ granule: 48000, eos: true, packets: [new Uint8Array([0x01])] });
|
||||
const d = new OggDemuxer();
|
||||
const packets = d.push(concat([head, tags, audio]));
|
||||
assertEqual(packets.length, 1, 'one audio packet');
|
||||
assertEqual(packets[0].isLastPage, true, 'EOS flagged');
|
||||
});
|
||||
|
||||
test('OggDemuxer reassembles a packet that spans two pages (255 last segment + continuation)', () => {
|
||||
const head = buildPage({ granule: 0, packets: [opusHeadPacket(2, 0)] });
|
||||
const tags = buildPage({ granule: 0, packets: [opusTagsPacket()] });
|
||||
// First audio page: one 255-byte segment that does NOT terminate (packet continues).
|
||||
const part1 = new Uint8Array(255).fill(0x11);
|
||||
const pageA = buildPage({ granule: -1, rawSegments: [255], rawPayload: part1 });
|
||||
// Second page (continued): a 10-byte terminating segment completes the packet.
|
||||
const part2 = new Uint8Array(10).fill(0x22);
|
||||
const pageB = buildPage({ granule: 24000, continued: true, rawSegments: [10], rawPayload: part2 });
|
||||
|
||||
const d = new OggDemuxer();
|
||||
const packets = d.push(concat([head, tags, pageA, pageB]));
|
||||
assertEqual(packets.length, 1, 'one reassembled packet');
|
||||
assertEqual(packets[0].data.length, 265, 'packet is 255 + 10 bytes');
|
||||
assertEqual(packets[0].data[0], 0x11, 'first byte from page A');
|
||||
assertEqual(packets[0].data[264], 0x22, 'last byte from page B');
|
||||
assertEqual(packets[0].pageGranule, 24000, 'granule from the completing page');
|
||||
});
|
||||
|
||||
test('OggDemuxer handles bytes split across push() calls (page straddles a network chunk)', () => {
|
||||
const head = buildPage({ granule: 0, packets: [opusHeadPacket(2, 0)] });
|
||||
const tags = buildPage({ granule: 0, packets: [opusTagsPacket()] });
|
||||
const audio = buildPage({ granule: 960, packets: [new Uint8Array([0x07, 0x08, 0x09])] });
|
||||
const full = concat([head, tags, audio]);
|
||||
|
||||
const d = new OggDemuxer();
|
||||
const cut = full.length - 2; // split mid-audio-page
|
||||
const first = d.push(full.subarray(0, cut));
|
||||
assertEqual(first.length, 0, 'no whole audio packet yet');
|
||||
const second = d.push(full.subarray(cut));
|
||||
assertEqual(second.length, 1, 'audio packet completes on the second push');
|
||||
assertArray(second[0].data, [0x07, 0x08, 0x09], 'reassembled across pushes');
|
||||
});
|
||||
|
||||
test('OggDemuxer.reset(continuation) treats the first page as audio (no setup expected)', () => {
|
||||
const audio = buildPage({ granule: 96000, packets: [new Uint8Array([0x42])] });
|
||||
const d = new OggDemuxer();
|
||||
d.reset(true);
|
||||
const packets = d.push(audio);
|
||||
assertEqual(packets.length, 1, 'continuation: first page is audio');
|
||||
assertArray(packets[0].data, [0x42], 'audio packet bytes');
|
||||
});
|
||||
|
||||
// --- extractOpusHead / opusHeadChannelCount: WebCodecs description from the sidecar -----------
|
||||
|
||||
test('extractOpusHead returns the OpusHead packet from the setup pages', () => {
|
||||
const head = buildPage({ granule: 0, packets: [opusHeadPacket(2, 312)] });
|
||||
const tags = buildPage({ granule: 0, packets: [opusTagsPacket()] });
|
||||
const setup = concat([head, tags]);
|
||||
const opusHead = assertNotNull(extractOpusHead(setup), 'OpusHead extracted');
|
||||
assertArray(opusHead.subarray(0, 8), [0x4f, 0x70, 0x75, 0x73, 0x48, 0x65, 0x61, 0x64], 'OpusHead magic');
|
||||
assertEqual(opusHeadChannelCount(opusHead), 2, 'channel count');
|
||||
});
|
||||
|
||||
test('extractOpusHead returns null when no OpusHead page is present', () => {
|
||||
const tags = buildPage({ granule: 0, packets: [opusTagsPacket()] });
|
||||
assertNull(extractOpusHead(tags), 'no OpusHead');
|
||||
});
|
||||
|
||||
// --- OpusStreamDecoder.totalDuration: available from the sidecar BEFORE the first push ----------
|
||||
//
|
||||
// Defect 1 (dead Opus seekbar): the C# layer locks the UI Duration on the first chunk whose result
|
||||
// carries a value, and AudioPlayer.processOpusChunk now surfaces `decoder.totalDuration` on that first
|
||||
// chunk rather than gating it on the (async, possibly-empty-on-chunk-1) decoded buffers. The load-bearing
|
||||
// guarantee that makes this correct is that `totalDuration` is known from the sidecar IMMEDIATELY — i.e.
|
||||
// before any push and without WebCodecs. These tests pin that contract; the WebCodecs decode itself stays
|
||||
// browser-verified. The constructor only stashes the context manager (totalDuration never touches it), so a
|
||||
// null-shaped stub is safe and no AudioDecoder is constructed.
|
||||
|
||||
const stubContextManager = {} as unknown as ConstructorParameters<typeof OpusStreamDecoder>[0];
|
||||
|
||||
test('OpusStreamDecoder.totalDuration is the sidecar duration, available before any push', () => {
|
||||
const sidecar = sidecarFrom({
|
||||
setupHeader: [0x4f, 0x70, 0x75, 0x73, 0x48, 0x65, 0x61, 0x64],
|
||||
totalByteLength: 500_000, totalDuration: 212.5, preSkip: 312,
|
||||
points: [{ granule: 312, byteOffset: 4096 }],
|
||||
});
|
||||
const decoder = new OpusStreamDecoder(stubContextManager, sidecar);
|
||||
// No push, no configure — the value the first chunk reports to C# must already be present.
|
||||
assertEqual(decoder.totalDuration, 212.5, 'totalDuration from sidecar, pre-push');
|
||||
});
|
||||
|
||||
test('OpusStreamDecoder.totalDuration is null when the sidecar carries no positive duration', () => {
|
||||
const sidecar = sidecarFrom({
|
||||
setupHeader: [0], totalByteLength: 0, totalDuration: 0, preSkip: 0, points: [],
|
||||
});
|
||||
const decoder = new OpusStreamDecoder(stubContextManager, sidecar);
|
||||
// A zero/absent sidecar duration must report null (not 0) so the chunk result carries no spurious
|
||||
// value — the WAV-header path, not a bogus Opus duration, then drives the UI.
|
||||
assertEqual(decoder.totalDuration, null, 'no positive duration -> null');
|
||||
});
|
||||
|
||||
// --- Phase 21.2b: Opus decode-ahead back-pressure (the stash-while-full half) ------------------
|
||||
//
|
||||
// When the shared scheduler is full, push() must NOT demux/decode ahead — it stashes the raw bytes
|
||||
// and returns nothing, so the WebCodecs decode queue and decodedQueue stay near-empty (OQ7). The
|
||||
// stash-while-full branch returns BEFORE ensureConfigured(), so it is testable without WebCodecs
|
||||
// (no AudioDecoder is constructed). The drain-on-resume path needs the real WebCodecs decoder and
|
||||
// stays browser-verified; here we pin the bound itself and the lifecycle resets.
|
||||
|
||||
// Access the private stash for white-box assertions (same idiom the scheduler tests use).
|
||||
function stashLength(decoder: OpusStreamDecoder): number {
|
||||
return (decoder as unknown as { pendingBytes: Uint8Array[] }).pendingBytes.length;
|
||||
}
|
||||
|
||||
// The stash-while-full branch returns synchronously at the top of push() (before any real await),
|
||||
// so the stash is observable immediately without awaiting the returned promise — keeping these
|
||||
// tests inside the synchronous inline harness (which does not await test bodies).
|
||||
test('push stashes bytes and decodes nothing while the scheduler is full (no decode-ahead)', () => {
|
||||
const sidecar = sidecarFrom({
|
||||
setupHeader: [0x4f, 0x70, 0x75, 0x73, 0x48, 0x65, 0x61, 0x64],
|
||||
totalByteLength: 500_000, totalDuration: 100, preSkip: 312,
|
||||
points: [{ granule: 312, byteOffset: 4096 }],
|
||||
});
|
||||
// Scheduler reports "full" → push must short-circuit before touching WebCodecs.
|
||||
const decoder = new OpusStreamDecoder(stubContextManager, sidecar, () => true);
|
||||
|
||||
void decoder.push(new Uint8Array([1, 2, 3]));
|
||||
void decoder.push(new Uint8Array([4, 5]));
|
||||
|
||||
assertEqual(stashLength(decoder), 2, 'both chunks stashed in arrival order');
|
||||
assertEqual(decoder.ready, false, 'decoder not even configured while throttled');
|
||||
});
|
||||
|
||||
test('reinitializeForRangeContinuation drops the pre-seek stash (C6 — no stale feed across reset)', () => {
|
||||
const sidecar = sidecarFrom({
|
||||
setupHeader: [0x4f, 0x70, 0x75, 0x73, 0x48, 0x65, 0x61, 0x64],
|
||||
totalByteLength: 500_000, totalDuration: 100, preSkip: 312,
|
||||
points: [{ granule: 312, byteOffset: 4096 }],
|
||||
});
|
||||
const decoder = new OpusStreamDecoder(stubContextManager, sidecar, () => true);
|
||||
void decoder.push(new Uint8Array([1, 2, 3])); // stash one chunk while full
|
||||
assertEqual(stashLength(decoder), 1, 'one chunk stashed pre-seek');
|
||||
|
||||
decoder.reinitializeForRangeContinuation(0, 5); // a seek
|
||||
assertEqual(stashLength(decoder), 0, 'pre-seek stash dropped on range-continuation');
|
||||
});
|
||||
|
||||
test('dispose clears the stash', () => {
|
||||
const sidecar = sidecarFrom({
|
||||
setupHeader: [0x4f, 0x70, 0x75, 0x73, 0x48, 0x65, 0x61, 0x64],
|
||||
totalByteLength: 500_000, totalDuration: 100, preSkip: 312,
|
||||
points: [{ granule: 312, byteOffset: 4096 }],
|
||||
});
|
||||
const decoder = new OpusStreamDecoder(stubContextManager, sidecar, () => true);
|
||||
void decoder.push(new Uint8Array([9]));
|
||||
assertEqual(stashLength(decoder), 1, 'stashed');
|
||||
decoder.dispose();
|
||||
assertEqual(stashLength(decoder), 0, 'stash cleared on dispose');
|
||||
});
|
||||
|
||||
function concat(arrs: Uint8Array[]): Uint8Array {
|
||||
let len = 0;
|
||||
for (const a of arrs) len += a.length;
|
||||
const out = new Uint8Array(len);
|
||||
let o = 0;
|
||||
for (const a of arrs) { out.set(a, o); o += a.length; }
|
||||
return out;
|
||||
}
|
||||
|
||||
// --- report ----------------------------------------------------------------------------------
|
||||
if (failures.length > 0) {
|
||||
console.error(failures.join('\n'));
|
||||
throw new Error(`${failures.length} test(s) failed, ${passed} passed`);
|
||||
}
|
||||
console.log(`ALL ${passed} TESTS PASSED`);
|
||||
@@ -0,0 +1,395 @@
|
||||
/**
|
||||
* OpusStreamDecoder - the WebCodecs streaming Opus decode pipeline.
|
||||
*
|
||||
* This replaces the fundamentally-broken per-segment `decodeAudioData` Opus model. Instead of cutting
|
||||
* the Ogg stream into page-runs and decoding each as a standalone file (which re-applies pre-skip and
|
||||
* starts from cold codec state at every boundary), it feeds a single stateful WebCodecs `AudioDecoder`
|
||||
* the demuxed Opus packets in order — correct pre-skip-once handling and full inter-frame continuity.
|
||||
*
|
||||
* Pipeline: OggDemuxer (pages -> Opus packets + granule) -> AudioDecoder (codec 'opus', configured
|
||||
* from the OpusHead in the sidecar) -> AudioData (48 kHz PCM) -> AudioBuffer -> PlaybackScheduler.
|
||||
*
|
||||
* Pre-skip (encoder delay): handled ONCE, by the decoder. WebCodecs decodes Opus with the OpusHead
|
||||
* passed as `AudioDecoderConfig.description`; the OpusHead carries `pre_skip`, and the WebCodecs Opus
|
||||
* decoder discards those leading samples itself. We do NOT re-trim per packet — doing so on top of the
|
||||
* decoder's own trim would double-count. This is the spec-intended path (W3C WebCodecs Opus registration).
|
||||
*
|
||||
* End-trim: the sidecar's `totalDurationSeconds` is the exact pre-skip-corrected stream length. We cap
|
||||
* cumulative emitted audio at that length so the final partial frame's padding does not leak past the
|
||||
* true end. (Granule-position end-trim from the EOS page is the alternative; capping on the known total
|
||||
* is equivalent and simpler, and the sidecar total is authoritative.)
|
||||
*
|
||||
* Sample rate: Opus always decodes at 48 kHz (RFC 7845). We force the AudioContext to 48 kHz at init so
|
||||
* the decoded AudioData needs no resampling before scheduling — the same `recreateWithSampleRate` seam
|
||||
* the WAV path uses for non-44.1 sources.
|
||||
*
|
||||
* BROWSER-VERIFIED. The actual decode/playback/trim correctness is verified in Daniel's browser
|
||||
* (WebCodecs cannot run in Node/headless here). The Ogg demux, packet timing, and end-trim *math* are
|
||||
* unit-tested; the WebCodecs glue (configure/decode/flush/AudioData->AudioBuffer) is browser-verified.
|
||||
*/
|
||||
|
||||
import { AudioContextManager } from './AudioContextManager.js';
|
||||
import { decodePressure } from './decodePressure.js';
|
||||
import { IStreamingDecoder } from './IStreamingDecoder.js';
|
||||
import { OggDemuxer, OpusPacket, extractOpusHead, opusHeadChannelCount } from './OggDemuxer.js';
|
||||
import { OpusSeekData, OPUS_SAMPLE_RATE } from './OpusSidecar.js';
|
||||
|
||||
/** Opus packet duration ceiling is 120 ms; at 48 kHz that is 5760 frames. Used only for chunk timestamps. */
|
||||
const MAX_PACKET_FRAMES = 5760;
|
||||
|
||||
export class OpusStreamDecoder implements IStreamingDecoder {
|
||||
private readonly contextManager: AudioContextManager;
|
||||
private readonly sidecar: OpusSeekData;
|
||||
// Phase 21.2b back-pressure hook: returns true when the shared scheduler is full (forward fill
|
||||
// over high-water). While full, push() stashes raw bytes WITHOUT demuxing/decoding so the
|
||||
// WebCodecs decode queue and decodedQueue stay near-empty behind a throttled socket (OQ7).
|
||||
// Null = no back-pressure (e.g. unit tests), in which case the decoder feeds eagerly as before.
|
||||
private readonly isSchedulerFull: (() => boolean) | null;
|
||||
|
||||
// Raw bytes received while the scheduler was full, held undemuxed until it drains. The C# read
|
||||
// loop also pauses above high-water, so this stash is bounded to at most the in-flight chunks
|
||||
// between the loop reading the productionPaused flag and actually stopping — a handful of KB,
|
||||
// not a decode-ahead. Drained (demuxed + decoded) on the next push once below high-water.
|
||||
private pendingBytes: Uint8Array[] = [];
|
||||
|
||||
private demuxer = new OggDemuxer();
|
||||
private decoder: AudioDecoder | null = null;
|
||||
private channelCount = 2;
|
||||
private configured = false;
|
||||
// OpusHead bytes used as the AudioDecoder `description`, captured once at first configure and reused
|
||||
// verbatim on a range-continuation reconfigure (avoids re-extracting / a non-null assertion).
|
||||
private opusHeadDescription: Uint8Array | null = null;
|
||||
|
||||
// Decoded AudioData awaiting conversion, filled by the AudioDecoder output callback.
|
||||
private decodedQueue: AudioData[] = [];
|
||||
private fatalError = false;
|
||||
|
||||
// Frames to discard from the head of the first post-seek decoded output (AC9 fine re-sync).
|
||||
// Set by reinitializeForRangeContinuation to (targetTimeSeconds - landingTimeSeconds) * 48000,
|
||||
// consumed frame-by-frame in audioDataToBuffer until exhausted (then zero for the rest of the stream).
|
||||
private leadTrimFrames = 0;
|
||||
|
||||
// Monotonic packet timestamp (microseconds) handed to each EncodedAudioChunk. WebCodecs requires
|
||||
// strictly increasing timestamps; the true value is irrelevant to us (we schedule by accumulation),
|
||||
// so a synthetic 48 kHz-derived counter suffices and stays exact.
|
||||
private nextTimestampUs = 0;
|
||||
|
||||
// Cumulative frames already emitted as AudioBuffers, for end-trim against the known total length.
|
||||
private emittedFrames = 0;
|
||||
private readonly totalFrames: number;
|
||||
|
||||
constructor(
|
||||
contextManager: AudioContextManager,
|
||||
sidecar: OpusSeekData,
|
||||
isSchedulerFull: (() => boolean) | null = null) {
|
||||
this.contextManager = contextManager;
|
||||
this.sidecar = sidecar;
|
||||
this.isSchedulerFull = isSchedulerFull;
|
||||
this.totalFrames = sidecar.totalDurationSeconds > 0
|
||||
? Math.round(sidecar.totalDurationSeconds * OPUS_SAMPLE_RATE)
|
||||
: Number.POSITIVE_INFINITY;
|
||||
}
|
||||
|
||||
get hasFatalError(): boolean {
|
||||
return this.fatalError;
|
||||
}
|
||||
|
||||
get ready(): boolean {
|
||||
return this.configured;
|
||||
}
|
||||
|
||||
get totalDuration(): number | null {
|
||||
return this.sidecar.totalDurationSeconds > 0 ? this.sidecar.totalDurationSeconds : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Lazily build + configure the WebCodecs decoder from the sidecar's OpusHead. Idempotent. Forces the
|
||||
* AudioContext to 48 kHz so decoded AudioData schedules without resampling. Returns false on a config
|
||||
* the browser cannot support (caller should never reach here — the capability gate runs first — but
|
||||
* we fail safe rather than throw into the stream loop).
|
||||
*/
|
||||
private async ensureConfigured(): Promise<boolean> {
|
||||
if (this.configured) return true;
|
||||
if (typeof AudioDecoder === 'undefined') {
|
||||
this.fatalError = true;
|
||||
return false;
|
||||
}
|
||||
|
||||
const opusHead = extractOpusHead(this.sidecar.setupHeaderBytes);
|
||||
if (!opusHead) {
|
||||
this.fatalError = true;
|
||||
return false;
|
||||
}
|
||||
this.channelCount = opusHeadChannelCount(opusHead);
|
||||
// Copy the OpusHead into a standalone buffer — the sidecar subarray is a view we keep.
|
||||
this.opusHeadDescription = opusHead.slice();
|
||||
|
||||
// Opus decodes at 48 kHz; align the context so no resample is needed. AudioPlayer.initializeStreaming
|
||||
// already aligned it to 48 kHz up front (the format is resolved before any bytes flow), so in the
|
||||
// common path this is an early-return no-op — the live graph is NOT close()'d and rebuilt mid-decode.
|
||||
// Kept as the defensive backstop for any path that reaches a configured decoder on a non-48 kHz
|
||||
// context (the same recreate seam the WAV path uses for non-44.1 sources).
|
||||
if (this.contextManager.sampleRate !== OPUS_SAMPLE_RATE) {
|
||||
await this.contextManager.recreateWithSampleRate(OPUS_SAMPLE_RATE);
|
||||
}
|
||||
|
||||
this.decoder = new AudioDecoder({
|
||||
output: (data) => this.decodedQueue.push(data),
|
||||
error: (err) => {
|
||||
console.error('Opus AudioDecoder error:', err.message);
|
||||
this.fatalError = true;
|
||||
}
|
||||
});
|
||||
this.decoder.configure(this.buildConfig());
|
||||
this.configured = true;
|
||||
return true;
|
||||
}
|
||||
|
||||
private buildConfig(): AudioDecoderConfig {
|
||||
return {
|
||||
codec: 'opus',
|
||||
sampleRate: OPUS_SAMPLE_RATE,
|
||||
numberOfChannels: this.channelCount,
|
||||
description: this.opusHeadDescription ?? undefined
|
||||
};
|
||||
}
|
||||
|
||||
async push(chunk: Uint8Array): Promise<AudioBuffer[]> {
|
||||
if (this.fatalError) return [];
|
||||
|
||||
// 21.2b back-pressure: while the scheduler is full, do NOT demux/decode ahead. Stash the
|
||||
// raw bytes in arrival order and return nothing — the WebCodecs decode queue and
|
||||
// decodedQueue stay near-empty (OQ7). The bytes are demuxed/decoded on a later push once
|
||||
// the scheduler has drained below low-water, in exactly the order received (Ogg demux is
|
||||
// order-sensitive). configure() is deferred too — no need to spin up the decoder while
|
||||
// throttled. The C# loop also stops reading above high-water, so the stash stays small.
|
||||
if (this.isSchedulerFull?.()) {
|
||||
this.pendingBytes.push(chunk);
|
||||
return [];
|
||||
}
|
||||
|
||||
if (!(await this.ensureConfigured())) return [];
|
||||
|
||||
// Drained below high-water: replay any stashed bytes first (preserving stream order), then
|
||||
// the new chunk, through the demuxer as one contiguous feed.
|
||||
const out: AudioBuffer[] = [];
|
||||
if (this.pendingBytes.length > 0) {
|
||||
const stashed = this.pendingBytes;
|
||||
this.pendingBytes = [];
|
||||
for (const bytes of stashed) {
|
||||
this.decodePackets(this.demuxer.push(bytes));
|
||||
}
|
||||
}
|
||||
|
||||
this.decodePackets(this.demuxer.push(chunk));
|
||||
// Wait until the WebCodecs decoder has processed the queued packets before draining.
|
||||
await this.yieldToDecoder();
|
||||
out.push(...this.drainDecoded());
|
||||
return out;
|
||||
}
|
||||
|
||||
async complete(): Promise<AudioBuffer[]> {
|
||||
if (this.fatalError) {
|
||||
return this.drainDecoded();
|
||||
}
|
||||
|
||||
// End-of-stream may arrive while still throttled with bytes stashed (e.g. a short track
|
||||
// that finished sending before the scheduler drained). Configure if needed and replay the
|
||||
// stash so the tail is decoded before flush — otherwise the final seconds would be lost.
|
||||
//
|
||||
// OQ7/AC1-Opus precision note: the stash here is drained in full without a water-mark
|
||||
// check. This is intentionally correct: the stream has ended — you cannot back-pressure a
|
||||
// finished stream — and the remainder is tail-only (bounded by whatever the throttled C#
|
||||
// loop left in flight, which is at most one push() worth of bytes). Adding a water-mark
|
||||
// gate to complete() would silently drop the track's tail and is therefore wrong.
|
||||
if (this.pendingBytes.length > 0) {
|
||||
if (await this.ensureConfigured()) {
|
||||
const stashed = this.pendingBytes;
|
||||
this.pendingBytes = [];
|
||||
for (const bytes of stashed) {
|
||||
this.decodePackets(this.demuxer.push(bytes));
|
||||
}
|
||||
} else {
|
||||
this.pendingBytes = [];
|
||||
}
|
||||
}
|
||||
|
||||
if (!this.decoder || this.decoder.state !== 'configured') {
|
||||
return this.drainDecoded();
|
||||
}
|
||||
try {
|
||||
await this.decoder.flush();
|
||||
} catch (err) {
|
||||
// A flush can reject if the decoder was reset/closed concurrently (track switch); the loop's
|
||||
// own cancellation handles that — surface nothing, just drain what we have.
|
||||
console.warn('Opus decoder flush interrupted:', (err as Error).message);
|
||||
}
|
||||
return this.drainDecoded();
|
||||
}
|
||||
|
||||
/**
|
||||
* Reinitialize for a Range-continuation stream after seek-beyond-buffer.
|
||||
*
|
||||
* @param landingTimeSeconds The actual page-start presentation time resolved from the seek index
|
||||
* (t_page ≤ targetTimeSeconds). This is the time at which the decoder
|
||||
* will begin emitting audio after reconfigure.
|
||||
* @param targetTimeSeconds The user-requested seek position. The difference
|
||||
* `(target - landing) * OPUS_SAMPLE_RATE` frames are trimmed from the
|
||||
* head of the decoded output so playback lands precisely at the target
|
||||
* (AC9 fine re-sync, §3.4a step 4).
|
||||
*
|
||||
* Pre-skip note: the reconfigure re-applies the WebCodecs Opus decoder's own pre-skip trim. The
|
||||
* W3C spec is non-normative on the exact sample count and browsers vary (~312 samples at 48 kHz in
|
||||
* practice). `leadTrimFrames` is computed from the sidecar's pre-skip-corrected presentation times
|
||||
* (via `presentationTimeSeconds`), so it does NOT double-count the per-reconfigure pre-skip; the
|
||||
* decoder handles that internally. If browser testing reveals a residual offset, adjust the
|
||||
* `leadTrimFrames` calculation here — this is the single point of control.
|
||||
*/
|
||||
reinitializeForRangeContinuation(landingTimeSeconds: number, targetTimeSeconds: number): void {
|
||||
// New 206 body starts on a page boundary with no setup pages; the codec config is unchanged but
|
||||
// inter-frame state must restart cleanly. AudioDecoder.reset() drops queued work and returns the
|
||||
// decoder to 'unconfigured', so we reconfigure with the cached config. The demuxer goes into
|
||||
// continuation mode (treat the first page's packets as audio — no setup pages in a 206 body).
|
||||
this.demuxer.reset(true);
|
||||
this.decodedQueue = [];
|
||||
// Drop any bytes stashed by back-pressure: they belong to the PRE-seek stream position and
|
||||
// must never be replayed against the post-seek (range-continuation) demux state (C6 — no
|
||||
// stale feed racing the reset).
|
||||
this.pendingBytes = [];
|
||||
this.emittedFrames = 0; // post-seek buffers are positioned by the scheduler's playbackOffset
|
||||
// Arm the lead trim: skip enough decoded frames to land at targetTimeSeconds, not at
|
||||
// landingTimeSeconds (the page start). Clamp to ≥0 to guard against floating-point rounding.
|
||||
this.leadTrimFrames = Math.max(0, Math.round((targetTimeSeconds - landingTimeSeconds) * OPUS_SAMPLE_RATE));
|
||||
if (this.decoder && this.decoder.state === 'configured') {
|
||||
this.decoder.reset();
|
||||
this.decoder.configure(this.buildConfig());
|
||||
}
|
||||
}
|
||||
|
||||
dispose(): void {
|
||||
for (const d of this.decodedQueue) {
|
||||
try { d.close(); } catch { /* already closed */ }
|
||||
}
|
||||
this.decodedQueue = [];
|
||||
this.pendingBytes = [];
|
||||
if (this.decoder && this.decoder.state !== 'closed') {
|
||||
try { this.decoder.close(); } catch { /* already closed */ }
|
||||
}
|
||||
this.decoder = null;
|
||||
this.configured = false;
|
||||
}
|
||||
|
||||
private decodePackets(packets: OpusPacket[]): void {
|
||||
if (!this.decoder || this.decoder.state !== 'configured') return;
|
||||
for (const pkt of packets) {
|
||||
if (pkt.data.length === 0) continue;
|
||||
// Every Opus packet is independently a "key" frame at the container level for WebCodecs's
|
||||
// purposes — Opus has no key/delta distinction; 'key' is the correct type for all packets.
|
||||
const chunk = new EncodedAudioChunk({
|
||||
type: 'key',
|
||||
timestamp: this.nextTimestampUs,
|
||||
data: pkt.data
|
||||
});
|
||||
// Advance the synthetic clock by a packet's max duration; exact value is immaterial to us.
|
||||
this.nextTimestampUs += Math.round((MAX_PACKET_FRAMES / OPUS_SAMPLE_RATE) * 1_000_000);
|
||||
try {
|
||||
this.decoder.decode(chunk);
|
||||
} catch (err) {
|
||||
console.error('Opus decode() threw:', (err as Error).message);
|
||||
this.fatalError = true;
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert every queued AudioData into an AudioBuffer at the context sample rate, applying
|
||||
* end-trim against the known total frame count and lead-trim for post-seek fine re-sync.
|
||||
*/
|
||||
private drainDecoded(): AudioBuffer[] {
|
||||
const out: AudioBuffer[] = [];
|
||||
const ctx = this.contextManager.getContext();
|
||||
|
||||
while (this.decodedQueue.length > 0) {
|
||||
const data = this.decodedQueue.shift()!;
|
||||
try {
|
||||
const buffer = this.audioDataToBuffer(ctx, data);
|
||||
if (buffer) out.push(buffer);
|
||||
} finally {
|
||||
try { data.close(); } catch { /* already closed */ }
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Copy an AudioData's PCM into a new AudioBuffer, applying:
|
||||
* 1. Lead-trim (post-seek fine re-sync): skip `leadTrimFrames` from the front so the audible
|
||||
* start lands at the requested seek position, not at the preceding page boundary (AC9).
|
||||
* 2. End-trim: cap cumulative output at `totalFrames` so the final partial frame's padding
|
||||
* does not leak past the true stream end.
|
||||
* Returns null when either trim leaves zero usable frames.
|
||||
*/
|
||||
private audioDataToBuffer(ctx: BaseAudioContext, data: AudioData): AudioBuffer | null {
|
||||
const frames = data.numberOfFrames;
|
||||
const channels = data.numberOfChannels;
|
||||
|
||||
// Lead-trim: consume frames from the front for post-seek fine re-sync (AC9).
|
||||
let skip = 0;
|
||||
if (this.leadTrimFrames > 0) {
|
||||
skip = Math.min(this.leadTrimFrames, frames);
|
||||
this.leadTrimFrames -= skip;
|
||||
}
|
||||
const available = frames - skip;
|
||||
if (available <= 0) return null;
|
||||
|
||||
// End-trim: cap cumulative output at totalFrames.
|
||||
let keep = available;
|
||||
if (Number.isFinite(this.totalFrames)) {
|
||||
const room = this.totalFrames - this.emittedFrames;
|
||||
if (room <= 0) return null;
|
||||
if (room < available) keep = room;
|
||||
}
|
||||
if (keep <= 0) return null;
|
||||
|
||||
const buffer = ctx.createBuffer(channels, keep, data.sampleRate);
|
||||
// Allocate only for the frames we actually copy; frameOffset skips the lead-trim region.
|
||||
const plane = new Float32Array(keep);
|
||||
for (let ch = 0; ch < channels; ch++) {
|
||||
data.copyTo(plane, { planeIndex: ch, frameOffset: skip, frameCount: keep, format: 'f32-planar' });
|
||||
buffer.copyToChannel(plane, ch);
|
||||
}
|
||||
this.emittedFrames += keep;
|
||||
return buffer;
|
||||
}
|
||||
|
||||
/**
|
||||
* Wait until the AudioDecoder's internal work queue drains (decodeQueueSize → 0), so output
|
||||
* callbacks have fired before we drain decodedQueue. Bounded to MAX_YIELD_ITERS × 4 ms to guard
|
||||
* against a stuck decoder; any outputs collected before the cap are still returned. `complete()`
|
||||
* uses decoder.flush() as its final barrier instead (flush() is the authoritative end-of-stream
|
||||
* drain).
|
||||
*/
|
||||
private yieldToDecoder(): Promise<void> {
|
||||
const MAX_YIELD_ITERS = 50; // 50 × 4 ms = 200 ms ceiling
|
||||
return new Promise<void>((resolve) => {
|
||||
let iters = 0;
|
||||
const poll = () => {
|
||||
if (!this.decoder || this.decoder.decodeQueueSize === 0 || iters >= MAX_YIELD_ITERS) {
|
||||
// Hitting the 200 ms ceiling with the decode queue still non-empty means the WebCodecs
|
||||
// decoder is falling behind realtime — the decode-starvation symptom that worsens with
|
||||
// HW accel off (software WebGL render contending for the main thread). Report it as
|
||||
// decode pressure so the visualizer throttles and yields the main thread back to decode.
|
||||
if (this.decoder && iters >= MAX_YIELD_ITERS && this.decoder.decodeQueueSize > 0) {
|
||||
decodePressure.report();
|
||||
}
|
||||
resolve();
|
||||
return;
|
||||
}
|
||||
iters++;
|
||||
setTimeout(poll, 4);
|
||||
};
|
||||
poll();
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,834 @@
|
||||
/**
|
||||
* PlaybackScheduler partial-eviction tests (Phase 21.1) — the anchor/index bookkeeping.
|
||||
*
|
||||
* The crux of 21.1 is that getCurrentPosition / playFromPosition / the schedule loop stay
|
||||
* exact against a buffer array that no longer begins at absolute time 0 after front eviction.
|
||||
* That math is pure given a clock and buffer durations, so it is testable in Node without a
|
||||
* browser by injecting fakes for AudioContextManager and AudioBuffer (the scheduler only ever
|
||||
* reads contextManager.currentTime, getGainNode(), getContext().createBufferSource(), and
|
||||
* buffer.duration).
|
||||
*
|
||||
* Same harness convention as OpusStreamDecoder.test.ts: no test runner in this repo, run a
|
||||
* copy from the COMPILED output so the `.js` import specifier resolves:
|
||||
*
|
||||
* dotnet build DeepDrftPublic/DeepDrftPublic.csproj
|
||||
* cp DeepDrftPublic/Interop/audio/PlaybackScheduler.test.ts DeepDrftPublic/wwwroot/js/audio/
|
||||
* node DeepDrftPublic/wwwroot/js/audio/PlaybackScheduler.test.ts
|
||||
*
|
||||
* A thrown error / non-zero exit signals failure; "ALL <n> TESTS PASSED" signals success.
|
||||
* Excluded from the production tsc build via tsconfig `exclude: Interop/ ** /*.test.ts`.
|
||||
*/
|
||||
|
||||
import { PlaybackScheduler } from './PlaybackScheduler.js';
|
||||
import type { AudioContextManager } from './AudioContextManager.js';
|
||||
|
||||
// --- tiny inline harness (no dependencies) ---------------------------------------------------
|
||||
let passed = 0;
|
||||
const failures: string[] = [];
|
||||
function test(name: string, fn: () => void): void {
|
||||
try {
|
||||
fn();
|
||||
passed++;
|
||||
} catch (e) {
|
||||
failures.push(`FAIL: ${name}\n ${(e as Error).message}`);
|
||||
}
|
||||
}
|
||||
function assertClose(actual: number, expected: number, msg?: string, eps = 1e-9): void {
|
||||
if (Math.abs(actual - expected) > eps) {
|
||||
throw new Error(`${msg ?? 'assertClose'}: expected ${expected}, got ${actual}`);
|
||||
}
|
||||
}
|
||||
function assertEqual(actual: unknown, expected: unknown, msg?: string): void {
|
||||
if (actual !== expected) {
|
||||
throw new Error(`${msg ?? 'assertEqual'}: expected ${String(expected)}, got ${String(actual)}`);
|
||||
}
|
||||
}
|
||||
|
||||
// --- fakes -----------------------------------------------------------------------------------
|
||||
|
||||
/** A buffer source that records start/stop and fires onended on demand. */
|
||||
class FakeSource {
|
||||
public buffer: unknown = null;
|
||||
public onended: (() => void) | null = null;
|
||||
public started = false;
|
||||
public stopped = false;
|
||||
connect(): void { /* no-op */ }
|
||||
start(): void { this.started = true; }
|
||||
stop(): void {
|
||||
this.stopped = true;
|
||||
// The real Web Audio fires onended when a source is stopped; the scheduler relies on
|
||||
// that for cleanup. Mirror it so handleSourceEnded paths are exercised.
|
||||
this.onended?.();
|
||||
}
|
||||
}
|
||||
|
||||
/** Controllable clock + the minimal AudioContext surface the scheduler touches. */
|
||||
class FakeContextManager {
|
||||
public now = 0;
|
||||
public sources: FakeSource[] = [];
|
||||
get currentTime(): number { return this.now; }
|
||||
getGainNode(): unknown { return {}; }
|
||||
getContext(): unknown {
|
||||
const self = this;
|
||||
return {
|
||||
createBufferSource(): FakeSource {
|
||||
const s = new FakeSource();
|
||||
self.sources.push(s);
|
||||
return s;
|
||||
}
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/** A decoded buffer is, for the scheduler's purposes, just a duration. */
|
||||
function buf(duration: number): AudioBuffer {
|
||||
return { duration } as AudioBuffer;
|
||||
}
|
||||
|
||||
/**
|
||||
* A decoded buffer carrying realistic byte-footprint fields (length + numberOfChannels) for the
|
||||
* OQ3 byte-ceiling test. Models 48 kHz stereo float PCM: length = duration × 48000 frames, 2 ch.
|
||||
*/
|
||||
function bufBytes(duration: number): AudioBuffer {
|
||||
return { duration, length: Math.round(duration * 48000), numberOfChannels: 2 } as AudioBuffer;
|
||||
}
|
||||
|
||||
function makeScheduler(cm: FakeContextManager): PlaybackScheduler {
|
||||
// The scheduler only uses the subset FakeContextManager implements.
|
||||
return new PlaybackScheduler(cm as unknown as AudioContextManager);
|
||||
}
|
||||
|
||||
/**
|
||||
* Drive the schedule cursor to the end of the buffer array WITHOUT running playback to
|
||||
* completion, then clear the live-source set so neither nextBufferIndex nor a live source
|
||||
* pins eviction. This isolates the back-retain threshold math from the live-frontier guards
|
||||
* (which are exercised by their own tests).
|
||||
*
|
||||
* The lookahead in scheduleBuffersFrom only schedules ~500ms ahead per call; pushing the clock
|
||||
* far back makes "lookahead" small so a single scheduleNewBuffers() call schedules everything
|
||||
* remaining. We then drop the (white-box) live-source list and reset the schedule cursor to the
|
||||
* end, leaving the array intact for a direct evictPlayedBuffers() call at a chosen position.
|
||||
*/
|
||||
function advanceCursorToEnd(s: PlaybackScheduler, cm: FakeContextManager): void {
|
||||
const priv = s as unknown as { nextScheduleTime: number; nextBufferIndex: number; scheduledSources: unknown[] };
|
||||
// Make the existing schedule anchor look "now" so the lookahead window is tiny, then let
|
||||
// the scheduler lay down every remaining buffer in one pass.
|
||||
priv.nextScheduleTime = cm.now;
|
||||
s.scheduleNewBuffers();
|
||||
// Repeat until the cursor reaches the end (lookahead may break early on long arrays).
|
||||
let guard = 0;
|
||||
while ((priv.nextBufferIndex as number) < s.getBufferCount() && guard++ < 1000) {
|
||||
priv.nextScheduleTime = cm.now;
|
||||
s.scheduleNewBuffers();
|
||||
}
|
||||
// Unpin the front: discard live sources without firing the onended cascade.
|
||||
cm.sources.forEach(x => { x.onended = null; x.stopped = true; });
|
||||
priv.scheduledSources.length = 0;
|
||||
}
|
||||
|
||||
// --- tests -----------------------------------------------------------------------------------
|
||||
|
||||
// Position correctness after eviction: query current position after the front of the buffer
|
||||
// array has been evicted; it must still equal wall-clock track time.
|
||||
test('position stays exact after a front eviction', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
s.setBackRetainSeconds(0); // retain nothing behind the playhead — evict aggressively
|
||||
// Ten 1s buffers, track [0,10).
|
||||
for (let i = 0; i < 10; i++) s.addBuffer(buf(1));
|
||||
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0); // schedules a 500ms lookahead worth of sources from index 0
|
||||
advanceCursorToEnd(s, cm);
|
||||
|
||||
cm.now = 3.0;
|
||||
const dropped = s.evictPlayedBuffers();
|
||||
if (dropped <= 0) throw new Error('expected front buffers to be evicted at t=3 with 0s retain');
|
||||
|
||||
// Absolute position must read 3.0 regardless of how many front buffers were dropped.
|
||||
assertClose(s.getCurrentPosition(), 3.0, 'position after eviction');
|
||||
// And buffers[0] no longer being the track start is reflected in the advanced offset.
|
||||
if (s.getPlaybackOffset() <= 0) {
|
||||
throw new Error('expected playbackOffset to advance past 0 after eviction');
|
||||
}
|
||||
});
|
||||
|
||||
// Eviction threshold respected: buffers older than back-retain are released; those within are
|
||||
// kept. With back-retain = 2s at position 5, end<=3 is droppable, end in (3,..] is retained.
|
||||
// Driven deterministically: advance the schedule cursor to the end (so nextBufferIndex does
|
||||
// not pin eviction), clear live sources, then call eviction directly at a known position.
|
||||
test('back-retain bound governs what is evicted', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
s.setBackRetainSeconds(2);
|
||||
for (let i = 0; i < 10; i++) s.addBuffer(buf(1)); // track [0,10)
|
||||
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0);
|
||||
advanceCursorToEnd(s, cm); // nextBufferIndex == 10, no live sources
|
||||
|
||||
cm.now = 5.0; // playhead at absolute t=5
|
||||
const evicted = s.evictPlayedBuffers();
|
||||
|
||||
// currentPosition is 5.0; backRetain 2 => evictBefore = 3. Buffers ending at 1,2,3 are
|
||||
// droppable (3 buffers); the buffer ending at 4 must be retained.
|
||||
assertEqual(evicted, 3, 'evicted count under 2s back-retain at t=5');
|
||||
assertEqual(s.getBufferCount(), 7, 'seven buffers retained');
|
||||
assertClose(s.getPlaybackOffset(), 3.0, 'offset == dropped duration');
|
||||
assertClose(s.getCurrentPosition(), 5.0, 'position unchanged by eviction');
|
||||
});
|
||||
|
||||
// Resume-after-pause with an evicted front: playFromPosition resumes at the correct absolute
|
||||
// time against the shortened array.
|
||||
test('resume after pause lands at correct absolute time post-eviction', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
s.setBackRetainSeconds(1);
|
||||
for (let i = 0; i < 10; i++) s.addBuffer(buf(1)); // [0,10)
|
||||
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0);
|
||||
advanceCursorToEnd(s, cm);
|
||||
|
||||
cm.now = 4.0;
|
||||
s.evictPlayedBuffers(); // back-retain 1 at t=4 => drops buffers ending <=3 (3 buffers)
|
||||
|
||||
// Pause at t=4: returns absolute position 4.0.
|
||||
const paused = s.pause();
|
||||
assertClose(paused, 4.0, 'pause returns absolute position');
|
||||
// Front was evicted, so offset advanced. The buffer-relative anchor must net to absolute 4.
|
||||
assertClose(s.getCurrentPosition(), 4.0, 'position holds at 4 while paused');
|
||||
|
||||
// Resume the way AudioPlayer.play does: buffer-relative = absolute - offset.
|
||||
cm.now = 4.0;
|
||||
const bufferRelative = paused - s.getPlaybackOffset();
|
||||
if (bufferRelative < 0) throw new Error('buffer-relative resume position went negative');
|
||||
s.playFromPosition(bufferRelative);
|
||||
cm.now = 4.0;
|
||||
assertClose(s.getCurrentPosition(), 4.0, 'resume restored absolute position');
|
||||
});
|
||||
|
||||
// Seek-back into still-retained buffers works: with back-retain holding recent audio, a short
|
||||
// backward seek stays in-buffer (queryable/playable), no clamp to the new front.
|
||||
test('short seek-back into retained region resolves in-buffer', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
s.setBackRetainSeconds(3);
|
||||
for (let i = 0; i < 10; i++) s.addBuffer(buf(1)); // [0,10)
|
||||
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0);
|
||||
advanceCursorToEnd(s, cm);
|
||||
|
||||
cm.now = 6.0;
|
||||
s.evictPlayedBuffers(); // back-retain 3 at t=6 => evictBefore=3, drops buffers ending <=3
|
||||
|
||||
const offset = s.getPlaybackOffset();
|
||||
// back-retain 3 at t=6 => evictBefore=3, so buffers ending <=3 dropped, offset==3.
|
||||
assertClose(offset, 3.0, 'offset after eviction with 3s retain');
|
||||
|
||||
// The retained region is [offset, totalEnd) == [3, 10). A seek back to t=4 is inside it.
|
||||
const seekTarget = 4.0;
|
||||
const bufferRelative = seekTarget - offset; // 1.0 into the retained array
|
||||
if (bufferRelative < 0) throw new Error('seek-back target fell below retained front (should be in-buffer)');
|
||||
cm.now = 6.0;
|
||||
s.playFromPosition(bufferRelative);
|
||||
cm.now = 6.0;
|
||||
assertClose(s.getCurrentPosition(), seekTarget, 'seek-back resolved to absolute target');
|
||||
});
|
||||
|
||||
// Eviction never crosses the live frontier: a buffer still referenced by an unstopped source
|
||||
// must not be dropped even if the clock says it is "behind".
|
||||
test('eviction does not drop buffers under live sources or past the schedule cursor', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
s.setBackRetainSeconds(0);
|
||||
for (let i = 0; i < 10; i++) s.addBuffer(buf(1));
|
||||
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0); // schedules ~first 500ms+ of sources; they remain live (not ended)
|
||||
|
||||
// Jump the clock far ahead WITHOUT ending the live sources.
|
||||
cm.now = 9.0;
|
||||
const before = s.getBufferCount();
|
||||
const dropped = s.evictPlayedBuffers();
|
||||
|
||||
// Nothing past the schedule cursor or under a live source may be dropped. The scheduled
|
||||
// (live) sources pin the front, so eviction is bounded — it must not strip the whole array.
|
||||
if (s.getBufferCount() < 0) throw new Error('buffer count went negative');
|
||||
assertEqual(s.getBufferCount(), before - dropped, 'count matches dropped');
|
||||
// The live sources start at index 0, so firstLiveIndex pins eviction at 0 — nothing drops.
|
||||
assertEqual(dropped, 0, 'no eviction while front sources are live');
|
||||
});
|
||||
|
||||
// handleSourceEnded cascade: eviction fires from the real production trigger (onended), not
|
||||
// via a direct evictPlayedBuffers() call. Confirms the anchor/index invariants hold end-to-end
|
||||
// through the scheduler's own event handling while playback is still active with a live source.
|
||||
//
|
||||
// Setup: 0.3s buffers so the 500ms lookahead window fits exactly two sources after
|
||||
// playFromPosition(0). Buffer 0 ends at ~0.31s, buffer 1 ends at ~0.61s — both are scheduled.
|
||||
// Clock is then advanced to t=0.6 so buffer 0's end (0.31) < evictBefore (0.6) while the live
|
||||
// source on buffer 1 pins firstLiveIndex=1, blocking further eviction. This is the mid-array
|
||||
// pinning scenario that later waves (21.2/21.3) build on.
|
||||
test('eviction via handleSourceEnded: position exact, live bufferIndex decremented, frontier respected', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
// Retain nothing behind the playhead — evict aggressively so the cascade fires.
|
||||
s.setBackRetainSeconds(0);
|
||||
|
||||
// Eight 0.3s buffers. scheduleBuffersFrom with lookaheadTarget=0.5s at t=0:
|
||||
// after buf 0: nextScheduleTime≈0.31, lookahead=0.31 < 0.5 → continues
|
||||
// after buf 1: nextScheduleTime≈0.61, lookahead=0.61 > 0.5 → breaks
|
||||
// → exactly two sources are live after playFromPosition.
|
||||
for (let i = 0; i < 8; i++) s.addBuffer(buf(0.3));
|
||||
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0);
|
||||
|
||||
// Reach inside to see which sources were scheduled and what bufferIndex they hold.
|
||||
const priv = s as unknown as {
|
||||
scheduledSources: Array<{ source: FakeSource; bufferIndex: number; startTime: number; endTime: number }>;
|
||||
nextBufferIndex: number;
|
||||
};
|
||||
|
||||
// Confirm two sources are live — the setup guarantee.
|
||||
if (priv.scheduledSources.length < 2) {
|
||||
throw new Error(`Expected ≥2 scheduled sources after playFromPosition, got ${priv.scheduledSources.length}`);
|
||||
}
|
||||
|
||||
// Identify the first and second scheduled sources by bufferIndex order.
|
||||
const sorted = [...priv.scheduledSources].sort((a, b) => a.bufferIndex - b.bufferIndex);
|
||||
const firstScheduled = sorted[0]; // bufferIndex 0
|
||||
const secondScheduled = sorted[1]; // bufferIndex 1
|
||||
const secondBufferIndexBefore = secondScheduled.bufferIndex; // must be 1
|
||||
|
||||
// Record the second FakeSource so we can assert it was not stopped by eviction.
|
||||
const secondFakeSource = secondScheduled.source as unknown as FakeSource;
|
||||
|
||||
// Advance clock to 0.6s. Buffer 0 ends at ~0.31s → evictBefore=0.6, end=0.31 ≤ 0.6 →
|
||||
// droppable. Buffer 1 ends at ~0.61s → its live source pins firstLiveIndex=1 → NOT dropped.
|
||||
cm.now = 0.6;
|
||||
|
||||
// Confirm playback is still active before firing the cascade.
|
||||
assertEqual(s.isActive(), true, 'isActive must be true before cascade');
|
||||
|
||||
// Fire the cascade via the production trigger: stop the first source, which calls onended,
|
||||
// which calls handleSourceEnded, which calls evictPlayedBuffers internally.
|
||||
(firstScheduled.source as unknown as FakeSource).stop();
|
||||
|
||||
// (a) Absolute position must remain exactly 0.6.
|
||||
assertClose(s.getCurrentPosition(), 0.6, 'position after handleSourceEnded cascade');
|
||||
|
||||
// (b) The second live source's bufferIndex must have been decremented by 1 (the one evicted
|
||||
// front buffer), shifting it from absolute index 1 to absolute index 0.
|
||||
const expectedSecondIndex = secondBufferIndexBefore - 1;
|
||||
assertEqual(secondScheduled.bufferIndex, expectedSecondIndex, 'live source bufferIndex decremented');
|
||||
|
||||
// (c) Eviction stopped at firstLiveIndex=1, not nextBufferIndex — the second buffer was
|
||||
// NOT dropped. Verify the second source was not stopped (it remained live throughout).
|
||||
assertEqual(secondFakeSource.stopped, false, 'live second source not stopped by eviction');
|
||||
// And the scheduler still has buffers (the array was not wiped past the frontier).
|
||||
if (s.getBufferCount() === 0) {
|
||||
throw new Error('eviction wiped all buffers — should have stopped at firstLiveIndex');
|
||||
}
|
||||
});
|
||||
|
||||
// === Phase 21.2 back-pressure: the forward water-mark signal =================================
|
||||
//
|
||||
// The signal is pure given the clock + buffer durations + the playhead position, so it is
|
||||
// testable in Node with the same fakes. We drive forward lookahead by adding buffers (fill) and
|
||||
// advancing the clock (drain), and assert the hysteresis latch and the OQ3 byte ceiling.
|
||||
|
||||
/**
|
||||
* Fill the scheduler with `count` 1 s buffers, start playback at t=0, and advance the schedule
|
||||
* cursor to the end so nextBufferIndex does not pin anything. Leaves all `count` buffers decoded
|
||||
* and the playhead at the clock position the caller sets afterwards.
|
||||
*/
|
||||
function fillAndStart(s: PlaybackScheduler, cm: FakeContextManager, count: number): void {
|
||||
for (let i = 0; i < count; i++) s.addBuffer(buf(1));
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0);
|
||||
advanceCursorToEnd(s, cm);
|
||||
}
|
||||
|
||||
// High-water reached → production pauses; the signal reflects the forward lookahead.
|
||||
test('evaluateProductionPause latches true when forward lookahead reaches high-water', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
s.setForwardWindow(10, 5, 0); // high 10s, low 5s, byte cap disabled
|
||||
fillAndStart(s, cm, 40); // 40s decoded, track [0,40)
|
||||
|
||||
cm.now = 0; // playhead at 0 → forward lookahead = 40s ≥ 10s high-water
|
||||
assertEqual(s.getForwardLookaheadSeconds(), 40, 'lookahead is full decoded tail at t=0');
|
||||
assertEqual(s.evaluateProductionPause(), true, 'pauses at/above high-water');
|
||||
});
|
||||
|
||||
// Below high-water but above low-water while NOT yet paused → stays unpaused (no premature pause).
|
||||
test('evaluateProductionPause stays false in the hysteresis band before the high-water crossing', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
s.setForwardWindow(10, 5, 0);
|
||||
fillAndStart(s, cm, 8); // 8s decoded
|
||||
|
||||
cm.now = 0; // lookahead 8s: between low(5) and high(10), never latched → unpaused
|
||||
assertEqual(s.evaluateProductionPause(), false, 'no pause until high-water is actually reached');
|
||||
});
|
||||
|
||||
// Hysteresis: once paused at high-water, stays paused through the band until lookahead drains
|
||||
// below low-water, then resumes. Drain is modeled by advancing the clock (playhead moves forward,
|
||||
// shrinking forward lookahead).
|
||||
test('evaluateProductionPause holds through the band and resumes only below low-water', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
s.setForwardWindow(10, 5, 0);
|
||||
fillAndStart(s, cm, 40); // track [0,40)
|
||||
|
||||
cm.now = 0;
|
||||
assertEqual(s.evaluateProductionPause(), true, 'latched at high-water (40s ahead)');
|
||||
|
||||
// Playhead at 32 → lookahead 8s: in the band (5..10) → must STAY paused (hysteresis).
|
||||
cm.now = 32;
|
||||
assertEqual(s.getForwardLookaheadSeconds(), 8, 'lookahead drained to 8s');
|
||||
assertEqual(s.evaluateProductionPause(), true, 'still paused inside the band');
|
||||
|
||||
// Playhead at 36 → lookahead 4s ≤ low-water 5 → resume.
|
||||
cm.now = 36;
|
||||
assertEqual(s.getForwardLookaheadSeconds(), 4, 'lookahead below low-water');
|
||||
assertEqual(s.evaluateProductionPause(), false, 'resumes below low-water');
|
||||
|
||||
// Refill back over high-water re-latches (the next chunk would re-pause).
|
||||
for (let i = 0; i < 20; i++) s.addBuffer(buf(1)); // +20s decoded ahead
|
||||
assertEqual(s.evaluateProductionPause(), true, 're-latches when fill exceeds high-water again');
|
||||
});
|
||||
|
||||
// OQ3 hard byte ceiling pauses production independent of the time window, and releases as soon as
|
||||
// the footprint is back under the cap (no separate low-water band on the byte guard).
|
||||
test('OQ3 byte ceiling pauses regardless of the time window', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
// Each 1s buffer here is 48000 frames × 2 ch × 4 bytes = 384000 bytes. Cap at ~1.5 MB ≈ 4 buffers.
|
||||
const perBuffer = 48000 * 2 * 4;
|
||||
s.setForwardWindow(1000, 500, perBuffer * 4); // time window huge so only the byte cap can fire
|
||||
for (let i = 0; i < 6; i++) s.addBuffer(bufBytes(1)); // 6 buffers > 4-buffer cap
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0);
|
||||
advanceCursorToEnd(s, cm);
|
||||
|
||||
cm.now = 0;
|
||||
if (s.getDecodedByteEstimate() <= perBuffer * 4) {
|
||||
throw new Error('test setup: byte estimate should exceed the cap');
|
||||
}
|
||||
assertEqual(s.evaluateProductionPause(), true, 'byte ceiling pauses even with a huge time window');
|
||||
});
|
||||
|
||||
// clear() / clearForSeek() release the latch so a fresh stream/seek starts unthrottled (C2).
|
||||
test('clear and clearForSeek release the back-pressure latch (C2 latency parity)', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
s.setForwardWindow(10, 5, 0);
|
||||
fillAndStart(s, cm, 40);
|
||||
cm.now = 0;
|
||||
assertEqual(s.evaluateProductionPause(), true, 'latched');
|
||||
|
||||
s.clear();
|
||||
// After clear there are no buffers, lookahead is 0, and the latch is reset → unpaused.
|
||||
assertEqual(s.evaluateProductionPause(), false, 'clear resets the latch and empties fill');
|
||||
|
||||
fillAndStart(s, cm, 40);
|
||||
cm.now = 0;
|
||||
assertEqual(s.evaluateProductionPause(), true, 'latched again after refill');
|
||||
s.clearForSeek();
|
||||
assertEqual(s.evaluateProductionPause(), false, 'clearForSeek resets the latch');
|
||||
});
|
||||
|
||||
// Production defaults (no setForwardWindow): the widened 60s/30s cushion. The byte cap is the
|
||||
// UNCHANGED hard OOM bound; these defaults only govern the time window. buf(1) carries no byte
|
||||
// fields, so getDecodedByteEstimate is NaN and the byte guard never fires — the time window alone
|
||||
// governs, which is exactly what we want to pin here.
|
||||
test('default forward window throttles at 60s and resumes at 30s (no setForwardWindow)', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
// Deliberately no setForwardWindow() — exercise the PRODUCTION defaults (high 60s / low 30s).
|
||||
for (let i = 0; i < 70; i++) s.addBuffer(buf(1)); // 70s decoded, track [0,70)
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0);
|
||||
advanceCursorToEnd(s, cm);
|
||||
|
||||
cm.now = 0; // forward lookahead = 70s ≥ 60s high-water
|
||||
assertEqual(s.evaluateProductionPause(), true, 'pauses at the 60s default high-water');
|
||||
cm.now = 35; // lookahead 35s: inside the 30..60 band → stays paused (hysteresis)
|
||||
assertEqual(s.evaluateProductionPause(), true, 'holds through the widened band');
|
||||
cm.now = 45; // lookahead 25s ≤ 30s low-water → resume
|
||||
assertEqual(s.evaluateProductionPause(), false, 'resumes at the 30s default low-water');
|
||||
});
|
||||
|
||||
// Lookahead correctness in the underrun state + the prime block hypothesis directly refuted: when the
|
||||
// playhead has drained the queue mid-stream, forward lookahead must read ~0 (not a stale-high value)
|
||||
// so production is NOT throttled while decoded audio is genuinely low.
|
||||
test('forward lookahead is exact during an underrun park and never trips a false pause', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
// 5s decoded, playback started, stream NOT complete.
|
||||
for (let i = 0; i < 5; i++) s.addBuffer(buf(1));
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0);
|
||||
|
||||
// Playhead advances past the decoded tail and the queue drains → mid-stream underrun park.
|
||||
cm.now = 6;
|
||||
drainAllSources(s, cm);
|
||||
assertEqual(s.isActive(), false, 'parked in underrun');
|
||||
|
||||
// At the park the playhead sits at the decoded tail: forward lookahead is 0, so production must
|
||||
// NOT be throttled (the "paused while decoded audio is low" hypothesis must not hold here).
|
||||
assertClose(s.getForwardLookaheadSeconds(), 0, 'lookahead is 0 at the underrun tail');
|
||||
assertEqual(s.evaluateProductionPause(), false, 'low decoded audio does not pause production');
|
||||
|
||||
// Refill arriving during the park grows the lead monotonically; lookahead reflects exactly it,
|
||||
// measured against the FROZEN playhead — not a stale pre-underrun position.
|
||||
s.addBuffer(buf(1));
|
||||
s.addBuffer(buf(1));
|
||||
assertClose(s.getForwardLookaheadSeconds(), 2, 'lookahead equals the freshly-accumulated lead');
|
||||
assertEqual(s.evaluateProductionPause(), false, 'still unthrottled well below the 60s high-water');
|
||||
});
|
||||
|
||||
// === False end-of-playback guard (Opus-startup misfire) ======================================
|
||||
//
|
||||
// The scheduler must distinguish a GENUINE end-of-track (stream complete AND queue drained) from a
|
||||
// transient startup/underrun gap (queue drained while bytes are still streaming — Opus decodes via
|
||||
// WebCodecs asynchronously, so the first buffers can lag the playback-start minimum). The end
|
||||
// callback fires only in the first case. These tests drive the real handleSourceEnded cascade via
|
||||
// FakeSource.stop() and assert onPlaybackEnded fires exactly when it should.
|
||||
|
||||
/** Drive the schedule cursor + live sources to a fully-drained queue at the buffer tail. */
|
||||
function drainAllSources(s: PlaybackScheduler, cm: FakeContextManager): void {
|
||||
const priv = s as unknown as { scheduledSources: Array<{ source: FakeSource }> };
|
||||
let guard = 0;
|
||||
while (priv.scheduledSources.length > 0 && guard++ < 10000) {
|
||||
// Stop the head source; its onended → handleSourceEnded removes it and schedules the next.
|
||||
priv.scheduledSources[0].source.stop();
|
||||
}
|
||||
}
|
||||
|
||||
// A drained queue MID-STREAM (streamComplete false) must NOT fire onPlaybackEnded — it parks in
|
||||
// underrun instead. This is the exact Opus-startup false-end.
|
||||
test('drained queue while still streaming does not fire onPlaybackEnded (no false end)', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
let ended = 0;
|
||||
s.onPlaybackEnded = () => { ended++; };
|
||||
|
||||
// A short run of buffers, playback started, but the stream is NOT marked complete.
|
||||
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0);
|
||||
|
||||
// Advance the clock past the buffered tail and drain every scheduled source.
|
||||
cm.now = 1.0;
|
||||
drainAllSources(s, cm);
|
||||
|
||||
assertEqual(ended, 0, 'no end callback fired mid-stream');
|
||||
assertEqual(s.isActive(), false, 'scheduler parked (inactive) on underrun');
|
||||
});
|
||||
|
||||
// After a mid-stream underrun, newly decoded buffers must RESUME playback (scheduleNewBuffers
|
||||
// re-anchors and re-activates) — not stay stuck, and still not fire a false end.
|
||||
test('underrun resumes when new buffers arrive', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
let ended = 0;
|
||||
s.onPlaybackEnded = () => { ended++; };
|
||||
|
||||
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0);
|
||||
cm.now = 1.0;
|
||||
drainAllSources(s, cm); // underrun
|
||||
assertEqual(s.isActive(), false, 'inactive after underrun');
|
||||
|
||||
// Decode catches up: enough buffers arrive to clear the 1s rebuffer lead (4 × 0.3 = 1.2s).
|
||||
for (let i = 0; i < 4; i++) s.addBuffer(buf(0.3));
|
||||
s.scheduleNewBuffers();
|
||||
|
||||
assertEqual(s.isActive(), true, 'resumed active after refill');
|
||||
assertEqual(ended, 0, 'still no false end after resume');
|
||||
const priv = s as unknown as { scheduledSources: unknown[] };
|
||||
if (priv.scheduledSources.length === 0) {
|
||||
throw new Error('expected new sources scheduled on resume');
|
||||
}
|
||||
});
|
||||
|
||||
// === Rebuffer hysteresis (Opus-startup thrash fix) ===========================================
|
||||
//
|
||||
// After a mid-stream underrun the scheduler must NOT resume on the first arriving buffer (which,
|
||||
// for ~20 ms Opus packets, plays one buffer, drains, and re-parks — the audible start/stop thrash).
|
||||
// It re-accumulates a healthy decoded LEAD (DEFAULT_MIN_PLAYBACK_LEAD_SECONDS = 1s) first. The
|
||||
// streamComplete override is the escape hatch so a genuine short tail still plays out, never parking
|
||||
// forever. These drive the real handleSourceEnded/scheduleNewBuffers/setStreamComplete paths.
|
||||
|
||||
// Below the rebuffer lead: a thin refill must keep the scheduler parked (no resume, no false end);
|
||||
// once the accumulated lead crosses the threshold, it resumes.
|
||||
test('underrun does not resume below the rebuffer lead, resumes once it is met', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
let ended = 0;
|
||||
s.onPlaybackEnded = () => { ended++; };
|
||||
|
||||
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0);
|
||||
cm.now = 1.0;
|
||||
drainAllSources(s, cm); // underrun
|
||||
assertEqual(s.isActive(), false, 'parked in underrun');
|
||||
|
||||
// Only 0.6s of fresh lead arrives — below the 1s rebuffer threshold. Must stay parked.
|
||||
for (let i = 0; i < 2; i++) s.addBuffer(buf(0.3));
|
||||
s.scheduleNewBuffers();
|
||||
assertEqual(s.isActive(), false, 'still parked — lead below the rebuffer threshold');
|
||||
assertEqual(ended, 0, 'no false end while re-accumulating lead');
|
||||
const priv = s as unknown as { scheduledSources: unknown[] };
|
||||
assertEqual(priv.scheduledSources.length, 0, 'nothing scheduled below the threshold');
|
||||
|
||||
// More lead arrives, crossing the threshold (0.6 + 0.6 = 1.2s ≥ 1s) → now resume.
|
||||
for (let i = 0; i < 2; i++) s.addBuffer(buf(0.3));
|
||||
s.scheduleNewBuffers();
|
||||
assertEqual(s.isActive(), true, 'resumes once the lead crosses the threshold');
|
||||
assertEqual(ended, 0, 'still no false end after resume');
|
||||
});
|
||||
|
||||
// Genuine-end tail SHORTER than the rebuffer lead: while parked, a small tail arrives AND the stream
|
||||
// completes. The threshold is overridden so the tail plays out and the genuine end fires exactly
|
||||
// once — the scheduler must never park forever waiting for a lead that will never come.
|
||||
test('streamComplete tail below the rebuffer lead still plays out and fires end once', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
let ended = 0;
|
||||
s.onPlaybackEnded = () => { ended++; };
|
||||
|
||||
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0);
|
||||
cm.now = 1.0;
|
||||
drainAllSources(s, cm); // underrun
|
||||
assertEqual(s.isActive(), false, 'parked in underrun');
|
||||
|
||||
// A short final tail (0.6s, below the 1s threshold) arrives; the hysteresis keeps it parked.
|
||||
for (let i = 0; i < 2; i++) s.addBuffer(buf(0.3));
|
||||
s.scheduleNewBuffers();
|
||||
assertEqual(s.isActive(), false, 'parked — tail below threshold, stream not yet complete');
|
||||
assertEqual(ended, 0, 'no end before completion');
|
||||
|
||||
// The stream completes: the threshold no longer applies → the tail schedules and plays out.
|
||||
s.setStreamComplete(true);
|
||||
assertEqual(s.isActive(), true, 'resumed to play out the final tail on completion');
|
||||
assertEqual(ended, 0, 'end not fired until the tail drains');
|
||||
|
||||
// Drain the tail → genuine end fires exactly once.
|
||||
cm.now = 2.0;
|
||||
drainAllSources(s, cm);
|
||||
assertEqual(ended, 1, 'genuine end fires exactly once after the tail drains');
|
||||
assertEqual(s.isActive(), false, 'inactive after genuine end');
|
||||
});
|
||||
|
||||
// GENUINE end: stream complete AND queue drains → onPlaybackEnded fires exactly once.
|
||||
test('genuine end (streamComplete + drained) fires onPlaybackEnded exactly once', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
let ended = 0;
|
||||
s.onPlaybackEnded = () => { ended++; };
|
||||
|
||||
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0);
|
||||
s.setStreamComplete(true); // all bytes in, no more buffers coming
|
||||
|
||||
cm.now = 1.0;
|
||||
drainAllSources(s, cm);
|
||||
|
||||
assertEqual(ended, 1, 'end fired once on genuine completion');
|
||||
assertEqual(s.isActive(), false, 'inactive after genuine end');
|
||||
});
|
||||
|
||||
// setStreamComplete arriving AFTER the queue has already drained mid-stream (the tail produced no
|
||||
// new buffers) must finalise immediately — the genuine-end signal that landed late.
|
||||
test('setStreamComplete after an already-drained queue finalises immediately', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
let ended = 0;
|
||||
s.onPlaybackEnded = () => { ended++; };
|
||||
|
||||
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0);
|
||||
cm.now = 1.0;
|
||||
drainAllSources(s, cm); // underrun, no end yet
|
||||
assertEqual(ended, 0, 'no end before completion signal');
|
||||
|
||||
s.setStreamComplete(true); // signal arrives now → finalise
|
||||
assertEqual(ended, 1, 'end fired when completion signalled post-drain');
|
||||
});
|
||||
|
||||
// clearForSeek must reset streamComplete so a post-seek refill cannot inherit a stale "complete"
|
||||
// and fire a premature end before its own bytes arrive.
|
||||
test('clearForSeek resets streamComplete (no inherited end on refill)', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
let ended = 0;
|
||||
s.onPlaybackEnded = () => { ended++; };
|
||||
|
||||
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0);
|
||||
s.setStreamComplete(true);
|
||||
|
||||
s.clearForSeek();
|
||||
s.setPlaybackOffset(5);
|
||||
// Post-seek continuation: fresh buffers, playback resumes, stream NOT yet complete.
|
||||
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
|
||||
cm.now = 5;
|
||||
s.playFromPosition(0);
|
||||
cm.now = 6.0;
|
||||
drainAllSources(s, cm);
|
||||
|
||||
assertEqual(ended, 0, 'no end fired — stale streamComplete was cleared by clearForSeek');
|
||||
});
|
||||
|
||||
// pause() during underrun: setStreamComplete must NOT fire end while the user is paused.
|
||||
// This is the narrow window the fix to pause() closes: without the underrun_ clear, a paused
|
||||
// scheduler that was mid-underrun satisfies the setStreamComplete immediate-finalise guard
|
||||
// (complete && underrun_ && drained) and fires TrackEnded / queue-advance while paused.
|
||||
test('pause during underrun: setStreamComplete does not fire end while paused', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
let ended = 0;
|
||||
s.onPlaybackEnded = () => { ended++; };
|
||||
|
||||
// A short run of buffers, drain them mid-stream → scheduler parks in underrun.
|
||||
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0);
|
||||
cm.now = 1.0;
|
||||
drainAllSources(s, cm); // queue drained, streamComplete still false → underrun
|
||||
assertEqual(s.isActive(), false, 'parked in underrun after drain');
|
||||
assertEqual(ended, 0, 'no end before pause');
|
||||
|
||||
// User pauses while the scheduler is parked in underrun.
|
||||
s.pause();
|
||||
|
||||
// Stream completes with no further buffers (the tail produced nothing new).
|
||||
// With the fix, pause() cleared underrun_ so this must NOT finalise immediately.
|
||||
s.setStreamComplete(true);
|
||||
|
||||
assertEqual(ended, 0, 'no end fired while paused — setStreamComplete must not fire during pause');
|
||||
assertEqual(s.isActive(), false, 'scheduler stays inactive after setStreamComplete during pause');
|
||||
});
|
||||
|
||||
// underrun → resume → genuine end fires exactly once: the full composition from a mid-stream gap
|
||||
// through resumed playback to completion. Confirms no double-fire and no stuck scheduler.
|
||||
test('underrun → resume → genuine end fires exactly once', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
let ended = 0;
|
||||
s.onPlaybackEnded = () => { ended++; };
|
||||
|
||||
// Drain initial buffers into underrun.
|
||||
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0);
|
||||
cm.now = 1.0;
|
||||
drainAllSources(s, cm);
|
||||
assertEqual(s.isActive(), false, 'underrun after initial drain');
|
||||
assertEqual(ended, 0, 'no end count during underrun');
|
||||
|
||||
// Decode catches up: enough buffers arrive to clear the 1s rebuffer lead (4 × 0.3 = 1.2s).
|
||||
for (let i = 0; i < 4; i++) s.addBuffer(buf(0.3));
|
||||
s.scheduleNewBuffers();
|
||||
assertEqual(s.isActive(), true, 'resumed active after refill');
|
||||
assertEqual(ended, 0, 'still no end after resume');
|
||||
|
||||
// Mark the stream complete, then drain the resumed sources to genuine end.
|
||||
s.setStreamComplete(true);
|
||||
cm.now = 2.0;
|
||||
drainAllSources(s, cm);
|
||||
|
||||
assertEqual(ended, 1, 'end fires exactly once after genuine completion');
|
||||
assertEqual(s.isActive(), false, 'inactive after genuine end');
|
||||
});
|
||||
|
||||
// === Complete-without-start (force-start fallback) ==========================================
|
||||
//
|
||||
// The C# producer calls StartStreamingPlayback after MarkStreamCompleteAsync when
|
||||
// _streamingPlaybackStarted is still false (total audio below the start threshold). The JS-side
|
||||
// effect is playFromPosition(0) called with streamComplete already true. This section covers the
|
||||
// scheduler-side guarantee: sub-threshold buffers + streamComplete already set + forced
|
||||
// playFromPosition drains and fires end exactly once, never zero, never twice.
|
||||
//
|
||||
// The C# transition itself is not exercisable here (requires StreamingAudioPlayerService +
|
||||
// AudioInteropService), so the test covers the scheduler drain-and-end-once contract directly.
|
||||
|
||||
// Forced start after completion: sub-threshold total audio, streamComplete set BEFORE
|
||||
// playFromPosition(0), sources drain and onPlaybackEnded fires exactly once.
|
||||
test('forced start on complete stream: sub-threshold buffers drain and fire end exactly once', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
let ended = 0;
|
||||
s.onPlaybackEnded = () => { ended++; };
|
||||
|
||||
// Sub-threshold buffers (0.4s total, below the 1s rebuffer lead). Never started.
|
||||
for (let i = 0; i < 2; i++) s.addBuffer(buf(0.2));
|
||||
|
||||
// Stream marks complete BEFORE playback starts — the C# completion-path ordering:
|
||||
// MarkStreamCompleteAsync fires first, then StartStreamingPlayback is called because
|
||||
// _streamingPlaybackStarted is false. setStreamComplete with underrun_=false returns
|
||||
// early (sets the flag but does not schedule/finalize — that is correct, nothing to drain yet).
|
||||
s.setStreamComplete(true);
|
||||
assertEqual(ended, 0, 'no end fired at setStreamComplete — playback not yet started');
|
||||
assertEqual(s.isActive(), false, 'scheduler inactive before forced start');
|
||||
|
||||
// Forced start: C# calls startStreamingPlayback() → playFromPosition(0).
|
||||
// With streamComplete already true and buffers present, this schedules all buffers.
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0);
|
||||
|
||||
const priv = s as unknown as { scheduledSources: unknown[] };
|
||||
if (priv.scheduledSources.length === 0) {
|
||||
throw new Error('expected sources scheduled after forced playFromPosition');
|
||||
}
|
||||
assertEqual(ended, 0, 'end not fired yet — sources must drain first');
|
||||
assertEqual(s.isActive(), true, 'scheduler active while sources are scheduled');
|
||||
|
||||
// Drain sources → streamComplete is true → genuine end fires exactly once.
|
||||
cm.now = 0.5;
|
||||
drainAllSources(s, cm);
|
||||
|
||||
assertEqual(ended, 1, 'end fires exactly once after forced-start drain');
|
||||
assertEqual(s.isActive(), false, 'scheduler inactive after genuine end');
|
||||
});
|
||||
|
||||
// No double-fire: calling setStreamComplete again after end has already fired is a no-op.
|
||||
test('setStreamComplete after forced-start drain is a no-op (no double end)', () => {
|
||||
const cm = new FakeContextManager();
|
||||
const s = makeScheduler(cm);
|
||||
let ended = 0;
|
||||
s.onPlaybackEnded = () => { ended++; };
|
||||
|
||||
for (let i = 0; i < 2; i++) s.addBuffer(buf(0.2));
|
||||
s.setStreamComplete(true);
|
||||
cm.now = 0;
|
||||
s.playFromPosition(0);
|
||||
cm.now = 0.5;
|
||||
drainAllSources(s, cm);
|
||||
assertEqual(ended, 1, 'end fired once after forced-start drain');
|
||||
|
||||
// A redundant setStreamComplete (e.g. called again from a stale C# path) must not re-fire.
|
||||
s.setStreamComplete(true);
|
||||
assertEqual(ended, 1, 'still exactly one end after redundant setStreamComplete');
|
||||
});
|
||||
|
||||
// --- run -------------------------------------------------------------------------------------
|
||||
if (failures.length > 0) {
|
||||
console.error(failures.join('\n'));
|
||||
console.error(`\n${failures.length} FAILED, ${passed} passed`);
|
||||
process.exit(1);
|
||||
} else {
|
||||
console.log(`ALL ${passed} TESTS PASSED`);
|
||||
}
|
||||
@@ -2,10 +2,94 @@
|
||||
* PlaybackScheduler - Manages AudioBuffer storage and playback scheduling.
|
||||
*
|
||||
* Single Responsibility: Store decoded buffers and schedule them for playback.
|
||||
* Supports pause/resume/seek by retaining all buffers.
|
||||
*
|
||||
* Memory model (Phase 21.1 — partial eviction)
|
||||
* --------------------------------------------
|
||||
* The scheduler is the single shared sink both decode paths feed (WAV/MP3/FLAC via
|
||||
* `IFormatDecoder`, Opus via the WebCodecs `IStreamingDecoder`); eviction lives here once
|
||||
* and serves both with zero format branches.
|
||||
*
|
||||
* THE INDEX/TIME-ANCHOR INVARIANT (the crux of 21.1):
|
||||
* `playbackOffset` is the absolute track time at which `buffers[0]` begins. Every
|
||||
* position query and scheduling decision is expressed as `playbackOffset` + a sum of
|
||||
* `buffers[i].duration` from index 0. Originally `buffers[0]` was always the track start,
|
||||
* so `playbackOffset` was 0 except after a seek-beyond-buffer. After partial eviction
|
||||
* `buffers[0]` is no longer the track start — so eviction MUST add the dropped buffers'
|
||||
* total duration to `playbackOffset`. That one move keeps `getCurrentPosition`,
|
||||
* `playFromPosition`, the `getTotalDuration`-based clamp/bounds, and the schedule loop all
|
||||
* exact against a buffer array that no longer starts at absolute time 0.
|
||||
*
|
||||
* The second half of the invariant is the array indices. `nextBufferIndex` and every live
|
||||
* `scheduledSources[].bufferIndex` are absolute positions into `buffers`; splicing `k`
|
||||
* buffers off the front shifts every surviving index down by `k`, so both must be
|
||||
* decremented by `k`. Eviction therefore never crosses the live frontier: it will not drop
|
||||
* a buffer at/after `nextBufferIndex`, nor one still referenced by a scheduled source.
|
||||
*/
|
||||
|
||||
import { AudioContextManager } from './AudioContextManager.js';
|
||||
import { decodePressure } from './decodePressure.js';
|
||||
|
||||
/**
|
||||
* Provisional back-retain default. The window-size POLICY (OQ1/OQ3) is not decided yet, so
|
||||
* this is intentionally a tunable seam (see setBackRetainSeconds), not a baked-in number —
|
||||
* 21.2 feeds real water-marks in later. The default keeps a few seconds of already-played
|
||||
* audio so a short seek-back stays in-buffer (UC3) without a network refetch.
|
||||
*/
|
||||
const DEFAULT_BACK_RETAIN_SECONDS = 10;
|
||||
|
||||
/**
|
||||
* Forward back-pressure water-marks (Phase 21.2 — the bound on the *unplayed* region).
|
||||
*
|
||||
* The single back-pressure signal is the scheduler's decoded forward lookahead: how many
|
||||
* seconds of decoded audio sit AHEAD of the playhead (OQ7). Production (the C# read loop and,
|
||||
* for Opus, the demux/decode feed) pauses above the high-water mark and resumes below the
|
||||
* low-water mark — classic hysteresis so the two producers do not chatter on/off per chunk.
|
||||
*
|
||||
* Time-based defaults — the cushion, NOT the memory bound:
|
||||
* - HIGH (60 s): the most decoded lookahead we hold ahead of the playhead before throttling.
|
||||
* Comfortably above the playback-start minimum (`AudioPlayer.minBuffersForPlayback = 6`
|
||||
* buffers, each typically 0.06 – 1 s depending on format/chunk size), so C2 holds — first
|
||||
* audio never waits on a throttle (the high-water is reached only well after playback runs).
|
||||
* - LOW (30 s): resume producing here. Kept generous so the forward fill never drains to the
|
||||
* ~500 ms scheduler lookahead under network/decode jitter (AC3 — no starvation).
|
||||
*
|
||||
* Why 60/30 and not the old 30/15: the time window is a CUSHION knob, not the memory guarantee —
|
||||
* the OQ3 byte ceiling below is the hard OOM bound. The old 30 s was sized for WAV's byte density
|
||||
* and needlessly starved the cushion for the async WebCodecs Opus path, whose decoded float
|
||||
* footprint is tiny (48 kHz stereo ≈ 0.37 MB/s, so 60 s ≈ 23 MB — a fraction of the 96 MB cap)
|
||||
* yet whose per-packet decode jitter (HW-accel-off software decode, main-thread AudioData copies)
|
||||
* needs a deeper buffer to stay ahead of the playhead. Doubling the window lets Opus use the memory
|
||||
* headroom the byte cap already permits. The byte cap is UNCHANGED, so a high-footprint stream
|
||||
* still pauses at exactly the same footprint as before — the OOM fix does not regress.
|
||||
*
|
||||
* OQ3 hard memory ceiling: an absolute byte cap on total decoded float held, independent of the
|
||||
* time window. This is the guard-rail that makes "1 GB never OOMs" a guarantee rather than a
|
||||
* tuning hope — production pauses on `lookahead >= high OR bytes > cap`, whichever fires first, so
|
||||
* the footprint can never exceed the cap regardless of the time window. The decoded f32 footprint
|
||||
* scales with sample rate × channels (not source codec), so for high-sample-rate / multichannel
|
||||
* audio the byte cap fires before 60 s (bounding memory exactly as the old 30 s window's byte
|
||||
* estimate did); for sparse 48 kHz stereo Opus the time window fires first, at ~23 MB. Estimated
|
||||
* as channels × frames × 4 (f32).
|
||||
*/
|
||||
const DEFAULT_FORWARD_HIGH_WATER_SECONDS = 60;
|
||||
const DEFAULT_FORWARD_LOW_WATER_SECONDS = 30;
|
||||
const DEFAULT_MAX_DECODED_BYTES = 96 * 1024 * 1024; // ~96 MB of decoded float PCM — the HARD OOM bound
|
||||
const BYTES_PER_FLOAT_SAMPLE = 4;
|
||||
|
||||
/**
|
||||
* Rebuffer hysteresis lead — the minimum SECONDS of decoded-but-unscheduled audio that must
|
||||
* accumulate ahead of the schedule cursor before playback may (re)start after a mid-stream underrun.
|
||||
*
|
||||
* Why seconds, not a buffer count: the per-buffer duration differs wildly by format. A WAV/lossless
|
||||
* segment is a sizeable slab (~0.1–0.4 s); a single Opus WebCodecs packet is ~20 ms. The old resume
|
||||
* path re-anchored on the FIRST arriving buffer, so for Opus it scheduled ~20 ms, drained it, parked,
|
||||
* resumed on the next ~20 ms, and so on — the audible start/stop thrash during the WebCodecs decode
|
||||
* ramp. Gating on a fixed LEAD in seconds gives a resume the same cushion a fresh start has,
|
||||
* independent of format. 1 s is the same order as the lossless playback-start lead (~6 segments) and
|
||||
* sits far below the 60 s forward high-water, so back-pressure never throttles production while the
|
||||
* scheduler is still re-accumulating this lead. Tunable; not magic.
|
||||
*/
|
||||
const DEFAULT_MIN_PLAYBACK_LEAD_SECONDS = 1.0;
|
||||
|
||||
interface ScheduledSource {
|
||||
source: AudioBufferSourceNode;
|
||||
@@ -26,11 +110,53 @@ export class PlaybackScheduler {
|
||||
private nextScheduleTime: number = 0; // AudioContext time for next buffer
|
||||
private isActive_: boolean = false; // Prevents scheduling during pause/stop
|
||||
|
||||
// Offset for seek-beyond-buffer scenarios
|
||||
// When seeking to position T beyond buffers, we clear buffers and set playbackOffset = T
|
||||
// The new stream starts at T, so buffer positions are relative to T
|
||||
// Offset for seek-beyond-buffer scenarios AND partial eviction.
|
||||
// This is the absolute track time at which buffers[0] begins. It is set on
|
||||
// seek-beyond-buffer (the new stream starts at T) and ADVANCED by eviction (when the
|
||||
// front k buffers are dropped, their total duration is added here so buffers[0] still
|
||||
// names the correct absolute time). See the index/time-anchor invariant in the header.
|
||||
private playbackOffset: number = 0;
|
||||
|
||||
// Back-retain bound (seconds of already-played audio kept un-evicted). Provisional seam;
|
||||
// 21.2 will drive this from the window policy. Not a hardcoded eviction decision.
|
||||
private backRetainSeconds: number = DEFAULT_BACK_RETAIN_SECONDS;
|
||||
|
||||
// Forward back-pressure water-marks + the OQ3 hard byte ceiling (Phase 21.2). This is the
|
||||
// single shared window policy (OQ6): both producers call evaluateProductionPause() and honor it
|
||||
// in their own way — the C# read loop stops ReadAsync, the Opus feed stops demux/decode.
|
||||
private forwardHighWaterSeconds: number = DEFAULT_FORWARD_HIGH_WATER_SECONDS;
|
||||
private forwardLowWaterSeconds: number = DEFAULT_FORWARD_LOW_WATER_SECONDS;
|
||||
private maxDecodedBytes: number = DEFAULT_MAX_DECODED_BYTES;
|
||||
|
||||
// Rebuffer hysteresis lead (seconds). The minimum decoded-but-unscheduled audio that must sit
|
||||
// ahead of the schedule cursor before playback may (re)start — at a fresh start AND after a
|
||||
// mid-stream underrun. Without it the underrun resume re-anchored on the first arriving buffer
|
||||
// and thrashed on the Opus decode ramp. See DEFAULT_MIN_PLAYBACK_LEAD_SECONDS.
|
||||
private minPlaybackLeadSeconds: number = DEFAULT_MIN_PLAYBACK_LEAD_SECONDS;
|
||||
|
||||
// Hysteresis latch for the production pause. Once forward fill crosses the high-water mark we
|
||||
// stay paused until it drains below the low-water mark, so the two producers do not flap
|
||||
// on/off around a single threshold (and a paused producer does not resume for one chunk only
|
||||
// to re-pause immediately). False until first crossing; flips on the band edges.
|
||||
// Mutated by evaluateProductionPause() — named to signal the state-advance on each call.
|
||||
private productionPaused_: boolean = false;
|
||||
|
||||
// True once the producer (C# read loop / Opus feed) has signalled that ALL bytes are in and
|
||||
// every decodable buffer has been added. This is the discriminator between a genuine
|
||||
// end-of-track and a transient gap. End-of-playback fires ONLY when this is true AND the
|
||||
// scheduled queue has drained — a drained queue while this is false is a startup/underrun gap,
|
||||
// not the end (Opus decodes via WebCodecs asynchronously, so the first AudioBuffer can lag the
|
||||
// playback-start minimum, briefly leaving zero scheduled sources before real playback). Reset
|
||||
// by clear/clearForSeek/resetToStart; set by setStreamComplete.
|
||||
private streamComplete: boolean = false;
|
||||
|
||||
// True while playback is logically running but the decoded queue ran dry mid-stream (underrun).
|
||||
// We stop the scheduler (isActive_ = false) so no source schedules against a stale anchor, but
|
||||
// remember we must re-anchor and resume the moment new buffers arrive — distinct from a paused/
|
||||
// stopped player, which clears this. Without it, scheduleNewBuffers would silently no-op on the
|
||||
// !isActive_ guard and playback would never recover from a starvation gap.
|
||||
private underrun_: boolean = false;
|
||||
|
||||
// Callbacks
|
||||
public onPlaybackEnded: (() => void) | null = null;
|
||||
|
||||
@@ -45,6 +171,38 @@ export class PlaybackScheduler {
|
||||
this.buffers.push(buffer);
|
||||
}
|
||||
|
||||
/**
|
||||
* Mark whether the byte stream is complete (all bytes received and all decodable buffers added).
|
||||
* The end-of-playback callback fires only when this is true AND the scheduled queue has drained —
|
||||
* so a drained queue while the stream is still in flight (startup/underrun) is never mistaken for
|
||||
* end-of-track. Set true by AudioPlayer on markStreamComplete / decoder isComplete; set false on a
|
||||
* fresh stream or a range-continuation reinit. Setting it true while playback has already drained
|
||||
* mid-stream finalises the track immediately (the genuine-end signal arrived after the queue
|
||||
* emptied — e.g. the very last buffers were the tail).
|
||||
*/
|
||||
setStreamComplete(complete: boolean): void {
|
||||
this.streamComplete = complete;
|
||||
// Only act when the genuine-end signal lands while we are parked in underrun (logically
|
||||
// playing but starved); a drained queue with no playback in flight — never started, or
|
||||
// already finished — is left untouched. Gated on underrun_, not isActive_, which is false
|
||||
// during a parked underrun.
|
||||
if (!complete || !this.underrun_) {
|
||||
return;
|
||||
}
|
||||
// The rebuffer threshold no longer applies — a complete stream yields no further buffers:
|
||||
// - tail buffers accumulated below the threshold while we were parked (the new hysteresis
|
||||
// kept us parked) → schedule them out; scheduleNewBuffers' underrun branch now resumes
|
||||
// because streamComplete overrides the lead gate, and handleSourceEnded fires the genuine
|
||||
// end when they drain. Without this the buffers would never schedule and we would park
|
||||
// forever (queue drained, isActive_ false, threshold never met).
|
||||
// - no tail at all (cursor already at the decoded end) → this drained state IS the end.
|
||||
if (this.nextBufferIndex < this.buffers.length) {
|
||||
this.scheduleNewBuffers();
|
||||
} else if (this.scheduledSources.length === 0) {
|
||||
this.finishPlayback();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get total duration of all stored buffers
|
||||
*/
|
||||
@@ -88,6 +246,169 @@ export class PlaybackScheduler {
|
||||
return this.playbackOffset;
|
||||
}
|
||||
|
||||
/**
|
||||
* Configure the back-retain bound (seconds of already-played audio kept un-evicted).
|
||||
* Provisional config seam — 21.2 feeds the real window policy in here. Negative values
|
||||
* are clamped to 0 (retain nothing behind the playhead).
|
||||
*/
|
||||
setBackRetainSeconds(seconds: number): void {
|
||||
this.backRetainSeconds = Math.max(0, seconds);
|
||||
}
|
||||
|
||||
/**
|
||||
* Configure the forward back-pressure water-marks (seconds of decoded lookahead) and the OQ3
|
||||
* hard byte ceiling. Provisional config seam — 21.4 tunes the numbers. Low is clamped below
|
||||
* high so the hysteresis band is always valid; non-positive byte cap disables the OQ3 guard.
|
||||
*/
|
||||
setForwardWindow(highWaterSeconds: number, lowWaterSeconds: number, maxDecodedBytes: number): void {
|
||||
this.forwardHighWaterSeconds = Math.max(0, highWaterSeconds);
|
||||
this.forwardLowWaterSeconds = Math.max(0, Math.min(lowWaterSeconds, this.forwardHighWaterSeconds));
|
||||
this.maxDecodedBytes = maxDecodedBytes;
|
||||
}
|
||||
|
||||
/**
|
||||
* Seconds of decoded audio sitting AHEAD of the current playhead — the forward fill. This is
|
||||
* the single back-pressure signal (OQ7): the absolute end time of the last decoded buffer
|
||||
* minus the current playback position. Never negative (clamped at 0 when the playhead has
|
||||
* caught up to or passed the decoded tail).
|
||||
*/
|
||||
getForwardLookaheadSeconds(): number {
|
||||
const decodedEnd = this.getTotalDuration() + this.playbackOffset;
|
||||
return Math.max(0, decodedEnd - this.getCurrentPosition());
|
||||
}
|
||||
|
||||
/**
|
||||
* Estimated bytes of decoded float PCM currently retained (OQ3 input). Web Audio AudioBuffers
|
||||
* are 32-bit float per sample per channel; frames = duration × sampleRate. Summed across the
|
||||
* retained buffers only — evicted buffers are already reclaimed, so this tracks the live
|
||||
* footprint, not the whole track.
|
||||
*/
|
||||
getDecodedByteEstimate(): number {
|
||||
let bytes = 0;
|
||||
for (const b of this.buffers) {
|
||||
bytes += b.length * b.numberOfChannels * BYTES_PER_FLOAT_SAMPLE;
|
||||
}
|
||||
return bytes;
|
||||
}
|
||||
|
||||
/**
|
||||
* The single shared production-pause decision (Phase 21.2, OQ6/OQ7). Both producers — the C#
|
||||
* read loop (21.2a) and the Opus demux/decode feed (21.2b) — call this and stop producing
|
||||
* while it returns true. Hysteresis: pause when forward lookahead exceeds the high-water mark
|
||||
* OR the decoded byte estimate exceeds the OQ3 ceiling; resume only once forward lookahead has
|
||||
* drained below the low-water mark AND the byte estimate is back under the ceiling. The
|
||||
* byte-ceiling test has no separate low-water band — it is the hard guard rail, so it releases
|
||||
* as soon as eviction brings the footprint back under the cap.
|
||||
*
|
||||
* Named `evaluateProductionPause` (not `isProductionPaused`) because each call may ADVANCE the
|
||||
* hysteresis latch (`productionPaused_`), making it a state-advancing evaluation, not a pure
|
||||
* read. `AudioPlayer.isProductionPaused()` is the pure-predicate wrapper exposed to callers
|
||||
* outside the scheduler.
|
||||
*/
|
||||
evaluateProductionPause(): boolean {
|
||||
const lookahead = this.getForwardLookaheadSeconds();
|
||||
const overByteCeiling = this.maxDecodedBytes > 0 && this.getDecodedByteEstimate() > this.maxDecodedBytes;
|
||||
|
||||
if (this.productionPaused_) {
|
||||
// Stay paused until BOTH the time window has drained below low-water AND the byte
|
||||
// footprint is back under the ceiling.
|
||||
if (lookahead <= this.forwardLowWaterSeconds && !overByteCeiling) {
|
||||
this.productionPaused_ = false;
|
||||
}
|
||||
} else if (lookahead >= this.forwardHighWaterSeconds || overByteCeiling) {
|
||||
this.productionPaused_ = true;
|
||||
}
|
||||
|
||||
return this.productionPaused_;
|
||||
}
|
||||
|
||||
/**
|
||||
* Drop already-played buffers from the front of the array, reclaiming their decoded float
|
||||
* memory, and advance the time anchor so all position/index bookkeeping stays exact.
|
||||
*
|
||||
* Eviction frontier: any buffer whose absolute END time is at or older than
|
||||
* (currentPosition - backRetainSeconds) is droppable. We evict a contiguous run from the
|
||||
* front only — buffers are appended in playback order, so the front is always the oldest.
|
||||
*
|
||||
* Two hard safety bounds keep the live frontier intact (the second half of the
|
||||
* index/time-anchor invariant):
|
||||
* 1. Never evict at/after `nextBufferIndex` — those are not yet scheduled; dropping them
|
||||
* would lose unplayed audio and corrupt the schedule cursor.
|
||||
* 2. Never evict a buffer still referenced by a live scheduled source — its
|
||||
* AudioBufferSourceNode is mid-flight and `handleSourceEnded` still tracks it.
|
||||
*
|
||||
* Returns the number of buffers evicted (0 if nothing was droppable).
|
||||
*
|
||||
* This is the SHARED eviction both decode paths get for free — no format branch. It does
|
||||
* not fetch, decode, or back-pressure (those are 21.2/21.3); with producers unchanged it
|
||||
* makes the *played* region provably memory-bounded on both paths.
|
||||
*/
|
||||
evictPlayedBuffers(): number {
|
||||
if (this.buffers.length === 0) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Absolute time before which a fully-ended buffer may be dropped.
|
||||
const evictBefore = this.getCurrentPosition() - this.backRetainSeconds;
|
||||
|
||||
// Lowest index still referenced by a live scheduled source (or buffers.length if none).
|
||||
// Eviction must not cross this — those sources are playing now.
|
||||
let firstLiveIndex = this.buffers.length;
|
||||
for (const scheduled of this.scheduledSources) {
|
||||
if (scheduled.bufferIndex < firstLiveIndex) {
|
||||
firstLiveIndex = scheduled.bufferIndex;
|
||||
}
|
||||
}
|
||||
|
||||
// Hard ceiling on how many front buffers we may drop: not past the schedule cursor,
|
||||
// and not past the oldest live source.
|
||||
const maxEvictable = Math.min(this.nextBufferIndex, firstLiveIndex);
|
||||
|
||||
// Walk the front, accumulating absolute end times, counting droppable buffers.
|
||||
let evictCount = 0;
|
||||
let accumulatedEnd = this.playbackOffset;
|
||||
for (let i = 0; i < maxEvictable; i++) {
|
||||
accumulatedEnd += this.buffers[i].duration;
|
||||
// Drop buffers whose END is at or behind the retain frontier (inclusive bound).
|
||||
if (accumulatedEnd <= evictBefore) {
|
||||
evictCount = i + 1;
|
||||
} else {
|
||||
break; // later buffers end even later — nothing more is droppable
|
||||
}
|
||||
}
|
||||
|
||||
if (evictCount === 0) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Sum the dropped duration BEFORE splicing, then advance the time anchor by it so
|
||||
// buffers[0] still names the correct absolute start time. This is the move that keeps
|
||||
// every position/scheduling query exact against a front-evicted array.
|
||||
let droppedDuration = 0;
|
||||
for (let i = 0; i < evictCount; i++) {
|
||||
droppedDuration += this.buffers[i].duration;
|
||||
}
|
||||
|
||||
this.buffers.splice(0, evictCount);
|
||||
|
||||
// Advance the absolute time anchor (offset) by the dropped duration AND drop the
|
||||
// buffer-relative anchor position by the same amount. These two move in lockstep:
|
||||
// getCurrentPosition() is (playbackAnchorPosition + playbackOffset + elapsed), so
|
||||
// adjusting only one would make the reported position jump by droppedDuration.
|
||||
// Moving both by +d / -d leaves the ABSOLUTE position unchanged while keeping
|
||||
// playbackAnchorPosition buffer-relative (the convention playFromPosition/pause use).
|
||||
this.playbackOffset += droppedDuration;
|
||||
this.playbackAnchorPosition -= droppedDuration;
|
||||
|
||||
// Every surviving absolute index shifts down by evictCount.
|
||||
this.nextBufferIndex -= evictCount;
|
||||
for (const scheduled of this.scheduledSources) {
|
||||
scheduled.bufferIndex -= evictCount;
|
||||
}
|
||||
|
||||
return evictCount;
|
||||
}
|
||||
|
||||
/**
|
||||
* Start or resume playback from a specific position
|
||||
*/
|
||||
@@ -111,18 +432,25 @@ export class PlaybackScheduler {
|
||||
}
|
||||
|
||||
if (startBufferIndex >= this.buffers.length) {
|
||||
// Position landed at or past the end of all buffers. Previously this
|
||||
// returned silently, leaving the player stuck "playing" with no source
|
||||
// scheduled — a pause near the end followed by play never recovered.
|
||||
// Treat this as end-of-track so listeners (UI / end callback) fire.
|
||||
this.isActive_ = false;
|
||||
this.playbackAnchorTime = 0;
|
||||
this.playbackAnchorPosition = 0;
|
||||
this.onPlaybackEnded?.();
|
||||
// Position landed at or past the end of all currently-decoded buffers. This is
|
||||
// end-of-track ONLY if the stream is complete; otherwise it is a startup/underrun
|
||||
// gap (decode hasn't caught up to the playhead yet) and firing onPlaybackEnded here
|
||||
// would be a FALSE end — exactly the Opus-startup misfire. When complete, finish;
|
||||
// when still streaming, park in underrun so scheduleNewBuffers resumes on the next
|
||||
// decoded buffer rather than the player being stuck "playing" with nothing scheduled.
|
||||
if (this.streamComplete) {
|
||||
this.finishPlayback();
|
||||
} else {
|
||||
this.underrun_ = true;
|
||||
this.playbackAnchorPosition = position;
|
||||
this.nextBufferIndex = startBufferIndex;
|
||||
this.isActive_ = false; // no source to schedule yet; resume() re-anchors on refill
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// Set timing anchors
|
||||
this.underrun_ = false;
|
||||
this.playbackAnchorPosition = position;
|
||||
this.playbackAnchorTime = this.contextManager.currentTime;
|
||||
this.nextScheduleTime = this.contextManager.currentTime + 0.01; // Small lookahead
|
||||
@@ -141,6 +469,34 @@ export class PlaybackScheduler {
|
||||
return; // No new buffers
|
||||
}
|
||||
|
||||
// Resume from a mid-stream underrun: the queue had drained ahead of decode and we parked
|
||||
// (isActive_ = false, underrun_ = true) instead of firing a false end. Newly decoded
|
||||
// buffers are now available at nextBufferIndex, so re-anchor the clock at the resume point
|
||||
// and re-enable scheduling. We re-anchor (rather than reusing the stale nextScheduleTime
|
||||
// captured before the gap) so the resumed audio is contiguous from "now" — a stale anchor
|
||||
// would schedule the next source in the past and the browser would drop or rush it.
|
||||
if (this.underrun_) {
|
||||
// Rebuffer hysteresis: do NOT resume on the first arriving buffer. With an empty scheduled
|
||||
// tail, resuming on a single buffer plays it (~20 ms for Opus) and immediately re-drains,
|
||||
// re-parking — the audible start/stop thrash on the Opus WebCodecs decode ramp. Stay parked
|
||||
// and keep accumulating until a healthy lead has rebuilt, so the resumed playback has the
|
||||
// same cushion a fresh start does. While parked the playhead is frozen, so each arriving
|
||||
// buffer grows the lead monotonically toward the threshold (no starvation/deadlock).
|
||||
//
|
||||
// streamComplete overrides the gate: a finished stream produces no further buffers, so a
|
||||
// tail shorter than the lead MUST still play out (here and via setStreamComplete) rather
|
||||
// than park forever. handleSourceEnded fires the genuine end once that tail drains.
|
||||
if (!this.streamComplete && !this.hasMinimumPlaybackLead()) {
|
||||
return; // still re-accumulating the rebuffer lead — remain parked
|
||||
}
|
||||
this.underrun_ = false;
|
||||
this.isActive_ = true;
|
||||
this.playbackAnchorTime = this.contextManager.currentTime;
|
||||
this.nextScheduleTime = this.contextManager.currentTime + 0.01;
|
||||
this.scheduleBuffersFrom(this.nextBufferIndex, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
// Use isActive_ as the sentinel for "playback is running", not nextScheduleTime === 0.
|
||||
// AudioContext.currentTime can legitimately be 0 at context creation, which would cause
|
||||
// nextScheduleTime === 0 to incorrectly reset a value already set by playFromPosition.
|
||||
@@ -214,26 +570,69 @@ export class PlaybackScheduler {
|
||||
this.scheduledSources.splice(index, 1);
|
||||
}
|
||||
|
||||
// A source just finished, so its buffer is now behind the playhead — the natural
|
||||
// point to reclaim played memory. Eviction is self-contained (no fetch/back-pressure)
|
||||
// and runs before re-scheduling so index bookkeeping is settled first. This is the
|
||||
// 21.1 trigger that keeps the PLAYED region bounded with producers unchanged.
|
||||
this.evictPlayedBuffers();
|
||||
|
||||
// Schedule more buffers if available
|
||||
if (this.nextBufferIndex < this.buffers.length) {
|
||||
this.scheduleBuffersFrom(this.nextBufferIndex, 0);
|
||||
}
|
||||
|
||||
// Check if all playback has finished
|
||||
// The scheduled queue drained AND the cursor caught up to every decoded buffer. Whether
|
||||
// this is the end depends on the stream:
|
||||
// - streamComplete: genuine end-of-track — finish and fire onPlaybackEnded.
|
||||
// - still streaming: a mid-stream UNDERRUN (decode fell behind the playhead — the Opus
|
||||
// WebCodecs startup gap, or a network stall). Firing onPlaybackEnded here is the false
|
||||
// end this guards against. Park in underrun; scheduleNewBuffers resumes on the next
|
||||
// decoded buffer.
|
||||
if (this.scheduledSources.length === 0 && this.nextBufferIndex >= this.buffers.length) {
|
||||
this.isActive_ = false;
|
||||
this.playbackAnchorTime = 0;
|
||||
this.playbackAnchorPosition = 0;
|
||||
this.onPlaybackEnded?.();
|
||||
if (this.streamComplete) {
|
||||
this.finishPlayback();
|
||||
} else {
|
||||
this.underrun_ = true;
|
||||
// Mid-stream underrun: the scheduled queue drained and decode has not caught up. Report it
|
||||
// as decode pressure so the visualizer throttles — a sustained run of these is exactly the
|
||||
// HW-accel-off starvation the auto-throttle protects against. The hysteresis in the signal
|
||||
// ignores a lone startup-ramp underrun; only a sustained run engages the throttle.
|
||||
decodePressure.report();
|
||||
// Hold the playhead at the decoded tail so getCurrentPosition stays exact during
|
||||
// the gap. isActive_ goes false so no stale-anchor scheduling occurs; resume
|
||||
// re-anchors at currentTime when buffers arrive.
|
||||
this.playbackAnchorPosition = this.getCurrentPosition() - this.playbackOffset;
|
||||
this.playbackAnchorTime = 0;
|
||||
this.isActive_ = false;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Finalise playback: stop the clock, reset anchors, and fire the end-of-playback callback. The
|
||||
* single genuine-end path, reached only when the stream is complete AND the queue has fully
|
||||
* drained (handleSourceEnded / setStreamComplete) or playback resumed past a complete stream's
|
||||
* end (playFromPosition). Never called for a transient startup/underrun gap.
|
||||
*/
|
||||
private finishPlayback(): void {
|
||||
this.isActive_ = false;
|
||||
this.underrun_ = false;
|
||||
this.playbackAnchorTime = 0;
|
||||
this.playbackAnchorPosition = 0;
|
||||
this.onPlaybackEnded?.();
|
||||
}
|
||||
|
||||
/**
|
||||
* Pause playback - saves position and stops sources
|
||||
*/
|
||||
pause(): number {
|
||||
const position = this.getCurrentPosition();
|
||||
this.isActive_ = false; // Prevent handleSourceEnded from scheduling more
|
||||
// Clear the underrun flag: if the queue drained mid-stream and the user pauses before new
|
||||
// buffers arrive, a subsequent setStreamComplete must not fire finishPlayback while still
|
||||
// paused. On resume, playFromPosition re-parks underrun if the decoded tail still hasn't
|
||||
// caught up, so no genuine end is lost by clearing it here.
|
||||
this.underrun_ = false;
|
||||
this.stopAllSources();
|
||||
// getCurrentPosition() returns absolute time (anchor + playbackOffset); the anchor
|
||||
// is buffer-relative, so strip the offset back out before storing it.
|
||||
@@ -262,6 +661,8 @@ export class PlaybackScheduler {
|
||||
*/
|
||||
resetToStart(): void {
|
||||
this.isActive_ = false;
|
||||
this.underrun_ = false;
|
||||
this.streamComplete = false;
|
||||
this.stopAllSources();
|
||||
this.playbackAnchorPosition = 0;
|
||||
this.playbackAnchorTime = 0;
|
||||
@@ -274,6 +675,8 @@ export class PlaybackScheduler {
|
||||
*/
|
||||
clear(): void {
|
||||
this.isActive_ = false;
|
||||
this.underrun_ = false;
|
||||
this.streamComplete = false;
|
||||
this.stopAllSources();
|
||||
this.buffers = [];
|
||||
this.playbackAnchorPosition = 0;
|
||||
@@ -281,6 +684,9 @@ export class PlaybackScheduler {
|
||||
this.nextBufferIndex = 0;
|
||||
this.nextScheduleTime = 0;
|
||||
this.playbackOffset = 0;
|
||||
// Release the back-pressure latch — a fresh stream must start unthrottled so its first
|
||||
// chunks decode immediately (C2: no throttle-induced first-audio stall).
|
||||
this.productionPaused_ = false;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -288,6 +694,11 @@ export class PlaybackScheduler {
|
||||
*/
|
||||
clearForSeek(): void {
|
||||
this.isActive_ = false;
|
||||
this.underrun_ = false;
|
||||
// The range continuation is a fresh byte stream — it is NOT complete until its own
|
||||
// markStreamComplete. Reset so a stale "complete" from the pre-seek stream cannot make the
|
||||
// post-seek refill fire a premature end before its bytes arrive.
|
||||
this.streamComplete = false;
|
||||
this.stopAllSources();
|
||||
this.buffers = [];
|
||||
this.playbackAnchorPosition = 0;
|
||||
@@ -295,6 +706,9 @@ export class PlaybackScheduler {
|
||||
this.nextBufferIndex = 0;
|
||||
this.nextScheduleTime = 0;
|
||||
// Note: playbackOffset is NOT reset - it will be set by the caller
|
||||
// Release the back-pressure latch — the post-seek continuation must refill from the new
|
||||
// offset without inheriting the pre-seek paused state.
|
||||
this.productionPaused_ = false;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -311,6 +725,24 @@ export class PlaybackScheduler {
|
||||
return this.buffers.length >= minCount;
|
||||
}
|
||||
|
||||
/**
|
||||
* True once at least `minPlaybackLeadSeconds` of decoded-but-unscheduled audio sits ahead of the
|
||||
* schedule cursor — the rebuffer-hysteresis gate for both a fresh playback start (cursor at 0, so
|
||||
* this measures the whole decoded head) and an underrun resume (cursor at the drained tail, so this
|
||||
* measures only the freshly-accumulated lead). Sums only up to the threshold and short-circuits, so
|
||||
* it is bounded (~one threshold's worth of buffers) regardless of how much is buffered ahead.
|
||||
*/
|
||||
hasMinimumPlaybackLead(): boolean {
|
||||
let lead = 0;
|
||||
for (let i = this.nextBufferIndex; i < this.buffers.length; i++) {
|
||||
lead += this.buffers[i].duration;
|
||||
if (lead >= this.minPlaybackLeadSeconds) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if playback is active
|
||||
*/
|
||||
|
||||
@@ -61,6 +61,16 @@ export class StreamDecoder {
|
||||
// at 4 GB by the 32-bit RIFF size field, so overflow is not a practical concern.
|
||||
private totalRawBytes: number = 0;
|
||||
private processedBytes: number = 0;
|
||||
|
||||
// Absolute count of raw bytes already DROPPED off the front of rawChunks (the memory bound).
|
||||
// processedBytes is an absolute cursor into the whole logical byte stream; rawChunks no longer
|
||||
// begins at stream byte 0 once consumed chunks are compacted away, so extractAlignedData walks
|
||||
// from discardedBytes (the absolute position of rawChunks[0]) rather than 0. totalRawBytes and
|
||||
// every offset stay absolute and unchanged — only the array's front moves. Without this, a long
|
||||
// WAV (e.g. a 92-min mix ≈ 970 MB raw) accumulates its ENTIRE decoded-from body in rawChunks
|
||||
// because consumed chunks were never released; Phase 21.2 bounds only the DECODED scheduler
|
||||
// queue, not this raw queue — so software (HW-accel-off) playback crashed the tab on memory.
|
||||
private discardedBytes: number = 0;
|
||||
private totalStreamLength: number = 0;
|
||||
private streamComplete: boolean = false;
|
||||
private headerError: string | null = null;
|
||||
@@ -94,6 +104,7 @@ export class StreamDecoder {
|
||||
this.rawChunks = [];
|
||||
this.totalRawBytes = 0;
|
||||
this.processedBytes = 0;
|
||||
this.discardedBytes = 0;
|
||||
this.totalStreamLength = totalStreamLength;
|
||||
this.streamComplete = false;
|
||||
this.headerBytesReceived = 0;
|
||||
@@ -228,6 +239,36 @@ export class StreamDecoder {
|
||||
this.totalRawBytes += data.length;
|
||||
}
|
||||
|
||||
/**
|
||||
* Drop fully-consumed raw chunks off the front of rawChunks, reclaiming their bytes. A chunk is
|
||||
* droppable only when its ENTIRE span lies at or before processedBytes (the decode cursor); a
|
||||
* chunk that straddles the cursor still has unconsumed tail bytes a later segment will read, so
|
||||
* the walk stops there. discardedBytes tracks the absolute start of rawChunks[0] so
|
||||
* extractAlignedData keeps reading the correct bytes after compaction. Splicing once at the end
|
||||
* (not per chunk) keeps this O(n) in the dropped count.
|
||||
*
|
||||
* This is the raw-side analogue of PlaybackScheduler.evictPlayedBuffers (the decoded side): both
|
||||
* keep their queue bounded to roughly the live window, so a long stream never balloons memory.
|
||||
*/
|
||||
private releaseConsumedChunks(): void {
|
||||
let dropCount = 0;
|
||||
let frontPos = this.discardedBytes;
|
||||
for (const chunk of this.rawChunks) {
|
||||
// Drop only when the whole chunk is behind the cursor (end <= processedBytes). A chunk
|
||||
// ending exactly at processedBytes has every byte consumed and is safe to drop.
|
||||
if (frontPos + chunk.length <= this.processedBytes) {
|
||||
frontPos += chunk.length;
|
||||
dropCount++;
|
||||
} else {
|
||||
break; // this chunk straddles the cursor (or is ahead) — stop.
|
||||
}
|
||||
}
|
||||
if (dropCount > 0) {
|
||||
this.rawChunks.splice(0, dropCount);
|
||||
this.discardedBytes = frontPos;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Try to decode the next segment of audio.
|
||||
*
|
||||
@@ -276,6 +317,9 @@ export class StreamDecoder {
|
||||
// Advance only after a successful decode so a thrown timeout/decode
|
||||
// failure does not silently drop the segment.
|
||||
this.processedBytes += alignedSize;
|
||||
// Release fully-consumed raw chunks now that the cursor has moved past them. This is the
|
||||
// memory bound: without it rawChunks retains the whole stream body (the OOM on long WAVs).
|
||||
this.releaseConsumedChunks();
|
||||
return { buffer, duration: buffer.duration };
|
||||
} catch (error) {
|
||||
// Re-throw typed errors so the outer drain loop in processChunk /
|
||||
@@ -339,7 +383,9 @@ export class StreamDecoder {
|
||||
let extractedOffset = 0;
|
||||
let remaining = size;
|
||||
let streamPosition = this.processedBytes;
|
||||
let currentPos = 0;
|
||||
// rawChunks[0] now begins at absolute stream byte `discardedBytes` (front-compaction has
|
||||
// dropped everything before it), so the walk starts there, not at 0.
|
||||
let currentPos = this.discardedBytes;
|
||||
|
||||
for (const chunk of this.rawChunks) {
|
||||
if (remaining <= 0) break;
|
||||
@@ -473,6 +519,7 @@ export class StreamDecoder {
|
||||
this.rawChunks = [];
|
||||
this.totalRawBytes = 0;
|
||||
this.processedBytes = 0;
|
||||
this.discardedBytes = 0;
|
||||
this.totalStreamLength = 0;
|
||||
this.streamComplete = false;
|
||||
this.headerBytesReceived = 0;
|
||||
@@ -501,6 +548,7 @@ export class StreamDecoder {
|
||||
this.rawChunks = [];
|
||||
this.totalRawBytes = 0;
|
||||
this.processedBytes = 0;
|
||||
this.discardedBytes = 0;
|
||||
this.streamComplete = false;
|
||||
this.headerBytesReceived = 0;
|
||||
this.headerSearchChunks = [];
|
||||
|
||||
@@ -0,0 +1,157 @@
|
||||
/**
|
||||
* decodePressure hysteresis tests — the Part-1 auto-throttle signal logic.
|
||||
*
|
||||
* These cover the four named behaviours that make the visualizer-throttle safe: it engages only on
|
||||
* SUSTAINED pressure, releases only after SUSTAINED recovery, never flaps on/off, and is a complete
|
||||
* no-op when decode is healthy. The clock is injected so every transition is asserted at an exact
|
||||
* timestamp — no real timers, fully deterministic.
|
||||
*
|
||||
* Run (no test runner configured; Node 22+ strips TS types natively — see OpusStreamDecoder.test.ts):
|
||||
* dotnet build DeepDrftPublic/DeepDrftPublic.csproj
|
||||
* cp DeepDrftPublic/Interop/audio/decodePressure.test.ts DeepDrftPublic/wwwroot/js/audio/
|
||||
* node DeepDrftPublic/wwwroot/js/audio/decodePressure.test.ts
|
||||
*
|
||||
* A thrown error / non-zero exit signals failure; "ALL <n> TESTS PASSED" signals success.
|
||||
*/
|
||||
|
||||
import {
|
||||
DecodePressureSignal,
|
||||
ENGAGE_EVENTS,
|
||||
ENGAGE_WINDOW_MS,
|
||||
RELEASE_QUIET_MS,
|
||||
MIN_ENGAGED_MS,
|
||||
} from './decodePressure.js';
|
||||
|
||||
// --- tiny inline harness (no dependencies) ---------------------------------------------------
|
||||
let passed = 0;
|
||||
const failures: string[] = [];
|
||||
function test(name: string, fn: () => void): void {
|
||||
try {
|
||||
fn();
|
||||
passed++;
|
||||
} catch (e) {
|
||||
failures.push(`FAIL: ${name}\n ${(e as Error).message}`);
|
||||
}
|
||||
}
|
||||
function assertTrue(actual: boolean, msg?: string): void {
|
||||
if (actual !== true) throw new Error(`${msg ?? 'assertTrue'}: expected true, got ${String(actual)}`);
|
||||
}
|
||||
function assertFalse(actual: boolean, msg?: string): void {
|
||||
if (actual !== false) throw new Error(`${msg ?? 'assertFalse'}: expected false, got ${String(actual)}`);
|
||||
}
|
||||
|
||||
/** A signal driven by a hand-advanced clock, so every transition is asserted at an exact time. */
|
||||
function makeSignal() {
|
||||
let now = 1000; // start at a non-zero base so "no prior stress" (-Infinity) is unambiguous
|
||||
const sig = new DecodePressureSignal(() => now);
|
||||
return {
|
||||
sig,
|
||||
at(ms: number) { now = ms; },
|
||||
advance(ms: number) { now += ms; },
|
||||
now() { return now; },
|
||||
};
|
||||
}
|
||||
|
||||
// --- no engage when healthy ------------------------------------------------------------------
|
||||
|
||||
test('healthy stream never engages (no reports at all)', () => {
|
||||
const { sig, advance } = makeSignal();
|
||||
for (let i = 0; i < 10; i++) {
|
||||
advance(1000);
|
||||
assertFalse(sig.isUnderPressure(), 'healthy must never be under pressure');
|
||||
}
|
||||
});
|
||||
|
||||
test('a single transient stress does not engage', () => {
|
||||
const { sig, advance } = makeSignal();
|
||||
sig.report();
|
||||
assertFalse(sig.isUnderPressure(), 'one event is not sustained');
|
||||
advance(500);
|
||||
assertFalse(sig.isUnderPressure(), 'still not sustained');
|
||||
});
|
||||
|
||||
test('fewer than ENGAGE_EVENTS within the window does not engage', () => {
|
||||
const { sig, advance } = makeSignal();
|
||||
for (let i = 0; i < ENGAGE_EVENTS - 1; i++) {
|
||||
sig.report();
|
||||
advance(10);
|
||||
}
|
||||
assertFalse(sig.isUnderPressure(), 'one short of the threshold must not engage');
|
||||
});
|
||||
|
||||
test('stress spread wider than the window never accumulates enough to engage', () => {
|
||||
const { sig, advance } = makeSignal();
|
||||
// One report per full window: the prune drops each before the next, so the live count never
|
||||
// reaches ENGAGE_EVENTS even after many reports.
|
||||
for (let i = 0; i < ENGAGE_EVENTS * 3; i++) {
|
||||
sig.report();
|
||||
assertFalse(sig.isUnderPressure(), 'spread-out stress is not sustained');
|
||||
advance(ENGAGE_WINDOW_MS);
|
||||
}
|
||||
});
|
||||
|
||||
// --- engages on sustained pressure -----------------------------------------------------------
|
||||
|
||||
test('ENGAGE_EVENTS within the window engages', () => {
|
||||
const { sig, advance } = makeSignal();
|
||||
for (let i = 0; i < ENGAGE_EVENTS; i++) {
|
||||
sig.report();
|
||||
advance(10); // all comfortably inside ENGAGE_WINDOW_MS
|
||||
}
|
||||
assertTrue(sig.isUnderPressure(), 'sustained pressure must engage');
|
||||
});
|
||||
|
||||
// --- releases after recovery -----------------------------------------------------------------
|
||||
|
||||
test('releases after sustained quiet past the min dwell', () => {
|
||||
const { sig, advance } = makeSignal();
|
||||
for (let i = 0; i < ENGAGE_EVENTS; i++) { sig.report(); advance(10); }
|
||||
assertTrue(sig.isUnderPressure(), 'engaged');
|
||||
|
||||
// Quiet long enough to satisfy BOTH the min engaged dwell and the release-quiet window.
|
||||
advance(Math.max(MIN_ENGAGED_MS, RELEASE_QUIET_MS) + 1);
|
||||
assertFalse(sig.isUnderPressure(), 'sustained recovery must release');
|
||||
});
|
||||
|
||||
test('re-engages after a release when a fresh burst arrives', () => {
|
||||
const { sig, advance } = makeSignal();
|
||||
for (let i = 0; i < ENGAGE_EVENTS; i++) { sig.report(); advance(10); }
|
||||
assertTrue(sig.isUnderPressure(), 'engaged first time');
|
||||
advance(Math.max(MIN_ENGAGED_MS, RELEASE_QUIET_MS) + 1);
|
||||
assertFalse(sig.isUnderPressure(), 'released');
|
||||
|
||||
for (let i = 0; i < ENGAGE_EVENTS; i++) { sig.report(); advance(10); }
|
||||
assertTrue(sig.isUnderPressure(), 'a fresh sustained burst re-engages');
|
||||
});
|
||||
|
||||
// --- no flap ---------------------------------------------------------------------------------
|
||||
|
||||
test('stays engaged during a brief quiet shorter than the release window', () => {
|
||||
const { sig, advance } = makeSignal();
|
||||
for (let i = 0; i < ENGAGE_EVENTS; i++) { sig.report(); advance(10); }
|
||||
assertTrue(sig.isUnderPressure(), 'engaged');
|
||||
|
||||
// A gap shorter than RELEASE_QUIET_MS must NOT release — that is the anti-flap guarantee.
|
||||
advance(RELEASE_QUIET_MS - 100);
|
||||
assertTrue(sig.isUnderPressure(), 'a brief quiet must not drop the throttle');
|
||||
});
|
||||
|
||||
test('continued stress holds the throttle engaged indefinitely', () => {
|
||||
const { sig, advance } = makeSignal();
|
||||
for (let i = 0; i < ENGAGE_EVENTS; i++) { sig.report(); advance(10); }
|
||||
assertTrue(sig.isUnderPressure(), 'engaged');
|
||||
|
||||
// Keep reporting at a cadence under the release window; it must never release.
|
||||
for (let i = 0; i < 20; i++) {
|
||||
advance(RELEASE_QUIET_MS - 100);
|
||||
sig.report();
|
||||
assertTrue(sig.isUnderPressure(), 'ongoing stress keeps it engaged');
|
||||
}
|
||||
});
|
||||
|
||||
// --- report ----------------------------------------------------------------------------------
|
||||
if (failures.length > 0) {
|
||||
console.error(failures.join('\n'));
|
||||
throw new Error(`${failures.length} test(s) failed, ${passed} passed`);
|
||||
}
|
||||
console.log(`ALL ${passed} TESTS PASSED`);
|
||||
@@ -0,0 +1,93 @@
|
||||
/**
|
||||
* Shared decode-pressure signal — the seam that lets the audio decode pipeline protect itself
|
||||
* from the WebGL visualizer under CPU contention.
|
||||
*
|
||||
* THE PROBLEM (browser-confirmed): with hardware acceleration OFF the WaveformVisualizer's WebGL2
|
||||
* lava-lamp software-renders on the main thread. WebCodecs Opus decode also runs on the main thread,
|
||||
* so a 60 fps software render starves decode → it falls behind realtime → playback underruns. Turning
|
||||
* the visualizer off makes decode keep up perfectly. With HW accel ON the render is on the GPU and
|
||||
* there is no contention; WAV/lossless decodes synchronously and never pressures decode either.
|
||||
*
|
||||
* THE SEAM: this module is a singleton shared by two otherwise-independent browser module graphs —
|
||||
* the audio pipeline (`js/audio/*`, the PRODUCER) and the visualizer (`js/visualizer/*`, the
|
||||
* CONSUMER) — because an ES module is instantiated once per URL. The producer reports decode stress;
|
||||
* the consumer reads {@link DecodePressureSignal.isUnderPressure} each frame and throttles its render
|
||||
* cadence so the main thread yields time back to decode. No routing through C#, no constructor growth.
|
||||
*
|
||||
* HYSTERESIS (no flap): the signal engages only on SUSTAINED stress (≥ ENGAGE_EVENTS reports within
|
||||
* ENGAGE_WINDOW_MS) and releases only after SUSTAINED recovery (no stress for RELEASE_QUIET_MS, and
|
||||
* never before a MIN_ENGAGED_MS dwell). A lone startup-ramp blip never engages; once engaged the
|
||||
* throttle cannot toggle off frame-to-frame.
|
||||
*
|
||||
* HEALTHY-CASE NO-OP: when decode keeps up nothing ever calls report(), so {@link isUnderPressure}
|
||||
* stays false forever and the consumer runs at full quality. This protection only activates under
|
||||
* genuine, sustained decode starvation.
|
||||
*/
|
||||
|
||||
/** Stress reports required within {@link ENGAGE_WINDOW_MS} to engage the throttle. */
|
||||
export const ENGAGE_EVENTS = 5;
|
||||
/** Sliding window (ms) over which {@link ENGAGE_EVENTS} stress reports count toward engaging. */
|
||||
export const ENGAGE_WINDOW_MS = 2500;
|
||||
/** Stress-free dwell (ms) required before the throttle releases. */
|
||||
export const RELEASE_QUIET_MS = 1500;
|
||||
/** Minimum engaged dwell (ms) before release is even considered — the anti-flap floor. */
|
||||
export const MIN_ENGAGED_MS = 1000;
|
||||
|
||||
type Clock = () => number;
|
||||
|
||||
export class DecodePressureSignal {
|
||||
// Timestamps of recent stress reports, pruned to the engage window. Length ≥ ENGAGE_EVENTS is the
|
||||
// "sustained pressure" condition. Bounded by the window, so this never grows unbounded.
|
||||
private stressTimestamps: number[] = [];
|
||||
private lastStressMs = Number.NEGATIVE_INFINITY;
|
||||
private engaged = false;
|
||||
private engagedAtMs = 0;
|
||||
|
||||
// Clock injectable purely for deterministic unit tests; production uses performance.now().
|
||||
constructor(private readonly now: Clock = () => performance.now()) {}
|
||||
|
||||
/**
|
||||
* Report one unit of decode stress — decode falling behind realtime. Called by the producer at
|
||||
* each genuine lag event: the WebCodecs decode queue staying non-empty past its yield ceiling
|
||||
* (OpusStreamDecoder) and the scheduler parking on a mid-stream underrun (PlaybackScheduler).
|
||||
*/
|
||||
report(): void {
|
||||
const t = this.now();
|
||||
this.lastStressMs = t;
|
||||
this.stressTimestamps.push(t);
|
||||
this.prune(t);
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether decode is under sustained pressure right now. Pure read for the caller, but it ADVANCES
|
||||
* the hysteresis latch (engage on sustained stress, release on sustained quiet past the min dwell)
|
||||
* — so the transition is evaluated lazily on the clock, identical whether called once or per frame.
|
||||
*/
|
||||
isUnderPressure(): boolean {
|
||||
const t = this.now();
|
||||
this.prune(t);
|
||||
|
||||
if (this.engaged) {
|
||||
const engagedFor = t - this.engagedAtMs;
|
||||
const quietFor = t - this.lastStressMs;
|
||||
if (engagedFor >= MIN_ENGAGED_MS && quietFor >= RELEASE_QUIET_MS) {
|
||||
this.engaged = false;
|
||||
}
|
||||
} else if (this.stressTimestamps.length >= ENGAGE_EVENTS) {
|
||||
this.engaged = true;
|
||||
this.engagedAtMs = t;
|
||||
}
|
||||
return this.engaged;
|
||||
}
|
||||
|
||||
/** Drop stress timestamps older than the engage window so the count reflects only the live window. */
|
||||
private prune(t: number): void {
|
||||
const cutoff = t - ENGAGE_WINDOW_MS;
|
||||
while (this.stressTimestamps.length > 0 && this.stressTimestamps[0] < cutoff) {
|
||||
this.stressTimestamps.shift();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** The process-wide signal both the audio pipeline and the visualizer share. */
|
||||
export const decodePressure = new DecodePressureSignal();
|
||||
@@ -3,6 +3,7 @@
|
||||
*/
|
||||
|
||||
import { AudioPlayer, AudioResult, StreamingResult, AudioState } from './AudioPlayer.js';
|
||||
import { canDecodeOggOpus } from './OpusCapability.js';
|
||||
|
||||
// Player instances by ID
|
||||
const audioPlayers = new Map<string, AudioPlayer>();
|
||||
@@ -31,12 +32,26 @@ const DeepDrftAudio = {
|
||||
}
|
||||
},
|
||||
|
||||
initializeStreaming: (playerId: string, totalStreamLength: number, contentType: string): AudioResult => {
|
||||
initializeStreaming: async (playerId: string, totalStreamLength: number, contentType: string): Promise<AudioResult> => {
|
||||
const player = audioPlayers.get(playerId);
|
||||
if (!player) return { success: false, error: 'Player not found' };
|
||||
return player.initializeStreaming(totalStreamLength, contentType);
|
||||
},
|
||||
|
||||
// Opus injection seam (wave 18.4). Wave 18.5 fetches the per-track sidecar (setup header +
|
||||
// seek index) over HTTP and hands the raw bytes here BEFORE initializeStreaming on an Opus
|
||||
// stream. This module never fetches the sidecar — it only parses + stores it on the player.
|
||||
setOpusSidecar: (playerId: string, sidecarBytes: Uint8Array): AudioResult => {
|
||||
const player = audioPlayers.get(playerId);
|
||||
if (!player) return { success: false, error: 'Player not found' };
|
||||
return player.setOpusSidecar(sidecarBytes);
|
||||
},
|
||||
|
||||
// Capability seam. Resolves whether this browser can stream-decode Ogg Opus via WebCodecs
|
||||
// (AudioDecoder + codec:'opus'; Safari < 16.4 / older Firefox cannot). The player consumes this
|
||||
// to choose lossless when unsupported; this module only reports the capability.
|
||||
canDecodeOggOpus: (): Promise<boolean> => canDecodeOggOpus(),
|
||||
|
||||
processStreamingChunk: async (playerId: string, chunk: Uint8Array): Promise<StreamingResult> => {
|
||||
const player = audioPlayers.get(playerId);
|
||||
if (!player) return { success: false, error: 'Player not found' };
|
||||
@@ -102,12 +117,39 @@ const DeepDrftAudio = {
|
||||
return player?.calculateByteOffset(positionSeconds) ?? 0;
|
||||
},
|
||||
|
||||
// "Load at timestamp" seam (Phase 18 wave 18.6 format switch). Resolve the file-absolute byte offset
|
||||
// to begin a stream at `position` with no playback/buffer state — the C# load-from-position path calls
|
||||
// this after initializeStreaming (Opus: sidecar resolves immediately; WAV: after a header probe) and
|
||||
// then streams from the returned offset via the seek/refill loop. seekBeyondBuffer:true + byteOffset.
|
||||
resolveStreamOffset: (playerId: string, position: number): AudioResult => {
|
||||
const player = audioPlayers.get(playerId);
|
||||
if (!player) return { success: false, error: 'Player not found' };
|
||||
return player.resolveStreamOffset(position);
|
||||
},
|
||||
|
||||
// Phase 21.2a back-pressure poll: the C# read loop calls this WHILE throttled to learn when
|
||||
// the scheduler has drained below low-water and reading may resume. A missing player reads as
|
||||
// "not paused" so a torn-down player never wedges a loop that is already exiting.
|
||||
isProductionPaused: (playerId: string): boolean => {
|
||||
const player = audioPlayers.get(playerId);
|
||||
return player?.isProductionPaused() ?? false;
|
||||
},
|
||||
|
||||
reinitializeFromOffset: (playerId: string, totalStreamLength: number, seekPosition: number): AudioResult => {
|
||||
const player = audioPlayers.get(playerId);
|
||||
if (!player) return { success: false, error: 'Player not found' };
|
||||
return player.reinitializeFromOffset(totalStreamLength, seekPosition);
|
||||
},
|
||||
|
||||
// Phase 21.3 / AC6: recover into a clean paused-but-loaded state after a window-miss refill
|
||||
// (seek-back past the retained tail) failed its Range fetch or reinit. Prevents the starved
|
||||
// scheduler from firing a silent false end; leaves the track loaded so a retry is possible.
|
||||
recoverFromFailedRefill: (playerId: string, seekPosition: number): AudioResult => {
|
||||
const player = audioPlayers.get(playerId);
|
||||
if (!player) return { success: false, error: 'Player not found' };
|
||||
return player.recoverFromFailedRefill(seekPosition);
|
||||
},
|
||||
|
||||
setVolume: (playerId: string, volume: number): AudioResult => {
|
||||
const player = audioPlayers.get(playerId);
|
||||
if (!player) return { success: false, error: 'Player not found' };
|
||||
|
||||
+82
@@ -0,0 +1,82 @@
|
||||
/**
|
||||
* Minimal ambient WebCodecs declarations.
|
||||
*
|
||||
* TypeScript 5.9's bundled lib.dom.d.ts does NOT yet ship the WebCodecs audio types
|
||||
* (`AudioDecoder`, `EncodedAudioChunk`, `AudioData`, `AudioDecoderConfig`), and this repo has no
|
||||
* package.json / node_modules to pull in `@types/dom-webcodecs`. Rather than add a dependency
|
||||
* toolchain for one feature, this declares exactly the slice of the WebCodecs surface the Opus
|
||||
* streaming decoder uses — nothing more. The shapes follow the W3C WebCodecs spec.
|
||||
*
|
||||
* These are runtime-optional: `AudioDecoder` is absent on Safari < 16.4 and older Firefox, so every
|
||||
* use site guards on `typeof AudioDecoder !== 'undefined'` before touching it (the capability gate).
|
||||
*/
|
||||
|
||||
interface AudioDecoderConfig {
|
||||
codec: string;
|
||||
sampleRate: number;
|
||||
numberOfChannels: number;
|
||||
/** Codec-specific setup bytes. For Opus this is the OpusHead identification header. */
|
||||
description?: BufferSource;
|
||||
}
|
||||
|
||||
interface AudioDecoderSupport {
|
||||
supported: boolean;
|
||||
config: AudioDecoderConfig;
|
||||
}
|
||||
|
||||
type AudioSampleFormat = 'u8' | 's16' | 's24' | 's32' | 'f32' | 'u8-planar' | 's16-planar' | 's24-planar' | 's32-planar' | 'f32-planar';
|
||||
|
||||
interface AudioDataCopyToOptions {
|
||||
planeIndex: number;
|
||||
frameOffset?: number;
|
||||
frameCount?: number;
|
||||
format?: AudioSampleFormat;
|
||||
}
|
||||
|
||||
interface AudioData {
|
||||
readonly format: AudioSampleFormat | null;
|
||||
readonly sampleRate: number;
|
||||
readonly numberOfFrames: number;
|
||||
readonly numberOfChannels: number;
|
||||
readonly duration: number;
|
||||
/** Presentation timestamp in microseconds. */
|
||||
readonly timestamp: number;
|
||||
allocationSize(options: AudioDataCopyToOptions): number;
|
||||
copyTo(destination: BufferSource, options: AudioDataCopyToOptions): void;
|
||||
close(): void;
|
||||
}
|
||||
|
||||
interface EncodedAudioChunkInit {
|
||||
type: 'key' | 'delta';
|
||||
/** Presentation timestamp in microseconds. */
|
||||
timestamp: number;
|
||||
duration?: number;
|
||||
data: BufferSource;
|
||||
}
|
||||
|
||||
declare class EncodedAudioChunk {
|
||||
constructor(init: EncodedAudioChunkInit);
|
||||
readonly type: 'key' | 'delta';
|
||||
readonly timestamp: number;
|
||||
readonly duration: number | null;
|
||||
readonly byteLength: number;
|
||||
}
|
||||
|
||||
interface AudioDecoderInit {
|
||||
output: (data: AudioData) => void;
|
||||
error: (error: DOMException) => void;
|
||||
}
|
||||
|
||||
type CodecState = 'unconfigured' | 'configured' | 'closed';
|
||||
|
||||
declare class AudioDecoder {
|
||||
constructor(init: AudioDecoderInit);
|
||||
readonly state: CodecState;
|
||||
readonly decodeQueueSize: number;
|
||||
configure(config: AudioDecoderConfig): void;
|
||||
decode(chunk: EncodedAudioChunk): void;
|
||||
flush(): Promise<void>;
|
||||
reset(): void;
|
||||
close(): void;
|
||||
static isConfigSupported(config: AudioDecoderConfig): Promise<AudioDecoderSupport>;
|
||||
}
|
||||
@@ -11,44 +11,20 @@
|
||||
* resets to 0 on `unobserve` (player minimized / disposed) so the spacer
|
||||
* collapses.
|
||||
*
|
||||
* COALESCING (Phase 20 theater-flash fix). `--player-height` has two consumers:
|
||||
* the layout spacer div AND the ambient WaveformVisualizer backdrop, whose
|
||||
* `bottom` inset is this var (WaveformVisualizer.razor.css `.mix-waveform-bg`).
|
||||
* Moving that inset changes the visualizer canvas's CSS box, which fires the
|
||||
* renderer's own canvas ResizeObserver — and a GL resize CLEARS the backing
|
||||
* store. That is correct and cheap for a discrete bar-height change (breakpoint
|
||||
* reflow, minimize/expand, error banner). But Theater Mode eases the player bar's
|
||||
* "now showing" band open/closed over ~0.45s via a CSS grid-rows transition, so
|
||||
* the bar height changes EVERY FRAME of the ease. Mirroring each intermediate
|
||||
* frame here would re-clear the GL backing store ~27×, reading as a flash.
|
||||
*
|
||||
* The fix coalesces the publish with a LEADING + TRAILING edge: the first change
|
||||
* after a quiet period is written immediately (so a discrete jump — the common
|
||||
* case — has zero added latency and the clip never lags), then a rapid STREAM of
|
||||
* further changes (an animated transition) is debounced and only its SETTLED
|
||||
* end-state is written. So a Theater ease resizes the visualizer at most twice
|
||||
* (leading 1px move + final settle) instead of once per frame. The settled value
|
||||
* is always the last write, so at-rest sizing/clip stays exact; and this remains
|
||||
* the SOLE writer of `--player-height`, so the renderer's ResizeObserver stays the
|
||||
* sole canvas size writer (its invariant is untouched).
|
||||
* SOLE CONSUMER (post visualizer-viewport-framing). `--player-height` now feeds
|
||||
* ONLY the layout spacer div (MainLayout.razor.css `.player-spacer.expanded`).
|
||||
* The ambient WaveformVisualizer backdrop is anchored `inset: 0` and no longer
|
||||
* reads this var, so a height change here only resizes the spacer's `height` — a
|
||||
* cheap, side-effect-free layout write. There is no GL backing store to clear and
|
||||
* no theater-flash to debounce against, so we publish every observed frame
|
||||
* directly: the spacer tracks the bar exactly through the Theater-Mode ease with
|
||||
* no settle lag, and this stays the SOLE writer of `--player-height`.
|
||||
*/
|
||||
|
||||
const HEIGHT_VAR = '--player-height';
|
||||
|
||||
/**
|
||||
* Quiet window (ms) after which a pending settled height is flushed. One change
|
||||
* then silence (a discrete reflow) flushes after this delay but was ALSO written
|
||||
* on the leading edge, so the trailing flush is a no-op — discrete jumps pay no
|
||||
* latency. A continuous transition keeps resetting this timer until it ends, then
|
||||
* flushes the final height once. ~80ms comfortably exceeds a frame interval (so a
|
||||
* mid-ease frame never trips an early flush) yet settles promptly after the ease.
|
||||
*/
|
||||
const SETTLE_MS = 80;
|
||||
|
||||
let observer: ResizeObserver | null = null;
|
||||
let lastWritten = -1;
|
||||
let pendingHeight = -1;
|
||||
let settleTimer: number | null = null;
|
||||
|
||||
function setVar(px: number): void {
|
||||
// Round up so sub-pixel heights never leave a hairline of overlap.
|
||||
@@ -58,28 +34,6 @@ function setVar(px: number): void {
|
||||
document.documentElement.style.setProperty(HEIGHT_VAR, `${rounded}px`);
|
||||
}
|
||||
|
||||
/**
|
||||
* Publish a measured height with leading + trailing coalescing. Leading: if no
|
||||
* settle is pending, this is the first change after a quiet period — write it now.
|
||||
* Trailing: (re)arm the settle timer so the final value of a rapid stream lands
|
||||
* once the stream stops.
|
||||
*/
|
||||
function publishHeight(px: number): void {
|
||||
pendingHeight = px;
|
||||
if (settleTimer === null) {
|
||||
// Leading edge — discrete jumps land immediately; the first frame of a
|
||||
// transition lands too (one resize), then the rest is debounced below.
|
||||
setVar(px);
|
||||
}
|
||||
if (settleTimer !== null) {
|
||||
clearTimeout(settleTimer);
|
||||
}
|
||||
settleTimer = window.setTimeout(() => {
|
||||
settleTimer = null;
|
||||
setVar(pendingHeight);
|
||||
}, SETTLE_MS);
|
||||
}
|
||||
|
||||
function measure(entry: ResizeObserverEntry): number {
|
||||
// Prefer the border-box measurement; fall back to contentRect on the
|
||||
// (older) engines that don't populate borderBoxSize.
|
||||
@@ -94,28 +48,17 @@ export function observe(element: Element): void {
|
||||
observer = new ResizeObserver(entries => {
|
||||
const entry = entries[0];
|
||||
if (!entry) return;
|
||||
publishHeight(measure(entry));
|
||||
setVar(measure(entry));
|
||||
});
|
||||
observer.observe(element);
|
||||
|
||||
// Seed synchronously so the spacer is correct on this frame, before the
|
||||
// first ResizeObserver callback fires. A fresh observe target is a discrete
|
||||
// change, so write it straight through (bypassing the debounce) — re-pointing
|
||||
// the observer (e.g. expanded <-> minimized) must not lag behind a settle.
|
||||
if (settleTimer !== null) {
|
||||
clearTimeout(settleTimer);
|
||||
settleTimer = null;
|
||||
}
|
||||
// first ResizeObserver callback fires.
|
||||
setVar(element.getBoundingClientRect().height);
|
||||
}
|
||||
|
||||
export function unobserve(): void {
|
||||
observer?.disconnect();
|
||||
observer = null;
|
||||
if (settleTimer !== null) {
|
||||
clearTimeout(settleTimer);
|
||||
settleTimer = null;
|
||||
}
|
||||
pendingHeight = -1;
|
||||
setVar(0);
|
||||
}
|
||||
|
||||
+2
-1
@@ -8,7 +8,8 @@
|
||||
* iframe) it returns null rather than throwing, and the caller simply sends no anonId. Over-counting is
|
||||
* the known, accepted direction of error (§3).
|
||||
*
|
||||
* Exposed on window.DeepDrftAnonId; imported once in App.razor alongside the audio engine and beacon.
|
||||
* Exposed on window.DeepDrftAnonId; served from js/session/anonid.js, imported once in App.razor
|
||||
* alongside the audio engine and the unload-lifecycle module.
|
||||
*/
|
||||
|
||||
const STORAGE_KEY = 'deepdrft.anonId';
|
||||
+18
-12
@@ -1,10 +1,16 @@
|
||||
/**
|
||||
* Telemetry beacon interop (Phase 16 §2.2). A thin wrapper over navigator.sendBeacon for fire-and-forget
|
||||
* play/share events, plus a page-unload handler that lets the player close an open play session as the
|
||||
* tab goes away. sendBeacon (not fetch) is the load-bearing choice: it survives page unload, where a
|
||||
* fetch would be cancelled — exactly the tab-close edge case the play metric must still record.
|
||||
* Page-lifecycle unload transport. A thin wrapper over navigator.sendBeacon for the single edge case where
|
||||
* an awaited fetch cannot run: the page is being torn down (tab close, navigation, bfcache freeze, mobile
|
||||
* backgrounding). It exposes a sendBeacon POST plus a page-unload handler that lets the player close an
|
||||
* open play session as the tab goes away. sendBeacon (not fetch) is the load-bearing choice here: it
|
||||
* survives page unload, where a fetch would be cancelled.
|
||||
*
|
||||
* Exposed on window.DeepDrftBeacon; imported once in App.razor alongside the audio engine.
|
||||
* Normal play closes (organic end / track-switch / stop) and all share events do NOT use this module —
|
||||
* they go over a first-party same-origin HttpClient POST from C#, which privacy/tracking heuristics do not
|
||||
* block. This module is named off the former telemetry/beacon path (DeepDrftLifecycle, served from
|
||||
* js/session/lifecycle.js) so even this retained unload fallback is not caught by name-based blockers.
|
||||
*
|
||||
* Exposed on window.DeepDrftLifecycle; imported once in App.razor alongside the audio engine.
|
||||
*/
|
||||
|
||||
// .NET interop type — a DotNetObjectReference the unload handler invokes back into.
|
||||
@@ -47,11 +53,11 @@ function wireUnloadOnce(): void {
|
||||
});
|
||||
}
|
||||
|
||||
const DeepDrftBeacon = {
|
||||
const DeepDrftLifecycle = {
|
||||
/**
|
||||
* Queue a fire-and-forget POST of a small JSON body. Returns false if the browser refused to queue
|
||||
* the beacon (e.g. over the per-origin byte budget) — callers ignore it; a dropped telemetry event
|
||||
* is acceptable by design.
|
||||
* Queue a fire-and-forget sendBeacon POST of a small JSON body, for the page-unload edge only. Returns
|
||||
* false if the browser refused to queue the beacon (e.g. over the per-origin byte budget) — callers
|
||||
* ignore it; a dropped telemetry event is acceptable by design.
|
||||
*/
|
||||
send: (url: string, json: string): boolean => {
|
||||
try {
|
||||
@@ -79,10 +85,10 @@ const DeepDrftBeacon = {
|
||||
|
||||
declare global {
|
||||
interface Window {
|
||||
DeepDrftBeacon: typeof DeepDrftBeacon;
|
||||
DeepDrftLifecycle: typeof DeepDrftLifecycle;
|
||||
}
|
||||
}
|
||||
|
||||
window.DeepDrftBeacon = DeepDrftBeacon;
|
||||
window.DeepDrftLifecycle = DeepDrftLifecycle;
|
||||
|
||||
export { DeepDrftBeacon };
|
||||
export { DeepDrftLifecycle };
|
||||
@@ -0,0 +1,33 @@
|
||||
/**
|
||||
* Listener-settings interop (Phase 18 wave 18.6). A safe, eval-free cookie helper for persisting
|
||||
* public-site preferences (streaming quality, and any future setting added under PublicSiteSettings).
|
||||
* The 365-day durable-truth seam dark mode uses — same mechanism, no eval.
|
||||
*
|
||||
* Exposed on window.DeepDrftSettings; imported once in App.razor.
|
||||
*/
|
||||
|
||||
const DeepDrftSettings = {
|
||||
/**
|
||||
* Write a cookie with the given name, value, and lifetime. Equivalent to the browser's
|
||||
* document.cookie assignment but without building JS via string interpolation or eval.
|
||||
* Path is always "/"; SameSite is always "Lax" — matches the dark-mode cookie semantics.
|
||||
*/
|
||||
setCookie: (name: string, value: string, days: number): void => {
|
||||
const expires = new Date();
|
||||
expires.setTime(expires.getTime() + days * 24 * 60 * 60 * 1000);
|
||||
document.cookie =
|
||||
`${encodeURIComponent(name)}=${encodeURIComponent(value)}` +
|
||||
`; expires=${expires.toUTCString()}` +
|
||||
`; path=/; SameSite=Lax`;
|
||||
},
|
||||
};
|
||||
|
||||
declare global {
|
||||
interface Window {
|
||||
DeepDrftSettings: typeof DeepDrftSettings;
|
||||
}
|
||||
}
|
||||
|
||||
window.DeepDrftSettings = DeepDrftSettings;
|
||||
|
||||
export { DeepDrftSettings };
|
||||
@@ -44,6 +44,13 @@
|
||||
* position while !isPlaying). The loop stops only on tab-hidden (visibilitychange) and dispose.
|
||||
*/
|
||||
|
||||
import { decodePressure } from '../audio/decodePressure.js';
|
||||
|
||||
// Re-exported so the Blazor bridge (WaveformVisualizer.razor.cs) reaches the HW-accel probe through
|
||||
// the same module reference it already imports for create() — one JS import surface, no second handle.
|
||||
// The probe itself (and its unit-tested pure classifier) lives in hwAccel.ts.
|
||||
export { detectHardwareAcceleration } from './hwAccel.js';
|
||||
|
||||
// ── Tuning anchors (see spec §B). These are the load-bearing constants. ──────────
|
||||
|
||||
/**
|
||||
@@ -148,6 +155,16 @@ const RIBBON_HALF_WIDTH_FRAC = 0.92;
|
||||
*/
|
||||
const MAX_DPR = 2;
|
||||
|
||||
/**
|
||||
* Minimum milliseconds between drawn frames WHILE decode is under sustained pressure (Part 1 —
|
||||
* auto-protect audio). 1000/15 ≈ 66.7 ms caps the loop at ~15 fps, cutting the main-thread WebGL
|
||||
* software-render + physics cost by ~75% so the synchronous WebCodecs Opus decode (which shares the
|
||||
* main thread when HW accel is off) gets the time it needs to keep up. The decodePressure signal is
|
||||
* false in the common case (HW accel on, or lossless), so this cap never applies and the loop draws
|
||||
* every frame at full quality. Tunable; the exact fps that clears starvation is browser-confirmed.
|
||||
*/
|
||||
const PRESSURE_THROTTLE_FRAME_MS = 1000 / 15;
|
||||
|
||||
// ════════════════════════════════════════════════════════════════════════════════════
|
||||
// R2 — the wax-blob lava physics (CPU step + uniform upload). The lava is now a real
|
||||
// Lagrangian particle system integrated each frame on the JS side and rendered as
|
||||
@@ -1679,6 +1696,10 @@ export function create(canvas: HTMLCanvasElement): WaveformVisualizerHandle {
|
||||
let rafId: number | null = null;
|
||||
let disposed = false;
|
||||
const startTimeMs = performance.now();
|
||||
// Wall-clock of the last DRAWN continuous-loop frame, for the decode-pressure throttle (Part 1).
|
||||
// While decodePressure.isUnderPressure() the loop draws at most once per PRESSURE_THROTTLE_FRAME_MS
|
||||
// so the main thread yields time back to a starved decode; unthrottled it draws every frame.
|
||||
let lastDrawMs = performance.now();
|
||||
// Wall-clock anchor for the physics dt (separate from the playhead decay clock).
|
||||
let lastPhysicsMs = performance.now();
|
||||
|
||||
@@ -1923,9 +1944,30 @@ export function create(canvas: HTMLCanvasElement): WaveformVisualizerHandle {
|
||||
rafId = null;
|
||||
return;
|
||||
}
|
||||
|
||||
// Auto-protect audio under decode pressure (Part 1). When the WebCodecs Opus decode pipeline
|
||||
// reports SUSTAINED lag (decodePressure.isUnderPressure()), throttle the draw cadence to
|
||||
// ~PRESSURE_THROTTLE_FRAME_MS so this loop's main-thread GL + physics cost yields time back to
|
||||
// decode; we still reschedule every frame so full cadence resumes the instant decode recovers.
|
||||
// A no-op when decode is healthy — isUnderPressure() stays false, the gate is always open, and
|
||||
// every frame draws exactly as before. Skipping a draw also skips the physics step (it runs
|
||||
// inside draw()), and its dt is clamped to PHYSICS_MAX_DT, so a throttled gap never lurches the
|
||||
// lava. redrawOnce() (idle/control-tweak stills) is intentionally NOT throttled — those are rare
|
||||
// one-shots, not the continuous loop.
|
||||
const nowMs = performance.now();
|
||||
if (!decodePressure.isUnderPressure() || nowMs - lastDrawMs >= PRESSURE_THROTTLE_FRAME_MS) {
|
||||
lastDrawMs = nowMs;
|
||||
drawFrame();
|
||||
}
|
||||
|
||||
rafId = requestAnimationFrame(frame);
|
||||
}
|
||||
|
||||
/** One drawn continuous-loop frame: the GL draw plus the gated FPS/lava diagnostic tally. */
|
||||
function drawFrame(): void {
|
||||
draw();
|
||||
|
||||
// FPS tally: count this callback, and once per elapsed second emit the rate.
|
||||
// FPS tally: count this drawn frame, and once per elapsed second emit the rate.
|
||||
// performance.now() is cheap (no GPU stall, unlike gl.getError); the gated log
|
||||
// fires at most once/sec, so this adds no meaningful per-frame cost.
|
||||
if (DEBUG) {
|
||||
@@ -1968,10 +2010,6 @@ export function create(canvas: HTMLCanvasElement): WaveformVisualizerHandle {
|
||||
fpsWindowStartMs = nowMs;
|
||||
}
|
||||
}
|
||||
|
||||
// Reschedule unconditionally — the loop runs continuously now (lava reframe Part C); it is
|
||||
// stopped only by dispose() or the tab going hidden, never by audio pausing.
|
||||
rafId = requestAnimationFrame(frame);
|
||||
}
|
||||
|
||||
// ── Tab-visibility gating (lava reframe Part C power-saving). ────────────────────
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user