Merge streaming-overhaul into dev (Opus low-data streaming, windowed streaming, HW-accel-off stabilization)

This commit is contained in:
daniel-c-harvey
2026-06-26 11:14:59 -04:00
97 changed files with 10086 additions and 591 deletions
+21 -4
View File
@@ -66,11 +66,28 @@ If step 1 succeeds and step 2 fails, audio is orphaned in the vault (no rollback
The player is not fetch-then-play:
1. Client calls `GET api/track/{id}` on DeepDrftContent and receives WAV bytes as a stream (`HttpCompletionOption.ResponseHeadersRead`).
2. `StreamingAudioPlayerService` reads in adaptive 1664 KB chunks, pushes each via `AudioInteropService.processStreamingChunk`.
3. TypeScript `StreamDecoder` parses WAV header, decodes chunks to `AudioBuffer`s. `PlaybackScheduler` schedules them on a Web Audio graph.
1. Client calls `GET api/track/{id}` on DeepDrftContent and receives the first bounded segment (`Range: bytes=0-{SegmentSizeBytes-1}`, 4 MB) via `HttpCompletionOption.ResponseHeadersRead`.
2. `StreamingAudioPlayerService` reads in adaptive 1664 KB chunks within each segment, pushes each via `AudioInteropService.processStreamingChunk`.
3. TypeScript `StreamDecoder` parses the format header, decodes chunks to `AudioBuffer`s. `PlaybackScheduler` schedules them on a Web Audio graph.
4. Playback starts as soon as a min buffer is queued; UI duration from parsed header (not waiting for full file).
5. **Seek beyond buffer**: if seek target is past what's decoded, client issues `GET api/track/{id}` with `Range: bytes={byteOffset}-`. Server streams raw bytes from that file-absolute offset with a `206 Partial Content` response. Player retains the parsed WAV header and feeds the raw PCM continuation into the existing decode pipeline.
5. **Seek beyond buffer**: if seek target is past what's decoded, `StreamingAudioPlayerService` issues a new bounded segment starting at the seek byte offset. Server responds 206; player retains the parsed header and feeds the raw continuation into the existing decode pipeline.
**Memory bounding (three complementary layers, all required):**
- **Raw-queue bound (`StreamDecoder`):** `releaseConsumedChunks()` front-compacts `rawChunks` after each aligned segment is decoded, using a `discardedBytes` absolute cursor so all offsets remain absolute even as the array's front moves. Without this, a long WAV (e.g. a 92-min mix ≈ 970 MB raw) accumulates its entire decoded-from body in `rawChunks` regardless of the decode-side bounds below.
- **Decoded-queue bound (`PlaybackScheduler`):** `evictPlayedBuffers()` discards already-played `AudioBuffer`s, capping the scheduler's forward fill to a 96 MB ceiling (Phase 21.2). Decoded PCM is larger than source (Web Audio uses 32-bit float — a 16-bit stereo WAV roughly doubles; Opus decodes to the same float footprint).
- **Network bound (segmented fetch, Phase 21 Direction B):** The forward stream is fetched as sequential bounded `bytes=cursor-{cursor+4MB-1}` Range requests via `RunSegmentedStreamAsync`; the next segment is fetched only after `DrainBackpressureAsync` confirms the scheduler is below low-water. Because each segment is fully consumed before the next is issued, the browser holds at most ~one segment of raw bytes. A per-chunk drain inside the segment loop (gated on `_streamingPlaybackStarted` so it cannot deadlock first-audio) additionally prevents high-density codecs (e.g. Opus, where a 4 MB segment is ~100 s of audio) from decoding the whole segment eagerly ahead of the playhead before the inter-segment gate runs.
**Playback stability invariants (streaming-stabilization arc):**
- **Back-pressure water marks:** forward fill 60s high / 30s low (`PlaybackScheduler.DEFAULT_FORWARD_HIGH_WATER_SECONDS` / `...LOW_WATER_SECONDS`). Production pauses on `lookahead ≥ high OR decoded bytes > 96 MB ceiling`, whichever first. The time window is a jitter cushion for Opus's async decode ramp; the byte cap is the hard OOM guarantee and is unchanged.
- **Genuine end-of-playback:** `PlaybackScheduler` uses a `streamComplete` flag (set by `setStreamComplete`) combined with an `underrun_` park/resume state to distinguish a drained-but-still-streaming queue (startup/underrun gap → park, resume on refill) from a truly finished track (stream complete AND queue drained → `finishPlayback()`, the single genuine-end path). Prevents the false `onPlaybackEnded` that previously fired when the Opus WebCodecs queue momentarily drained during the decode ramp.
- **Rebuffer hysteresis:** Opus playback start and underrun-resume are gated on `hasMinimumPlaybackLead()` (1s decoded lead, `DEFAULT_MIN_PLAYBACK_LEAD_SECONDS`); WAV keeps `hasMinimumBuffers(6)`. `streamComplete` overrides the gate so a short tail still plays out. `StreamingAudioPlayerService.TryStartPlaybackAsync()` force-starts after `MarkStreamCompleteAsync` when the threshold was never crossed (ultra-short track protection).
- **Opus `AudioContext` pre-aligned to 48 kHz** in `AudioPlayer.initializeStreaming` before any bytes flow, eliminating the mid-decode context teardown that previously OOM'd the tab under software rendering.
**Visualizer / decode contention (decodePressure + hwAccel):**
- `DeepDrftPublic/Interop/audio/decodePressure.ts` exports a shared `DecodePressureSignal` singleton. The audio pipeline (`OpusStreamDecoder` yield-cap events, `PlaybackScheduler` underrun-parking events) calls `report()` on sustained lag; the visualizer calls `isUnderPressure()` each rAF frame. The signal engages only after ≥ 5 reports within 2500 ms with a 1s minimum hold (hysteresis); a lone startup-ramp blip never engages.
- Under pressure, `WaveformVisualizer.ts` throttles its rAF loop to ~15 fps (`PRESSURE_THROTTLE_FRAME_MS = 1000/15`), cutting main-thread WebGL software-render + physics cost so WebCodecs decode recovers. A no-op under HW accel.
- `DeepDrftPublic/Interop/visualizer/hwAccel.ts` probes `WEBGL_debug_renderer_info.UNMASKED_RENDERER_WEBGL` for software-renderer signatures (SwiftShader, llvmpipe, softpipe, Microsoft Basic Render, etc.). Absence of the debug extension is treated as accelerated — lava is disabled only on positive evidence. On first interactive render, `WaveformVisualizerControlState.ApplyCapabilityDefault(bool)` applies a one-time scoped default: `LavaEnabled = false` (the expensive subsystem) when no HW accel is detected, `WaveformEnabled` stays on. Guarded by `_capabilityDefaultApplied`; never overrides an explicit user toggle.
- **Known limitation / deferred escalation:** HW-accel-off Opus playback is made usable by defaulting lava off and throttling under pressure. If starvation recurs (e.g. waveform-only path also proves too costly, or a software renderer slips the probe), the documented next step is moving Opus WebCodecs decode off the main thread (Web Worker / AudioWorklet) so it stops competing with main-thread rendering. Not a current implementation — a recorded fallback.
Keep this seam clean — it is the most architecturally load-bearing part of the playback path.
+79
View File
@@ -47,6 +47,83 @@ Newest entries at the top. Group by phase/wave header (mirroring `PLAN.md` / `CM
---
## Phase 21 — Windowed Streaming Buffer (landed 2026-06-24)
**Landed:** 2026-06-24 on `streaming-overhaul`.
- **What:** Bounded client memory for long audio streams. Playing a 1 GB+ DJ Mix (the Phase 9 `Mix` medium — a single long track) no longer accumulates the full decoded PCM in the browser; instead the player holds only a sliding forward window and discards what has already played. Four waves, all landed. Public listener site only (`DeepDrftPublic.Client` player stack + `DeepDrftPublic` TypeScript audio interop); no CMS, no API endpoint, no schema change.
- **Why:** The `PlaybackScheduler` held an `AudioBuffer[]` it never evicted — both decode paths (`StreamDecoder`/`IFormatDecoder` for WAV/MP3/FLAC, and `OggDemuxer``OpusStreamDecoder` for Opus) pushed into it without limit. Decoded PCM is larger than the source (Web Audio is 32-bit float per sample/channel — a 16-bit stereo WAV roughly doubles once decoded; a low-data Opus mix decodes to the same float footprint regardless of how few compressed bytes arrived), so a 1 GB WAV or Opus mix could accumulate ~2 GB of retained float in the browser. The Opus path also had a second upstream accumulation locus: the WebCodecs `AudioDecoder` work queue and `decodedQueue: AudioData[]`.
- **Shape (by wave):**
- **21.1 — Partial eviction in `PlaybackScheduler` (cold-start; load-bearing; shared by both paths).** Drop already-played buffers while keeping position/index/time-anchor bookkeeping exact against a buffer array that no longer begins at absolute time 0. Written once, serves both decode paths (they `addBuffer` identically). This is the hardest correctness work in the phase. No refill yet.
- **21.2 — Back-pressure (one fill signal, two throttle sites).** Bound the unplayed region by stopping production above a high-water mark and resuming below low-water, driven by `PlaybackScheduler.evaluateProductionPause()` — a single shared scheduler-fill signal (OQ6/OQ7 resolved: shared controller, per-path hook). **21.2a**`StreamingAudioPlayerService` stops fetching the next segment above high-water, polling `evaluateProductionPause()` at 100 ms cadence until the fill drains below low-water (serves both paths). **21.2b** — the Opus demux/decode feed is additionally stopped when the same signal is set, so the WebCodecs decode queue and `decodedQueue` do not balloon behind a paused segment loop (Opus only; no WAV analogue). Together with 21.1 this bounds both the played and unplayed sides on both formats.
- **21.3 — Seek-back-past-window refill + clean failure recovery.** When a backward seek lands earlier than the retained tail, the existing seek-beyond-buffer path is reused pointed at the earlier offset (whichever resolver the active path ships: `IFormatDecoder`/`StreamDecoder` for WAV; the live `resolveOpusByteOffset` + `OpusStreamDecoder.reinitializeForRangeContinuation` for Opus). Minimal AC6 refill-failure handling added: `RecoverFromFailedRefill` clears the scheduler and surfaces a user-visible error rather than leaving the player wedged.
- **21.4 — Validation + Direction A→B pivot (the critical wave — see as-built divergence below).** Network-memory bounding confirmed in Daniel's browser run: the segmented approach delivers ~4 MB segments pacing with playback, with the browser holding ~one segment of raw bytes rather than the full artifact.
- **As-built divergence — Direction A→B pivot (important).** The spec recommended **Direction A** (sliding window on one open-ended forward stream, relying on pausing `ReadAsync` / the segment loop to backpressure the socket) with **Direction B** (discrete bounded `Range: bytes=start-end` segments, each fetched only when the scheduler drains below low-water) held as the documented fallback "if Direction A's back-pressure proves leaky in practice."
**21.4 browser validation proved it leaky.** On Blazor WASM, the browser `fetch` API buffers the entire HTTP response body regardless of how slowly the application reads it: pausing reads bounded the *decode* but not the *network download*, so the entire ~970 MB body accumulated in browser memory even though the application only decoded a window of it. Direction A's core assumption — that pausing `ReadAsync` closes the TCP flow-control window before the browser caches the whole body — does not hold in the WASM `fetch` runtime.
**We pivoted to Direction B.** The forward stream now issues sequential bounded `bytes=cursor-(cursor+SegmentSizeBytes-1)` Range requests (`SegmentSizeBytes = 4 MB`), each fetched only after the scheduler drains below low-water — via `RunSegmentedStreamAsync` in `StreamingAudioPlayerService`. Because each 4 MB request is fully consumed before the next is issued, the browser holds at most ~one segment of raw bytes regardless of file size. The decode-side windowing (21.1/21.2) pairs with it: the segment loop's segment-gate replaces the raw `ReadAsync` pause as the production throttle (21.2a), and the Opus decode-ahead throttle (21.2b) hooks the same `evaluateProductionPause()` signal. Seek and refill converge on the same segmented loop. Direction B is recorded as the shipped approach; Direction A is recorded as tried-in-validation and found insufficient for the WASM `fetch` runtime.
- **What Phase 21 does NOT include:** the full AC matrix re-run (Opus seek-storm, visualizer, rapid-seek concurrency under the new segmented loop) beyond the network-memory bounding confirmed in 21.4. Those acceptance criteria remain the validation baseline for any follow-on work touching this seam.
- **Design memo:** `product-notes/phase-21-windowed-streaming-buffer.md` (note: the spec recommended Direction A; the as-built pivot to Direction B is annotated at the top of that doc and in §3.2/§3.3).
### Phase 21 — Post-landing: raw-queue and per-chunk bounds (landed 2026-06-24)
**Landed:** 2026-06-24 on `streaming-overhaul`. Fixes a Chrome tab crash (main-process OOM with hardware acceleration OFF) on long-track playback. Two unbounded memory queues remained after Phase 21 landed; this patch adds complementary bounds on both.
1. **`StreamingAudioPlayerService` — per-chunk drain inside segments.** Phase 21 Direction B's inter-segment back-pressure gate is matched to WAV byte density (~24 s per 4 MB segment), but not to Opus: at 320 kbps a 4 MB segment is ~100 s of decodable audio. The inner chunk loop had no fill check, so the entire segment decoded eagerly ahead of the playhead — piling decoded f32 PCM into main-process RAM before the inter-segment gate ever ran. Fix: an extracted `DrainBackpressureAsync` helper (shared with the inter-segment gate) is called per chunk once the scheduler is over high-water, gated on `_streamingPlaybackStarted` so it cannot deadlock first-audio (the playhead doesn't advance until playback starts, so draining to low-water would block indefinitely before the first buffer plays). Also added load-generation diagnostic logging (`_loadGeneration`) to confirm single-load-per-play-action in-browser.
2. **`StreamDecoder.ts``releaseConsumedChunks()` front-compaction of `rawChunks`.** `rawChunks` retained the entire decoded-from stream body because consumed chunks were never released — a 92-min WAV mix (≈ 970 MB raw) accumulated all of it in the browser. Phase 21.2's 96 MB decoded-side ceiling does not reach this raw queue. Fix: `releaseConsumedChunks()` walks `rawChunks` after each aligned segment is decoded and splices off all fully-consumed chunks (any chunk whose end ≤ `processedBytes`), advancing `discardedBytes` (an absolute cursor into the logical stream) so `extractAlignedData` can continue walking from the correct position even though `rawChunks[0]` no longer begins at byte 0. This is the raw-side analogue of `PlaybackScheduler.evictPlayedBuffers` (the decoded side).
Together with the existing decoded-side 96 MB ceiling (Phase 21.2) and the segmented fetch (Phase 21 Direction B), these complete the three-layer memory bound: raw queue, decoded queue, and network.
### Phase 21 — Post-landing: streaming stabilization (landed 2026-06-26)
**Landed:** 2026-06-26 on `streaming-overhaul`. Resolves playback instability that surfaced specifically with Opus + hardware acceleration off: the Opus underrun/false-end cycle, visualizer starvation of WebCodecs decode, and an OOM path from mid-decode `AudioContext` teardown. All changes are TypeScript-only (`DeepDrftPublic/Interop/audio/` + `DeepDrftPublic/Interop/visualizer/`) plus the C# service that calls them; no API, no schema, no CMS change.
1. **Back-pressure water marks widened to 60s high / 30s low.** `PlaybackScheduler.ts` `DEFAULT_FORWARD_HIGH_WATER_SECONDS` / `DEFAULT_FORWARD_LOW_WATER_SECONDS` doubled from 30/15. The 96 MB decoded-byte ceiling is unchanged and remains the hard OOM bound; production pauses on `lookahead ≥ high OR bytes > cap`, whichever first. The wider time window absorbs per-packet WebCodecs decode jitter (Opus 48 kHz stereo ≈ 0.37 MB/s → 60 s ≈ 23 MB — well within the cap) without false back-pressure on the Opus decode ramp.
2. **End-of-playback gated on genuine completion (`streamComplete` + `underrun_`).** `PlaybackScheduler` now distinguishes a drained-but-still-streaming queue (startup/underrun gap) from a truly finished track (stream complete AND queue drained). `streamComplete` (set by `setStreamComplete`, called from `AudioPlayer.markStreamComplete`) is the discriminator. When the queue drains while `streamComplete` is false the scheduler parks in `underrun_` state (anchors frozen, `isActive_` false) and resumes from `scheduleNewBuffers` when buffers arrive. `finishPlayback()` is the single genuine-end path, reached only when both conditions hold simultaneously. Previously a drained Opus queue during the WebCodecs startup ramp fired `onPlaybackEnded`, nulling Duration and advancing the queue.
3. **Rebuffer hysteresis — 1s decoded-lead gate.** `hasMinimumPlaybackLead()` gates Opus playback START and underrun RESUME (`DEFAULT_MIN_PLAYBACK_LEAD_SECONDS = 1.0`), preventing the decode-ramp thrash (resume on a 20 ms Opus packet → drain immediately → re-park → repeat). `streamComplete` overrides the gate so a short tail plays out rather than parking forever. WAV keeps its `hasMinimumBuffers(6)` buffer-count start gate unchanged — large synchronous WAV segments rarely underrun and that gate's character must not change. **Complete-without-start force-start:** `StreamingAudioPlayerService.TryStartPlaybackAsync()` is called after `MarkStreamCompleteAsync` when `_streamingPlaybackStarted` is still false, preventing an ultra-short Opus track (total audio below the 1s lead) from sitting loaded-but-not-playing.
4. **Opus `AudioContext` pre-aligned to 48 kHz.** `AudioPlayer.initializeStreaming` calls `contextManager.recreateWithSampleRate(OPUS_SAMPLE_RATE)` when the content type resolves as Opus, before any bytes flow. The context is already at 48 kHz when the decoder's own lazy `recreateWithSampleRate` check runs, so it short-circuits (no-op). Eliminates the mid-decode context teardown-and-rebuild that previously double-decoded the stream and OOM'd the tab under HW accel off.
5. **`decodePressure` singleton + `WaveformVisualizer` auto-throttle (~15 fps under pressure).** `DeepDrftPublic/Interop/audio/decodePressure.ts` exports a process-wide `DecodePressureSignal` singleton shared between the audio pipeline (producer: `OpusStreamDecoder` yield-cap events and `PlaybackScheduler` underrun-parking events) and the visualizer (consumer). Engages only on sustained stress (≥ 5 `report()` calls within a 2500 ms window) and holds for a 1000 ms minimum dwell before releasing, suppressing lone startup-ramp blips. `WaveformVisualizer.ts` checks `decodePressure.isUnderPressure()` each rAF frame; when engaged it skips frames so at most one renders per `PRESSURE_THROTTLE_FRAME_MS` (`1000/15` ms, ~15 fps), cutting main-thread WebGL software-render + physics cost so WebCodecs decode recovers. A no-op under HW accel — healthy decode never calls `report()` and `isUnderPressure()` stays false indefinitely.
6. **Hardware-acceleration detection + `ApplyCapabilityDefault`.** `DeepDrftPublic/Interop/visualizer/hwAccel.ts` probes `WEBGL_debug_renderer_info.UNMASKED_RENDERER_WEBGL` for known software-renderer signatures (SwiftShader, llvmpipe, softpipe, Microsoft Basic Render Driver, Mesa Offscreen, "software"). Absence of the debug extension is treated as accelerated — the probe disables lava only on positive evidence of software rendering, not on uncertainty. `WaveformVisualizerControlState.ApplyCapabilityDefault(bool hardwareAccelerated)` is called once per session by the visualizer bridge on first interactive render: when `hardwareAccelerated` is false it sets `LavaEnabled = false` (the expensive main-thread subsystem) while leaving `WaveformEnabled` on, coerces Theater Mode, and raises `Changed` once so all observers see the default in a single cycle. Guarded by `_capabilityDefaultApplied` so it runs at most once per session and never overrides an explicit in-session user toggle. With HW accel present the call is a no-op.
---
## Phase 18 — Opus Low-Data Streaming (landed 2026-06-23)
_Note: Two distinct efforts share the "Phase 18" label — phase numbers are organisational, not sequential. See also **Phase 18 — Theme / Dark-Mode Remediation (landed 2026-06-19)** below._
**Landed:** 2026-06-23 on `streaming-overhaul`.
- **What:** Dual-format audio delivery — the existing lossless WAV path plus a new low-data Ogg Opus (fullband, 320 kbps) path — giving listeners a quality choice. Opus is the default (capability-gated). Six waves, all landed.
- **Why:** Lossless WAV works but imposes high bandwidth cost, particularly for long DJ Mixes. Opus at 320 kbps is indistinguishable for typical listening and an order of magnitude smaller, making the site usable on metered or constrained connections. The bespoke Web Audio decode→schedule graph is retained by deliberate choice; Opus feeds the same `IFormatDecoder` seam rather than an HTML `<media>` element or MSE.
- **Shape (by wave):**
- **18.1 — Ingest transcode + seek-index sidecar.** New `OpusTranscodeService`/processor in `DeepDrftContent`, invoked post-store from `UnifiedTrackService.UploadAsync` as a background job (failure-tolerant; the track plays lossless first). Produces the Ogg Opus 320 artifact and a combined seek/setup **sidecar** (0.5 s-bucketed granule→byte index + `OpusHead`/`OpusTags` setup header), both stored in a new `track-opus` vault. **ffmpeg/libopus** added to the deploy bootstrap as a runtime prereq on the API host.
- **18.2 — Storage + lookup contract.** Server-side "given `EntryKey` + format, return the right artifact + content-type, with lossless fallback when no Opus artifact exists."
- **18.3 — Delivery: `?format=opus|lossless` + sidecar serving + proxy threading.** `?format=` param on the stream endpoint (Range preserved); sidecar-serving endpoint; param forwarded through `TrackProxyController`; `TrackMediaClient` sends the format param.
- **18.4 — Client Opus decoder + index-based seek + capability detection.** New `OpusFormatDecoder` (`IFormatDecoder`): Ogg-page demux, setup-header carry from the cached sidecar, `calculateByteOffset` binary-searching the precomputed seek index (VBR-safe accurate seek — not interpolation). Capability gate falls back to the lossless path on browsers without Opus support. Testing-phase fixes also landed: a valid ffmpeg-generated capability-probe sample and correct duration reporting (fixed the seekbar + visualizer in Opus mode).
- **18.5 — End-to-end integration.** Player format selection (Opus-by-default, capability-gated); sidecar fetch/inject on track load; **Backfill-Opus** CMS bulk action (third sibling to Generate-Profiles / Backfill-High-res); replace-audio regenerates Opus + sidecar.
- **18.6 — Settings menu + quality toggle + CMS status.** New public-site **Settings menu** (app-bar trigger + settings-item abstraction + `ListenerSettings`, persisted via the dark-mode-pattern seam: `streamQuality` cookie → prerender-read → `PersistentComponentState` → client cookie service); streaming-quality toggle (Low-data/Lossless) as its first occupant. CMS **Opus-status badge** on the track grid. CMS upload meter **Post-Processing** phase visualizing background Opus transcode.
- **As-built divergence — the Opus decoder (important).** The originally-spec'd "segment-and-decode-each-page via `decodeAudioData`" model was implemented and then **replaced**: browser testing showed it glitches at every segment boundary and broke seeking (Opus has pre-skip + inter-frame state, so independent per-segment decode is architecturally wrong). The landed decoder is a **WebCodecs `AudioDecoder` streaming pipeline** (Ogg demux → packets → stateful continuous decode), with index-driven seek and a WebCodecs-based capability gate. Daniel's call at the decision point — offered a decode-whole stopgap (Option A) vs. the proper WebCodecs streaming decoder (Option B), he chose B: "Let's go for B, why do you like half-solutions so much" (the same no-stopgap posture as the Mix visualizer WebGL2 decision, where he said "no stopgap — WebGL as step 1, no pussyfooting"). The segmented approach is recorded as tried-and-replaced.
- **What Phase 18 does NOT include:** windowed / bounded-memory streaming. Phase 18 still fetches the whole (smaller) Opus artifact; bounding client memory to a sliding forward window is **Phase 21's job**.
- **Design memo:** `product-notes/phase-18-opus-low-data-streaming.md`.
---
## Phase 20 — Theater Mode (landed 2026-06-20)
**Landed:** 2026-06-20 on dev. Pending: final manual browser/GPU smoke-test on dev.
@@ -76,6 +153,8 @@ Newest entries at the top. Group by phase/wave header (mirroring `PLAN.md` / `CM
## Phase 18 — Theme / Dark-Mode Remediation (landed 2026-06-19)
_Note: Two distinct efforts share the "Phase 18" label — phase numbers are organisational, not sequential. See also **Phase 18 — Opus Low-Data Streaming (landed 2026-06-23)** above._
**Landed:** 2026-06-19 on dev (Wave 1 + Wave 2 + Wave 3).
- **What:** A DRY token pass resolving six theming symptoms (five in dark mode, one in light) that all traced to three root causes: neutral page surfaces bound to constant brand tokens, the play chip bound to a constant light-grey, and no theme-aware popover-surface token. Resolved as one coherent pass via a shared token layer rather than per-component patches.
+156 -2
View File
@@ -5,6 +5,7 @@ using DeepDrftContent.Constants;
using DeepDrftContent.FileDatabase.Services;
using DeepDrftContent.FileDatabase.Models;
using DeepDrftContent.Processors;
using DeepDrftContent.Processors.Opus;
using DeepDrftData;
using DeepDrftModels.DTOs;
using DeepDrftModels.Enums;
@@ -20,6 +21,7 @@ public class TrackController : ControllerBase
private readonly UnifiedTrackService _unifiedService;
private readonly ITrackService _sqlTrackService;
private readonly WaveformProfileService _waveformProfileService;
private readonly TrackFormatResolver _formatResolver;
private readonly UploadStagingDirectory _stagingDirectory;
private readonly ILogger<TrackController> _logger;
@@ -35,6 +37,7 @@ public class TrackController : ControllerBase
UnifiedTrackService unifiedService,
ITrackService sqlTrackService,
WaveformProfileService waveformProfileService,
TrackFormatResolver formatResolver,
UploadStagingDirectory stagingDirectory,
ILogger<TrackController> logger)
{
@@ -43,6 +46,7 @@ public class TrackController : ControllerBase
_unifiedService = unifiedService;
_sqlTrackService = sqlTrackService;
_waveformProfileService = waveformProfileService;
_formatResolver = formatResolver;
_stagingDirectory = stagingDirectory;
_logger = logger;
}
@@ -245,6 +249,40 @@ public class TrackController : ControllerBase
return Ok(status);
}
// GET api/track/opus-status ([ApiKeyAuthorize])
// Admin Post-Processing view (18.6): returns every track with a flag for whether it carries a COMPLETE
// Opus artifact — both the Opus audio AND the seek/setup sidecar present (TrackFormatResolver.HasOpusAsync,
// the same completeness rule the 18.5 Backfill-Opus pass enqueues against; a half-derived track counts as
// missing). Mirrors GET waveform-status exactly: same ApiKey auth, same unpaged whole-catalogue shape, same
// literal-route placement before "{trackId}". The CMS reads it to show the Backfill-Opus "missing N" badge
// and to poll per-track Post-Processing status after an upload.
[ApiKeyAuthorize]
[HttpGet("opus-status")]
public async Task<ActionResult> GetOpusStatus()
{
var tracks = await _sqlTrackService.GetAll();
if (!tracks.Success || tracks.Value is null)
{
var error = tracks.Messages.FirstOrDefault()?.Message ?? "Unknown error";
_logger.LogError("GetOpusStatus failed to load tracks: {Error}", error);
return StatusCode(500, "Failed to load tracks");
}
var status = new List<OpusStatusDto>(tracks.Value.Count);
foreach (var track in tracks.Value)
{
status.Add(new OpusStatusDto
{
TrackId = track.Id,
EntryKey = track.EntryKey,
TrackName = track.TrackName,
HasOpus = await _formatResolver.HasOpusAsync(track.EntryKey),
});
}
return Ok(status);
}
// POST api/track/duration/backfill ([ApiKeyAuthorize], no body)
// One-time admin backfill: for every track whose SQL duration is still null, read the duration from
// the vault audio and write it to SQL. Mirrors the waveform backfill posture. Idempotent — a re-run
@@ -265,6 +303,27 @@ public class TrackController : ControllerBase
return Ok(new { updated = result.Value.Updated, skipped = result.Value.Skipped });
}
// POST api/track/opus/backfill ([ApiKeyAuthorize], no body)
// Backfill-Opus (18.5, OQ4): enqueue a background Opus derive for every track lacking a complete Opus
// artifact (audio + sidecar). Mirrors the duration-backfill posture — enqueue-only and non-blocking, the
// transcodes run on the shared serial worker. Idempotent: a re-run only schedules tracks still missing
// Opus. Returns { enqueued, skipped }. Declared in the literal-route block (before "{trackId}") so the
// "opus/backfill" segment is never treated as a trackId; distinct shape from "{trackId}/opus" (per-track).
[ApiKeyAuthorize]
[HttpPost("opus/backfill")]
public async Task<ActionResult> BackfillOpus(CancellationToken cancellationToken)
{
var result = await _unifiedService.BackfillOpusAsync(cancellationToken);
if (!result.Success)
{
var error = result.Messages.FirstOrDefault()?.Message ?? "Unknown error";
_logger.LogError("BackfillOpus failed: {Error}", error);
return StatusCode(500, error);
}
return Ok(new { enqueued = result.Value.Enqueued, skipped = result.Value.Skipped });
}
// POST api/track/upload: raw audio in (multipart/form-data) + metadata → persisted TrackDto out.
// Accepts .wav, .mp3, and .flac. Used by the CMS upload flow on DeepDrftManager; that host
// proxies the upload here so it never touches the vault disk path or SQL directly.
@@ -642,10 +701,27 @@ public class TrackController : ControllerBase
// --- Parameterized routes ---
// GET api/track/{trackId}?format=opus|lossless (unauthenticated)
// Streams the track's audio bytes with HTTP Range support. The optional `format` selector (Phase 18.3)
// picks the delivery rendering: absent or unrecognized ⇒ Lossless (byte-identical to pre-Phase-18 —
// the existing zero-copy disk-stream path, untouched); `opus` ⇒ the derived Ogg Opus 320 artifact
// when present, falling back to lossless when it is not (C2 — never 404/silence). The Opus path serves
// the resolved in-memory bytes via File(..., enableRangeProcessing: true) so Range: bytes=X- still
// yields 206 (load-bearing for streaming + seek), matching the lossless disk-stream's range behavior.
[HttpGet("{trackId}")]
public async Task<ActionResult> GetTrack(string trackId)
public async Task<ActionResult> GetTrack(string trackId, [FromQuery] string? format = null)
{
_logger.LogInformation("GetTrack called with trackId: {TrackId}", trackId);
_logger.LogInformation("GetTrack called with trackId: {TrackId}, format: {Format}", trackId, format);
// Only `opus` diverges from today's behavior; everything else (null, "lossless", garbage) takes the
// unchanged lossless disk-stream path below, preserving the large-file zero-copy streaming. Routing
// lossless through the resolver would force the whole source (up to ~1 GB) into memory per request —
// a regression the resolver's in-memory byte[] result is fine for Opus (small) but not for lossless.
if (Enum.TryParse<AudioFormat>(format, ignoreCase: true, out var requestedFormat)
&& requestedFormat == AudioFormat.Opus)
{
return await GetTrackOpusAsync(trackId);
}
try
{
@@ -700,6 +776,58 @@ public class TrackController : ControllerBase
}
}
// The ?format=opus arm of GetTrack. Resolves the Opus artifact (or the lossless fallback when none
// exists, C2) via TrackFormatResolver and serves the resolved bytes with explicit range processing.
// enableRangeProcessing:true is the load-bearing detail the 18.2 reviewer flagged: File(byte[], ...)
// does NOT get ASP.NET's automatic range handling unless asked, so without this flag a Range: bytes=X-
// would silently return the whole body as 200 instead of a 206 slice — breaking seek for the Opus path
// (and Phase 21 windowing). The resolver reports the *actually-served* format via ResolvedAudio, so the
// content-type matches the bytes (audio/ogg on a hit, the source MIME on a fallback) and the eventual
// client decoder dispatches correctly.
private async Task<ActionResult> GetTrackOpusAsync(string trackId)
{
try
{
var resolved = await _formatResolver.ResolveAsync(trackId, AudioFormat.Opus);
if (resolved is null)
{
_logger.LogWarning("Track not found for Opus request: {TrackId}", trackId);
return NotFound();
}
_logger.LogInformation(
"Streaming track {TrackId} as {Format} ({Size} bytes, {ContentType})",
trackId, resolved.ResolvedFormat, resolved.Audio.Buffer.Length, resolved.ContentType);
return File(resolved.Audio.Buffer, resolved.ContentType, enableRangeProcessing: true);
}
catch (Exception ex)
{
_logger.LogError(ex, "Error retrieving track as Opus: {TrackId}", trackId);
return StatusCode(500, "Internal server error");
}
}
// GET api/track/{trackId}/opus/seekdata (unauthenticated)
// Returns the Opus setup-header + granule→byte seek-index sidecar bytes (Phase 18.3). The client
// fetches this once on track load and parses it into OpusSeekData (18.4) before issuing any Opus seek.
// Raw octet-stream — the bytes are the OpusSidecar blob exactly as 18.1 stored them. 404 when no sidecar
// is stored (no Opus artifact yet, or an older derive predating the sidecar); the client then degrades
// to lossless, mirroring the C2 posture of the audio path. Same public auth posture as the audio stream.
// The "opus/seekdata" literal suffix keeps this distinct from the audio and waveform routes.
[HttpGet("{trackId}/opus/seekdata")]
public async Task<ActionResult> GetOpusSeekData(string trackId)
{
var sidecar = await _formatResolver.GetOpusSidecarAsync(trackId);
if (sidecar is null)
{
_logger.LogInformation("No Opus sidecar for track: {TrackId}", trackId);
return NotFound();
}
return File(sidecar, "application/octet-stream");
}
// GET api/track/{trackId}/waveform (unauthenticated)
// Returns the stored waveform loudness profile for a track, base64-encoded. Public listener
// data, same auth posture as GET api/track/{trackId} streaming. 404 when no profile is stored
@@ -816,6 +944,32 @@ public class TrackController : ControllerBase
return Ok();
}
// POST api/track/{trackId}/opus ([ApiKeyAuthorize])
// Per-track Opus (re)derive trigger (18.5): schedule a single track's background transcode. Enqueue-only
// and non-blocking — the transcode runs on the shared serial worker; this returns as soon as it is
// scheduled. Re-runnable: overwrites any prior artifact in place. trackId is the EntryKey. 404 when the
// track id is unknown. The "opus" literal suffix keeps this distinct from the audio/waveform routes and
// from the parameterized PUT "{trackId}". Returns 202 Accepted — the work is queued, not done inline.
[ApiKeyAuthorize]
[HttpPost("{trackId}/opus")]
public async Task<ActionResult> GenerateOpus(string trackId, CancellationToken cancellationToken)
{
var result = await _unifiedService.EnqueueOpusAsync(trackId, cancellationToken);
if (result.Success)
{
return Accepted();
}
var error = result.Messages.FirstOrDefault()?.Message ?? "Unknown error";
if (string.Equals(error, UnifiedTrackService.TrackNotFoundMessage, StringComparison.Ordinal))
{
return NotFound();
}
_logger.LogError("GenerateOpus failed for {TrackId}: {Error}", trackId, error);
return StatusCode(500, error);
}
[ApiKeyAuthorize]
[HttpPut("{trackId}")]
public async Task<ActionResult> PutTrack(string trackId, [FromBody] AudioBinaryDto track)
+8
View File
@@ -4,6 +4,7 @@ using DeepDrftAPI;
using DeepDrftAPI.Middleware;
using DeepDrftAPI.Models;
using DeepDrftAPI.Services;
using DeepDrftAPI.Services.Opus;
using DeepDrftData;
using DeepDrftData.Data;
using DeepDrftData.Repositories;
@@ -66,6 +67,13 @@ builder.Services
.AddScoped<ITrackService>(sp => sp.GetRequiredService<TrackManager>());
builder.Services.AddScoped<UnifiedTrackService>();
// Background Opus transcode (Phase 18.1, OQ6). One singleton is both the enqueue seam
// (IOpusTranscodeQueue, injected into the scoped UnifiedTrackService) and the hosted drain loop
// (IHostedService). It resolves OpusTranscodeService — a singleton — so no scope is captured.
builder.Services.AddSingleton<OpusTranscodeBackgroundService>();
builder.Services.AddSingleton<IOpusTranscodeQueue>(sp => sp.GetRequiredService<OpusTranscodeBackgroundService>());
builder.Services.AddHostedService(sp => sp.GetRequiredService<OpusTranscodeBackgroundService>());
// Phase 16 anonymous telemetry — append-only event logs + incremental play-counter rollup (all SQL).
// EventManager is the IEventService boundary; EventRepository owns the EF writes and the
// release-resolution + counter-bump transaction.
@@ -0,0 +1,18 @@
namespace DeepDrftAPI.Services.Opus;
/// <summary>
/// The enqueue seam for the background Opus transcode (OQ6 / §3.1a). <see cref="UnifiedTrackService"/>
/// depends only on this thin interface — not on the worker — so adding the background derive to the
/// upload/replace paths costs one small dependency, not the whole transcode graph. Enqueuing is
/// non-blocking and best-effort: a freshly uploaded track is already persisted and playable losslessly
/// before anything is enqueued, and the transcode runs off the request thread.
/// </summary>
public interface IOpusTranscodeQueue
{
/// <summary>
/// Schedules a background Opus derive for the track identified by <paramref name="entryKey"/>. Returns
/// immediately. A dropped or failed enqueue must not affect the caller — the track remains
/// lossless-only and eligible for backfill.
/// </summary>
void Enqueue(string entryKey);
}
@@ -0,0 +1,72 @@
using System.Threading.Channels;
using DeepDrftContent.Processors.Opus;
namespace DeepDrftAPI.Services.Opus;
/// <summary>
/// The background worker behind <see cref="IOpusTranscodeQueue"/> (OQ6 / §3.1a). An unbounded in-process
/// channel buffers EntryKeys enqueued by the upload and replace-audio paths; a single hosted loop drains
/// them one at a time and runs <see cref="OpusTranscodeService.TranscodeAndStoreAsync"/> for each. Serial
/// by design — a transcode is CPU-heavy (§3.1), so running them concurrently would starve request
/// handling; one-at-a-time keeps the derive strictly off the hot path without saturating the host.
///
/// This worker IS the queue (implements <see cref="IOpusTranscodeQueue"/>) so enqueue and drain share one
/// channel with no extra indirection. It is registered as a singleton and surfaced under both the
/// interface and <see cref="IHostedService"/>.
/// </summary>
public sealed class OpusTranscodeBackgroundService : BackgroundService, IOpusTranscodeQueue
{
private readonly Channel<string> _channel =
Channel.CreateUnbounded<string>(new UnboundedChannelOptions { SingleReader = true });
private readonly OpusTranscodeService _transcodeService;
private readonly ILogger<OpusTranscodeBackgroundService> _logger;
public OpusTranscodeBackgroundService(
OpusTranscodeService transcodeService,
ILogger<OpusTranscodeBackgroundService> logger)
{
_transcodeService = transcodeService;
_logger = logger;
}
public void Enqueue(string entryKey)
{
if (string.IsNullOrWhiteSpace(entryKey))
return;
if (!_channel.Writer.TryWrite(entryKey))
{
// Unbounded writer only rejects after Complete(), i.e. during shutdown. The track stays
// lossless-only and is eligible for backfill, so a dropped enqueue is non-fatal — log it.
_logger.LogWarning("Opus transcode: could not enqueue {EntryKey} (queue closed).", entryKey);
}
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
await foreach (var entryKey in _channel.Reader.ReadAllAsync(stoppingToken))
{
try
{
await _transcodeService.TranscodeAndStoreAsync(entryKey, stoppingToken);
}
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
{
break; // host shutting down
}
catch (Exception ex)
{
// TranscodeAndStoreAsync already swallows expected failures; this guards the loop against
// anything unexpected so one bad track never kills the worker.
_logger.LogError(ex, "Opus transcode: unhandled failure draining {EntryKey}; worker continues.", entryKey);
}
}
}
public override Task StopAsync(CancellationToken cancellationToken)
{
_channel.Writer.TryComplete();
return base.StopAsync(cancellationToken);
}
}
@@ -1,6 +1,8 @@
using DeepDrftAPI.Services.Opus;
using DeepDrftContent;
using DeepDrftContent.Constants;
using DeepDrftContent.Processors;
using DeepDrftContent.Processors.Opus;
using DeepDrftData;
using DeepDrftModels.DTOs;
using DeepDrftModels.Enums;
@@ -39,6 +41,8 @@ public class UnifiedTrackService
private readonly ITrackService _sqlTrackService;
private readonly FileDb _fileDatabase;
private readonly WaveformProfileService _waveformProfileService;
private readonly IOpusTranscodeQueue _opusTranscodeQueue;
private readonly TrackFormatResolver _formatResolver;
private readonly ILogger<UnifiedTrackService> _logger;
public UnifiedTrackService(
@@ -46,12 +50,16 @@ public class UnifiedTrackService
ITrackService sqlTrackService,
FileDb fileDatabase,
WaveformProfileService waveformProfileService,
IOpusTranscodeQueue opusTranscodeQueue,
TrackFormatResolver formatResolver,
ILogger<UnifiedTrackService> logger)
{
_contentTrackContentService = contentTrackContentService;
_sqlTrackService = sqlTrackService;
_fileDatabase = fileDatabase;
_waveformProfileService = waveformProfileService;
_opusTranscodeQueue = opusTranscodeQueue;
_formatResolver = formatResolver;
_logger = logger;
}
@@ -219,6 +227,11 @@ public class UnifiedTrackService
// frontend, so a failure here is logged and swallowed — never fails the upload.
await TryStoreWaveformDatumsAsync(unpersisted.EntryKey, ct);
// Schedule the low-data Opus derive (OQ6 / §3.1a): the track is persisted and lossless-playable
// NOW; the transcode + seek-index build run on a background worker. Non-blocking and best-effort
// — the upload response never waits on it, and a transcode failure leaves the track lossless-only.
_opusTranscodeQueue.Enqueue(unpersisted.EntryKey);
return saveResult;
}
@@ -297,6 +310,11 @@ public class UnifiedTrackService
return Result.CreateFailResult("Audio replaced but duration metadata could not be updated.");
}
// The stale Opus artifact (if any) no longer matches the new source. Schedule a background
// regenerate — the transcode service overwrites the prior artifacts in place keyed by the same
// EntryKey. Best-effort, off the request thread, mirrors the waveform regen above.
_opusTranscodeQueue.Enqueue(entryKey);
return Result.CreatePassResult();
}
@@ -379,6 +397,69 @@ public class UnifiedTrackService
return ResultContainer<(int, int)>.CreatePassResult((updated, skipped));
}
/// <summary>
/// Backfill-Opus (18.5, OQ4): enqueue a background Opus derive for every non-deleted track that lacks a
/// complete Opus artifact (missing audio OR missing sidecar — a half-derived track is treated as missing
/// and re-derived). Mirrors the duration-backfill posture: enumerate SQL rows, check each against the
/// <c>track-opus</c> vault, schedule the misses. Enqueue-only and non-blocking — the actual transcodes run
/// on the shared background worker, serially (the same queue the upload/replace paths feed), so this
/// returns as soon as the misses are scheduled rather than waiting on CPU-heavy transcodes. Idempotent:
/// a re-run only enqueues tracks still missing Opus, and already-queued/in-flight derives simply overwrite
/// in place. Returns (enqueued, skipped) — skipped = tracks that already have a complete Opus artifact.
/// </summary>
public async Task<ResultContainer<(int Enqueued, int Skipped)>> BackfillOpusAsync(CancellationToken ct)
{
var all = await _sqlTrackService.GetAll();
if (!all.Success || all.Value is null)
{
var error = all.Messages.FirstOrDefault()?.Message ?? "Unknown error";
_logger.LogError("BackfillOpusAsync: failed to load tracks: {Error}", error);
return ResultContainer<(int, int)>.CreateFailResult($"Could not load tracks: {error}");
}
var enqueued = 0;
var skipped = 0;
foreach (var track in all.Value)
{
ct.ThrowIfCancellationRequested();
if (await _formatResolver.HasOpusAsync(track.EntryKey))
{
skipped++;
continue;
}
_opusTranscodeQueue.Enqueue(track.EntryKey);
enqueued++;
}
_logger.LogInformation("BackfillOpusAsync complete: {Enqueued} enqueued, {Skipped} already had Opus.",
enqueued, skipped);
return ResultContainer<(int, int)>.CreatePassResult((enqueued, skipped));
}
/// <summary>
/// Per-track Opus (re)derive trigger (18.5): schedule a background transcode for one track. Returns false
/// only when the track id is unknown; the enqueue itself is non-blocking and best-effort, like the bulk
/// backfill. Re-runnable — overwrites any prior artifact in place.
/// </summary>
public async Task<Result> EnqueueOpusAsync(string entryKey, CancellationToken ct)
{
var lookup = await _sqlTrackService.GetByEntryKey(entryKey);
if (!lookup.Success)
{
var error = lookup.Messages.FirstOrDefault()?.Message ?? "Unknown error";
_logger.LogError("EnqueueOpusAsync: lookup failed for {EntryKey}: {Error}", entryKey, error);
return Result.CreateFailResult("Failed to load track.");
}
if (lookup.Value is null)
return Result.CreateFailResult(TrackNotFoundMessage);
_opusTranscodeQueue.Enqueue(entryKey);
return Result.CreatePassResult();
}
/// <summary>
/// Delete a track's SQL row, then its vault entry. SQL is the source of truth: a SQL delete
/// failure fails the operation (and leaves the vault untouched), but a subsequent vault delete
+27
View File
@@ -4,6 +4,7 @@ using DeepDrftContent.Constants;
using DeepDrftContent.FileDatabase.Models;
using DeepDrftContent.FileDatabase.Services;
using DeepDrftContent.Processors;
using DeepDrftContent.Processors.Opus;
using Microsoft.Extensions.Logging;
using NetBlocks.Utilities.Environment;
@@ -44,6 +45,7 @@ namespace DeepDrftAPI
InitializeTrackVault(db).GetAwaiter().GetResult();
InitializeImageVault(db).GetAwaiter().GetResult();
InitializeTrackWaveformsVault(db).GetAwaiter().GetResult();
InitializeTrackOpusVault(db).GetAwaiter().GetResult();
return db;
});
@@ -65,6 +67,21 @@ namespace DeepDrftAPI
Environment.SetEnvironmentVariable("ASPNETCORE_TEMP", stagingPath);
builder.Services.AddSingleton(new UploadStagingDirectory(stagingPath));
// Opus low-data transcode (Phase 18.1). The domain service lives in DeepDrftContent; the host
// owns only the engine config and the background worker. Bitrate/ffmpeg-path come from the
// OpusTranscode config section; StagingPath is forced to the same data-disk staging directory
// the upload path uses so large transcode temp files never land on the /tmp tmpfs.
builder.Services.Configure<OpusTranscodeOptions>(
builder.Configuration.GetSection(nameof(OpusTranscodeOptions)));
builder.Services.PostConfigure<OpusTranscodeOptions>(o => o.StagingPath = stagingPath);
builder.Services.AddSingleton<FfmpegOpusEncoder>();
builder.Services.AddSingleton<OpusTranscodeService>();
// Opus delivery format resolution + sidecar lookup (Phase 18.2). The seam 18.3 calls behind
// the ?format= stream param and the sidecar path. Stateless over the FileDatabase + content
// service singletons; the lossless branch reuses the existing read path unchanged (C2).
builder.Services.AddSingleton<TrackFormatResolver>();
return Task.CompletedTask;
}
@@ -107,5 +124,15 @@ namespace DeepDrftAPI
await fileDatabase.CreateVaultAsync(VaultConstants.TrackWaveforms, MediaVaultType.Media);
}
}
// Ensure the track-opus vault exists (Phase 18.1). Holds the derived low-data Ogg Opus artifacts
// — the Opus audio bytes and the setup-header + seek-index sidecar — keyed by the track's EntryKey.
private static async Task InitializeTrackOpusVault(FileDatabase fileDatabase)
{
if (!fileDatabase.HasVault(VaultConstants.TrackOpus))
{
await fileDatabase.CreateVaultAsync(VaultConstants.TrackOpus, MediaVaultType.Audio);
}
}
}
}
+1 -3
View File
@@ -1,9 +1,7 @@
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning",
"DeepDrftContent.Controllers.TrackController": "Information"
"Default": "Information"
}
},
"AllowedHosts": "*",
@@ -28,4 +28,13 @@ public static class VaultConstants
/// The datum resolution is duration-derived (≈333 samples/sec, see <c>WaveformResolution</c>).
/// </summary>
public const string TrackWaveforms = "track-waveforms";
/// <summary>
/// Vault name for the derived low-data Ogg Opus artifacts, keyed by the track's EntryKey (Phase 18,
/// S2). Holds two entries per track: the Opus audio bytes (<c>.opus</c>) and the combined setup-header
/// + granule→byte seek-index sidecar (<c>.opusidx</c>). Both are best-effort derived artifacts —
/// regenerable, and a track without them still plays losslessly. Distinct from the source <c>tracks</c>
/// vault so the source means exactly one thing (mirrors the <c>track-waveforms</c> precedent).
/// </summary>
public const string TrackOpus = "track-opus";
}
@@ -206,6 +206,7 @@ public static class MimeTypeExtensions
{ ".flac", "audio/flac" },
{ ".aac", "audio/aac" },
{ ".ogg", "audio/ogg" },
{ ".opus", "audio/ogg" },
{ ".m4a", "audio/mp4" }
};
@@ -0,0 +1,140 @@
using System.Diagnostics;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
namespace DeepDrftContent.Processors.Opus;
/// <summary>
/// Encodes a source audio file (any format the source vault holds — WAV/MP3/FLAC) to Ogg Opus fullband
/// 320 kbps by shelling out to FFmpeg (libopus). FFmpeg is chosen over a managed encoder because it
/// muxes a correct Ogg container with accurate granule positions across every input format — the page
/// structure the seek-index walk depends on — which a raw libopus binding does not provide. The external
/// <c>ffmpeg</c> binary is therefore a host runtime prerequisite (flagged in the wave handoff).
/// </summary>
public sealed class FfmpegOpusEncoder
{
private readonly OpusTranscodeOptions _options;
private readonly ILogger<FfmpegOpusEncoder> _logger;
public FfmpegOpusEncoder(IOptions<OpusTranscodeOptions> options, ILogger<FfmpegOpusEncoder> logger)
{
_options = options.Value;
_logger = logger;
}
/// <summary>
/// Transcodes <paramref name="sourcePath"/> to an Ogg Opus file at <paramref name="destinationPath"/>.
/// Returns true on a clean exit with a non-empty output. Returns false (logged) on a non-zero exit,
/// a timeout, a missing ffmpeg binary, or any process failure — a transcode failure must never throw
/// to the caller (C6); the background worker treats false as "leave the track lossless-only".
/// </summary>
public async Task<bool> EncodeAsync(string sourcePath, string destinationPath, CancellationToken ct)
{
var ffmpeg = string.IsNullOrWhiteSpace(_options.FfmpegPath) ? "ffmpeg" : _options.FfmpegPath;
// -vn drops any cover-art video stream; -map a:0 takes the first audio stream; -ar 48000 forces
// fullband (Opus internally resamples to 48 kHz anyway, but stating it keeps granulepos math
// unambiguous); libopus VBR at the target bitrate; -f ogg for an explicit Ogg container; -y
// overwrites the (pre-created, empty) destination temp file.
var args = new[]
{
"-hide_banner", "-nostdin", "-loglevel", "error",
"-i", sourcePath,
"-vn", "-map", "a:0",
"-c:a", "libopus", "-b:a", $"{_options.BitrateKbps}k",
"-ar", "48000",
"-f", "ogg",
"-y", destinationPath,
};
var psi = new ProcessStartInfo(ffmpeg)
{
RedirectStandardError = true,
RedirectStandardOutput = true,
UseShellExecute = false,
CreateNoWindow = true,
};
foreach (var arg in args)
psi.ArgumentList.Add(arg);
using var process = new Process { StartInfo = psi };
try
{
if (!process.Start())
{
_logger.LogError("Opus transcode: ffmpeg failed to start ({Ffmpeg}).", ffmpeg);
return false;
}
}
catch (Exception ex)
{
// Most commonly a missing binary (Win32Exception "file not found"). This is the ops
// prerequisite failing — log loudly so it is unmistakable in the deploy logs.
_logger.LogError(ex,
"Opus transcode: could not launch ffmpeg ({Ffmpeg}). Is the ffmpeg binary installed on the host?",
ffmpeg);
return false;
}
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(ct);
timeout.CancelAfter(TimeSpan.FromSeconds(_options.TimeoutSeconds));
// Drain stderr concurrently — ffmpeg can block writing diagnostics if the pipe is not read.
var stderrTask = process.StandardError.ReadToEndAsync(timeout.Token);
try
{
await process.WaitForExitAsync(timeout.Token);
}
catch (OperationCanceledException) when (ct.IsCancellationRequested)
{
TryKill(process);
await SafeStderr(stderrTask); // observe to avoid unobserved-task warnings
throw; // genuine shutdown cancellation — let it propagate
}
catch (OperationCanceledException)
{
TryKill(process);
await SafeStderr(stderrTask); // observe to avoid unobserved-task warnings
_logger.LogError("Opus transcode: ffmpeg exceeded the {Timeout}s timeout for {Source}.",
_options.TimeoutSeconds, sourcePath);
return false;
}
var stderr = await SafeStderr(stderrTask);
if (process.ExitCode != 0)
{
_logger.LogError("Opus transcode: ffmpeg exited {Code} for {Source}. stderr: {Stderr}",
process.ExitCode, sourcePath, stderr);
return false;
}
if (!File.Exists(destinationPath) || new FileInfo(destinationPath).Length == 0)
{
_logger.LogError("Opus transcode: ffmpeg exited 0 but produced no output for {Source}.", sourcePath);
return false;
}
return true;
}
private void TryKill(Process process)
{
try
{
if (!process.HasExited)
process.Kill(entireProcessTree: true);
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Opus transcode: failed to kill timed-out ffmpeg process.");
}
}
private static async Task<string> SafeStderr(Task<string> stderrTask)
{
try { return await stderrTask; }
catch { return "<stderr unavailable>"; }
}
}
@@ -0,0 +1,65 @@
namespace DeepDrftContent.Processors.Opus;
/// <summary>
/// Wire-format constants for the Ogg-Opus derived artifacts. Centralised so the seek-index codec,
/// the page walker, and the tests agree on one set of magic numbers.
/// </summary>
public static class OggOpusConstants
{
/// <summary>Opus granule positions are always sample counts at 48 kHz, regardless of input rate.</summary>
public const double OpusSampleRate = 48000.0;
/// <summary>One seek-index entry per this many seconds of audio (OQ7 — 0.5 s buckets).</summary>
public const double SeekBucketSeconds = 0.5;
/// <summary>The Ogg page capture pattern "OggS" — every page starts with these four bytes.</summary>
public static ReadOnlySpan<byte> CapturePattern => "OggS"u8;
/// <summary>Magic signature opening an OpusHead identification header packet.</summary>
public static ReadOnlySpan<byte> OpusHeadSignature => "OpusHead"u8;
/// <summary>Magic signature opening an OpusTags comment header packet.</summary>
public static ReadOnlySpan<byte> OpusTagsSignature => "OpusTags"u8;
/// <summary>
/// Fixed size of an Ogg page header before the segment table: capture(4) + version(1) +
/// header-type(1) + granulepos(8) + serial(4) + sequence(4) + checksum(4) + page-segments(1).
/// </summary>
public const int OggPageHeaderSize = 27;
/// <summary>Byte offset of the 64-bit granule position within an Ogg page header.</summary>
public const int GranulePositionOffset = 6;
/// <summary>Byte offset of the page-segment count (the segment-table length) within the header.</summary>
public const int PageSegmentCountOffset = 26;
/// <summary>Sentinel granule position for a page that ends mid-packet (no usable timestamp).</summary>
public const ulong NoGranulePosition = 0xFFFFFFFFFFFFFFFFUL;
/// <summary>
/// Minimum byte length of an <c>OpusHead</c> packet payload to safely read <c>pre_skip</c>.
/// RFC 7845 §5.1: "OpusHead"(8) + version(1) + channels(1) + pre_skip(2) = 12 bytes minimum.
/// </summary>
public const int OpusHeadMinSize = 12;
/// <summary>
/// Byte offset of <c>pre_skip</c> within the full <c>OpusHead</c> packet payload (including the
/// magic). RFC 7845 §5.1: "OpusHead"(8) + version(1) + channels(1) = 10 bytes before pre_skip.
/// </summary>
public const int OpusHeadPreSkipOffset = 10;
/// <summary>
/// Header size of the serialized seek-index blob:
/// totalBytes(8) + duration(8) + count(4) + preSkip(2) + reserved(2) = 24 bytes.
/// </summary>
public const int SeekIndexHeaderSize = 24;
/// <summary>Size of one serialized seek point: granulepos(8) + byteOffset(8).</summary>
public const int SeekPointSize = 16;
/// <summary>Vault-resource extension for the Opus audio bytes.</summary>
public const string OpusExtension = ".opus";
/// <summary>Vault-resource extension for the combined setup-header + seek-index sidecar.</summary>
public const string SidecarExtension = ".opusidx";
}
@@ -0,0 +1,146 @@
using System.Buffers.Binary;
namespace DeepDrftContent.Processors.Opus;
/// <summary>
/// The result of walking an encoded Ogg Opus stream once: the captured setup header (the leading
/// <c>OpusHead</c> + <c>OpusTags</c> pages, verbatim) and the bucketed granule→byte seek index. This
/// is everything the sidecar artifact carries (§3.4a) — built at transcode time so delivery never
/// re-walks the stream.
/// </summary>
/// <param name="SetupHeaderBytes">The leading setup pages (OpusHead + OpusTags), exactly as they
/// appear at the start of the stream, ready to prepend to any mid-stream page run before decode.</param>
/// <param name="SeekIndex">The accurate, 0.5 s-bucketed granule→byte transfer function.</param>
public sealed record OggOpusWalk(byte[] SetupHeaderBytes, OggOpusSeekIndex SeekIndex);
/// <summary>
/// Pure Ogg-Opus stream walker. Reads the page structure directly (the <c>OggS</c> capture pattern and
/// the 27-byte page header) to (1) capture the setup-header pages and (2) record, for every audio page,
/// its end granule position and exact byte offset — bucketed to 0.5 s with each bucket boundary snapped
/// to the nearest enclosing page start. No external dependency: the encoder (FFmpeg) produces the bytes;
/// this turns them into the seek artifact deterministically, so it is unit-testable without a codec.
/// </summary>
public static class OggOpusParser
{
/// <summary>
/// Walks <paramref name="oggBytes"/> and produces the setup header + seek index, or null if the
/// bytes are not a recognisable Ogg Opus stream (no setup header, no audio pages, or truncated
/// structure). A null is the caller's signal to treat the transcode as failed and leave the track
/// lossless-only (C6) — it does not throw for malformed input.
/// </summary>
public static OggOpusWalk? Walk(ReadOnlySpan<byte> oggBytes)
{
var setupHeaderEnd = -1;
var sawOpusHead = false;
var sawOpusTags = false;
ushort preSkip = 0;
var points = new List<OpusSeekPoint>();
ulong lastGranule = 0;
var nextBucketTime = 0.0;
var firstAudioPointTaken = false;
var offset = 0;
while (offset + OggOpusConstants.OggPageHeaderSize <= oggBytes.Length)
{
var page = oggBytes.Slice(offset);
if (!page[..4].SequenceEqual(OggOpusConstants.CapturePattern))
{
// Not on a page boundary — the encoder writes contiguous pages, so this means the
// stream is malformed or we mis-stepped. Either way it is unrecoverable here.
return null;
}
var segmentCount = page[OggOpusConstants.PageSegmentCountOffset];
var segmentTableEnd = OggOpusConstants.OggPageHeaderSize + segmentCount;
if (segmentTableEnd > page.Length)
return null; // truncated header
var payloadSize = 0;
for (var i = 0; i < segmentCount; i++)
payloadSize += page[OggOpusConstants.OggPageHeaderSize + i];
var pageTotalSize = segmentTableEnd + payloadSize;
if (pageTotalSize > page.Length)
return null; // truncated payload
var payload = page.Slice(segmentTableEnd, payloadSize);
var granule = BinaryPrimitives.ReadUInt64LittleEndian(
page.Slice(OggOpusConstants.GranulePositionOffset, 8));
// The setup pages carry no audio granule (OpusHead has granulepos 0; OpusTags too). They
// are the leading pages whose payload opens with the Opus magic signatures.
if (!sawOpusHead && StartsWith(payload, OggOpusConstants.OpusHeadSignature))
{
sawOpusHead = true;
setupHeaderEnd = offset + pageTotalSize;
// RFC 7845 §5.1 — OpusHead layout after the 8-byte "OpusHead" magic:
// [0] version (1 byte), [1] channel count (1 byte),
// [2-3] pre_skip (little-endian uint16) ← at packet bytes 10-11
// pre_skip is the number of decoder samples to discard before presenting audio;
// all granule→time conversions must subtract it (RFC 7845 §4.3).
if (payload.Length >= OggOpusConstants.OpusHeadMinSize)
preSkip = BinaryPrimitives.ReadUInt16LittleEndian(
payload.Slice(OggOpusConstants.OpusHeadPreSkipOffset, 2));
}
else if (sawOpusHead && !sawOpusTags && StartsWith(payload, OggOpusConstants.OpusTagsSignature))
{
sawOpusTags = true;
setupHeaderEnd = offset + pageTotalSize;
}
else if (sawOpusHead && sawOpusTags)
{
// Audio page. Record the first audio page unconditionally (the seek anchor at t=0),
// then one entry per 0.5 s bucket. A page with no end-granule (mid-packet continuation,
// granulepos == -1) is skipped for indexing — its time is unknown — but still advances
// the byte cursor.
if (granule != OggOpusConstants.NoGranulePosition)
{
// RFC 7845 §4.3: presentation time = max(0, granule preSkip) / 48000.
// Use this corrected time for bucketing so that a stream with pre-skip 3840 (~80 ms)
// does not systematically offset every indexed time by that amount.
var correctedTime = Math.Max(0.0,
(granule - (double)preSkip) / OggOpusConstants.OpusSampleRate);
if (!firstAudioPointTaken)
{
// Anchor the first seek point at corrected time = 0 by storing the granule as
// preSkip. This guarantees that a binary search for t=0 ("largest entry with
// corrected time ≤ 0") always resolves to the first audio page's byte offset —
// even when the real granule is slightly above preSkip due to encoder lead-in.
points.Add(new OpusSeekPoint(preSkip, (ulong)offset));
firstAudioPointTaken = true;
nextBucketTime = OggOpusConstants.SeekBucketSeconds;
}
else if (correctedTime >= nextBucketTime)
{
points.Add(new OpusSeekPoint(granule, (ulong)offset));
// Advance past every bucket this page crossed so a long page does not emit a
// backlog of entries; the next bucket is the first boundary strictly after it.
while (nextBucketTime <= correctedTime)
nextBucketTime += OggOpusConstants.SeekBucketSeconds;
}
lastGranule = granule;
}
}
offset += pageTotalSize;
}
if (!sawOpusHead || setupHeaderEnd < 0 || points.Count == 0)
return null;
var setupHeader = oggBytes[..setupHeaderEnd].ToArray();
// RFC 7845 §4.3: total duration is also pre-skip-corrected, matching the time a listener
// experiences (the last audio page's corrected time, clamped to ≥ 0).
var totalDuration = Math.Max(0.0,
(lastGranule - (double)preSkip) / OggOpusConstants.OpusSampleRate);
var index = new OggOpusSeekIndex(points, totalDuration, (ulong)oggBytes.Length, preSkip);
return new OggOpusWalk(setupHeader, index);
}
private static bool StartsWith(ReadOnlySpan<byte> payload, ReadOnlySpan<byte> signature) =>
payload.Length >= signature.Length && payload[..signature.Length].SequenceEqual(signature);
}
@@ -0,0 +1,124 @@
using System.Buffers.Binary;
namespace DeepDrftContent.Processors.Opus;
/// <summary>
/// A single seek-index entry: an authoritative 48 kHz <see cref="GranulePosition"/> (Opus granule
/// positions are always sample counts at 48 kHz) paired with the exact byte offset of the Ogg page that
/// carries it. Every <see cref="ByteOffset"/> is a real page-start boundary, so a
/// <c>Range: bytes={ByteOffset}-</c> fetch lands the decoder Ogg-sync-aligned.
/// </summary>
/// <remarks>
/// Per RFC 7845 §4.3, the PCM presentation time is <c>(granulepos preSkip) / 48000</c>. The raw
/// <see cref="GranulePosition"/> is stored here as-is; callers should subtract the containing
/// <see cref="OggOpusSeekIndex.PreSkip"/> before converting to a presentation time. Use
/// <see cref="OggOpusSeekIndex.PresentationTimeSeconds"/> for the corrected value.
/// </remarks>
/// <param name="GranulePosition">The page's end granule position (48 kHz sample count).</param>
/// <param name="ByteOffset">The byte offset of the page start in the Opus file.</param>
public readonly record struct OpusSeekPoint(ulong GranulePosition, ulong ByteOffset)
{
/// <summary>
/// Raw granule-position-to-time conversion (granulepos / 48 kHz). Does NOT subtract pre-skip — use
/// <see cref="OggOpusSeekIndex.PresentationTimeSeconds"/> for the RFC 7845-correct presentation time.
/// </summary>
public double RawTimeSeconds => GranulePosition / OggOpusConstants.OpusSampleRate;
}
/// <summary>
/// The accurate, precomputed transfer function from seek-time to true file byte offset for one Ogg
/// Opus stream (§3.4a A). Built once at transcode time by walking the encoded stream; the client reads
/// it back and binary-searches <see cref="Points"/> instead of doing inaccurate VBR byte-rate math.
/// One entry per 0.5 s of audio (<see cref="OggOpusConstants.SeekBucketSeconds"/>), each snapped to the
/// nearest enclosing page start, plus the totals needed to clamp a seek to range.
/// </summary>
/// <param name="Points">Ordered (granulepos, byteOffset) entries, ascending. The first entry always
/// has <see cref="OpusSeekPoint.GranulePosition"/> == <paramref name="PreSkip"/> (corrected time = 0)
/// and points at the first audio page start, ensuring a seek to t=0 always resolves.</param>
/// <param name="TotalDurationSeconds">
/// Pre-skip-corrected total stream duration: <c>max(0, lastGranule preSkip) / 48000</c>.
/// </param>
/// <param name="TotalByteLength">Total Opus file byte length, for clamping a seek past the end.</param>
/// <param name="PreSkip">
/// The <c>pre_skip</c> value from the <c>OpusHead</c> identification header (RFC 7845 §5.1). Opus
/// decoders must discard this many samples from the decoded start before presenting audio. The client
/// (wave 18.4) needs this to trim the first decoded buffer; storing it here avoids a re-parse of the
/// Ogg stream at delivery time.
/// </param>
public sealed record OggOpusSeekIndex(
IReadOnlyList<OpusSeekPoint> Points,
double TotalDurationSeconds,
ulong TotalByteLength,
ushort PreSkip)
{
/// <summary>
/// Returns the RFC 7845-correct presentation time for a seek point: <c>max(0, granule preSkip) / 48000</c>.
/// Use this for all time comparisons; raw <see cref="OpusSeekPoint.RawTimeSeconds"/> omits the pre-skip.
/// </summary>
public double PresentationTimeSeconds(OpusSeekPoint point) =>
Math.Max(0.0, (point.GranulePosition - (double)PreSkip) / OggOpusConstants.OpusSampleRate);
/// <summary>
/// Serializes the index to the compact little-endian binary blob the sidecar stores. Layout:
/// <c>[uint64 totalByteLength][double totalDurationSeconds][uint32 pointCount][uint16 preSkip][uint16 reserved]</c>
/// then <c>pointCount × (uint64 granulepos, uint64 byteOffset)</c>. The four-byte preSkip+reserved
/// region pads the header to 24 bytes, keeping the point table 8-byte-aligned.
/// Fixed-width records keep the client parse to a single typed-array read.
/// </summary>
public byte[] ToBytes()
{
var size = OggOpusConstants.SeekIndexHeaderSize + Points.Count * OggOpusConstants.SeekPointSize;
var bytes = new byte[size];
var span = bytes.AsSpan();
BinaryPrimitives.WriteUInt64LittleEndian(span[..8], TotalByteLength);
BinaryPrimitives.WriteDoubleLittleEndian(span.Slice(8, 8), TotalDurationSeconds);
BinaryPrimitives.WriteUInt32LittleEndian(span.Slice(16, 4), (uint)Points.Count);
BinaryPrimitives.WriteUInt16LittleEndian(span.Slice(20, 2), PreSkip);
// bytes 22-23: reserved (zero-initialized by array allocation)
var cursor = OggOpusConstants.SeekIndexHeaderSize;
foreach (var point in Points)
{
BinaryPrimitives.WriteUInt64LittleEndian(span.Slice(cursor, 8), point.GranulePosition);
BinaryPrimitives.WriteUInt64LittleEndian(span.Slice(cursor + 8, 8), point.ByteOffset);
cursor += OggOpusConstants.SeekPointSize;
}
return bytes;
}
/// <summary>
/// Parses a blob produced by <see cref="ToBytes"/>. Returns null if the blob is too short or its
/// declared point count does not fit — the storage contract is exact, so a malformed blob is a
/// corruption signal, not a recoverable shape. (Provided so tests and any future server-side reader
/// share one codec with the writer.)
/// </summary>
public static OggOpusSeekIndex? FromBytes(ReadOnlySpan<byte> bytes)
{
if (bytes.Length < OggOpusConstants.SeekIndexHeaderSize)
return null;
var totalByteLength = BinaryPrimitives.ReadUInt64LittleEndian(bytes[..8]);
var totalDuration = BinaryPrimitives.ReadDoubleLittleEndian(bytes.Slice(8, 8));
var count = BinaryPrimitives.ReadUInt32LittleEndian(bytes.Slice(16, 4));
var preSkip = BinaryPrimitives.ReadUInt16LittleEndian(bytes.Slice(20, 2));
// bytes 22-23: reserved — ignored on read for forward-compatibility
var expected = OggOpusConstants.SeekIndexHeaderSize + (long)count * OggOpusConstants.SeekPointSize;
if (bytes.Length < expected)
return null;
var points = new OpusSeekPoint[count];
var cursor = OggOpusConstants.SeekIndexHeaderSize;
for (var i = 0; i < count; i++)
{
var granule = BinaryPrimitives.ReadUInt64LittleEndian(bytes.Slice(cursor, 8));
var offset = BinaryPrimitives.ReadUInt64LittleEndian(bytes.Slice(cursor + 8, 8));
points[i] = new OpusSeekPoint(granule, offset);
cursor += OggOpusConstants.SeekPointSize;
}
return new OggOpusSeekIndex(points, totalDuration, totalByteLength, preSkip);
}
}
@@ -0,0 +1,57 @@
using System.Buffers.Binary;
namespace DeepDrftContent.Processors.Opus;
/// <summary>
/// The single derived sidecar artifact per track (§3.4a B, recommended design): the Opus setup header
/// (<c>OpusHead</c> + <c>OpusTags</c>) followed by the granule→byte seek index. The client fetches this
/// once on track load and parses it into its <c>OpusSeekData</c>, so it always has both the setup bytes
/// (to prepend to any mid-stream slice) and the accurate seek transfer function before it ever issues a
/// Range fetch — including a window that opens away from byte 0 (UC9).
/// </summary>
/// <param name="SetupHeaderBytes">The verbatim OpusHead + OpusTags pages.</param>
/// <param name="SeekIndex">The bucketed granule→byte seek index.</param>
public sealed record OpusSidecar(byte[] SetupHeaderBytes, OggOpusSeekIndex SeekIndex)
{
/// <summary>
/// Serializes to <c>[uint32 setupHeaderLength][setup-header bytes][seek-index blob]</c>. The
/// length prefix lets the client split the two regions with one read; the seek-index blob carries
/// its own self-describing header (<see cref="OggOpusSeekIndex.ToBytes"/>), so it needs no trailing
/// length.
/// </summary>
public byte[] ToBytes()
{
var indexBytes = SeekIndex.ToBytes();
var bytes = new byte[4 + SetupHeaderBytes.Length + indexBytes.Length];
var span = bytes.AsSpan();
BinaryPrimitives.WriteUInt32LittleEndian(span[..4], (uint)SetupHeaderBytes.Length);
SetupHeaderBytes.CopyTo(span.Slice(4));
indexBytes.CopyTo(span.Slice(4 + SetupHeaderBytes.Length));
return bytes;
}
/// <summary>
/// Parses a blob produced by <see cref="ToBytes"/>. Returns null on any structural inconsistency
/// (short blob, length prefix that overruns, or an unparseable index) — the format is exact, so a
/// malformed blob is corruption.
/// </summary>
public static OpusSidecar? FromBytes(ReadOnlySpan<byte> bytes)
{
if (bytes.Length < 4)
return null;
var setupLength = BinaryPrimitives.ReadUInt32LittleEndian(bytes[..4]);
var indexStart = 4 + (long)setupLength;
if (bytes.Length < indexStart)
return null;
var setupHeader = bytes.Slice(4, (int)setupLength).ToArray();
var index = OggOpusSeekIndex.FromBytes(bytes.Slice((int)indexStart));
if (index is null)
return null;
return new OpusSidecar(setupHeader, index);
}
}
@@ -0,0 +1,33 @@
namespace DeepDrftContent.Processors.Opus;
/// <summary>
/// Host-supplied configuration for the Opus transcode. The only operationally significant knob is
/// <see cref="FfmpegPath"/> — the transcode shells out to FFmpeg (libopus), which must be present on the
/// DeepDrftAPI host (see the wave handoff notes). Defaults target Ogg Opus fullband (48 kHz) at 320 kbps,
/// the artifact the spec fixes (§1).
/// </summary>
public sealed class OpusTranscodeOptions
{
/// <summary>
/// Path to the ffmpeg executable. Empty/null resolves to <c>"ffmpeg"</c> (found on PATH). Override
/// with an absolute path when the binary is not on the host PATH.
/// </summary>
public string FfmpegPath { get; set; } = "ffmpeg";
/// <summary>Target Opus bitrate in kbps. 320 kbps fullband is the fixed artifact quality (§1).</summary>
public int BitrateKbps { get; set; } = 320;
/// <summary>
/// Directory for the transient source/output files the transcode stages. Defaults to the system
/// temp path; the host overrides it to the data-disk upload-staging directory so large files never
/// land on the small RAM-backed <c>/tmp</c> tmpfs (same constraint the upload path already honours).
/// </summary>
public string StagingPath { get; set; } = Path.GetTempPath();
/// <summary>
/// Hard ceiling on a single transcode, in seconds. A run that exceeds it is killed and the track
/// stays lossless-only (C6). Generous by default — a 1 GB mix is CPU-expensive (§3.1) — but bounded
/// so a hung ffmpeg never wedges the background worker.
/// </summary>
public int TimeoutSeconds { get; set; } = 3600;
}
@@ -0,0 +1,154 @@
using DeepDrftContent.Constants;
using DeepDrftContent.FileDatabase.Models;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using FileDb = DeepDrftContent.FileDatabase.Services.FileDatabase;
namespace DeepDrftContent.Processors.Opus;
/// <summary>
/// Derives and persists a track's low-data Ogg Opus artifacts (Phase 18.1). Mirrors
/// <see cref="WaveformProfileService"/>'s derived-artifact lifecycle: compute from the stored source,
/// store in a dedicated vault keyed by <c>EntryKey</c>, regenerable, failure-tolerant. For one track it
/// produces two entries in the <see cref="VaultConstants.TrackOpus"/> vault — the Opus audio bytes and a
/// combined setup-header + seek-index sidecar (§3.4a). Strictly additive: the source <c>tracks</c> vault
/// is never touched, and a failure here leaves the track lossless-only and eligible for backfill (C2/C6).
/// </summary>
public sealed class OpusTranscodeService
{
private readonly FileDb _fileDatabase;
private readonly FfmpegOpusEncoder _encoder;
private readonly OpusTranscodeOptions _options;
private readonly ILogger<OpusTranscodeService> _logger;
public OpusTranscodeService(
FileDb fileDatabase,
FfmpegOpusEncoder encoder,
IOptions<OpusTranscodeOptions> options,
ILogger<OpusTranscodeService> logger)
{
_fileDatabase = fileDatabase;
_encoder = encoder;
_options = options.Value;
_logger = logger;
}
/// <summary>
/// Reads the source audio for <paramref name="entryKey"/> from the <c>tracks</c> vault, transcodes it
/// to Ogg Opus 320, walks the encoded stream to build the seek index + capture the setup header, and
/// stores the Opus bytes and the sidecar in the <see cref="VaultConstants.TrackOpus"/> vault under the
/// same key. Re-runnable — a second call overwrites the prior artifacts (backfill / replace-audio).
/// Returns false (logged) on any failure; never throws for expected failure modes (C6). The only
/// propagated exception is <see cref="OperationCanceledException"/> on genuine shutdown.
/// </summary>
public async Task<bool> TranscodeAndStoreAsync(string entryKey, CancellationToken ct)
{
var source = await _fileDatabase.LoadResourceAsync<AudioBinary>(VaultConstants.Tracks, entryKey);
if (source is null)
{
_logger.LogWarning("Opus transcode: no source audio in vault for {EntryKey}; skipping.", entryKey);
return false;
}
Directory.CreateDirectory(_options.StagingPath);
var sourcePath = Path.Combine(_options.StagingPath, $"opus-src-{Guid.NewGuid():N}{source.Extension}");
var opusPath = Path.Combine(_options.StagingPath, $"opus-out-{Guid.NewGuid():N}{OggOpusConstants.OpusExtension}");
try
{
await File.WriteAllBytesAsync(sourcePath, source.Buffer, ct);
if (!await _encoder.EncodeAsync(sourcePath, opusPath, ct))
return false; // encoder already logged the cause
var opusBytes = await File.ReadAllBytesAsync(opusPath, ct);
var walk = OggOpusParser.Walk(opusBytes);
if (walk is null)
{
_logger.LogError(
"Opus transcode: ffmpeg produced output but the Ogg stream could not be walked for {EntryKey}; " +
"no artifacts stored.", entryKey);
return false;
}
await EnsureVaultAsync();
var opusBitrate = source.Duration > 0
? (int)(opusBytes.Length * 8 / source.Duration / 1000)
: _options.BitrateKbps;
var audioBinary = new AudioBinary(new AudioBinaryParams(
opusBytes, opusBytes.Length, OggOpusConstants.OpusExtension, source.Duration, opusBitrate));
var sidecar = new OpusSidecar(walk.SetupHeaderBytes, walk.SeekIndex).ToBytes();
var sidecarBinary = new MediaBinary(new MediaBinaryParams(
sidecar, sidecar.Length, OggOpusConstants.SidecarExtension));
// Store the audio first, then the sidecar. If the sidecar write fails the Opus bytes are
// present but unseekable — treat that as a failed derive (return false) so a backfill re-runs
// it; do not leave a half-derived track that the delivery layer would treat as complete.
var audioStored = await _fileDatabase.RegisterResourceAsync(
VaultConstants.TrackOpus, OpusAudioKey(entryKey), audioBinary);
if (!audioStored)
{
_logger.LogError("Opus transcode: vault write of Opus audio failed for {EntryKey}.", entryKey);
return false;
}
var sidecarStored = await _fileDatabase.RegisterResourceAsync(
VaultConstants.TrackOpus, OpusSidecarKey(entryKey), sidecarBinary);
if (!sidecarStored)
{
_logger.LogError("Opus transcode: vault write of sidecar failed for {EntryKey}.", entryKey);
return false;
}
_logger.LogInformation(
"Opus transcode complete for {EntryKey}: {OpusBytes} bytes, {Points} seek points, {Duration:F1}s.",
entryKey, opusBytes.Length, walk.SeekIndex.Points.Count, walk.SeekIndex.TotalDurationSeconds);
return true;
}
catch (OperationCanceledException) when (ct.IsCancellationRequested)
{
throw;
}
catch (Exception ex)
{
_logger.LogError(ex, "Opus transcode failed for {EntryKey}; track stays lossless-only.", entryKey);
return false;
}
finally
{
TryDelete(sourcePath);
TryDelete(opusPath);
}
}
/// <summary>The vault entry key under which a track's Opus audio bytes are stored.</summary>
public static string OpusAudioKey(string entryKey) => entryKey;
/// <summary>The vault entry key under which a track's setup-header + seek-index sidecar is stored.</summary>
public static string OpusSidecarKey(string entryKey) => $"{entryKey}-sidecar";
private async Task EnsureVaultAsync()
{
// The TrackOpus vault is created at host startup (Startup.cs), so this guard is normally a
// no-op for the upload path. It is retained for the backfill path, which may run via a
// standalone CLI or a host that skips vault pre-creation, where the vault might not exist.
if (!_fileDatabase.HasVault(VaultConstants.TrackOpus))
await _fileDatabase.CreateVaultAsync(VaultConstants.TrackOpus, MediaVaultType.Audio);
}
private void TryDelete(string path)
{
try
{
if (File.Exists(path))
File.Delete(path);
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Opus transcode: failed to delete staging file {Path}.", path);
}
}
}
@@ -0,0 +1,23 @@
using DeepDrftContent.FileDatabase.Models;
using DeepDrftModels.Enums;
namespace DeepDrftContent.Processors.Opus;
/// <summary>
/// The outcome of resolving a track + requested <see cref="AudioFormat"/> to a concrete artifact
/// (Phase 18.2). Carries the bytes, the content-type that matches <em>what was actually returned</em>,
/// and the format actually served — which may differ from the requested one when the C2 fallback fires
/// (Opus requested, no Opus artifact → the lossless artifact + its content-type). The delivery layer
/// (18.3) sets the response <c>Content-Type</c> from <see cref="ContentType"/> so the eventual decoder
/// picks the right decoder for the bytes it receives, not the bytes the listener asked for.
/// </summary>
/// <param name="Audio">The resolved audio artifact (never null when a resolution succeeds).</param>
/// <param name="ContentType">The MIME type of <paramref name="Audio"/> (e.g. <c>audio/ogg</c> for Opus,
/// or the source's real MIME for lossless).</param>
/// <param name="ResolvedFormat">The format actually returned. Equal to the requested format on a direct
/// hit; <see cref="AudioFormat.Lossless"/> when an Opus request fell back.</param>
public sealed record ResolvedAudio(AudioBinary Audio, string ContentType, AudioFormat ResolvedFormat)
{
/// <summary>True when an Opus request was served the lossless artifact because no Opus existed (C2).</summary>
public bool DidFallBack(AudioFormat requested) => requested != ResolvedFormat;
}
@@ -0,0 +1,110 @@
using DeepDrftContent.Constants;
using DeepDrftContent.FileDatabase.Models;
using DeepDrftModels.Enums;
using Microsoft.Extensions.Logging;
using FileDb = DeepDrftContent.FileDatabase.Services.FileDatabase;
namespace DeepDrftContent.Processors.Opus;
/// <summary>
/// The server-side format resolution + sidecar lookup seam (Phase 18.2). Given a track's
/// <c>EntryKey</c> and a requested <see cref="AudioFormat"/>, returns the correct audio artifact and the
/// content-type that matches it; given an <c>EntryKey</c>, returns the Opus seek/setup sidecar bytes.
/// Downstream waves call this — 18.3 wires it behind the <c>?format=</c> stream param and serves the
/// sidecar over HTTP; this wave delivers only the seam, not the HTTP surface.
/// <para>
/// Additive and non-breaking (C2): the lossless branch reads the source exactly as the existing stream
/// path does (via <see cref="TrackContentService.GetAudioBinaryAsync"/>), and an Opus request for a track
/// with no Opus artifact falls back to lossless rather than failing. Mirrors the
/// <see cref="WaveformProfileService"/> derived-artifact lookup precedent: read from the dedicated vault,
/// swallow misses to null (FileDatabase convention), let the caller decide.
/// </para>
/// </summary>
public sealed class TrackFormatResolver
{
private readonly FileDb _fileDatabase;
private readonly TrackContentService _trackContentService;
private readonly ILogger<TrackFormatResolver> _logger;
public TrackFormatResolver(
FileDb fileDatabase,
TrackContentService trackContentService,
ILogger<TrackFormatResolver> logger)
{
_fileDatabase = fileDatabase;
_trackContentService = trackContentService;
_logger = logger;
}
/// <summary>
/// Resolves <paramref name="entryKey"/> + <paramref name="requestedFormat"/> to the audio artifact to
/// serve plus its content-type. <see cref="AudioFormat.Lossless"/> resolves the source artifact in the
/// <c>tracks</c> vault with its real MIME (WAV/MP3/FLAC). <see cref="AudioFormat.Opus"/> resolves the
/// derived Opus artifact (<c>audio/ogg</c>) when present, and <strong>falls back to lossless</strong>
/// when it is not (C2). Returns null only when even the lossless source is missing — i.e. the track has
/// no audio at all (an unknown key or a genuinely empty vault), the one case the caller treats as 404.
/// </summary>
public async Task<ResolvedAudio?> ResolveAsync(string entryKey, AudioFormat requestedFormat)
{
if (requestedFormat == AudioFormat.Opus)
{
var opus = await _fileDatabase.LoadResourceAsync<AudioBinary>(
VaultConstants.TrackOpus, OpusTranscodeService.OpusAudioKey(entryKey));
if (opus is not null)
return new ResolvedAudio(opus, MimeTypeExtensions.GetMimeType(opus.Extension), AudioFormat.Opus);
// C2 fallback: no Opus artifact yet (legacy row, not backfilled, or transcode failed). Degrade
// to lossless rather than 404 — Opus is strictly additive; its absence never means "no audio".
_logger.LogInformation(
"Opus requested for {EntryKey} but no Opus artifact exists; falling back to lossless.", entryKey);
}
return await ResolveLosslessAsync(entryKey);
}
/// <summary>
/// Resolves the lossless source artifact and its real MIME — the existing read path, unchanged. Shared
/// by the explicit-lossless branch and the Opus fallback so both produce identical bytes + content-type.
/// </summary>
private async Task<ResolvedAudio?> ResolveLosslessAsync(string entryKey)
{
var source = await _trackContentService.GetAudioBinaryAsync(entryKey);
if (source is null)
return null;
return new ResolvedAudio(source, MimeTypeExtensions.GetMimeType(source.Extension), AudioFormat.Lossless);
}
/// <summary>
/// Returns the Opus setup-header + seek-index sidecar bytes for <paramref name="entryKey"/>, or null
/// when no sidecar is stored (no Opus artifact yet, or an older derive predating the sidecar). 18.3
/// serves these on their own path; 18.4 fetches them once on track load and parses them into the
/// client's <c>OpusSeekData</c>. The bytes are the raw <see cref="OpusSidecar"/> blob
/// (<c>[uint32 setupHeaderLength][setup-header][seek-index]</c>) exactly as 18.1 stored them.
/// </summary>
public async Task<byte[]?> GetOpusSidecarAsync(string entryKey)
{
var sidecar = await _fileDatabase.LoadResourceAsync<MediaBinary>(
VaultConstants.TrackOpus, OpusTranscodeService.OpusSidecarKey(entryKey));
return sidecar?.Buffer;
}
/// <summary>
/// Reports whether <paramref name="entryKey"/> already has a complete Opus derive — both the audio bytes
/// AND the seek/setup sidecar present in the <c>track-opus</c> vault. The Backfill-Opus pass (18.5) uses
/// this to enqueue only tracks that are missing or half-derived (audio without sidecar = unseekable, so
/// treated as incomplete and re-derived). Both halves are required because the transcode stores them in
/// sequence and a sidecar-write failure leaves a track the delivery layer must not treat as Opus-ready.
/// </summary>
public async Task<bool> HasOpusAsync(string entryKey)
{
var audio = await _fileDatabase.LoadResourceAsync<AudioBinary>(
VaultConstants.TrackOpus, OpusTranscodeService.OpusAudioKey(entryKey));
if (audio is null)
return false;
var sidecar = await _fileDatabase.LoadResourceAsync<MediaBinary>(
VaultConstants.TrackOpus, OpusTranscodeService.OpusSidecarKey(entryKey));
return sidecar is not null;
}
}
@@ -518,7 +518,10 @@
{
var row = _tracks[i];
if (row.Status == BatchRowStatus.Done)
// Skip rows already processed in a prior submit attempt. PostProcessing counts as processed:
// the track persisted successfully (only its background Opus derive is still settling), so a
// re-submit after a partial failure must NOT re-upload it and mint a duplicate.
if (row.Status is BatchRowStatus.Done or BatchRowStatus.PostProcessing)
{
_processedCount++;
continue;
@@ -638,7 +641,12 @@
}
}
row.Status = BatchRowStatus.Done;
// §3.1a: a new-track upload persists the track (live + lossless) and the server
// derives Opus in the background, so the row enters the visible Post-Processing
// phase — same as BatchUpload. (The metadata-only update path above stays Done: it
// changes no audio, so it triggers no transcode.) Non-blocking; the Releases view
// polls the durable Opus status to settle it.
row.Status = BatchRowStatus.PostProcessing;
succeeded++;
}
}
@@ -40,4 +40,9 @@ public class BatchRowModel
: 0;
}
public enum BatchRowStatus { Queued, Uploading, Done, Failed }
// Done is the terminal success state (track persisted + playable losslessly). PostProcessing is the
// visible third upload phase (§3.1a): the byte transfer and server persist are finished and the track is
// live, but the server-side background Opus transcode is still running. It is NOT a failure and never
// blocks completion — the form may navigate away while a row sits in PostProcessing; the Releases browse
// view polls the durable Opus status from there.
public enum BatchRowStatus { Queued, Uploading, PostProcessing, Done, Failed }
@@ -126,6 +126,8 @@
{
BatchRowStatus.Uploading => @<MudChip T="string" Size="Size.Small" Color="Color.Info" Variant="Variant.Text">
<MudProgressCircular Indeterminate="true" Size="Size.Small" Class="mr-1" />Uploading</MudChip>,
BatchRowStatus.PostProcessing => @<MudChip T="string" Size="Size.Small" Color="Color.Info" Variant="Variant.Text">
<MudProgressCircular Indeterminate="true" Size="Size.Small" Class="mr-1" />Post-Processing</MudChip>,
BatchRowStatus.Done => @<MudChip T="string" Size="Size.Small" Color="Color.Success" Variant="Variant.Text" Icon="@Icons.Material.Filled.CheckCircle">Done</MudChip>,
BatchRowStatus.Failed => @<MudChip T="string" Size="Size.Small" Color="Color.Error" Variant="Variant.Text" Icon="@Icons.Material.Filled.Error">Failed</MudChip>,
_ => @<MudChip T="string" Size="Size.Small" Color="Color.Default" Variant="Variant.Text">Queued</MudChip>
@@ -69,6 +69,15 @@
Value="@_tracks[0].UploadPercent"
aria-label="Uploading track" />
}
else if (_tracks[0].Status == BatchRowStatus.PostProcessing)
{
@* §3.1a: track is live + plays lossless; the Opus transcode runs in the background.
Indeterminate (no client-side progress for a server-side job) and non-blocking. *@
<MudStack Row="true" AlignItems="AlignItems.Center" Spacing="2">
<MudProgressCircular Indeterminate="true" Size="Size.Small" />
<MudText Typo="Typo.caption">Post-Processing (deriving Opus)…</MudText>
</MudStack>
}
}
</MudStack>
</MudPaper>
@@ -519,7 +528,13 @@
}
}
row.Status = BatchRowStatus.Done;
// §3.1a: the byte transfer + server persist are done and the track is live and plays
// losslessly — the upload is successful HERE. The server then derives Opus on a
// background worker, so the row enters the visible Post-Processing phase rather than
// jumping straight to Done. This never blocks: the loop continues and the form may
// navigate away while rows sit in Post-Processing; the Releases view polls the durable
// Opus status to settle each one.
row.Status = BatchRowStatus.PostProcessing;
succeeded++;
}
}
@@ -9,6 +9,7 @@
@inject ISnackbar Snackbar
@inject ILogger<Releases> Logger
@inject NavigationManager NavigationManager
@implements IDisposable
@attribute [Authorize]
<PageTitle>Releases — Deep DRFT Management</PageTitle>
@@ -51,6 +52,28 @@
<span>Backfill High-res (@MissingHighResCount)</span>
}
</MudButton>
@* Backfill-Opus (Phase 18.5 + 18.6 badge). The Opus derive runs on a server-side background
worker: pressing the button enqueues every track lacking a complete Opus artifact and reports
the (enqueued / skipped) outcome. The "missing N" badge (18.6) reads the same opus-status map
the page polls, giving visual parity with the two waveform backfill buttons — but unlike them,
the count is informational, not a per-track client loop (the work is scheduled, not driven from
here). The button stays pressable at N=0 (a no-op re-run is harmless); it only disables while a
press is in flight or another bulk run holds the page. *@
<MudButton Variant="Variant.Outlined"
Color="Color.Primary"
StartIcon="@Icons.Material.Filled.GraphicEq"
Disabled="@(_bulkRunning || _highResBulkRunning || _opusBackfillRunning)"
OnClick="BackfillOpusAsync">
@if (_opusBackfillRunning)
{
<MudProgressCircular Class="mr-2" Size="Size.Small" Indeterminate="true" />
<span>Scheduling…</span>
}
else
{
<span>Backfill Opus (@MissingOpusCount)</span>
}
</MudButton>
</MudStack>
</MudStack>
@@ -140,6 +163,22 @@
private int MissingProfileCount => _waveformStatus.Count(s => !s.HasProfile);
private int MissingHighResCount => _waveformStatus.Count(s => !s.HasHighRes);
// EntryKey → HasOpus (a complete audio+sidecar derive). Loaded alongside the waveform status on init and
// re-read after a Backfill-Opus run so the "missing N" badge settles. Also the source the Post-Processing
// poll watches: a freshly uploaded track lands here with HasOpus=false and flips to true once the
// server-side background transcode finishes — the durable surface for the upload meter's Post-Processing
// phase after the form returns the admin to this view (§3.1a).
private IReadOnlyList<OpusStatusDto> _opusStatus = Array.Empty<OpusStatusDto>();
private int MissingOpusCount => _opusStatus.Count(s => !s.HasOpus);
// Post-Processing poll: while any track is still missing Opus (a transcode in flight or not yet
// backfilled), re-read the opus-status map on an interval so the "missing N" badge and any per-track
// Post-Processing indicator settle without a manual refresh. Stops itself once nothing is missing, and is
// torn down on dispose. Non-blocking — it never gates an upload or a button; it only refreshes a count.
private const int OpusPollIntervalMs = 4000;
private CancellationTokenSource? _opusPollCts;
// Local state for the parent-owned "Generate All Profiles" bulk run.
private bool _bulkRunning;
private int _bulkTotal;
@@ -150,6 +189,11 @@
private int _highResBulkTotal;
private int _highResBulkDone;
// Local state for the "Backfill Opus" action. The Opus derive is server-side and background-queued, so
// there is no client-side per-track loop or progress total — this flag only guards the button while the
// single scheduling call is in flight.
private bool _opusBackfillRunning;
protected override async Task OnInitializedAsync()
{
// Seed the active tab from ?medium= so a catalogue card deep-links straight to its medium. Panel 0
@@ -170,9 +214,81 @@
_waveformStatus = result.Success && result.Value is not null
? result.Value
: Array.Empty<WaveformStatusDto>();
await RefreshOpusStatusAsync();
StateHasChanged();
}
/// <summary>
/// Re-read the per-track Opus derive status and (re)arm the Post-Processing poll when work is still
/// pending. Called on init, after a Backfill-Opus run, and from the poll itself. Best-effort: a failed
/// fetch leaves the previous map in place rather than zeroing the badge on a transient API blip. Does not
/// call StateHasChanged itself — callers batch it with their own render (the poll path renders explicitly).
/// </summary>
private async Task RefreshOpusStatusAsync()
{
var opusResult = await CmsTrackService.GetOpusStatusAsync();
if (opusResult.Success && opusResult.Value is not null)
{
_opusStatus = opusResult.Value;
}
if (MissingOpusCount > 0)
{
EnsureOpusPollRunning();
}
}
// Start the Post-Processing poll if it is not already running. The loop re-reads opus-status every
// OpusPollIntervalMs and renders; it exits as soon as nothing is missing (or on cancel/dispose). Guarded
// by a non-null CTS so overlapping callers (init + backfill) cannot start two loops.
private void EnsureOpusPollRunning()
{
if (_opusPollCts is not null)
{
return;
}
_opusPollCts = new CancellationTokenSource();
_ = PollOpusStatusAsync(_opusPollCts.Token);
}
private async Task PollOpusStatusAsync(CancellationToken ct)
{
try
{
while (!ct.IsCancellationRequested && MissingOpusCount > 0)
{
await Task.Delay(OpusPollIntervalMs, ct);
var result = await CmsTrackService.GetOpusStatusAsync();
if (result.Success && result.Value is not null)
{
_opusStatus = result.Value;
await InvokeAsync(StateHasChanged);
}
}
}
catch (OperationCanceledException)
{
// Expected on dispose / navigation away — nothing to do.
}
catch (Exception ex)
{
Logger.LogWarning(ex, "Opus Post-Processing poll stopped on an unexpected error.");
}
finally
{
_opusPollCts?.Dispose();
_opusPollCts = null;
}
}
public void Dispose()
{
_opusPollCts?.Cancel();
}
// Invalidates the cached per-track waveform status on all embedded grids so the next row expand
// re-fetches fresh data. Called after each catalogue-wide bulk run so already-expanded rows
// reflect the new waveform state on the next expand interaction.
@@ -291,4 +407,54 @@
Snackbar.Add($"Backfilled {succeeded} high-res datum(s); {failures} failed.", Severity.Warning);
}
}
/// <summary>
/// Kick off the catalogue-wide Backfill-Opus pass. The API enumerates the tracks lacking a complete Opus
/// artifact, enqueues a background derive for each, and returns the (enqueued, skipped) counts. This is a
/// single scheduling call — the transcodes run server-side afterward — so there is no per-track progress
/// to render here, just a busy flag and a result snackbar. Re-runnable: a second press only schedules
/// tracks still missing Opus.
/// </summary>
private async Task BackfillOpusAsync()
{
_opusBackfillRunning = true;
StateHasChanged();
try
{
var result = await CmsTrackService.BackfillOpusAsync();
if (!result.Success)
{
var error = result.Messages.FirstOrDefault()?.Message ?? "Failed to start the Opus backfill.";
Snackbar.Add(error, Severity.Error);
return;
}
// Re-read the status map so the "missing N" badge reflects the just-enqueued work and the
// Post-Processing poll arms to watch the transcodes settle from N→0 as each finishes.
await RefreshOpusStatusAsync();
var (enqueued, skipped) = (result.Value.Enqueued, result.Value.Skipped);
if (enqueued == 0)
{
Snackbar.Add($"All {skipped} track(s) already have Opus — nothing to backfill.", Severity.Info);
}
else
{
Snackbar.Add(
$"Scheduled {enqueued} Opus transcode(s) in the background ({skipped} already had Opus). " +
"They will appear as each finishes.",
Severity.Success);
}
}
catch (Exception ex)
{
Logger.LogError(ex, "Opus backfill failed to start");
Snackbar.Add("Failed to start the Opus backfill.", Severity.Error);
}
finally
{
_opusBackfillRunning = false;
StateHasChanged();
}
}
}
@@ -765,6 +765,89 @@ public class CmsTrackService : ICmsTrackService
}
}
public async Task<ResultContainer<OpusBackfillResult>> BackfillOpusAsync(CancellationToken ct = default)
{
var client = _httpClientFactory.CreateClient(ContentCmsClientName);
HttpResponseMessage response;
try
{
response = await client.PostAsync("api/track/opus/backfill", null, ct);
}
catch (Exception ex)
{
_logger.LogError(ex, "Content API call failed for Opus backfill");
return ResultContainer<OpusBackfillResult>.CreateFailResult("Content API is unreachable.");
}
using (response)
{
if (!response.IsSuccessStatusCode)
{
var body = await response.Content.ReadAsStringAsync(ct);
_logger.LogError("Content API Opus backfill failed: {Status} {Body}", (int)response.StatusCode, body);
return ResultContainer<OpusBackfillResult>.CreateFailResult("Failed to start the Opus backfill.");
}
OpusBackfillResult payload;
try
{
payload = await response.Content.ReadFromJsonAsync<OpusBackfillResult>(ct);
}
catch (Exception ex)
{
_logger.LogError(ex, "Failed to deserialize Opus backfill response from Content API");
return ResultContainer<OpusBackfillResult>.CreateFailResult("Content API returned an unexpected response.");
}
return ResultContainer<OpusBackfillResult>.CreatePassResult(payload);
}
}
public async Task<ResultContainer<OpusStatusDto[]>> GetOpusStatusAsync(CancellationToken ct = default)
{
var client = _httpClientFactory.CreateClient(ContentCmsClientName);
HttpResponseMessage response;
try
{
response = await client.GetAsync("api/track/opus-status", ct);
}
catch (Exception ex)
{
_logger.LogError(ex, "Content API call failed for Opus status");
return ResultContainer<OpusStatusDto[]>.CreateFailResult("Content API is unreachable.");
}
using (response)
{
if (!response.IsSuccessStatusCode)
{
_logger.LogError("Content API Opus status failed: {Status}", (int)response.StatusCode);
return ResultContainer<OpusStatusDto[]>.CreateFailResult("Failed to load Opus status.");
}
OpusStatusDto[]? status;
try
{
status = await response.Content.ReadFromJsonAsync<OpusStatusDto[]>(ct);
}
catch (Exception ex)
{
_logger.LogError(ex, "Failed to deserialize Opus status from Content API response");
return ResultContainer<OpusStatusDto[]>.CreateFailResult("Content API returned an unexpected response.");
}
if (status is null)
{
_logger.LogError("Content API returned a null Opus status list");
return ResultContainer<OpusStatusDto[]>.CreateFailResult("Content API returned an empty response.");
}
return ResultContainer<OpusStatusDto[]>.CreatePassResult(status);
}
}
public async Task<ResultContainer<List<ReleaseDto>>> GetReleasesAsync(CancellationToken ct = default)
{
var client = _httpClientFactory.CreateClient(ContentCmsClientName);
@@ -152,6 +152,23 @@ public interface ICmsTrackService
/// </summary>
Task<Result> GenerateHighResWaveformAsync(string entryKey, CancellationToken ct = default);
/// <summary>
/// Trigger the catalogue-wide Backfill-Opus pass via <c>POST api/track/opus/backfill</c> (Phase 18.5).
/// The API enqueues a background Opus derive for every track lacking a complete Opus artifact and returns
/// the (enqueued, skipped) counts. Enqueue-only — the transcodes run server-side on a serial background
/// worker, so this call returns as soon as the work is scheduled, not when transcoding finishes. The
/// <c>Enqueued</c> count is how many derives were scheduled; <c>Skipped</c> is how many already had Opus.
/// </summary>
Task<ResultContainer<OpusBackfillResult>> BackfillOpusAsync(CancellationToken ct = default);
/// <summary>
/// Fetch per-track Opus derive status from <c>GET api/track/opus-status</c> (Phase 18.6) for the CMS
/// Post-Processing surfaces. Unpaged — the admin catalogue is small. Each row's <c>HasOpus</c> is true only
/// when the track carries a complete Opus artifact (audio + sidecar). Drives the Backfill-Opus "missing N"
/// badge and the post-upload Post-Processing poll. Idempotent read — safe to poll on an interval.
/// </summary>
Task<ResultContainer<OpusStatusDto[]>> GetOpusStatusAsync(CancellationToken ct = default);
/// <summary>Returns all releases with track counts from GET api/track/albums.</summary>
Task<ResultContainer<List<ReleaseDto>>> GetReleasesAsync(CancellationToken ct = default);
@@ -160,3 +177,11 @@ public interface ICmsTrackService
/// </summary>
Task<ResultContainer<int>> GetTrackCountAsync(CancellationToken ct = default);
}
/// <summary>
/// Outcome of a Backfill-Opus pass (Phase 18.5): how many tracks had a background derive scheduled
/// (<paramref name="Enqueued"/>) and how many were skipped because they already carry a complete Opus
/// artifact (<paramref name="Skipped"/>). Both are counts of tracks, not finished transcodes — the work
/// runs asynchronously on the API's background worker after this returns.
/// </summary>
public readonly record struct OpusBackfillResult(int Enqueued, int Skipped);
+1 -2
View File
@@ -1,8 +1,7 @@
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning"
"Default": "Warning"
}
},
"AllowedHosts": "*",
+19
View File
@@ -0,0 +1,19 @@
namespace DeepDrftModels.DTOs;
/// <summary>
/// Per-track Opus derive status for the CMS Post-Processing surfaces (Phase 18.6). Mirrors
/// <see cref="WaveformStatusDto"/>: one row per track, flagging whether the track already carries a
/// <strong>complete</strong> Opus artifact. "Complete" means BOTH the Opus audio bytes AND the seek/setup
/// sidecar are present in the <c>track-opus</c> vault — a half-derived track (audio without sidecar) is
/// unseekable and counts as missing, so the Backfill-Opus pass re-derives it. <see cref="EntryKey"/> is the
/// vault key the per-track enqueue trigger and the polling Post-Processing affordance key on.
/// </summary>
public class OpusStatusDto
{
public long TrackId { get; set; }
public string EntryKey { get; set; } = string.Empty;
public string TrackName { get; set; } = string.Empty;
/// <summary>True only when both the Opus audio and the seek/setup sidecar are stored (a complete derive).</summary>
public bool HasOpus { get; set; }
}
+24
View File
@@ -0,0 +1,24 @@
namespace DeepDrftModels.Enums;
/// <summary>
/// The delivery format a listener requests for a track's audio (Phase 18). One <c>TrackEntity</c> /
/// <c>EntryKey</c> addresses both renderings — "one source, multiple views" applied to delivery (C5).
/// Lives here, not in the content library, because it is a cross-boundary contract: the API stream
/// endpoint (18.3) parses it off the <c>?format=</c> query param, the WASM client (18.4 / 18.6) selects
/// it, and the content-side resolver (18.2) resolves it to bytes — all three reference one enum.
/// </summary>
public enum AudioFormat
{
/// <summary>
/// The existing source artifact in the <c>tracks</c> vault, served byte-for-byte with its real MIME
/// (WAV/MP3/FLAC — do not assume WAV). The universal, always-present rendering.
/// </summary>
Lossless,
/// <summary>
/// The derived low-data Ogg Opus 320 artifact in the <c>track-opus</c> vault (<c>audio/ogg</c>). A
/// best-effort derived artifact: when absent (not yet transcoded, or transcode failed) a request for
/// it falls back to <see cref="Lossless"/> rather than 404ing (C2).
/// </summary>
Opus
}
+1 -1
View File
@@ -51,7 +51,7 @@ All interactive UI for the site. Blazor WebAssembly. Pages, controls, the stream
- `StreamingAudioPlayerService`: Production implementation. Chunked stream from `TrackMediaClient`, adaptive 1664 KB buffer, early-playback, **seek-beyond-buffer** via offset request to the content API.
- `AudioInteropService`: JS interop wrapper over `window.DeepDrftAudio`. Manages `DotNetObjectReference` lifetimes for progress, end-of-playback, spectrum callbacks.
- Dark-mode services: `DarkModeServiceBase` (cookie name constant), `DarkModeCookieService` (JS cookie read/write).
- `WaveformVisualizerControlState`: Scoped session-persistent holder for the visualizer's **eight** continuous control positions, **two subsystem on/off toggles** (Phase 15), and one **Theater-Mode flag** (Phase 20): `ScrollSpeed`, `GradientRotationSpeed`, `LavaGravity`, `LavaHeat`, `FluidAmount` (wax count/volume), `FluidViscosity` (cohesion — the second half of the Phase 10 "bubbles" split; `BlobDensity` is gone), `CollisionStrength`, `WaveformWidth`, `LavaEnabled` (bool, default `true`), `WaveformEnabled` (bool, default `true`), `TheaterMode` (bool, default `false``DefaultTheaterMode`). Each has a matching `Default*` const. `Changed` event is the decoupling seam — controls mutate state + raise `Changed`; the bridge (`WaveformVisualizer`) subscribes and pushes the affected uniform or subsystem-enable; the Theater observers (the three detail pages and `AudioPlayerBar`) subscribe to react to `TheaterMode`. **`CoerceTheaterMode()`**: enforces the invariant that Theater Mode cannot remain on when both subsystems are off — called from `WaveformVisualizerControls.ToggleLava`/`ToggleWaveform` **before** `NotifyChanged()` so all observers see a consistent, coerced state in the same `Changed` cycle. `TheaterMode` is a page-chrome presentation flag; the visualizer bridge ignores it. Scoped DI so state survives SPA nav within a session and resets on fresh page load. **Phase 20 Wave 2 — playing-release predicates** live in `ReleaseDetailBase` / `CutDetailBase` (not in this state holder): `IsThisReleasePlaying` (`PlayerService?.CurrentTrack?.Release?.EntryKey == EntryKey`), `IsContentHidden` (`TheaterMode && IsThisReleasePlaying`), `ShowTheaterToggle` (`(LavaEnabled || WaveformEnabled) && IsThisReleasePlaying`). Both base classes also subscribe to `IStreamingPlayerService.StateChanged` (idempotent, reference-guarded, disposed) so the predicates re-evaluate live when playback moves between releases.
- `WaveformVisualizerControlState`: Scoped session-persistent holder for the visualizer's **eight** continuous control positions, **two subsystem on/off toggles** (Phase 15), and one **Theater-Mode flag** (Phase 20): `ScrollSpeed`, `GradientRotationSpeed`, `LavaGravity`, `LavaHeat`, `FluidAmount` (wax count/volume), `FluidViscosity` (cohesion — the second half of the Phase 10 "bubbles" split; `BlobDensity` is gone), `CollisionStrength`, `WaveformWidth`, `LavaEnabled` (bool, default `true`), `WaveformEnabled` (bool, default `true`), `TheaterMode` (bool, default `false``DefaultTheaterMode`). Each has a matching `Default*` const. `Changed` event is the decoupling seam — controls mutate state + raise `Changed`; the bridge (`WaveformVisualizer`) subscribes and pushes the affected uniform or subsystem-enable; the Theater observers (the three detail pages and `AudioPlayerBar`) subscribe to react to `TheaterMode`. **`CoerceTheaterMode()`**: enforces the invariant that Theater Mode cannot remain on when both subsystems are off — called from `WaveformVisualizerControls.ToggleLava`/`ToggleWaveform` **before** `NotifyChanged()` so all observers see a consistent, coerced state in the same `Changed` cycle. **`ApplyCapabilityDefault(bool hardwareAccelerated)`**: one-time scoped capability default (guarded by `_capabilityDefaultApplied`; never re-applies on SPA navigation, never overrides an explicit in-session toggle). When `hardwareAccelerated` is false (positive software-renderer match from `hwAccel.ts`'s `UNMASKED_RENDERER_WEBGL` probe, or total WebGL failure) sets `LavaEnabled = false` while leaving `WaveformEnabled` at its default on, then calls `CoerceTheaterMode()` + `NotifyChanged()` once so all observers see the default in a single cycle. Called by the visualizer bridge on first interactive render once JS interop (the HW-accel probe via `detectHardwareAcceleration()` exported from `WaveformVisualizer.ts`) is available; a no-op when HW accel is present. `TheaterMode` is a page-chrome presentation flag; the visualizer bridge ignores it. Scoped DI so state survives SPA nav within a session and resets on fresh page load. **Phase 20 Wave 2 — playing-release predicates** live in `ReleaseDetailBase` / `CutDetailBase` (not in this state holder): `IsThisReleasePlaying` (`PlayerService?.CurrentTrack?.Release?.EntryKey == EntryKey`), `IsContentHidden` (`TheaterMode && IsThisReleasePlaying`), `ShowTheaterToggle` (`(LavaEnabled || WaveformEnabled) && IsThisReleasePlaying`). Both base classes also subscribe to `IStreamingPlayerService.StateChanged` (idempotent, reference-guarded, disposed) so the predicates re-evaluate live when playback moves between releases.
- `PlayTracker`: Per-session play-session tracker (Phase 16 wave 16.1). Opens on playback start, advances a high-water position on each progress tick (from `StreamingAudioPlayerService` — not the HTTP layer, so seek-beyond-buffer re-fetches are the same play), closes on track-switch / stop / organic-end / page-unload. Engagement floor: ≥3 s OR ≥5% of duration. Three-bucket classification (`partial`/`sampled`/`complete`). Emits at most one event per session via `IPlayEventSink`. No player or JS dependency — testable against a fake sink.
- `ShareTracker`: Per-session share tracker (Phase 16 wave 16.1). Called by `SharePopover` after a successful clipboard write; applies a 60-second per-(target, channel) debounce. Sends via `BeaconInterop`. Scoped so debounce memory resets on fresh page load. **Wave 16.3:** injects `IAnonIdProvider`; attaches `_anonId.Current` to `ShareEventDto.AnonId` (omitted when null).
- `BeaconInterop`: `navigator.sendBeacon` JS interop wrapper (Phase 16 wave 16.1). Fires JSON payloads to `api/event/{play,share}` fire-and-forget. Also wires a page-unload handler that flushes any pending play event when the page is torn down.
@@ -2,6 +2,8 @@ using System.Net;
using System.Net.Http.Headers;
using System.Net.Http.Json;
using DeepDrftModels.DTOs;
using DeepDrftModels.Enums;
using Microsoft.AspNetCore.Components.WebAssembly.Http;
using Microsoft.Extensions.DependencyInjection;
using NetBlocks.Models;
@@ -18,13 +20,24 @@ public class TrackMediaResponse : IDisposable
/// </summary>
public string ContentType { get; }
/// <summary>
/// The total file length in bytes, parsed from the 206 response's <c>Content-Range:
/// bytes start-end/TOTAL</c> header (Phase 21 Direction B). Null when the server returned
/// 200 (no Content-Range) — callers fall back to <see cref="ContentLength"/> as the total.
/// This is the EOF boundary the segment loop advances its cursor toward, and the full
/// logical length the JS decoder must see (so a bounded segment's small Content-Length
/// never trips the decoder's byte-count completion early).
/// </summary>
public long? TotalLength { get; }
private readonly HttpResponseMessage _response;
public TrackMediaResponse(Stream stream, long contentLength, string contentType, HttpResponseMessage response)
public TrackMediaResponse(Stream stream, long contentLength, string contentType, long? totalLength, HttpResponseMessage response)
{
Stream = stream;
ContentLength = contentLength;
ContentType = contentType;
TotalLength = totalLength;
_response = response;
}
@@ -45,24 +58,61 @@ public class TrackMediaClient
}
/// <summary>
/// Fetches the WAV stream for a track via an HTTP Range request starting at a
/// Fetches the audio stream for a track via an HTTP Range request starting at a
/// file-absolute byte offset. <paramref name="byteOffset"/> is the position from
/// the start of the file on disk (including the WAV header) — callers seeking into
/// audio data must add the header size themselves. The cancellation token aborts
/// the in-flight server connection rather than leaving the server draining bytes
/// into a dead socket.
/// the start of the file on disk (including any container/header bytes) — callers
/// seeking into audio data must add the header size themselves. The cancellation
/// token aborts the in-flight server connection rather than leaving the server
/// draining bytes into a dead socket.
/// <para>
/// <paramref name="byteEnd"/> (Phase 21 Direction B) bounds the request to a single
/// segment: when set, the Range header is <c>bytes={byteOffset}-{byteEnd}</c> (inclusive),
/// so the browser holds at most ~one segment of raw bytes regardless of file size — the
/// network-memory bound this phase exists for. When null the request is open-ended
/// (<c>bytes={byteOffset}-</c>), the pre-Direction-B behaviour. Either way the response's
/// <c>Content-Range</c> total is surfaced via <see cref="TrackMediaResponse.TotalLength"/>
/// so the caller knows the EOF boundary and the full logical length the decoder must see.
/// </para>
/// <para>
/// <paramref name="format"/> selects the delivery rendering (Phase 18): the default
/// <see cref="AudioFormat.Lossless"/> sends no <c>format</c> query param, so existing
/// callers hit the byte-identical pre-Phase-18 endpoint; <see cref="AudioFormat.Opus"/>
/// requests the low-data Ogg Opus artifact, which the server resolves and falls back to
/// lossless when absent (C2). The response <see cref="TrackMediaResponse.ContentType"/>
/// reports the format actually served, so the JS decoder dispatches on the real bytes.
/// </para>
/// </summary>
public async Task<ApiResult<TrackMediaResponse>> GetTrackMedia(
string trackId,
long byteOffset = 0,
long? byteEnd = null,
AudioFormat format = AudioFormat.Lossless,
CancellationToken cancellationToken = default)
{
try
{
// Same URL for every seek — only the Range header differs. byteOffset 0 is
// Same URL for every fetch — only the Range header differs. byteOffset 0 is
// not special-cased: "bytes=0-" requests the whole file from the start.
using var request = new HttpRequestMessage(HttpMethod.Get, $"api/track/{trackId}");
request.Headers.Range = new RangeHeaderValue(byteOffset, null);
// Lossless omits the format param entirely so the request is byte-identical to
// the pre-Phase-18 endpoint; only Opus appends ?format=opus.
var uri = format == AudioFormat.Lossless
? $"api/track/{trackId}"
: $"api/track/{trackId}?format={format.ToString().ToLowerInvariant()}";
using var request = new HttpRequestMessage(HttpMethod.Get, uri);
// Bounded (byteEnd set) → "bytes=start-end" so the server returns a finite 206
// slice and the browser buffers only that segment; open-ended (byteEnd null) →
// "bytes=start-". The server honours both via File(..., enableRangeProcessing: true),
// which parses the full RFC 7233 range grammar and slices accordingly.
request.Headers.Range = new RangeHeaderValue(byteOffset, byteEnd);
// Stream the response body incrementally instead of buffering it whole (Phase 21.4 fix).
// In Blazor WebAssembly the HttpClient is backed by the browser fetch API; without this the
// browser buffers the ENTIRE body before the response stream yields a byte. With Direction B
// each request is already bounded to one segment, so the body is small regardless — but
// streaming still lets us read it incrementally and is harmless on the SSR server-to-server
// path (SocketsHttpHandler ignores the unknown option). Kept for both the initial and the
// seek/refill paths since both share this method.
request.SetBrowserResponseStreamingEnabled(true);
// Use HttpCompletionOption.ResponseHeadersRead to get stream immediately
var response = await _http.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, cancellationToken);
@@ -72,11 +122,15 @@ public class TrackMediaClient
// Default to WAV when the server omits the header — the only format shipping
// today — so the JS factory always receives a usable media type.
var contentType = response.Content.Headers.ContentType?.MediaType ?? "audio/wav";
// Content-Range "bytes start-end/TOTAL" carries the full file length on a 206; on a 200
// there is no Content-Range, so TotalLength is null and callers use ContentLength.
var totalLength = response.Content.Headers.ContentRange?.Length;
var stream = await response.Content.ReadAsStreamAsync(cancellationToken);
// TrackMediaResponse takes ownership of both stream and response;
// do NOT dispose response here — the caller disposes via TrackMediaResponse.Dispose().
return ApiResult<TrackMediaResponse>.CreatePassResult(new TrackMediaResponse(stream, contentLength, contentType, response));
return ApiResult<TrackMediaResponse>.CreatePassResult(
new TrackMediaResponse(stream, contentLength, contentType, totalLength, response));
}
catch (Exception e)
{
@@ -115,4 +169,33 @@ public class TrackMediaClient
return ApiResult<WaveformProfileDto>.CreateFailResult(e.Message);
}
}
/// <summary>
/// Fetches a track's Opus seek/setup sidecar — the combined OpusHead/OpusTags setup header plus the
/// granule→byte seek index (Phase 18). The caller (18.5 player wiring) fetches this once on track load
/// and parses it into the JS-side OpusSeekData before issuing any Opus seek. A 404 means no Opus
/// artifact / sidecar exists for the track (legacy row, not backfilled, or transcode failed); callers
/// treat that as "this track has no Opus seek data — stay on lossless" rather than an error, so it
/// surfaces as a fail result with a stable message rather than throwing (mirrors GetWaveformProfileAsync).
/// </summary>
public async Task<ApiResult<byte[]>> GetOpusSidecarAsync(string trackId, CancellationToken cancellationToken = default)
{
try
{
var response = await _http.GetAsync($"api/track/{trackId}/opus/seekdata", cancellationToken);
if (response.StatusCode == HttpStatusCode.NotFound)
{
return ApiResult<byte[]>.CreateFailResult("No Opus sidecar available");
}
response.EnsureSuccessStatusCode();
var bytes = await response.Content.ReadAsByteArrayAsync(cancellationToken);
return ApiResult<byte[]>.CreatePassResult(bytes);
}
catch (Exception e)
{
return ApiResult<byte[]>.CreateFailResult(e.Message);
}
}
}
@@ -0,0 +1,29 @@
using Microsoft.AspNetCore.Components;
namespace DeepDrftPublic.Client.Common;
/// <summary>
/// The single public-site listener-settings object (Phase 18 wave 18.6, §4a). The generalized analogue of
/// <see cref="DarkModeSettings"/>: one scoped holder for every remembered listener preference, seeded at
/// server prerender, carried into WASM via <see cref="PersistentState"/>, and persisted to a cookie on
/// change. Today it carries one preference — streaming quality; tomorrow dark mode (and whatever follows)
/// folds in here as another property without disturbing the menu that reads it.
/// <para>
/// Built design-for-adaptability per §4a: a new preference is a new <c>[PersistentState]</c> property here
/// plus a new <see cref="Components.SettingsItem"/> in the menu — not a rewire. Dark mode is intentionally
/// <em>not</em> migrated in now (it keeps its own <see cref="DarkModeSettings"/> seam); this object is shaped
/// so that consolidation is later a merge of two identical seams, not a reconciliation of two different ones.
/// </para>
/// </summary>
public class PublicSiteSettings
{
/// <summary>
/// The listener's streaming-quality preference. Defaults to <see cref="StreamQuality.LowData"/> (Opus,
/// capability-gated — OQ2). Seeded from the <c>streamQuality</c> cookie at prerender; persisted on change
/// by the client cookie service. The player reads this to decide which <c>?format=</c> to request, but
/// the capability gate and C2 fallback still apply on top, so a <see cref="StreamQuality.LowData"/>
/// preference never forces an unplayable stream.
/// </summary>
[PersistentState]
public StreamQuality StreamQuality { get; set; } = StreamQuality.LowData;
}
@@ -0,0 +1,12 @@
using Microsoft.AspNetCore.Components;
namespace DeepDrftPublic.Client.Common;
/// <summary>
/// One entry in the public-site Settings menu (Phase 18 wave 18.6, §4a). The settings-item abstraction the
/// menu renders instead of a hard-coded control list: a <see cref="Label"/> plus a <see cref="Control"/>
/// fragment bound to a persisted preference. Adding a future tenant (e.g. dark mode) is appending one of
/// these — not rewiring the menu. The control fragment owns its own binding to <see cref="PublicSiteSettings"/>
/// and its own persistence call, so each item is self-contained and the menu stays preference-agnostic.
/// </summary>
public sealed record SettingsItem(string Label, RenderFragment Control);
@@ -0,0 +1,18 @@
namespace DeepDrftPublic.Client.Common;
/// <summary>
/// The listener's streaming-quality preference (Phase 18 wave 18.6, §4). This is the user's <em>intent</em>,
/// not the wire format that ultimately gets served: <see cref="LowData"/> means "give me Opus if you can,"
/// but the player still capability-gates and C2-falls-back to lossless when Opus can't play (a browser that
/// can't decode Ogg Opus, or a track with no Opus artifact). It is therefore deliberately distinct from
/// <c>DeepDrftModels.Enums.AudioFormat</c> (the delivery rendering resolved per request): one is the
/// remembered preference, the other is what a given stream request actually asks for.
/// </summary>
public enum StreamQuality
{
/// <summary>Bandwidth-friendly Opus (capability-gated; the default before any choice — OQ2).</summary>
LowData,
/// <summary>The lossless WAV path, always playable everywhere.</summary>
Lossless
}
@@ -1,3 +1,4 @@
using DeepDrftPublic.Client.Common;
using DeepDrftPublic.Client.Services;
using DeepDrftPublic.Client.Clients;
using Microsoft.AspNetCore.Components;
@@ -13,6 +14,7 @@ public partial class AudioPlayerProvider : ComponentBase, IAsyncDisposable
[Inject] public required BeaconInterop Beacon { get; set; }
[Inject] public required IPlayEventSink PlayEventSink { get; set; }
[Inject] public required IAnonIdProvider AnonId { get; set; }
[Inject] public required PublicSiteSettings Settings { get; set; }
private IStreamingPlayerService? _audioPlayerService;
private QueueService? _queueService;
@@ -26,7 +28,12 @@ public partial class AudioPlayerProvider : ComponentBase, IAsyncDisposable
// EnsureInitializedAsync — that path is correct because audio contexts
// require a user gesture anyway. Initializing eagerly here causes 4+
// SignalR round-trips before any content is stable.
var player = new StreamingAudioPlayerService(AudioInterop, TrackMediaClient, Logger);
// Construct the preference-aware player (Phase 18 wave 18.6): it honours the listener's streaming-
// quality choice via the ResolveStreamFormatAsync seam while inheriting the 18.5 capability gate and
// C2 fallback. PublicSiteSettings is scoped data (already prerender-seeded + WASM-bridged), so passing
// it through the constructor is cheap and carries no lifecycle — the telemetry tracker still binds
// post-construction below, exactly as before.
var player = new PreferenceAwareStreamingPlayerService(AudioInterop, TrackMediaClient, Logger, Settings);
// Phase 16: bind the play-session tracker to the player after construction, the same way the
// queue binds — the player is built with `new`, not DI, so threading telemetry through its
@@ -0,0 +1,55 @@
@using DeepDrftPublic.Client.Common
@using DeepDrftPublic.Client.Controls.Settings
@using DeepDrftPublic.Client.Services
@*
The public-site Settings menu (Phase 18 wave 18.6, §4a). An app-bar trigger opening a MudMenu that renders
a settings-item list — NOT a hard-coded control stack. Each entry is a SettingsItem (label + a control
fragment bound to a persisted preference), so a future tenant (dark mode) plugs in as a new list entry, not
a menu rewire. Today the list holds one item: the streaming-quality toggle.
The MudMenu items carry OnClick="@(() => {})" + OnTouch so a click inside a control row does not dismiss the
menu (MudMenu auto-closes on item activation otherwise), keeping the radio group usable.
*@
<div class="dd-accent-icon">
<MudMenu Icon="@Icons.Material.Filled.Settings"
Color="Color.Inherit"
AnchorOrigin="Origin.BottomRight"
TransformOrigin="Origin.TopRight"
AriaLabel="Settings"
Class="dd-settings-menu">
<div class="dd-settings-panel">
<div class="dd-settings-heading">Settings</div>
@foreach (var item in _items)
{
<div class="dd-settings-item">
<div class="dd-settings-item-label">@item.Label</div>
@item.Control
</div>
}
</div>
</MudMenu>
</div>
@code {
// The active player, cascaded by AudioPlayerProvider. SettingsMenu sits in the app bar inside the
// provider, so it receives the cascade here — but the MudMenu PANEL content below is portaled to
// <MudPopoverProvider> (outside the provider), so a cascade cannot reach it. We thread the player into
// the setting as an explicit parameter instead: an explicit value captured in the item fragment flows
// into portaled content fine, where a [CascadingParameter] would land null.
[CascadingParameter] public IStreamingPlayerService? Player { get; set; }
// The settings-item list. Built once; adding a preference is appending one SettingsItem with its control
// fragment — the menu body above renders whatever is here without knowing what each item is. The item
// fragment reads Player at render time (when the menu opens), so it picks up the cascaded instance even
// though the list itself is initialized before the cascade is set.
private readonly List<SettingsItem> _items;
public SettingsMenu()
{
_items =
[
new SettingsItem("Streaming quality", @<StreamQualitySetting Player="@Player" />)
];
}
}
@@ -0,0 +1,97 @@
@using DeepDrftPublic.Client.Common
@using DeepDrftPublic.Client.Services
@*
The streaming-quality control (Phase 18 wave 18.6, §4) — the first occupant of the Settings menu. Binds
the listener's choice to PublicSiteSettings.StreamQuality and persists it via the cookie seam. Honest
capability gate (OQ2 / AC7): on a browser that cannot decode Ogg Opus the Low-data option still selects,
but a note tells the listener the effective stream is lossless — we never let the choice silently imply a
format that can't play.
*@
<div class="dd-setting-control">
<MudRadioGroup T="StreamQuality" Value="_quality" ValueChanged="OnQualityChanged">
<MudRadio T="StreamQuality" Value="StreamQuality.LowData" Color="Color.Primary" Dense="true">
Low-data (Opus)
</MudRadio>
<MudRadio T="StreamQuality" Value="StreamQuality.Lossless" Color="Color.Primary" Dense="true">
Lossless (WAV)
</MudRadio>
</MudRadioGroup>
@if (_opusUnavailable && _quality == StreamQuality.LowData)
{
<div class="dd-setting-note">
This browser can't decode Opus — you'll stream lossless.
</div>
}
<div class="d-flex flex-align-right">
<MudButton Disabled="!IsApplyEnabled"
Color="Color.Primary"
Variant="Variant.Filled"
OnClick="@ApplyStreamQualitySetting">
APPLY
</MudButton>
</div>
</div>
@code {
[Inject] public required PublicSiteSettings Settings { get; set; }
[Inject] public required SettingsCookieService CookieService { get; set; }
[Inject] public required AudioInteropService AudioInterop { get; set; }
// The active player, threaded in from SettingsMenu (which reads it off the AudioPlayerProvider cascade).
// It is an explicit [Parameter], NOT a [CascadingParameter], because this control renders inside the
// MudMenu panel, which MudBlazor portals to <MudPopoverProvider> — outside AudioPlayerProvider's cascade
// scope, so a cascade would land null here. Null during prerender or when no provider is present — Apply
// then just persists the preference, with no live track to restart.
[Parameter] public IStreamingPlayerService? Player { get; set; }
private StreamQuality _quality;
public bool IsApplyEnabled => _quality != Settings.StreamQuality;
// Null until the capability probe runs (post-render JS interop). false → can decode Opus; true → cannot.
private bool _opusUnavailable;
protected override void OnInitialized()
{
// Read the current preference (already seeded at prerender + bridged into WASM).
_quality = Settings.StreamQuality;
}
protected override async Task OnAfterRenderAsync(bool firstRender)
{
if (!firstRender) return;
// Capability probe is JS interop — only valid once interactive. Surfaces the honest note when the
// browser can't decode Ogg Opus, so a Low-data pick reads as "effectively lossless" rather than
// silently failing. The player applies the same gate independently; this is purely the UI honesty.
var canDecodeOpus = await AudioInterop.CanDecodeOggOpus();
if (canDecodeOpus == _opusUnavailable)
{
_opusUnavailable = !canDecodeOpus;
StateHasChanged();
}
}
private async Task OnQualityChanged(StreamQuality quality)
{
_quality = quality;
}
private async Task ApplyStreamQualitySetting(MouseEventArgs arg)
{
// Persist the choice first so the cookie + in-memory PublicSiteSettings.StreamQuality both reflect
// the new value BEFORE the restart: the reload re-resolves the delivery format via
// PreferenceAwareStreamingPlayerService, which reads PublicSiteSettings.StreamQuality fresh.
await CookieService.SetStreamQualityAsync(_quality);
// Switch the currently-playing track to the new format immediately, preserving the listener's
// position. No-op inside the player when nothing is loaded — the new preference then simply
// applies to the next track played. Fire-and-forget: the reload runs the streaming loop for the
// life of the track, so awaiting it would pin this handler open; UI updates flow through the
// player's state notifications, not this await.
_ = Player?.ReloadPreservingPositionAsync();
}
}
@@ -265,6 +265,24 @@ public partial class WaveformVisualizer : ComponentBase, IAsyncDisposable
return;
}
// Apply the hardware-capability default ONCE per session before seeding the controls: if the
// browser has no WebGL hardware acceleration, the lava subsystem (which software-renders on
// the main thread and starves audio decode) defaults off while the waveform stays on. The
// probe lives in JS (it needs a real WebGL context); the scoped state guards the one-time
// application, so a remounted visualizer never re-applies and a later explicit toggle is
// never clobbered. Sequenced before PushControlsAsync so the seed already carries the
// corrected enables; ApplyCapabilityDefault also raises Changed for the controls UI.
try
{
var hardwareAccelerated = await _module.InvokeAsync<bool>("detectHardwareAcceleration");
ControlState.ApplyCapabilityDefault(hardwareAccelerated);
}
catch (JSException ex)
{
// A probe failure must not regress the HW-accel majority: leave the defaults (lava on).
Logger.LogWarning(ex, "WaveformVisualizer: hardware-acceleration probe failed; leaving lava at its default.");
}
// Seed the module with the current state now that it exists. All control values (the eight
// dials + the two Phase 15 subsystem enables) come from the shared (session-persisted) state,
// so a mix opened mid-session seeds the module with the knob/toggle positions the listener
@@ -1,5 +1,6 @@
@using DeepDrftPublic.Client.Common
@using DeepDrftPublic.Client.Controls
@using DeepDrftPublic.Client.Controls.Settings
@using DeepDrftPublic.Client.Services
@* Desktop Menu *@
@@ -42,6 +43,7 @@
<div class="dd-nav-actions">
<StreamNowButton ButtonClass="dd-nav-cta" ButtonLabel="Stream Now &#9654;"/>
<SettingsMenu />
<MudIconButton Icon="@(DarkLightModeButtonIcon)" Color="Color.Inherit" OnClick="@DarkModeToggle"/>
</div>
</nav>
@@ -74,7 +76,8 @@
@onclick="ToggleMobileMenu">
<span></span><span></span><span></span>
</button>
<SettingsMenu />
<MudIconButton Icon="@(DarkLightModeButtonIcon)" Color="Color.Inherit" OnClick="@DarkModeToggle"/>
</div>
+14 -2
View File
@@ -42,6 +42,7 @@
@code {
private string _audioPlayerClass = "minimized";
private const string DarkModeKey = "darkMode";
private const string StreamQualityKey = "streamQuality";
private bool _isDarkMode = false;
private bool? _lastAppliedDarkMode = null;
private PersistingComponentStateSubscription _persistingSubscription;
@@ -49,6 +50,7 @@
[Inject] public required PersistentComponentState PersistentState { get; set; }
[Inject] public required DarkModeSettings DarkModeSettings { get; set; }
[Inject] public required PublicSiteSettings PublicSiteSettings { get; set; }
[Inject] public required IJSRuntime JS { get; set; }
protected override void OnInitialized()
@@ -66,8 +68,17 @@
_isDarkMode = DarkModeSettings.IsDarkMode;
}
// Restore the prerender-seeded streaming-quality preference (Phase 18 wave 18.6). Same bridge dark
// mode uses: the server SettingsService seeded PublicSiteSettings from the streamQuality cookie, and
// this carries it into WASM so the client boots already knowing the preference (no re-read flash, no
// wrong default before the first stream).
if (PersistentState.TryTakeFromJson<StreamQuality>(StreamQualityKey, out var restoredQuality))
{
PublicSiteSettings.StreamQuality = restoredQuality;
}
// Register to persist state when prerendering completes
_persistingSubscription = PersistentState.RegisterOnPersisting(PersistDarkMode);
_persistingSubscription = PersistentState.RegisterOnPersisting(PersistState);
}
// Sync dark mode class on <body> so portaled MudBlazor elements (popovers, menus, selects)
@@ -91,9 +102,10 @@
// Theme wrapper class for CSS targeting
private string ThemeWrapperClass => _isDarkMode ? "deepdrft-theme-dark" : "deepdrft-theme-light";
private Task PersistDarkMode()
private Task PersistState()
{
PersistentState.PersistAsJson(DarkModeKey, _isDarkMode);
PersistentState.PersistAsJson(StreamQualityKey, PublicSiteSettings.StreamQuality);
return Task.CompletedTask;
}
@@ -70,6 +70,37 @@ public class AudioInteropService : IAsyncDisposable
return await InvokeJsAsync<AudioOperationResult>("DeepDrftAudio.initializeStreaming", playerId, totalStreamLength, contentType);
}
/// <summary>
/// Probes whether this browser can stream-decode Ogg Opus via WebCodecs (<c>AudioDecoder</c> +
/// <c>codec:'opus'</c>; Safari &lt; 16.4 / older Firefox cannot). Phase 18 capability gate (OQ2): the
/// player only requests Opus when this returns true, otherwise it stays on the universal lossless path
/// (AC7 — no listener ever gets silence over a codec gap). Probe failures degrade to <c>false</c>
/// (assume incapable) so an interop error can never silence playback.
/// </summary>
public async Task<bool> CanDecodeOggOpus()
{
try
{
return await _jsRuntime.InvokeAsync<bool>("DeepDrftAudio.canDecodeOggOpus");
}
catch
{
return false;
}
}
/// <summary>
/// Hands the raw Opus seek/setup sidecar bytes (setup header + granule→byte seek index) to the JS player
/// so the next Opus stream's decoder has them BEFORE init (the 18.4 set-before-init contract). The player
/// parses and stashes them; <see cref="InitializeStreaming"/> applies them when it builds the Opus decoder.
/// Must be called before <see cref="InitializeStreaming"/> on an Opus stream. Returns the parse result —
/// a failure means the bytes were not a valid sidecar, and the caller falls back to lossless.
/// </summary>
public async Task<AudioOperationResult> SetOpusSidecar(string playerId, byte[] sidecarBytes)
{
return await InvokeJsAsync<AudioOperationResult>("DeepDrftAudio.setOpusSidecar", playerId, sidecarBytes);
}
public async Task<StreamingResult> ProcessStreamingChunk(string playerId, byte[] audioChunk)
{
return await InvokeJsAsync<StreamingResult>("DeepDrftAudio.processStreamingChunk", playerId, audioChunk);
@@ -115,6 +146,18 @@ public class AudioInteropService : IAsyncDisposable
return await InvokeJsAsync<SeekResult>("DeepDrftAudio.seek", playerId, position);
}
/// <summary>
/// Resolve the file-absolute byte offset to begin a stream at <paramref name="position"/> with no
/// active playback or buffered audio — the "load at timestamp" seam (Phase 18 wave 18.6 format switch).
/// Returns <see cref="SeekResult.ByteOffset"/> on success; <see cref="AudioOperationResult.Success"/> is
/// false when the decoder cannot yet resolve an offset (e.g. a WAV stream whose header has not been
/// parsed), so the caller can feed header bytes and retry.
/// </summary>
public async Task<SeekResult> ResolveStreamOffsetAsync(string playerId, double position)
{
return await InvokeJsAsync<SeekResult>("DeepDrftAudio.resolveStreamOffset", playerId, position);
}
// New methods for seek-beyond-buffer support
public async Task<double> GetBufferedDuration(string playerId)
{
@@ -128,11 +171,42 @@ public class AudioInteropService : IAsyncDisposable
}
}
/// <summary>
/// Phase 21.2a back-pressure poll: ask whether the scheduler is still over its forward
/// high/low-water band. The read loop calls this only WHILE already throttled, to learn when it
/// may resume reading — the steady-state loop reads the piggybacked <c>ProductionPaused</c> flag
/// off each chunk result instead. Defaults to false on any interop failure so a torn-down player
/// never wedges a loop that is exiting anyway.
/// </summary>
public async Task<bool> IsProductionPaused(string playerId)
{
try
{
return await _jsRuntime.InvokeAsync<bool>("DeepDrftAudio.isProductionPaused", playerId);
}
catch
{
return false;
}
}
public async Task<AudioOperationResult> ReinitializeFromOffset(string playerId, long totalStreamLength, double seekPosition)
{
return await InvokeJsAsync<AudioOperationResult>("DeepDrftAudio.reinitializeFromOffset", playerId, totalStreamLength, seekPosition);
}
/// <summary>
/// Phase 21.3 / AC6 clean-failure recovery: after a window-miss refill (seek-back past the
/// retained tail) fails its Range fetch or reinit, halt the starved scheduler and leave the
/// player paused-but-loaded at <paramref name="seekPosition"/> so no silent false end fires and a
/// retry is possible. Routes through <see cref="InvokeJsAsync{T}"/> so an interop failure during
/// recovery still yields a failure result rather than throwing into the seek path.
/// </summary>
public async Task<AudioOperationResult> RecoverFromFailedRefill(string playerId, double seekPosition)
{
return await InvokeJsAsync<AudioOperationResult>("DeepDrftAudio.recoverFromFailedRefill", playerId, seekPosition);
}
public async Task<AudioOperationResult> SetVolumeAsync(string playerId, double volume)
{
return await InvokeJsAsync<AudioOperationResult>("DeepDrftAudio.setVolume", playerId, volume);
@@ -388,6 +462,11 @@ public class StreamingResult : AudioOperationResult
public bool HeaderParsed { get; set; }
public int BufferCount { get; set; }
public double? Duration { get; set; } // Duration in seconds calculated from WAV header
// Phase 21.2a back-pressure: true when the scheduler's forward decoded fill is over the
// high-water mark and the C# read loop should stop calling ReadAsync until it drains. Read off
// the chunk result the loop already awaits — no extra interop hop in the unthrottled steady state.
public bool ProductionPaused { get; set; }
}
public class AudioPlayerState
@@ -118,6 +118,16 @@ public abstract class AudioPlayerService : IPlayerService, IAsyncDisposable
IsPlaying = true;
IsPaused = false;
}
else if (IsPaused)
{
// Play failed while the player is paused — the scheduler may be empty after a
// failed refill (AC6 recovery). Re-issue a seek at the current position: the
// seek path routes to seekBeyondBuffer when the scheduler is empty (Phase 21.3
// fix), triggering a real refetch rather than returning "Streaming not ready".
// We return early here; Seek owns its own state mutations and NotifyStateChanged.
await Seek(CurrentTime);
return;
}
}
if (!result.Success)
@@ -128,7 +138,7 @@ public abstract class AudioPlayerService : IPlayerService, IAsyncDisposable
{
ErrorMessage = null;
}
await NotifyStateChanged();
}
catch (Exception ex)
@@ -298,7 +308,6 @@ public abstract class AudioPlayerService : IPlayerService, IAsyncDisposable
/// </summary>
protected virtual void OnPlaybackEnded() { }
protected async Task EnsureInitializedAsync()
{
if (!IsInitialized)
@@ -14,9 +14,7 @@ public class DarkModeCookieService(DarkModeSettings darkModeSetting, IJSRuntime
public async ValueTask SetDarkModeAsync(bool isDarkMode)
{
var expires = DateTime.UtcNow.AddDays(EXPIRY_DAYS).ToString("R");
await js.InvokeVoidAsync("eval",
$"document.cookie = '{COOKIE_NAME}={isDarkMode.ToString().ToLower()}; expires={expires}; path=/; SameSite=Lax'");
await js.InvokeVoidAsync("DeepDrftSettings.setCookie", COOKIE_NAME, isDarkMode.ToString().ToLower(), EXPIRY_DAYS);
darkModeSetting.IsDarkMode = isDarkMode;
}
}
@@ -91,4 +91,16 @@ public interface IStreamingPlayerService : IPlayerService
/// <see cref="IPlayerService.CurrentTrack"/> and notifies; performs no JS interop.
/// </summary>
Task StageTrack(TrackDto track);
/// <summary>
/// Re-streams the current track in the freshly-resolved delivery format while preserving the
/// listener's playback position (Phase 18 wave 18.6 — the Settings "Apply" restart). The format is
/// re-resolved on the new load via the <c>ResolveStreamFormatAsync</c> seam, so this picks up a just-
/// changed streaming-quality preference. A cross-format byte offset can only be resolved once the NEW
/// format's decoder has parsed its header, so the reload runs a fresh load from byte 0 to initialize
/// that decoder, then seeks back to the saved position through the existing seek-beyond-buffer path.
/// No-op when no track is loaded (nothing playing to switch). Safe to call while a track is playing;
/// a track switch during the brief restore window abandons the position restore.
/// </summary>
Task ReloadPreservingPositionAsync();
}
@@ -0,0 +1,49 @@
using DeepDrftModels.Enums;
using DeepDrftPublic.Client.Clients;
using DeepDrftPublic.Client.Common;
using Microsoft.Extensions.Logging;
namespace DeepDrftPublic.Client.Services;
/// <summary>
/// The production player that honours the listener's streaming-quality preference (Phase 18 wave 18.6).
/// Extends <see cref="StreamingAudioPlayerService"/> through the single deliberately-overridable seam,
/// <see cref="StreamingAudioPlayerService.ResolveStreamFormatAsync"/>, so the rest of the streaming stack
/// (seek, telemetry, the seek-beyond-buffer format reuse) is inherited verbatim.
/// <para>
/// The override is one branch: a <see cref="StreamQuality.Lossless"/> preference returns
/// <see cref="AudioFormat.Lossless"/> immediately; anything else falls through to <c>base</c>, which keeps
/// the 18.5 invariants intact — the capability gate (AC7: a browser that can't decode Ogg Opus gets lossless)
/// and the sidecar-absent → lossless fallback (C2: a legacy / un-backfilled / failed-transcode track gets
/// lossless). So a Lossless pick always yields lossless; a Low-data pick yields Opus only when it can
/// actually play, and lossless otherwise. No path produces an unplayable stream.
/// </para>
/// </summary>
public class PreferenceAwareStreamingPlayerService : StreamingAudioPlayerService
{
private readonly PublicSiteSettings _settings;
public PreferenceAwareStreamingPlayerService(
AudioInteropService audioInterop,
TrackMediaClient trackMediaClient,
ILogger<StreamingAudioPlayerService> logger,
PublicSiteSettings settings)
: base(audioInterop, trackMediaClient, logger)
{
_settings = settings;
}
protected override async Task<AudioFormat> ResolveStreamFormatAsync(string entryKey, CancellationToken cancellationToken)
{
// Listener explicitly chose lossless — request it directly, no Opus probe / sidecar fetch needed.
if (_settings.StreamQuality == StreamQuality.Lossless)
{
return AudioFormat.Lossless;
}
// Low-data preference: defer to the base capability-gated resolution, which probes Opus support and
// the sidecar's presence and degrades to lossless when either is missing. Both 18.5 invariants are
// inherited here, not re-implemented.
return await base.ResolveStreamFormatAsync(entryKey, cancellationToken);
}
}
@@ -0,0 +1,32 @@
using DeepDrftPublic.Client.Common;
using Microsoft.JSInterop;
namespace DeepDrftPublic.Client.Services;
/// <summary>
/// Client-side runtime writer for public-site settings (Phase 18 wave 18.6), the analogue of
/// <see cref="DarkModeCookieService"/>. Reads the current preference off the in-memory
/// <see cref="PublicSiteSettings"/> (already seeded at prerender and bridged into WASM), and writes a
/// 365-day cookie via <c>document.cookie</c> interop when the listener changes it in the Settings menu —
/// the same durable-truth seam dark mode uses, so the choice survives the session and seeds the next visit's
/// prerender (no flash).
/// </summary>
public class SettingsCookieService(PublicSiteSettings settings, IJSRuntime js) : SettingsServiceBase
{
private const int ExpiryDays = 365;
public StreamQuality GetStreamQuality() => settings.StreamQuality;
public async ValueTask SetStreamQualityAsync(StreamQuality quality)
{
if (settings.StreamQuality == quality) return;
await WriteCookieAsync(StreamQualityCookieName, FormatStreamQuality(quality));
settings.StreamQuality = quality;
}
private async ValueTask WriteCookieAsync(string name, string value)
{
await js.InvokeVoidAsync("DeepDrftSettings.setCookie", name, value, ExpiryDays);
}
}
@@ -0,0 +1,28 @@
using DeepDrftPublic.Client.Common;
namespace DeepDrftPublic.Client.Services;
/// <summary>
/// Shared cookie contract for the public-site settings seam (Phase 18 wave 18.6), the analogue of
/// <see cref="DarkModeServiceBase"/>. Holds the cookie names and the (de)serialization for each preference
/// so the server prerender-read service and the client cookie-write service agree on one wire format —
/// the load-bearing reason this is shared rather than duplicated. Each new preference adds its cookie name
/// and a parse/format pair here, keeping the round-trip in one place.
/// </summary>
public abstract class SettingsServiceBase
{
protected const string StreamQualityCookieName = "streamQuality";
/// <summary>
/// Parses the <c>streamQuality</c> cookie value into <see cref="StreamQuality"/>, defaulting to
/// <see cref="StreamQuality.LowData"/> (the OQ2 default) for an absent, empty, or unrecognized value so
/// a missing/garbled cookie never produces a surprising preference.
/// </summary>
protected static StreamQuality ParseStreamQuality(string? cookieValue) =>
Enum.TryParse<StreamQuality>(cookieValue, ignoreCase: true, out var parsed)
? parsed
: StreamQuality.LowData;
/// <summary>Formats a <see cref="StreamQuality"/> for cookie storage (round-trips with <see cref="ParseStreamQuality"/>).</summary>
protected static string FormatStreamQuality(StreamQuality quality) => quality.ToString();
}
@@ -1,4 +1,5 @@
using DeepDrftModels.DTOs;
using DeepDrftModels.Enums;
using DeepDrftPublic.Client.Clients;
using System.Buffers;
using Microsoft.Extensions.Logging;
@@ -15,6 +16,36 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
// Adaptive chunk sizing
private const int MinBufferSize = 16 * 1024; // 16KB minimum
private const int MaxBufferSize = 64 * 1024; // 64KB maximum
// Phase 21.2a back-pressure poll interval. While the scheduler is over its forward high-water
// mark, the segment loop stops fetching the next segment and polls IsProductionPaused at this
// cadence until the fill drains below low-water. 100 ms is well under the low-water lookahead
// (seconds), so resume is prompt relative to the playhead — no starvation (AC3) — while keeping
// the poll cheap. The poll honors the loop's cancellation token, so a track switch/seek during a
// pause exits through the same drain discipline as a pause during ReadAsync (C6).
private const int BackpressurePollMs = 100;
// Phase 21 Direction B — forward Range-segment size. The forward stream is fetched as a
// sequence of bounded "bytes=cursor-(cursor+SegmentSizeBytes-1)" 206 requests, the next issued
// only when the scheduler drains below low-water. Because each request is bounded and fully
// consumed before the next is issued, the browser fetch holds AT MOST ~one segment of raw bytes
// regardless of file size — this is the network-memory bound the phase exists for (the open-ended
// single GET buffered the whole ~970 MB body in the browser even when reads were paused, the
// 21.4 finding). 4 MB balances request overhead (a 1 GB mix is ~250 segments) against memory:
// at the 30 s high-water mark a fast connection holds well under a segment of unplayed raw bytes,
// so the bound is the segment size, not the decoded window. Tunable; not magic.
private const long SegmentSizeBytes = 4 * 1024 * 1024;
// Phase 18 wave 18.6 — position-preserving format switch ("load at timestamp"). When the listener
// changes streaming quality mid-track, the new-format stream is started DIRECTLY at the saved
// position rather than from byte 0: the load resolves the byte offset for the target time in the
// freshly-initialized decoder and streams from there (never audibly playing the start). For WAV the
// byte-offset math needs the header, so the byte-0 segment is probed (fed to the decoder WITHOUT
// starting playback) until the header parses; this caps that probe so a header that never appears
// (corrupt stream) can't read unbounded. The decoder's own header-search ceiling is 256 KB, so this
// matches it. Opus needs no probe (its sidecar resolves offsets immediately after init).
private const int MaxHeaderProbeBytes = 256 * 1024;
private int _currentBufferSize = DefaultBufferSize;
private int _consecutiveSlowReads = 0;
@@ -33,6 +64,12 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
private readonly ILogger<StreamingAudioPlayerService> _logger;
private string? _currentTrackId;
// The delivery format the active load resolved to (Phase 18). Captured once per LoadTrackStreaming and
// reused by the seek-beyond-buffer re-fetch so the Range continuation requests the SAME artifact the
// initial stream did — a seek must never switch formats mid-track (the JS decoder, the cached setup
// header, and the byte offsets all belong to one artifact). Defaults to Lossless until a load resolves.
private AudioFormat _currentFormat = AudioFormat.Lossless;
// Phase 16 play-session telemetry (§2.1). The tracker observes the playback lifecycle and emits at
// most one bucketed play event per session, behind the engagement floor. Attached after construction
// by AudioPlayerProvider (the player is not DI-registered), mirroring how QueueService binds — no
@@ -129,7 +166,30 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
await NotifyStateChanged();
}
private async Task LoadTrackStreaming(TrackDto track)
/// <inheritdoc />
public async Task ReloadPreservingPositionAsync()
{
// Nothing playing → nothing to switch. The new preference simply takes effect on the next play.
if (CurrentTrack is not { } track || !IsStreamingMode) return;
// Capture the position to restore before the reload resets streaming state. Near the very start
// there is nothing worth preserving — a plain restart in the new format is simpler and avoids a
// needless seek-offset resolution.
var resumeAt = CurrentTime;
await EnsureInitializedAsync();
await _audioInterop.EnsureAudioContextReady(PlayerId);
await NotifyTrackSelected();
// Reload the same track in the newly-resolved delivery format. A near-start position restarts
// from byte 0; otherwise the load begins DIRECTLY at the saved position (no audible playback
// from the start). LoadTrackStreaming runs the whole forward segment loop, so this is the last
// meaningful await — the caller already fires this fire-and-forget.
await LoadTrackStreaming(track, startPosition: resumeAt > 1.0 ? resumeAt : null);
await NotifyStateChanged();
}
private async Task LoadTrackStreaming(TrackDto track, double? startPosition = null)
{
// Always reset to clean state before loading new track. ResetToIdle
// both cancels and awaits any in-flight streaming loop, so by the time
@@ -174,22 +234,35 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
await NotifyStateChanged();
// Pass the streaming token to the HTTP layer so a navigation/track switch
// aborts the server connection instead of leaving it draining bytes.
var mediaResult = await _trackMediaClient.GetTrackMedia(
// Resolve the delivery format for this load BEFORE requesting bytes (Phase 18, default policy
// OQ2). When Opus is chosen the sidecar is fetched and injected into the JS player here, ahead of
// InitializeStreaming, honouring the 18.4 set-before-init contract. The result is captured so the
// seek-beyond-buffer re-fetch reuses the same artifact.
_currentFormat = await ResolveStreamFormatAsync(track.EntryKey, loadCts.Token);
// Direction B: fetch the FIRST bounded segment to learn the total file length and the
// content type. The 206 Content-Range carries the total; the segment loop advances its
// cursor toward it. The decoder is initialized with the TOTAL length (not the segment
// length) so a bounded segment's small Content-Length never trips its byte-count
// completion early — segment boundaries are invisible to the decoder, which sees one
// continuous in-order byte stream. Passing the streaming token aborts the server
// connection on a navigation/track switch instead of leaving it draining bytes.
var firstSegment = await _trackMediaClient.GetTrackMedia(
track.EntryKey,
byteOffset: 0,
byteEnd: SegmentSizeBytes - 1,
format: _currentFormat,
cancellationToken: loadCts.Token);
if (!mediaResult.Success)
if (!firstSegment.Success)
{
var technicalError = mediaResult.GetMessage();
var technicalError = firstSegment.GetMessage();
_logger.LogError("Failed to get track media for {TrackId}: {Error}",
track.EntryKey, technicalError);
ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(technicalError);
return;
}
if (mediaResult.Value == null)
if (firstSegment.Value == null)
{
const string technicalError = "No audio returned from server";
_logger.LogError("No audio data returned for track {TrackId}", track.EntryKey);
@@ -197,13 +270,22 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
return;
}
using var audio = mediaResult.Value;
// Ownership of the first segment transfers to the segment loop, which disposes it (and
// every subsequent segment). No `using` here — a double dispose is avoided and the socket
// is released the moment the loop finishes consuming the segment.
var audio = firstSegment.Value;
// Initialize streaming mode with content length and media type (drives
// JS format-decoder selection).
var streamingResult = await _audioInterop.InitializeStreaming(PlayerId, audio.ContentLength, audio.ContentType);
// The total logical length the decoder must see. On a 206 the Content-Range carries it;
// a 200 (server ignored Range / file ≤ one segment) has no Content-Range, so fall back to
// the body's own Content-Length — that body IS the whole file in that case.
var totalLength = audio.TotalLength ?? audio.ContentLength;
// Initialize streaming mode with the TOTAL length and media type (drives JS
// format-decoder selection). See above: total, not segment, length.
var streamingResult = await _audioInterop.InitializeStreaming(PlayerId, totalLength, audio.ContentType);
if (!streamingResult.Success)
{
audio.Dispose();
var technicalError = $"Failed to initialize streaming: {streamingResult.Error}";
_logger.LogError("Streaming initialization failed for track {TrackId}: {Error}",
track.EntryKey, technicalError);
@@ -211,8 +293,23 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
return;
}
_activeStreamingTask = StreamAudioWithEarlyPlayback(audio, loadCts.Token);
await _activeStreamingTask;
if (startPosition is { } startAt)
{
// "Load at timestamp" (Phase 18 wave 18.6 format switch): begin the stream DIRECTLY at
// startAt rather than byte 0, so the listener never hears the track restart from the
// beginning. The byte-0 segment in hand is used only to parse the header for byte-offset
// math (WAV) — Opus resolves the offset from its sidecar with no probe — and then a fresh
// segment is fetched from the resolved offset and pumped via the shared seek/refill loop.
await StartFromPositionAsync(track.EntryKey, audio, totalLength, startAt, loadCts.Token);
}
else
{
// Forward segmentation from byte 0. The first segment is already in hand; the loop pumps
// it, then fetches subsequent bounded segments gated on the scheduler fill signal.
_activeStreamingTask = RunSegmentedStreamAsync(
track.EntryKey, audio, cursor: 0, totalLength, seekPosition: null, loadCts.Token);
await _activeStreamingTask;
}
}
catch (OperationCanceledException) when (loadCts.IsCancellationRequested)
{
@@ -233,10 +330,33 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
catch (Exception ex)
{
StreamingErrorHandler.LogError(_logger, ex, "LoadTrackStreaming", track.EntryKey);
ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(ex.Message);
LoadProgress = 0;
IsLoaded = false;
IsStreamingMode = false;
var userError = StreamingErrorHandler.GetUserFriendlyMessage(ex.Message);
// Mid-stream failure (playback was already underway): halt the JS scheduler into a clean
// paused-but-loaded state exactly as the seek path does via RecoverFromFailedRefill, rather
// than resetting to unloaded and letting the scheduler's buffered tail drain into a silent
// false end (AC6). Apply only when this load is still the active operation — a superseding
// seek owns state and has already replaced _streamingCancellation with its own CTS.
if (_streamingPlaybackStarted && ReferenceEquals(_streamingCancellation, loadCts))
{
await RecoverFromFailedRefill(CurrentTime, userError);
}
else if (ReferenceEquals(_streamingCancellation, loadCts))
{
// First-segment failure (nothing buffered / playing yet), still the active operation:
// the normal unload-to-error path is correct — nothing is in the scheduler to halt.
ErrorMessage = userError;
LoadProgress = 0;
IsLoaded = false;
IsStreamingMode = false;
}
else
{
// Superseded load: a newer seek (or track switch) has already claimed _streamingCancellation
// and owns all shared state. Writing IsLoaded/IsStreamingMode here would corrupt the live
// operation — mirror the OCE catch's identity guard and do nothing to shared state.
_logger.LogDebug("Generic throw on superseded load for track {TrackId} — newer operation owns state, skipping unload", track.EntryKey);
}
}
finally
{
@@ -250,6 +370,178 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
}
}
/// <summary>
/// Begin streaming the freshly-initialized track DIRECTLY at <paramref name="startPosition"/> instead
/// of byte 0 (Phase 18 wave 18.6 — the position-preserving format switch). The decoder has already been
/// built by <c>InitializeStreaming</c>; this resolves the file-absolute byte offset for the target time
/// and then converges onto the shared seek/refill loop (<see cref="RunSegmentedStreamAsync"/> with a
/// non-null seekPosition), which reinitializes the decoder for a header-less Range continuation and
/// starts playback at the target — so nothing is ever audibly played from the start.
/// <para>
/// Opus resolves the offset from its sidecar immediately; WAV needs its header, so the byte-0 segment
/// already in hand is fed to the decoder (WITHOUT starting playback) until the header parses, then the
/// offset resolves. The probe is bounded by <see cref="MaxHeaderProbeBytes"/>. The byte-0 segment is
/// disposed once the header is in hand; the continuation is a fresh fetch from the resolved offset.
/// </para>
/// </summary>
private async Task StartFromPositionAsync(
string trackId,
TrackMediaResponse headerSegment,
long totalLength,
double startPosition,
CancellationToken cancellationToken)
{
// Resolve the byte offset for the target time. Opus answers immediately from its sidecar; WAV
// returns failure until its header is parsed, so we probe the byte-0 segment and retry. The
// byte-0 segment is disposed once it has served its purpose (header probe / nothing for Opus),
// even if the probe throws, so its socket never leaks before the continuation fetch.
SeekResult resolved;
try
{
resolved = await _audioInterop.ResolveStreamOffsetAsync(PlayerId, startPosition);
if (!resolved.Success || !resolved.SeekBeyondBuffer)
{
await ProbeHeaderAsync(headerSegment, cancellationToken);
resolved = await _audioInterop.ResolveStreamOffsetAsync(PlayerId, startPosition);
}
}
finally
{
headerSegment.Dispose();
}
if (!resolved.Success || !resolved.SeekBeyondBuffer)
{
// Could not resolve an offset even after probing — the stream is unusable for a positioned
// start. Surface as an error rather than silently restarting from 0 (which would contradict
// the "preserve position" contract the listener invoked). The catch in LoadTrackStreaming
// settles the error state.
throw new Exception(resolved.Error ?? "Could not resolve a stream offset for the requested position");
}
// Fetch the FIRST bounded segment from the resolved offset and pump it through the shared loop
// exactly as a seek-beyond-buffer does (reinit for the header-less continuation happens inside,
// and playback starts at startPosition). Reuse the format the load already resolved to.
var byteOffset = resolved.ByteOffset;
var firstSegment = await _trackMediaClient.GetTrackMedia(
trackId,
byteOffset,
byteEnd: byteOffset + SegmentSizeBytes - 1,
format: _currentFormat,
cancellationToken: cancellationToken);
if (!firstSegment.Success || firstSegment.Value == null)
{
var technicalError = firstSegment.GetMessage() ?? "Failed to load audio from position";
_logger.LogError("Failed to get track media from offset {Offset} for {TrackId}: {Error}",
byteOffset, trackId, technicalError);
throw new Exception(technicalError);
}
var audio = firstSegment.Value;
// The absolute EOF boundary the segment loop targets. On a 206 the Content-Range carries the file
// total; on a 200 (single-segment tail) fall back to the offset plus this body's length.
var continuationTotal = audio.TotalLength ?? (byteOffset + audio.ContentLength);
// Fresh playback-start transition for the positioned stream (it has not started yet).
_streamingPlaybackStarted = false;
CanStartStreaming = false;
BufferedChunks = 0;
// Reflect the landing position immediately so the UI seek bar shows the right spot while the
// first post-offset buffers decode.
CurrentTime = startPosition;
_activeStreamingTask = RunSegmentedStreamAsync(
trackId, audio, cursor: byteOffset, continuationTotal, seekPosition: startPosition, cancellationToken);
await _activeStreamingTask;
}
/// <summary>
/// Feed bytes from the byte-0 <paramref name="segment"/> into the decoder until its header parses
/// (<see cref="HeaderParsed"/>), WITHOUT starting playback — the WAV byte-offset math needs the header
/// before <see cref="AudioInteropService.ResolveStreamOffsetAsync"/> can answer. Bounded by
/// <see cref="MaxHeaderProbeBytes"/> so a stream that never yields a header cannot read unbounded. The
/// decoded buffers this queues are dropped by the subsequent Range-continuation reinit (clearForSeek),
/// so nothing is audible and nothing leaks. Opus never reaches here (its offset resolves pre-probe).
/// </summary>
private async Task ProbeHeaderAsync(TrackMediaResponse segment, CancellationToken cancellationToken)
{
var buffer = ArrayPool<byte>.Shared.Rent(MaxBufferSize);
try
{
var probed = 0;
while (!HeaderParsed && probed < MaxHeaderProbeBytes)
{
var read = await segment.Stream.ReadAsync(buffer, 0, _currentBufferSize, cancellationToken);
if (read <= 0) break; // segment exhausted before a header — let the caller surface the failure.
probed += read;
// Slice to the exact bytes read (the pooled buffer may carry stale tail bytes).
var chunk = buffer.AsSpan(0, read).ToArray();
var result = await _audioInterop.ProcessStreamingChunk(PlayerId, chunk);
if (!result.Success)
{
throw new Exception($"Failed to process header probe chunk: {result.Error}");
}
HeaderParsed = result.HeaderParsed;
// Capture the once-only duration the header yields so the UI and play session have it.
if (result.Duration.HasValue && Duration == null)
{
Duration = result.Duration.Value;
_playTracker?.SetDuration(result.Duration.Value);
}
}
}
finally
{
ArrayPool<byte>.Shared.Return(buffer);
}
}
/// <summary>
/// Resolves which delivery format this load should request (Phase 18 default policy, OQ2): Opus when the
/// browser can decode Ogg Opus AND a sidecar exists for the track, otherwise lossless. When Opus is
/// chosen the sidecar is injected into the JS player here (set-before-init, the 18.4 contract) so the
/// decoder has its setup header + seek index before <c>InitializeStreaming</c> builds it.
/// <para>
/// This is the single, deliberately-overridable seam for the listener quality preference (wave 18.6).
/// 18.6 overrides this to honour the user's "streaming quality" toggle — returning lossless when the
/// listener picked it, and otherwise falling through to this capability-gated default. The capability
/// gate (AC7) and the sidecar-absent → lossless fallback (C2) stay here so any override inherits both:
/// a browser that cannot decode Opus, or a track with no sidecar, always lands on lossless and plays.
/// </para>
/// </summary>
protected virtual async Task<AudioFormat> ResolveStreamFormatAsync(string entryKey, CancellationToken cancellationToken)
{
// Capability gate first (AC7): never hand Ogg Opus to a browser that cannot decode it.
if (!await _audioInterop.CanDecodeOggOpus())
{
return AudioFormat.Lossless;
}
// The sidecar must be present (and parseable by the JS decoder) to seek an Opus stream. Its absence
// means the track has no Opus artifact yet (legacy / not backfilled / transcode failed) — request
// lossless rather than Opus-without-a-sidecar (the server would C2-fall-back anyway, but asking for
// lossless keeps the request honest and avoids a wasted Opus-then-fallback round-trip).
var sidecar = await _trackMediaClient.GetOpusSidecarAsync(entryKey, cancellationToken);
if (!sidecar.Success || sidecar.Value is not { Length: > 0 } sidecarBytes)
{
return AudioFormat.Lossless;
}
// Inject BEFORE InitializeStreaming (the set-before-init contract). A parse failure here means the
// bytes are not a usable sidecar — fall back to lossless so a malformed sidecar never breaks playback.
var injected = await _audioInterop.SetOpusSidecar(PlayerId, sidecarBytes);
if (!injected.Success)
{
_logger.LogWarning("Opus sidecar for {EntryKey} failed to parse ({Error}); falling back to lossless.",
entryKey, injected.Error);
return AudioFormat.Lossless;
}
return AudioFormat.Opus;
}
/// <summary>
/// Fetches and decodes the track's waveform loudness profile, then notifies state so the
/// seek zone re-renders with real bars. Best-effort: a 404 (no stored profile) or any other
@@ -311,112 +603,223 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
return profile;
}
private async Task StreamAudioWithEarlyPlayback(TrackMediaResponse audio, CancellationToken cancellationToken)
/// <summary>
/// Phase 21 Direction B — the single segmented forward read loop, shared by the initial load and
/// the seek/refill path (the convergence C1/C5 require: one cursor, one fetch mechanism, no forked
/// path). It pumps the FIRST segment (already fetched by the caller), then fetches subsequent
/// bounded <c>bytes=cursor-(cursor+SegmentSizeBytes-1)</c> 206 segments — each only AFTER the
/// scheduler drains below low-water — until the cursor reaches <paramref name="totalLength"/>.
/// Because each segment is bounded and fully consumed before the next is requested, the browser
/// holds at most ~one segment of raw bytes (the network-memory bound), while the decoder sees one
/// continuous in-order byte stream across segment boundaries (the demuxer/decoder buffer partial
/// frames/pages across the boundary exactly as for arbitrary chunks today — no per-segment reinit).
/// </summary>
/// <param name="firstSegment">The already-fetched first segment (byte <paramref name="cursor"/>).
/// Owned by this method, which disposes it; subsequent segments are fetched and disposed inline.</param>
/// <param name="cursor">File-absolute byte offset the first segment starts at (0 for a fresh load,
/// the resolved seek offset for a refill).</param>
/// <param name="totalLength">Total file length in bytes — the EOF boundary the cursor advances
/// toward. The decoder is initialized/reinitialized against this, not the per-segment length.</param>
/// <param name="seekPosition">Non-null for a seek/refill: the decoder is reinitialized for the
/// header-less Range continuation at this time before the first segment's bytes are fed (WAV
/// retains its header, Opus re-applies the cached setup + lead-trim). Null for a forward load from
/// byte 0, where the first segment carries the header and no reinit is needed.</param>
private async Task RunSegmentedStreamAsync(
string trackId,
TrackMediaResponse firstSegment,
long cursor,
long totalLength,
double? seekPosition,
CancellationToken cancellationToken)
{
byte[]? buffer = null;
var segment = firstSegment;
try
{
long totalBytesRead = 0;
buffer = ArrayPool<byte>.Shared.Rent(MaxBufferSize); // Rent larger buffer to accommodate adaptive sizing
int currentBytes;
var readTimer = System.Diagnostics.Stopwatch.StartNew();
do
// Seek/refill: reinitialize the active decoder for the header-less continuation ONCE,
// before any continuation bytes are fed. Forward-from-zero (seekPosition null) skips this
// — its first segment carries the real header the decoder parses. Done here, inside the
// single loop, so seek and forward share the same fetch+pump mechanism (no forked path).
if (seekPosition is { } resumeAt)
{
readTimer.Restart();
currentBytes = await audio.Stream.ReadAsync(buffer, 0, _currentBufferSize, cancellationToken);
readTimer.Stop();
// Adapt buffer size based on read performance
AdaptBufferSize(currentBytes, readTimer.ElapsedMilliseconds);
if (currentBytes > 0)
// The decoder byte-counts the header-less continuation against the bytes REMAINING
// from the range start to EOF (total cursor), not the absolute total — that is what
// reinitializeForRangeContinuation expects (StreamDecoder.remainingByteLength). The
// loop's own cursor still targets the absolute totalLength for EOF.
var remainingBytes = Math.Max(0, totalLength - cursor);
var reinitResult = await _audioInterop.ReinitializeFromOffset(PlayerId, remainingBytes, resumeAt);
if (!reinitResult.Success)
{
totalBytesRead += currentBytes;
// Always slice to the exact number of bytes read. The pooled buffer
// is rented at MaxBufferSize and may carry stale bytes past
// currentBytes from a prior rental — handing the full array to JS
// interop would serialise that garbage into the audio stream.
var actualBuffer = buffer.AsSpan(0, currentBytes).ToArray();
// Process chunk for streaming
var chunkResult = await _audioInterop.ProcessStreamingChunk(PlayerId, actualBuffer);
if (!chunkResult.Success)
{
var error = $"Failed to process streaming chunk: {chunkResult.Error}";
_logger.LogWarning("Chunk processing failed: {Error}", error);
throw new Exception(error);
}
// Update streaming state
CanStartStreaming = chunkResult.CanStartStreaming;
HeaderParsed = chunkResult.HeaderParsed;
BufferedChunks = chunkResult.BufferCount;
// Set duration from WAV header when available (only set once)
if (chunkResult.Duration.HasValue && Duration == null)
{
Duration = chunkResult.Duration.Value;
_logger.LogInformation("Duration set from WAV header: {Duration:F2} seconds", Duration);
// Feed the same once-only duration to the play session so it can compute the
// completion fraction at close. Safe before/after session open — SetDuration
// is a no-op when no session is open and idempotent otherwise.
_playTracker?.SetDuration(chunkResult.Duration.Value);
}
// Start playback as soon as we can
if (!_streamingPlaybackStarted && CanStartStreaming)
{
var playbackResult = await _audioInterop.StartStreamingPlayback(PlayerId);
if (playbackResult.Success)
{
_streamingPlaybackStarted = true;
IsPlaying = true;
IsPaused = false;
IsLoaded = true; // Track is loaded and ready to play (even if still downloading)
ErrorMessage = null;
// Open the play session exactly once per load, at the moment playback truly
// begins (§2.1). The _sessionOpened guard keeps the SeekBeyondBuffer re-stream
// — which re-enters this transition with _streamingPlaybackStarted reset —
// from opening a second session for the same play. Duration may already be
// known from a prior chunk, so re-feed it after opening.
if (!_sessionOpened && _currentTrackId is { } trackKey)
{
_sessionOpened = true;
_playTracker?.OnPlaybackStarted(trackKey);
if (Duration is { } d)
_playTracker?.SetDuration(d);
}
await NotifyStateChanged(); // Immediate notification for critical state change
}
else
{
var technicalError = $"Failed to start streaming playback: {playbackResult.Error}";
_logger.LogError("Failed to start playback: {Error}", technicalError);
ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(technicalError);
}
}
// Update progress
if (audio.ContentLength > 0)
{
LoadProgress = Math.Min(1.0, (double)totalBytesRead / audio.ContentLength);
}
await ThrottledNotifyStateChanged();
throw new Exception($"Failed to reinitialize for offset streaming: {reinitResult.Error}");
}
} while (currentBytes > 0);
}
// Notify the JS decoder that the stream is finished. When the server omits
// Content-Length the StreamDecoder cannot determine completion via byte counting
// alone; this explicit signal ensures the tail-decoding path (streamComplete=true)
// fires regardless of whether Content-Length was present.
buffer = ArrayPool<byte>.Shared.Rent(MaxBufferSize); // larger rental to fit adaptive sizing
var readTimer = System.Diagnostics.Stopwatch.StartNew();
// Segment loop. Each iteration fully consumes one bounded 206 body, advancing the cursor by
// the bytes received. The next segment is fetched only when the scheduler is below
// high-water (the inter-segment gate). EOF is the cursor reaching totalLength, or a short
// segment (server returned fewer bytes than requested — the final slice).
while (true)
{
long segmentBytesRead = 0;
int currentBytes;
do
{
readTimer.Restart();
currentBytes = await segment.Stream.ReadAsync(buffer, 0, _currentBufferSize, cancellationToken);
readTimer.Stop();
AdaptBufferSize(currentBytes, readTimer.ElapsedMilliseconds);
if (currentBytes > 0)
{
segmentBytesRead += currentBytes;
// Slice to the exact bytes read: the pooled buffer is rented at MaxBufferSize
// and may carry stale bytes past currentBytes from a prior rental — handing the
// full array to JS would serialise that garbage into the audio stream.
var actualBuffer = buffer.AsSpan(0, currentBytes).ToArray();
var chunkResult = await _audioInterop.ProcessStreamingChunk(PlayerId, actualBuffer);
if (!chunkResult.Success)
{
var error = $"Failed to process streaming chunk: {chunkResult.Error}";
_logger.LogWarning("Chunk processing failed: {Error}", error);
throw new Exception(error);
}
CanStartStreaming = chunkResult.CanStartStreaming;
HeaderParsed = chunkResult.HeaderParsed;
BufferedChunks = chunkResult.BufferCount;
// Set duration from header when available (only set once)
if (chunkResult.Duration.HasValue && Duration == null)
{
Duration = chunkResult.Duration.Value;
_logger.LogInformation("Duration set from header: {Duration:F2} seconds", Duration);
// Feed the once-only duration to the play session for the completion
// fraction. No-op when no session is open; idempotent otherwise.
_playTracker?.SetDuration(chunkResult.Duration.Value);
}
// Start playback as soon as we can — at the min-buffer threshold, exactly as
// before (C2: first audio is not gated on the segment boundary; the first
// segment alone clears the threshold).
if (!_streamingPlaybackStarted && CanStartStreaming)
{
await TryStartPlaybackAsync();
}
// Progress against the total file length (cursor + bytes consumed so far).
if (totalLength > 0)
{
LoadProgress = Math.Min(1.0, (double)(cursor + segmentBytesRead) / totalLength);
}
await ThrottledNotifyStateChanged();
// Per-chunk back-pressure — the bound that actually holds for high-density codecs.
// The inter-segment gate alone is matched to WAV's byte density (~24 s of audio per
// 4 MB segment) but NOT to Opus: at 320 kbps a 4 MB segment is ~100 s of decodable
// audio. The inner loop has the whole segment's bytes already in hand, so with no
// network wait to pace it, it would decode the ENTIRE segment eagerly — piling tens
// of MB of decoded f32 PCM AHEAD of a playhead that has barely moved, before the
// inter-segment gate ever runs. With HW accel off that lookahead lives in main-
// process RAM, and the byte ceiling cannot save us because nothing on this path
// polls it. So drain to low-water per chunk once the scheduler is over high-water.
//
// Gated on _streamingPlaybackStarted so this can NEVER block first audio (C2): until
// playback starts the playhead does not advance, so the forward fill would never
// drain and the loop would deadlock. The 30 s high-water sits far above the
// 6-buffer playback-start minimum, so in practice the gate is not even reached
// before playback begins — the guard is the correctness backstop, not the common
// case. Reads the piggybacked flag (no extra interop hop) to DECIDE to drain; the
// drain helper then polls IsProductionPaused — the same steady-state-reads-flag /
// throttled-state-polls split the inter-segment gate uses.
if (_streamingPlaybackStarted && chunkResult.ProductionPaused)
{
await DrainBackpressureAsync(cancellationToken);
}
}
} while (currentBytes > 0);
// Segment fully consumed; advance the cursor and release this segment's stream/socket
// before deciding whether to fetch the next. Disposing here keeps exactly one segment's
// raw bytes resident at a time.
cursor += segmentBytesRead;
segment.Dispose();
segment = null!;
// EOF: cursor reached the total file length. This is the sole forward-EOF condition.
// A short segment body (segmentBytesRead < SegmentSizeBytes) is NOT an EOF signal —
// the inner read loop fully drains the HTTP body, so a short body means the server
// sent fewer bytes than the bounded range we requested. While cursor < totalLength that
// can only be a connection drop / truncated stream, NOT the file tail — route it to
// the same clean-failure recovery as a fetch error rather than silently completing.
var reachedTotal = totalLength > 0 && cursor >= totalLength;
if (reachedTotal)
{
break;
}
// Guard: if the body was short but we haven't reached totalLength, the stream was
// truncated mid-segment (connection drop / premature close). Surface as an error so
// the scheduler is halted rather than left to drain its buffered tail into a false end.
if (segmentBytesRead < SegmentSizeBytes)
{
throw new Exception(
$"Stream truncated at byte {cursor} of {totalLength}: received {segmentBytesRead} bytes " +
$"but expected up to {SegmentSizeBytes} and have not reached EOF");
}
// Inter-segment back-pressure gate (Phase 21.2 fill signal, gating SEGMENT FETCH). Do not
// fetch the next segment while the scheduler is over high-water; wait until it drains
// below low-water. Because the browser only buffers bounded segments and we hold off
// requesting the next one, raw network memory stays at ~one segment. Shares the same
// drain helper as the per-chunk gate above. No _streamingPlaybackStarted guard is needed
// here (unlike the per-chunk gate): reaching this point means a full segment was consumed,
// which is ~24 s (WAV) / ~100 s (Opus) of audio — far past the 6-buffer playback-start
// minimum — so playback is always running by now and the fill can drain. A file that fits
// in one segment hits EOF and breaks above, never reaching this gate.
await DrainBackpressureAsync(cancellationToken);
// Fetch the next bounded segment. The end offset is clamped implicitly by the server
// (a request past EOF yields the available tail as a short slice, caught above).
var nextEnd = cursor + SegmentSizeBytes - 1;
var nextResult = await _trackMediaClient.GetTrackMedia(
trackId,
byteOffset: cursor,
byteEnd: nextEnd,
format: _currentFormat,
cancellationToken: cancellationToken);
if (!nextResult.Success || nextResult.Value == null)
{
var technicalError = nextResult.GetMessage() ?? "Failed to fetch next stream segment";
_logger.LogError("Failed to fetch segment at offset {Offset} for {TrackId}: {Error}",
cursor, trackId, technicalError);
throw new Exception(technicalError);
}
segment = nextResult.Value;
}
// Notify the JS decoder that the stream is finished. The decoder marks completion by byte
// count against the total it was initialized with; this explicit signal flushes the
// residual tail and covers the (rare) case where the total was unknown.
await _audioInterop.MarkStreamCompleteAsync(PlayerId);
// Mark as fully loaded
// Complete-without-start fallback: if the track's total decodable audio never crossed the
// start threshold (e.g. total Opus audio < 1s lead, or WAV < 6 buffers), the in-loop
// CanStartStreaming check never fired and _streamingPlaybackStarted is still false. Now that
// streamComplete is set on the JS scheduler, calling StartStreamingPlayback lets it drain
// the accumulated buffers and fires onPlaybackEnded exactly once — same transition the
// normal path uses, so session/_sessionOpened/Duration handling is identical.
if (!_streamingPlaybackStarted)
{
await TryStartPlaybackAsync();
}
LoadProgress = 1.0;
await NotifyStateChanged();
}
@@ -427,7 +830,7 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
}
catch (Exception ex)
{
StreamingErrorHandler.LogError(_logger, ex, "StreamAudioWithEarlyPlayback");
StreamingErrorHandler.LogError(_logger, ex, "RunSegmentedStreamAsync");
ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(ex.Message);
LoadProgress = 0;
IsLoaded = false;
@@ -437,6 +840,8 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
}
finally
{
// Release the last segment (if a fetch failed mid-loop it may still be held) and the buffer.
segment?.Dispose();
if (buffer != null)
{
ArrayPool<byte>.Shared.Return(buffer);
@@ -444,6 +849,46 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
}
}
/// <summary>
/// Call <c>StartStreamingPlayback</c> on the JS player and apply the resulting state transitions.
/// This is the single playback-start transition shared by the in-loop threshold path and the
/// completion-path fallback — both callers set the guard and apply session/Duration handling
/// identically so neither path diverges.
/// </summary>
private async Task TryStartPlaybackAsync()
{
var playbackResult = await _audioInterop.StartStreamingPlayback(PlayerId);
if (playbackResult.Success)
{
_streamingPlaybackStarted = true;
IsPlaying = true;
IsPaused = false;
IsLoaded = true; // loaded and ready, even while still downloading
ErrorMessage = null;
// Open the play session exactly once per load, at the moment playback
// truly begins (§2.1). The _sessionOpened guard keeps a seek/refill
// re-stream — which re-enters this transition with
// _streamingPlaybackStarted reset — from opening a second session for
// the same play. Duration may already be known, so re-feed it.
if (!_sessionOpened && _currentTrackId is { } trackKey)
{
_sessionOpened = true;
_playTracker?.OnPlaybackStarted(trackKey);
if (Duration is { } d)
_playTracker?.SetDuration(d);
}
await NotifyStateChanged(); // immediate — critical state change
}
else
{
var technicalError = $"Failed to start streaming playback: {playbackResult.Error}";
_logger.LogError("Failed to start playback: {Error}", technicalError);
ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(technicalError);
}
}
/// <summary>
/// In streaming mode, Stop fully resets to Idle state since audio data is consumed.
/// This is equivalent to Unload for streaming playback.
@@ -515,6 +960,10 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
return;
}
// Capture into a non-null local: _currentTrackId is the field a track-switch could clear, but
// this seek operates against the track loaded NOW; the segment loop needs a stable id.
var trackId = _currentTrackId;
IsSeekingBeyondBuffer = true;
// Cancel the current streaming loop AND wait for it to fully exit before
@@ -538,46 +987,68 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
await DrainActiveStreamingTaskAsync();
oldCts?.Dispose();
// Single-writer discipline (C6/AC8): all three failure exits must share the same guard.
// TrackMediaClient.GetTrackMedia swallows OperationCanceledException and returns
// Success==false, so a superseded seek lands in the media-fetch-fail branch below
// rather than in the OCE catch. Without the guard those branches would call
// RecoverFromFailedRefill — running clearForSeek + setPlaybackOffset against the player
// state the NEWER seek now owns. A local predicate keeps all three exits symmetric so a
// future exit cannot forget the check.
bool IsStillActiveSeek() => ReferenceEquals(_streamingCancellation, seekCts);
try
{
// Update UI immediately
CurrentTime = seekPosition;
await NotifyStateChanged();
// Request new stream from offset
var mediaResult = await _trackMediaClient.GetTrackMedia(
_currentTrackId,
// Request the FIRST bounded segment from the resolved offset (Direction B — converged with
// the forward path). Reuse the format the initial load resolved to (_currentFormat): an
// Opus seek must come back as Opus bytes so the cached setup header + page-aligned
// byteOffset (resolved JS-side from the Opus seek index) match the continuation; WAV resolves
// its offset from the header — one seam, format-appropriate math (AC9 / §3.4a C). The
// segment loop then continues forward segmentation from this offset exactly as a fresh load
// does from 0 — no forked fetch path (C1/C5).
var firstSegment = await _trackMediaClient.GetTrackMedia(
trackId,
byteOffset,
byteEnd: byteOffset + SegmentSizeBytes - 1,
format: _currentFormat,
cancellationToken: seekCts.Token);
if (!mediaResult.Success || mediaResult.Value == null)
if (!firstSegment.Success || firstSegment.Value == null)
{
var technicalError = mediaResult.GetMessage() ?? "Failed to load audio from position";
var technicalError = firstSegment.GetMessage() ?? "Failed to load audio from position";
_logger.LogError("Failed to get track media from offset {Offset}: {Error}", byteOffset, technicalError);
ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(technicalError);
IsSeekingBeyondBuffer = false;
// Guard: a superseded seek must NOT touch shared state. The newer seek owns teardown.
if (IsStillActiveSeek())
{
await RecoverFromFailedRefill(seekPosition, StreamingErrorHandler.GetUserFriendlyMessage(technicalError));
}
else
{
_logger.LogDebug("Media-fetch failed on superseded seek to {Position} — newer seek owns state, skipping recovery", seekPosition);
}
return;
}
using var audio = mediaResult.Value;
var audio = firstSegment.Value;
// The absolute EOF boundary the segment loop's cursor targets. On a 206 the Content-Range
// carries the file total; on a 200 (single-segment file) fall back to cursor + body length.
var totalLength = audio.TotalLength ?? (byteOffset + audio.ContentLength);
// Reinitialize JS player for offset streaming
var reinitResult = await _audioInterop.ReinitializeFromOffset(PlayerId, audio.ContentLength, seekPosition);
if (!reinitResult.Success)
{
_logger.LogError("Failed to reinitialize for offset streaming: {Error}", reinitResult.Error);
ErrorMessage = "Failed to seek to position";
IsSeekingBeyondBuffer = false;
return;
}
// Reset streaming state for new stream
// Reset streaming state for the new stream. The decoder reinit for the header-less
// continuation happens INSIDE RunSegmentedStreamAsync (seekPosition non-null), so seek and
// forward share one fetch+pump+reinit mechanism. A reinit failure there throws and lands in
// the catch below, which recovers when still the active seek — the same clean-failure path
// (AC6) the old explicit reinit branch had, now unified with the fetch-failure path.
_streamingPlaybackStarted = false;
CanStartStreaming = false;
HeaderParsed = false;
BufferedChunks = 0;
// Stream audio from offset
_activeStreamingTask = StreamAudioWithEarlyPlayback(audio, seekCts.Token);
// Stream from offset via the shared segment loop. Ownership of `audio` transfers to it.
_activeStreamingTask = RunSegmentedStreamAsync(
trackId, audio, cursor: byteOffset, totalLength, seekPosition, seekCts.Token);
await _activeStreamingTask;
IsSeekingBeyondBuffer = false;
@@ -588,22 +1059,65 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
// still the active seek — if _streamingCancellation has been replaced, a
// newer seek is in progress and owns the flag.
_logger.LogDebug("Seek beyond buffer cancelled");
if (ReferenceEquals(_streamingCancellation, seekCts))
if (IsStillActiveSeek())
{
IsSeekingBeyondBuffer = false;
}
}
catch (Exception ex)
{
// A refill fetch can fail deep into a long mix (the listener didn't initiate it). Recover
// into a clean paused-but-loaded state (AC6) rather than leaving the starved scheduler to
// fire a silent false end. Only when we are still the active seek — a superseding seek owns
// the state and the OCE catch above handles its own teardown.
_logger.LogError(ex, "Error during seek beyond buffer to position {Position}", seekPosition);
ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(ex.Message);
IsSeekingBeyondBuffer = false;
await NotifyStateChanged();
if (IsStillActiveSeek())
{
await RecoverFromFailedRefill(seekPosition, StreamingErrorHandler.GetUserFriendlyMessage(ex.Message));
}
}
}
/// <summary>
/// Single method to reset all state - called by both Stop and Unload.
/// Clean-failure recovery for a window-miss refill (Phase 21.3 / AC6). A backward seek past the
/// retained tail re-fetches via the existing Range path; that mid-stream fetch the listener did not
/// initiate can fail deep into a long mix. When it does, the pre-seek loop has already been
/// cancelled and drained, but the JS scheduler is still holding stale pre-seek buffers and still
/// "playing" — left alone it drains them and fires a silent false end (the wedged/starved state AC6
/// forbids). This halts the scheduler into a paused-but-loaded state at <paramref name="seekPosition"/>,
/// surfaces a clear error, and leaves the track loaded so the listener can retry the seek or pick
/// another track. Mirrors <c>PlaybackScheduler.playFromPosition</c>'s end-of-buffer recovery: stop
/// pretending to play.
/// </summary>
private async Task RecoverFromFailedRefill(double seekPosition, string userFacingError)
{
// Halt the starved scheduler JS-side (stop sources, drop stale buffers, anchor at the target).
// Best-effort: if even this interop fails the player is no worse off, and we still surface the
// error and settle C# state below.
var recovered = await _audioInterop.RecoverFromFailedRefill(PlayerId, seekPosition);
if (!recovered.Success)
{
_logger.LogWarning("Refill-failure recovery interop did not succeed: {Error}", recovered.Error);
}
// Settle C# into the matching recoverable state: not playing, paused at the target, still loaded
// and still in streaming mode. IsLoaded = true and IsStreamingMode = true are both load-bearing —
// the "paused-but-loaded" contract lets the listener retry the seek (Seek early-returns when
// !IsLoaded || !IsStreamingMode), resume via TogglePlayPause, or pick another track. Resetting
// either to false would wedge at least one of the three retry routes (AC6 / Phase 21.3).
ErrorMessage = userFacingError;
IsPlaying = false;
IsPaused = true;
IsLoaded = true;
IsStreamingMode = true;
CurrentTime = seekPosition;
IsSeekingBeyondBuffer = false;
await NotifyStateChanged();
}
/// <summary>
/// Single method to reset all state - called by both Stop and Unload, and as the prologue of a new
/// load.
/// </summary>
private async Task ResetToIdle()
{
@@ -653,6 +1167,7 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
_streamingPlaybackStarted = false;
IsSeekingBeyondBuffer = false;
_currentTrackId = null;
_currentFormat = AudioFormat.Lossless;
await NotifyStateChanged();
}
@@ -691,6 +1206,27 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS
}
}
/// <summary>
/// Block the segment loop while the scheduler's decoded forward fill is over high-water, resuming
/// once it drains below low-water (Phase 21.2 hysteresis). Shared by the per-chunk gate (inside a
/// segment) and the inter-segment gate so both honor identical drain discipline — a guard present on
/// one path and absent on the other would let one path overshoot the memory bound.
/// <para>
/// The poll awaits on <paramref name="cancellationToken"/>, so a track switch/seek mid-wait throws
/// OCE and unwinds through the existing drain discipline (C6). UC5: a user pause freezes the playhead
/// so the fill never drains on its own — hold here until playback resumes (IsPaused clears) OR the
/// fill drains. Returns immediately when nothing is throttled (the steady-state common case).
/// </para>
/// </summary>
private async Task DrainBackpressureAsync(CancellationToken cancellationToken)
{
while (IsPaused || await _audioInterop.IsProductionPaused(PlayerId))
{
cancellationToken.ThrowIfCancellationRequested();
await Task.Delay(BackpressurePollMs, cancellationToken);
}
}
private async Task ThrottledNotifyStateChanged()
{
var now = DateTime.UtcNow;
@@ -160,6 +160,47 @@ public sealed class WaveformVisualizerControlState
/// </summary>
public event Action? Changed;
// Whether the one-time, capability-driven default has been applied this session. The default-set
// (lava off when the browser has no WebGL hardware acceleration) must run exactly once — on the
// first interactive render, before the listener has touched a toggle — so it sets the *initial
// default* and never clobbers a later explicit in-session toggle. Scoped with the rest of this
// state, so it survives SPA navigation (a remounted visualizer does not re-apply) and resets on a
// fresh page load (F5 re-probes).
private bool _capabilityDefaultApplied;
/// <summary>
/// Applies the hardware-capability default exactly once per session: when the browser reports no
/// WebGL hardware acceleration, the lava subsystem (the expensive, main-thread software-rendered
/// part that starves audio decode) defaults <c>off</c> while the waveform stays <c>on</c>. With
/// acceleration present this is a no-op — lava keeps its <see cref="DefaultLavaEnabled"/> on-state.
///
/// <para>
/// Idempotent and guarded: only the FIRST call this session has any effect, so it sets the initial
/// default and never overrides a listener's explicit toggle (the control remains fully functional —
/// a user on a software renderer may re-enable lava at their own risk). Mutates, coerces
/// Theater Mode, then raises <see cref="Changed"/> once so the controls UI, the visualizer bridge,
/// and the Theater observers all reflect the default in a single cycle. Called by the visualizer
/// bridge on first interactive render, once JS interop (the probe) is available.
/// </para>
/// </summary>
/// <param name="hardwareAccelerated">
/// The probe result — <c>true</c> when WebGL hardware acceleration is present (or the renderer is
/// unknown/masked, favoring the common case), <c>false</c> only on a positive software-renderer
/// match or total WebGL failure.
/// </param>
public void ApplyCapabilityDefault(bool hardwareAccelerated)
{
if (_capabilityDefaultApplied) return;
_capabilityDefaultApplied = true;
// Accelerated (or unknown): keep the as-shipped defaults — no observer churn.
if (hardwareAccelerated) return;
LavaEnabled = false; // the expensive subsystem off; WaveformEnabled stays at its default (true).
CoerceTheaterMode();
NotifyChanged();
}
/// <summary>
/// Enforces the Theater-Mode invariant: Theater Mode cannot remain on when both visualizer
/// subsystems are off (there is nothing to go to theater FOR). Call this after mutating
+7
View File
@@ -14,6 +14,13 @@ public static class Startup
services.AddScoped<DarkModeSettings>();
services.AddScoped<DarkModeCookieService>();
// Public-site listener settings (Phase 18 wave 18.6). PublicSiteSettings is the generalized,
// prerender-seeded preference object (today: streaming quality); SettingsCookieService writes the
// 365-day cookie at runtime. Same scoped lifetime + cookie seam as the dark-mode pair above, so the
// preference survives SPA nav within a session and seeds the next visit's prerender.
services.AddScoped<PublicSiteSettings>();
services.AddScoped<SettingsCookieService>();
// Track Client. The HTTP-backed ITrackDataService is used by both WASM and SSR
// prerender — both call DeepDrftAPI over the "DeepDrft.API" client.
services.AddScoped<TrackClient>();
+3
View File
@@ -25,6 +25,7 @@
<script src="_framework/blazor.web.js"></script>
<script src=@Assets["_content/MudBlazor/MudBlazor.min.js"]></script>
<script type="module">
import('./js/settings/settings.js');
import('./js/audio/index.js');
import('./js/telemetry/beacon.js');
import('./js/telemetry/anonid.js');
@@ -37,6 +38,7 @@
[Inject] public required DarkModeService DarkModeService { get; set; }
[Inject] public required SeoEnvironment SeoEnvironment { get; set; }
[Inject] public required IWebHostEnvironment HostEnvironment { get; set; }
[Inject] public required SettingsService SettingsService { get; set; }
protected override void OnInitialized()
{
@@ -45,6 +47,7 @@
// Seed the environment-gated robots bridge during prerender; [PersistentState] rounds it to WASM
// so both render passes resolve the same default robots (Production → index, else noindex).
SeoEnvironment.IsProduction = HostEnvironment.IsProduction();
SettingsService.CheckSettings();
}
}
@@ -205,21 +205,26 @@ public class TrackProxyController : ControllerBase
/// <summary>
/// Proxies audio streaming from DeepDrftAPI as a transparent HTTP Range relay.
/// Forwards the incoming Range header upstream and relays the upstream status
/// (200 full, 206 partial, 416 unsatisfiable) and range-related response headers
/// back to the browser verbatim. The proxy does not slice — the upstream already did.
/// Forwards the incoming Range header upstream and the optional <c>format</c> selector
/// (Phase 18.3 — <c>opus|lossless</c>, threaded the same way the listing params are),
/// and relays the upstream status (200 full, 206 partial, 416 unsatisfiable) and
/// range-related response headers back to the browser verbatim. The proxy does not
/// slice — the upstream already did.
/// </summary>
[HttpGet("{trackId}")]
public async Task<ActionResult> GetTrack(
string trackId,
[FromQuery] string? format = null,
CancellationToken ct = default)
{
var rangeHeader = Request.Headers.Range.ToString();
_logger.LogInformation("Proxying track {TrackId} range '{Range}'", trackId, rangeHeader);
_logger.LogInformation("Proxying track {TrackId} range '{Range}' format '{Format}'", trackId, rangeHeader, format);
var request = new HttpRequestMessage(
HttpMethod.Get,
$"api/track/{Uri.EscapeDataString(trackId)}");
var path = $"api/track/{Uri.EscapeDataString(trackId)}";
if (!string.IsNullOrWhiteSpace(format))
path += $"?format={Uri.EscapeDataString(format)}";
var request = new HttpRequestMessage(HttpMethod.Get, path);
// Forward the browser's Range header upstream so DeepDrftAPI slices the file.
// TryAddWithoutValidation avoids RangeHeaderValue reparsing — we relay the raw
@@ -355,4 +360,40 @@ public class TrackProxyController : ControllerBase
return Content(json, "application/json");
}
}
/// <summary>
/// Proxies a track's Opus seek/setup sidecar (raw bytes) from DeepDrftAPI (Phase 18.3). Unauthenticated,
/// same posture as the audio stream forward. The sidecar is a small one-time fetch (≤ ~115 KB), so it is
/// buffered and relayed; a 404 (no Opus artifact / no sidecar stored) passes through so the client
/// degrades to lossless rather than treating it as an error. The "opus/seekdata" 3-segment route makes a
/// collision with the parameterized "{trackId}" audio route impossible.
/// </summary>
[HttpGet("{trackId}/opus/seekdata")]
public async Task<ActionResult> GetOpusSeekData(string trackId, CancellationToken ct = default)
{
var path = $"api/track/{Uri.EscapeDataString(trackId)}/opus/seekdata";
HttpResponseMessage upstream;
try
{
upstream = await _upstream.GetAsync(path, HttpCompletionOption.ResponseHeadersRead, ct);
}
catch (Exception ex)
{
_logger.LogError(ex, "Upstream call to DeepDrftAPI track/{TrackId}/opus/seekdata failed", trackId);
return StatusCode(502, "Upstream unavailable");
}
using (upstream)
{
if (!upstream.IsSuccessStatusCode)
{
_logger.LogWarning("DeepDrftAPI track/{TrackId}/opus/seekdata returned {Status}", trackId, (int)upstream.StatusCode);
return StatusCode((int)upstream.StatusCode);
}
var bytes = await upstream.Content.ReadAsByteArrayAsync(ct);
return File(bytes, "application/octet-stream");
}
}
}
@@ -0,0 +1,298 @@
/**
* AudioPlayer window-miss refill tests (Phase 21.3) the seek-dispatch TRIGGER and the AC6
* clean-failure recovery.
*
* What this pins (the genuinely-new 21.3 work):
* - The window-miss TRIGGER. AudioPlayer.seek() routes by whether the target falls inside the
* retained window [playbackOffset, playbackOffset + totalDuration]. After 21.1 partial eviction
* playbackOffset is the absolute start of the retained back-window tail, so:
* * seek back WITHIN the tail -> seekWithinBuffer, NO refetch (UC3 / AC4),
* * seek back PAST the tail -> seekBeyondBuffer with the EARLIER resolved offset (UC4 / AC5),
* using whichever resolver the active path ships (WAV calculateByteOffset; Opus
* resolveOpusByteOffset over the sidecar index),
* * seek forward past the decoded end -> seekBeyondBuffer forward, unchanged (UC2/UC5).
* - The AC6 recovery. recoverFromFailedRefill() halts the scheduler (clearForSeek), anchors the
* offset at the seek target, and leaves the player paused-but-loaded so no silent false end fires.
*
* The seek dispatch and recovery are pure given the scheduler + active decoder, so they are testable
* in Node by white-box-injecting fakes for `scheduler`, `streamDecoder`, and `opusDecoder` (the same
* private-field injection idiom the scheduler/Opus tests use). The AudioPlayer constructor itself is
* Node-safe: it builds AudioContextManager/StreamDecoder/PlaybackScheduler, none of which touch Web
* Audio until initialize(). No AudioContext, no WebCodecs.
*
* Same harness convention as the sibling tests (no runner in this repo); run a copy from the COMPILED
* output so the `.js` import specifiers resolve:
*
* dotnet build DeepDrftPublic/DeepDrftPublic.csproj
* cp DeepDrftPublic/Interop/audio/AudioPlayer.test.ts DeepDrftPublic/wwwroot/js/audio/
* node DeepDrftPublic/wwwroot/js/audio/AudioPlayer.test.ts
*
* A thrown error / non-zero exit signals failure; "ALL <n> TESTS PASSED" signals success.
* Excluded from the production tsc build via tsconfig `exclude: Interop/ ** /*.test.ts`.
*/
import { AudioPlayer } from './AudioPlayer.js';
import { parseSidecar } from './OpusSidecar.js';
import type { OpusSeekData } from './OpusSidecar.js';
// --- tiny inline harness (no dependencies) ---------------------------------------------------
let passed = 0;
const failures: string[] = [];
function test(name: string, fn: () => void): void {
try {
fn();
passed++;
} catch (e) {
failures.push(`FAIL: ${name}\n ${(e as Error).message}`);
}
}
function assertEqual(actual: unknown, expected: unknown, msg?: string): void {
if (actual !== expected) {
throw new Error(`${msg ?? 'assertEqual'}: expected ${String(expected)}, got ${String(actual)}`);
}
}
function assertTrue(cond: boolean, msg?: string): void {
if (!cond) throw new Error(msg ?? 'assertTrue failed');
}
// --- fakes -----------------------------------------------------------------------------------
/**
* A scheduler stand-in exposing only what AudioPlayer.seek / seekWithinBuffer / seekBeyondBuffer /
* recoverFromFailedRefill read or call. The retained window is [offset, offset + total]. Records the
* methods that mutate so the recovery test can assert the cleanup happened.
*/
class FakeScheduler {
private offset: number;
private total: number;
// hasBuffers reflects whether the scheduler holds decoded audio. Starts true when total > 0
// (a populated window), set to false by clearForSeek() (recovery drains the buffers).
private _hasBuffers: boolean;
public clearedForSeek = false;
public stoppedAllSources = false;
public offsetSetTo: number | null = null;
constructor(offset: number, total: number) {
this.offset = offset;
this.total = total;
this._hasBuffers = total > 0;
}
getPlaybackOffset(): number { return this.offset; }
getTotalDuration(): number { return this.total; }
hasBuffers(): boolean { return this._hasBuffers; }
stopAllSources(): void { this.stoppedAllSources = true; }
// seekWithinBuffer calls playFromPosition only when wasPlaying; isPlaying is false in these
// unit constructions, so it is never invoked — present for completeness.
playFromPosition(_position: number): void { /* no-op */ }
clearForSeek(): void { this.clearedForSeek = true; this._hasBuffers = false; }
setPlaybackOffset(o: number): void { this.offset = o; this.offsetSetTo = o; }
}
/** A StreamDecoder stand-in for the WAV path: a format is parsed and byte math is identity-scaled. */
class FakeStreamDecoder {
private hasFormat: boolean;
private bytesPerSecond: number;
public requestedOffsetFor: number | null = null;
constructor(hasFormat: boolean, bytesPerSecond: number) { this.hasFormat = hasFormat; this.bytesPerSecond = bytesPerSecond; }
getFormatInfo(): unknown { return this.hasFormat ? { ok: true } : null; }
calculateByteOffset(position: number): number {
this.requestedOffsetFor = position;
return Math.round(position * this.bytesPerSecond);
}
}
function makePlayer(): AudioPlayer {
// Constructor is Node-safe (no Web Audio until initialize()).
return new AudioPlayer();
}
/** Inject the seek-relevant private fields and put the player in a loaded/streaming/playing state. */
function arm(
player: AudioPlayer,
opts: {
scheduler: FakeScheduler;
duration: number;
streamDecoder?: FakeStreamDecoder;
opusDecoder?: object | null;
sidecar?: OpusSeekData | null;
}
): void {
const priv = player as unknown as Record<string, unknown>;
priv.scheduler = opts.scheduler;
priv.duration = opts.duration;
priv.isStreamingMode = true;
priv.isPlaying = false; // keep dispatch pure (no real playFromPosition needed)
if (opts.streamDecoder) priv.streamDecoder = opts.streamDecoder;
priv.opusDecoder = opts.opusDecoder ?? null;
priv.activeOpusSidecar = opts.sidecar ?? null;
}
/** Read back private fields the recovery sets. */
function priv(player: AudioPlayer): Record<string, unknown> {
return player as unknown as Record<string, unknown>;
}
// A minimal real sidecar (parsed) so the Opus resolver returns a deterministic page offset.
// Index: t=0 -> byte 4096, t=1s -> byte 9000 (granule uses 48 kHz + preSkip).
function makeOpusSidecar(): OpusSeekData {
const setup = [0x4f, 0x70, 0x75, 0x73, 0x48, 0x65, 0x61, 0x64];
const SEEK_INDEX_HEADER_SIZE = 24;
const SEEK_POINT_SIZE = 16;
const preSkip = 312;
const points = [
{ granule: preSkip, byteOffset: 4096 }, // t = 0
{ granule: preSkip + 48000, byteOffset: 9000 }, // t = 1 s
];
const total = 4 + setup.length + SEEK_INDEX_HEADER_SIZE + points.length * SEEK_POINT_SIZE;
const bytes = new Uint8Array(total);
const view = new DataView(bytes.buffer);
view.setUint32(0, setup.length, true);
bytes.set(setup, 4);
let p = 4 + setup.length;
const writeU64 = (off: number, v: number) => {
view.setUint32(off, v >>> 0, true);
view.setUint32(off + 4, Math.floor(v / 0x100000000), true);
};
writeU64(p, 500_000);
view.setFloat64(p + 8, 100, true);
view.setUint32(p + 16, points.length, true);
view.setUint16(p + 20, preSkip, true);
p += SEEK_INDEX_HEADER_SIZE;
for (const pt of points) { writeU64(p, pt.granule); writeU64(p + 8, pt.byteOffset); p += SEEK_POINT_SIZE; }
const parsed = parseSidecar(bytes);
if (!parsed) throw new Error('test setup: sidecar failed to parse');
return parsed;
}
// --- TRIGGER: within-window vs past-tail vs forward ------------------------------------------
// UC3 / AC4: a backward seek INTO the retained tail resolves from buffer — NO seekBeyondBuffer,
// NO refetch signal. Window is [30, 60); target 40 is inside.
test('seek back within retained tail resolves in-buffer (no refetch) — AC4', () => {
const player = makePlayer();
const scheduler = new FakeScheduler(30, 30); // retained window [30, 60)
arm(player, { scheduler, duration: 120, streamDecoder: new FakeStreamDecoder(true, 1000) });
const result = player.seek(40);
assertEqual(result.success, true, 'seek succeeds');
assertEqual(result.seekBeyondBuffer ?? false, false, 'within-window seek does NOT signal a refetch');
// No clearForSeek / no offset request — the retained window served it.
assertEqual(scheduler.clearedForSeek, false, 'no clearForSeek for an in-buffer seek');
});
// UC4 / AC5 (WAV): a backward seek PAST the retained tail signals a refill at the EARLIER resolved
// offset, using the WAV resolver. Window [30, 60); target 10 is before the tail.
test('seek back past retained tail refetches at the WAV-resolved earlier offset — AC5', () => {
const player = makePlayer();
const scheduler = new FakeScheduler(30, 30); // retained window [30, 60)
const wav = new FakeStreamDecoder(true, 2000); // 2000 bytes/sec
arm(player, { scheduler, duration: 120, streamDecoder: wav });
const result = player.seek(10); // earlier than the retained tail start (30)
assertEqual(result.success, true, 'seek succeeds');
assertEqual(result.seekBeyondBuffer, true, 'past-tail back seek signals a refill (window miss)');
assertEqual(wav.requestedOffsetFor, 10, 'WAV resolver consulted for the EARLIER target');
assertEqual(result.byteOffset, 20000, 'refill offset is the WAV-resolved earlier byte offset');
});
// UC4 / AC5 (Opus): the same window miss on the Opus path uses resolveOpusByteOffset over the
// sidecar index (the live seek), not WAV byte math. Target 0.3 s resolves to the t=0 page (4096).
test('seek back past retained tail refetches at the Opus index-resolved offset — AC5', () => {
const player = makePlayer();
const scheduler = new FakeScheduler(30, 30); // retained window [30, 60)
arm(player, {
scheduler,
duration: 100,
opusDecoder: {}, // presence routes seekBeyondBuffer down the Opus branch
sidecar: makeOpusSidecar(),
});
const result = player.seek(0.3); // earlier than the retained tail (30) -> window miss
assertEqual(result.success, true, 'seek succeeds');
assertEqual(result.seekBeyondBuffer, true, 'past-tail back seek signals a refill on Opus too');
assertEqual(result.byteOffset, 4096, 'Opus index resolved the t=0 page start for the earlier target');
// The landing time of the resolved page is captured for the decoder lead-trim (AC9 reuse).
assertEqual(priv(player)._seekLandingTime, 0, 'landing time of the resolved page captured for lead-trim');
});
// UC2/UC5: a forward seek past the decoded end still routes to seekBeyondBuffer forward, unchanged.
test('forward seek past decoded end still routes to seekBeyondBuffer (unchanged)', () => {
const player = makePlayer();
const scheduler = new FakeScheduler(30, 30); // decoded [30, 60)
const wav = new FakeStreamDecoder(true, 1500);
arm(player, { scheduler, duration: 120, streamDecoder: wav });
const result = player.seek(90); // past the decoded end (60)
assertEqual(result.seekBeyondBuffer, true, 'forward-beyond-buffer still signals a fetch');
assertEqual(wav.requestedOffsetFor, 90, 'forward target resolved through the same WAV resolver');
assertEqual(result.byteOffset, 135000, 'forward offset is the resolved later byte offset');
});
// --- AC6: clean-failure recovery -------------------------------------------------------------
// A failed refill must leave the player recoverable: scheduler halted (clearForSeek), offset anchored
// at the seek target, paused-but-loaded — never a starved "playing" scheduler that fires a false end.
test('recoverFromFailedRefill halts the scheduler and leaves a paused-but-loaded state — AC6', () => {
const player = makePlayer();
const scheduler = new FakeScheduler(30, 30);
arm(player, { scheduler, duration: 120, streamDecoder: new FakeStreamDecoder(true, 1000) });
// Simulate the pre-failure "playing" state the drained pre-seek loop leaves behind.
priv(player).isPlaying = true;
priv(player).isPaused = false;
priv(player).streamingStarted = true;
const result = player.recoverFromFailedRefill(15);
assertEqual(result.success, true, 'recovery succeeds');
assertTrue(scheduler.clearedForSeek, 'stale buffers dropped (no false end can fire)');
assertEqual(scheduler.offsetSetTo, 15, 'offset anchored at the seek target for a retry');
assertEqual(priv(player).isPlaying, false, 'not playing after recovery');
assertEqual(priv(player).isPaused, true, 'paused after recovery');
assertEqual(priv(player).pausePosition, 15, 'pause anchor is the seek target');
assertEqual(priv(player).streamingStarted, false, 'streaming flagged not-started for a clean retry');
});
// --- AC6 retry contract: same-target seek after recovery refetches -------------------------
// After recoverFromFailedRefill the scheduler is empty (clearForSeek was called). A seek to
// the SAME position (seekPosition == playbackOffset) must route to seekBeyondBuffer — not
// seekWithinBuffer, which would be a silent no-op against the degenerate [P,P] empty window.
test('same-target seek after recovery routes to seekBeyondBuffer (AC6 retry)', () => {
const player = makePlayer();
const wav = new FakeStreamDecoder(true, 1000);
// Start with a populated window [30, 60), then simulate recovery at position 15:
// clearForSeek empties the scheduler; setPlaybackOffset anchors it to 15.
const scheduler = new FakeScheduler(30, 30);
arm(player, { scheduler, duration: 120, streamDecoder: wav });
// Drive recovery state manually (the same state recoverFromFailedRefill leaves).
player.recoverFromFailedRefill(15);
// At this point: scheduler.hasBuffers() == false, playbackOffset == 15, totalDuration == 0.
// A seek to 15 (the recovery anchor) must refetch, not silently resolve from the empty window.
const result = player.seek(15);
assertEqual(result.success, true, 'seek succeeds after recovery');
assertEqual(result.seekBeyondBuffer, true, 'same-target seek after recovery signals a refetch (AC6 retry)');
assertEqual(wav.requestedOffsetFor, 15, 'WAV resolver used for the retry offset');
});
// AC4 not regressed: a seek within a POPULATED retained window still resolves from buffer.
// This is the same test as the existing AC4 test but named explicitly to confirm the
// hasBuffers() guard does not affect the populated case.
test('seek within populated retained window still resolves in-buffer — AC4 not regressed', () => {
const player = makePlayer();
// Populated window [30, 60) — hasBuffers() starts true (total=30 > 0).
const scheduler = new FakeScheduler(30, 30);
arm(player, { scheduler, duration: 120, streamDecoder: new FakeStreamDecoder(true, 1000) });
const result = player.seek(45); // inside [30, 60)
assertEqual(result.success, true, 'seek succeeds');
assertEqual(result.seekBeyondBuffer ?? false, false, 'populated in-window seek does NOT signal a refetch');
assertEqual(scheduler.clearedForSeek, false, 'scheduler not cleared for an in-buffer seek (no refetch)');
});
// --- run -------------------------------------------------------------------------------------
if (failures.length > 0) {
console.error(failures.join('\n'));
console.error(`\n${failures.length} FAILED, ${passed} passed`);
process.exit(1);
} else {
console.log(`ALL ${passed} TESTS PASSED`);
}
+300 -24
View File
@@ -11,9 +11,12 @@ import { AudioContextManager } from './AudioContextManager.js';
import { StreamDecoder } from './StreamDecoder.js';
import { PlaybackScheduler } from './PlaybackScheduler.js';
import { IFormatDecoder } from './IFormatDecoder.js';
import { IStreamingDecoder } from './IStreamingDecoder.js';
import { WavFormatDecoder } from './WavFormatDecoder.js';
import { Mp3FormatDecoder } from './Mp3FormatDecoder.js';
import { FlacFormatDecoder } from './FlacFormatDecoder.js';
import { OpusStreamDecoder } from './OpusStreamDecoder.js';
import { OpusSeekData, parseSidecar, resolveOpusByteOffset, OpusSeekResolution, OPUS_SAMPLE_RATE } from './OpusSidecar.js';
export interface AudioResult {
success: boolean;
@@ -27,6 +30,10 @@ export interface StreamingResult extends AudioResult {
headerParsed?: boolean;
bufferCount?: number;
duration?: number;
// Phase 21.2a back-pressure signal piggybacked on the chunk result the C# read loop already
// awaits — true means the scheduler's forward fill is over the high-water mark and the loop
// should stop calling ReadAsync until it drains (no extra interop hop in the common case).
productionPaused?: boolean;
}
export interface AudioState {
@@ -45,6 +52,19 @@ export class AudioPlayer {
private streamDecoder: StreamDecoder;
private scheduler: PlaybackScheduler;
// The Opus WebCodecs decode path (IStreamingDecoder seam), used INSTEAD of streamDecoder when the
// active stream is Ogg Opus. Null for WAV/MP3/FLAC, which keep the streamDecoder path unchanged.
// Holding both is deliberate: the change is the decode stage only; the same scheduler/Web Audio
// graph feeds from whichever decoder is active for the current stream.
private opusDecoder: IStreamingDecoder | null = null;
// The sidecar in effect for the active Opus stream (its seek index resolves byte offsets). Distinct
// from pendingOpusSidecar, which is the one set for the NEXT stream init.
private activeOpusSidecar: OpusSeekData | null = null;
// The landing time of the most recent seek-beyond-buffer page resolution (seconds). Set by
// seekBeyondBuffer, consumed by reinitializeFromOffset to trim leading decoded frames so the
// audible position matches the requested seek target (AC9 fine re-sync, §3.4a step 4).
private _seekLandingTime: number = 0;
// Playback state
private isPlaying: boolean = false;
private isPaused: boolean = false;
@@ -62,6 +82,11 @@ export class AudioPlayer {
private onEndCallback: EndCallback | null = null;
private progressInterval: number | null = null;
// Pending Opus sidecar (setup header + seek index), parsed from the one-time sidecar fetch and
// applied to the OpusFormatDecoder when the next Opus stream initializes. Wave 18.5 sets this
// (via setOpusSidecar) before initializeStreaming; this class never fetches it.
private pendingOpusSidecar: OpusSeekData | null = null;
constructor() {
this.contextManager = new AudioContextManager();
this.streamDecoder = new StreamDecoder(this.contextManager);
@@ -93,17 +118,53 @@ export class AudioPlayer {
// ==================== Streaming ====================
initializeStreaming(totalStreamLength: number, contentType: string): AudioResult {
async initializeStreaming(totalStreamLength: number, contentType: string): Promise<AudioResult> {
try {
// Full cleanup before starting new stream
this.stopProgressTracking();
this.scheduler.clear();
this.streamDecoder.reset();
this.disposeOpusDecoder();
this.resetState();
// Initialize new stream with the format decoder selected from Content-Type.
this.isStreamingMode = true;
const formatDecoder = AudioPlayer.createFormatDecoder(contentType);
// Opus routes to the WebCodecs streaming seam (IStreamingDecoder); WAV/MP3/FLAC keep the
// StreamDecoder wrap-and-decode path byte-for-byte. The sidecar (setup header + seek index)
// must already be set (setOpusSidecar, before init) — without it Opus cannot be decoded or
// seeked, so we fall back by leaving opusDecoder null and using the StreamDecoder path,
// which the server's C2 fallback (lossless bytes) matches. In practice the C# resolver only
// selects Opus when the sidecar parsed, so the null branch is defensive.
if (this.isOpusContentType(contentType) && this.pendingOpusSidecar) {
this.activeOpusSidecar = this.pendingOpusSidecar;
// Align the AudioContext to 48 kHz NOW, before any Opus bytes flow — the format is
// already resolved (C# resolves Opus + injects the sidecar before this call), so the
// target rate is known up front. Done here, the decoder's own lazy
// recreateWithSampleRate(48000) in ensureConfigured hits its sampleRate-equal early
// return and is a no-op; the live graph is never close()'d and rebuilt mid-decode (the
// teardown that double-decoded the stream and OOM'd the tab with HW accel off). The
// recreate seam itself stays — it is the WAV path's mechanism for non-44.1 sources and
// remains the defensive backstop here.
if (this.contextManager.sampleRate !== OPUS_SAMPLE_RATE) {
await this.contextManager.recreateWithSampleRate(OPUS_SAMPLE_RATE);
}
// Pass the shared back-pressure signal (21.2b): the Opus decoder stops demuxing/
// decoding new packets while the scheduler is full, so the WebCodecs decode queue
// and decodedQueue do not balloon behind a throttled socket (OQ7). Same signal the
// C# read loop honors — one policy, two thin hooks.
this.opusDecoder = new OpusStreamDecoder(
this.contextManager,
this.pendingOpusSidecar,
() => this.scheduler.evaluateProductionPause());
return { success: true };
}
// Non-Opus (or Opus-without-sidecar): the existing StreamDecoder path, unchanged. The
// context sample rate is untouched here, so the WAV/lossless path is byte-for-byte
// unaffected by the Opus up-front alignment above.
const formatDecoder = this.createFormatDecoder(contentType);
this.streamDecoder.initialize(totalStreamLength, formatDecoder);
return { success: true };
} catch (error) {
@@ -111,10 +172,41 @@ export class AudioPlayer {
}
}
private isOpusContentType(contentType: string): boolean {
return contentType.includes('audio/ogg') || contentType.includes('audio/opus');
}
private disposeOpusDecoder(): void {
if (this.opusDecoder) {
this.opusDecoder.dispose();
this.opusDecoder = null;
}
this.activeOpusSidecar = null;
}
/**
* Select a format decoder from the response Content-Type.
* Inject the Opus sidecar (setup header + seek index) for the next Opus stream. Wave 18.5 calls
* this with the raw sidecar bytes (from its one-time HTTP fetch) BEFORE initializeStreaming; the
* parsed result is applied to the OpusFormatDecoder when the stream initializes. This is the
* injection seam the player owns no transport, only the parse + hand-off.
*
* @returns success:false with an error if the bytes are not a valid sidecar blob.
*/
private static createFormatDecoder(contentType: string): IFormatDecoder {
setOpusSidecar(sidecarBytes: Uint8Array): AudioResult {
const parsed = parseSidecar(sidecarBytes);
if (!parsed) {
return { success: false, error: 'Invalid Opus sidecar blob' };
}
this.pendingOpusSidecar = parsed;
return { success: true };
}
/**
* Select a format decoder from the response Content-Type for the StreamDecoder (wrap-and-decode)
* path. Opus is NOT handled here it routes to the WebCodecs IStreamingDecoder seam in
* initializeStreaming. This factory serves WAV/MP3/FLAC only.
*/
private createFormatDecoder(contentType: string): IFormatDecoder {
if (contentType.includes('audio/mpeg') || contentType.includes('audio/mp3')) {
return new Mp3FormatDecoder();
}
@@ -133,16 +225,24 @@ export class AudioPlayer {
*/
async markStreamComplete(): Promise<StreamingResult> {
try {
const results = await this.streamDecoder.markStreamComplete();
const results = this.opusDecoder
? await this.opusDecoder.complete()
: (await this.streamDecoder.markStreamComplete()).map(r => r.buffer);
if (results.length > 0) {
for (const result of results) {
this.scheduler.addBuffer(result.buffer);
for (const buffer of results) {
this.scheduler.addBuffer(buffer);
}
if (this.streamingStarted && this.isPlaying) {
this.scheduler.scheduleNewBuffers();
}
}
this.streamingCompleted = true;
// Hand the genuine-end signal to the scheduler AFTER the tail buffers are added and
// scheduled: now an empty scheduled queue is a real end-of-track, not a startup gap, so
// the scheduler may fire onPlaybackEnded when its queue drains. If the queue was already
// empty at this point (the tail produced no buffers, or they were already played),
// setStreamComplete finalises immediately.
this.scheduler.setStreamComplete(true);
return { success: true, bufferCount: this.scheduler.getBufferCount() };
} catch (error) {
return { success: false, error: (error as Error).message };
@@ -150,6 +250,66 @@ export class AudioPlayer {
}
async processStreamingChunk(chunk: Uint8Array): Promise<StreamingResult> {
return this.opusDecoder
? this.processOpusChunk(chunk)
: this.processFormatChunk(chunk);
}
/** Opus (WebCodecs) chunk path. Mirrors processFormatChunk's add->schedule->report shape. */
private async processOpusChunk(chunk: Uint8Array): Promise<StreamingResult> {
try {
const decoder = this.opusDecoder!;
const buffers = await decoder.push(chunk);
// Duration is known up front from the sidecar — surface it as soon as the decoder reports it,
// NOT gated on the first decoded buffers. The C# layer locks Duration on the first chunk whose
// result carries a value (the `Duration == null` guard), and WebCodecs decode is async, so the
// earliest chunks can return zero buffers; gating duration on buffers means C# captures the
// initial 0 and never overwrites it — the WAV header path sets duration on chunk 1 because its
// header parses synchronously, which is the asymmetry this closes. Set once so a seek (which
// reinitialises the decoder) cannot overwrite it.
if (this.duration === 0 && decoder.totalDuration) {
this.duration = decoder.totalDuration;
}
if (buffers.length > 0) {
for (const buffer of buffers) {
this.scheduler.addBuffer(buffer);
}
if (this.streamingStarted && this.isPlaying) {
this.scheduler.scheduleNewBuffers();
}
}
if (decoder.hasFatalError) {
return { success: false, error: 'Opus decode failed' };
}
// "headerParsed" maps to the decoder being configured (codec ready). canStart needs a
// healthy decoded lead before first playback — measured in SECONDS, not a buffer count.
// An Opus WebCodecs packet is ~20 ms, so the WAV-tuned 6-BUFFER minimum is only ~0.12 s of
// lead: playback would start, drain it before the async decode ramps, and underrun
// immediately. The seconds-based lead gate (same threshold the scheduler's underrun-resume
// hysteresis uses) gives Opus the cushion its decode ramp needs. WAV keeps the buffer-count
// gate below — its large synchronous segments rarely underrun and its start must not change.
const headerParsed = decoder.ready;
const canStart = headerParsed && this.scheduler.hasMinimumPlaybackLead();
return {
success: true,
canStartStreaming: canStart,
headerParsed,
bufferCount: this.scheduler.getBufferCount(),
duration: this.duration,
productionPaused: this.scheduler.evaluateProductionPause()
};
} catch (error) {
return { success: false, error: (error as Error).message };
}
}
/** WAV/MP3/FLAC (StreamDecoder) chunk path — unchanged from before the Opus seam split. */
private async processFormatChunk(chunk: Uint8Array): Promise<StreamingResult> {
try {
const results = await this.streamDecoder.processChunk(chunk);
@@ -172,9 +332,13 @@ export class AudioPlayer {
}
}
// Check if streaming is complete
// Check if streaming is complete. The StreamDecoder self-detects completion by byte
// count (WAV/MP3/FLAC); propagate that to the scheduler so a drained queue past this
// point is treated as a genuine end. Buffers from this chunk were already added above,
// so any final end fires through handleSourceEnded when they drain.
if (this.streamDecoder.isComplete) {
this.streamingCompleted = true;
this.scheduler.setStreamComplete(true);
}
const canStart = this.streamDecoder.headerParsed &&
@@ -185,7 +349,8 @@ export class AudioPlayer {
canStartStreaming: canStart,
headerParsed: this.streamDecoder.headerParsed,
bufferCount: this.scheduler.getBufferCount(),
duration: this.duration
duration: this.duration,
productionPaused: this.scheduler.evaluateProductionPause()
};
} catch (error) {
return { success: false, error: (error as Error).message };
@@ -278,6 +443,7 @@ export class AudioPlayer {
try {
this.scheduler.clear();
this.streamDecoder.reset();
this.disposeOpusDecoder();
this.resetState();
this.stopProgressTracking();
@@ -296,18 +462,36 @@ export class AudioPlayer {
return { success: false, error: 'Invalid seek position' };
}
// bufferStart is the absolute track time at which buffers[0] begins. Under Phase 21.1
// partial eviction this is the start of the RETAINED BACK-WINDOW TAIL — eviction advances
// playbackOffset as it drops played buffers off the front — so [bufferStart, bufferEnd] is
// exactly the window currently held in memory.
const bufferStart = this.scheduler.getPlaybackOffset();
const bufferEnd = this.scheduler.getTotalDuration() + bufferStart;
// Position must be within [bufferStart, bufferEnd] to use buffered content.
// A lower-bound check is required: after a seek-beyond-buffer, bufferStart is
// set to the prior seek position. Seeking to a position below bufferStart would
// produce a negative bufferRelativePosition in seekWithinBuffer, silently
// clamping to position 0 of the offset buffer instead of the requested time.
if (position >= bufferStart && position <= bufferEnd) {
// The window-miss test for BOTH directions, and the 21.3 refill trigger for backward seeks.
// Position must be within [bufferStart, bufferEnd] AND the scheduler must hold buffers to
// resolve from the retained window:
// - position >= bufferStart AND hasBuffers : UC3 — seek back within the retained back-window.
// Served from buffer with NO network refetch. (The lower bound is load-bearing: after
// eviction or a prior seek-beyond-buffer, bufferStart > 0, and a target below it would
// otherwise produce a negative bufferRelativePosition in seekWithinBuffer, silently clamping
// to position 0.)
// - position < bufferStart : UC4 — seek back PAST the retained tail (the window was evicted).
// Falls through to seekBeyondBuffer, which is the existing Range path run toward an EARLIER
// offset. This is the 21.3 window-miss refill: "a seek the listener didn't initiate" reuses
// the same per-path resolver + reinit a forward seek-beyond-buffer uses, no new mechanism.
// - position > bufferEnd : UC2/UC5 — forward seek beyond buffer, unchanged.
// - !hasBuffers (degenerate [P,P] window post-recovery): the window check above would
// spuriously route ANY target to seekWithinBuffer (bufferStart==bufferEnd==seekPosition
// after recoverFromFailedRefill). Force seekBeyondBuffer so a same-target retry actually
// refetches (AC6 retry contract). The !hasBuffers guard only fires in the degenerate case —
// a populated retained window has buffers and is unaffected (AC4 not regressed).
if (position >= bufferStart && position <= bufferEnd && this.scheduler.hasBuffers()) {
return this.seekWithinBuffer(position);
} else {
// Seeking outside buffered window - signal C# to fetch new stream
// Seeking outside the retained window, or to any position in an empty scheduler —
// signal C# to fetch a new stream from the resolved offset.
return this.seekBeyondBuffer(position);
}
}
@@ -339,8 +523,25 @@ export class AudioPlayer {
*/
private seekBeyondBuffer(position: number): AudioResult {
try {
// The header must be parsed for byte-offset math; without it we cannot
// build a valid Range request.
// Opus: resolve the offset from the precomputed seek index (the accurate VBR-safe transfer
// function). The returned offset is a real page start, so the Range continuation lands the
// demuxer/decoder Ogg-sync-aligned. Also capture the landing time (t_page ≤ position) so
// reinitializeFromOffset can trim the leading decoded frames and land precisely at `position`
// (AC9 fine re-sync, §3.4a step 4).
if (this.opusDecoder) {
if (!this.activeOpusSidecar) {
return { success: false, error: 'Cannot calculate byte offset' };
}
const resolution: OpusSeekResolution = resolveOpusByteOffset(this.activeOpusSidecar, position);
this._seekLandingTime = resolution.landingTimeSeconds;
return {
success: true,
seekBeyondBuffer: true,
byteOffset: resolution.byteOffset
};
}
// WAV/MP3/FLAC: the header must be parsed for byte-offset math.
if (!this.streamDecoder.getFormatInfo()) {
return { success: false, error: 'Cannot calculate byte offset' };
}
@@ -361,6 +562,22 @@ export class AudioPlayer {
}
}
/**
* Resolve the file-absolute byte offset to begin a stream at `position`, WITHOUT requiring active
* playback or buffered audio (the "load at timestamp" entry point Phase 18 wave 18.6 format switch).
* Unlike seek(), it has no duration guard and never routes to the within-buffer path: a fresh load has
* no scheduler window, so the answer is always "start the byte stream here". For Opus the sidecar
* resolves the offset (and captures the page landing time for the lead-trim) immediately after init; for
* WAV the header must already be parsed (feed the byte-0 segment first). Returns success:false when the
* decoder cannot yet resolve an offset (no header / no sidecar), so the caller can probe and retry.
*/
resolveStreamOffset(position: number): AudioResult {
if (!this.isStreamingMode) {
return { success: false, error: 'Not in streaming mode' };
}
return this.seekBeyondBuffer(position);
}
/**
* Get the total buffered duration (for C# to check if seek is within buffer)
*/
@@ -368,10 +585,26 @@ export class AudioPlayer {
return this.scheduler.getTotalDuration() + this.scheduler.getPlaybackOffset();
}
/**
* The shared back-pressure signal (Phase 21.2a), polled by the C# read loop WHILE it is
* already throttled to learn when the forward fill has drained below the low-water mark and it
* may resume reading. The steady-state (unthrottled) loop never calls this it reads the
* piggybacked productionPaused flag off each chunk result instead, so there is no extra
* interop hop until back-pressure actually engages.
*/
isProductionPaused(): boolean {
return this.scheduler.evaluateProductionPause();
}
/**
* Calculate byte offset for a time position (for C# layer)
*/
calculateByteOffset(positionSeconds: number): number {
if (this.opusDecoder) {
return this.activeOpusSidecar
? resolveOpusByteOffset(this.activeOpusSidecar, positionSeconds).byteOffset
: 0;
}
if (!this.streamDecoder.getFormatInfo()) return 0;
return this.streamDecoder.calculateByteOffset(positionSeconds);
}
@@ -384,17 +617,21 @@ export class AudioPlayer {
try {
// Stop current playback
this.stopProgressTracking();
const wasPlaying = this.isPlaying;
this.isPlaying = false;
// Clear buffers and set new offset
this.scheduler.clearForSeek();
this.scheduler.setPlaybackOffset(seekPosition);
// Reinitialize decoder for the Range-continuation stream. totalStreamLength
// here is the 206 Content-Length (range start → EOF), not the full file size —
// the decoder uses it to detect stream-complete against raw audio bytes.
this.streamDecoder.reinitializeForRangeContinuation(totalStreamLength);
// Reinitialize the active decoder for the Range-continuation stream (206 body, no header/
// setup pages). Opus resets demux + codec state (keeping the cached config) and arms the
// lead-trim so decoded audio starts at `seekPosition`, not at the page boundary (AC9). The
// StreamDecoder path uses totalStreamLength (the 206 Content-Length) to detect completion.
if (this.opusDecoder) {
this.opusDecoder.reinitializeForRangeContinuation(this._seekLandingTime, seekPosition);
} else {
this.streamDecoder.reinitializeForRangeContinuation(totalStreamLength);
}
// Update state
this.pausePosition = seekPosition;
@@ -407,6 +644,44 @@ export class AudioPlayer {
}
}
/**
* Recover the player into a clean, paused-but-loaded state after a window-miss REFILL failed
* (Phase 21.3 / AC6). A refill is "a seek the listener didn't initiate"; when its Range fetch or
* reinit fails mid-stream, the pre-seek loop has already been cancelled and drained, but the
* scheduler is still holding stale pre-seek buffers and is still `isActive_`. Left alone it would
* play the retained tail to exhaustion and fire `onPlaybackEnded` a SILENT FALSE END (the
* "wedged playing with a starved scheduler" AC6 forbids).
*
* The recovery mirrors `PlaybackScheduler.playFromPosition`'s end-of-buffer recovery in spirit:
* stop pretending to play. We stop all sources and clear the buffers for a seek (clearForSeek
* keeps no stale audio but is ready to accept a fresh continuation), set the offset to the
* requested seek position, and leave the player paused there. The track stays loaded so the
* listener can retry the seek or pick another track no new transport control, only a recoverable
* stop (C4). A subsequent seek to the same target re-enters seekBeyondBuffer cleanly because the
* offset names the seek position and the scheduler is empty (so it routes to a fresh fetch).
*
* @param seekPosition The seek target the failed refill was aiming for; becomes the resume anchor.
*/
recoverFromFailedRefill(seekPosition: number): AudioResult {
try {
this.stopProgressTracking();
// Halt the starved scheduler and drop the stale pre-seek buffers so no false end can fire.
this.scheduler.clearForSeek();
this.scheduler.setPlaybackOffset(seekPosition);
// Paused-but-loaded: not playing, not mid-seek-stream. pausePosition anchors a retry.
this.isPlaying = false;
this.isPaused = true;
this.pausePosition = seekPosition;
this.streamingStarted = false;
this.streamingCompleted = false;
return { success: true };
} catch (error) {
return { success: false, error: (error as Error).message };
}
}
// ==================== Volume ====================
setVolume(volume: number): AudioResult {
@@ -510,6 +785,7 @@ export class AudioPlayer {
this.isStreamingMode = false;
this.streamingStarted = false;
this.streamingCompleted = false;
this._seekLandingTime = 0;
}
private handlePlaybackEnded(): void {
@@ -36,6 +36,8 @@ export interface FormatInfo {
* MP3 VBR: Xing/VBRI TOC (100-entry Uint8Array, values are file-percentage * 255).
* FLAC: SeekTable (array of {sampleNumber: number, streamOffset: number} stream_offset
* is bytes from the start of audio frames, i.e. after all metadata blocks).
* Opus does NOT flow through this seam it uses the WebCodecs IStreamingDecoder path and resolves
* seek offsets via OpusSidecar.resolveOpusByteOffset, not FormatInfo.seekData.
*/
seekData?: Mp3VbrSeekData | FlacSeekData | null;
}
@@ -0,0 +1,63 @@
/**
* IStreamingDecoder - the stateful streaming-decode seam, parallel to IFormatDecoder.
*
* Why a second seam. `IFormatDecoder` (WAV/MP3/FLAC) is a *wrap-and-decode-each-segment* strategy:
* `StreamDecoder` cuts the stream into independently-decodable segments, `wrapSegment` makes each a
* standalone file, and `decodeAudioData` decodes each in isolation. That model is correct for raw PCM
* (WAV) and independently-decodable frames (FLAC), but it is fundamentally wrong for Opus: Opus has
* pre-skip (encoder delay) and inter-frame state (MDCT overlap-add, SILK/CELT continuity), so decoding
* page-runs independently re-applies the pre-skip and starts from cold codec state at every boundary
* audible glitching and a broken timeline.
*
* A WebCodecs `AudioDecoder` is the right tool: one stateful decoder fed packets sequentially, decoding
* continuously with correct pre-skip-once handling and full inter-frame continuity. But it does NOT fit
* `IFormatDecoder` it is async/callback-driven and owns its own buffering. So Opus gets this seam
* instead. `AudioPlayer` dispatches by content-type: WAV/MP3/FLAC keep the `StreamDecoder` path
* byte-for-byte; Opus routes here. Both feed the SAME `PlaybackScheduler` the change is the decode
* stage only, never the schedule/playback stage.
*
* The seam is intentionally minimal and mirrors the lifecycle `StreamDecoder` already exposes so
* `AudioPlayer` can treat the two uniformly: initialize -> push chunks -> mark complete, plus a
* range-continuation reinit for seek-beyond-buffer.
*/
export interface IStreamingDecoder {
/**
* Decoded buffers ready to schedule, drained by AudioPlayer after each push/flush. Each entry is a
* standard AudioBuffer at the AudioContext's sample rate, ready for PlaybackScheduler.addBuffer.
*/
readonly hasFatalError: boolean;
/** True once the decoder has enough to begin playback (header/config established). */
readonly ready: boolean;
/** Total stream duration in seconds if known up front (Opus knows it from the sidecar), else null. */
readonly totalDuration: number | null;
/**
* Push raw stream bytes. Returns decoded AudioBuffers that became ready (possibly empty WebCodecs
* decode is async, so a push may return nothing and a later push returns several).
*/
push(chunk: Uint8Array): Promise<AudioBuffer[]>;
/**
* Signal end-of-stream. Flushes the decoder and returns any residual decoded buffers (including the
* end-trimmed final buffer).
*/
complete(): Promise<AudioBuffer[]>;
/**
* Reinitialize for a Range-continuation after seek-beyond-buffer. The 206 body begins on an Ogg page
* boundary and carries no setup pages the decoder reuses the cached config and resets demux/codec
* state so inter-frame continuity restarts cleanly from the new offset.
*
* @param landingTimeSeconds The actual presentation time of the resolved seek page (t_page target).
* @param targetTimeSeconds The user-requested seek position. The decoder trims the leading
* `(target - landing) * sampleRate` frames so playback lands at target
* (AC9 fine re-sync, §3.4a step 4).
*/
reinitializeForRangeContinuation(landingTimeSeconds: number, targetTimeSeconds: number): void;
/** Tear down the underlying WebCodecs decoder and release resources. */
dispose(): void;
}
+298
View File
@@ -0,0 +1,298 @@
/**
* OggDemuxer - streaming Ogg-page -> Opus-packet demuxer for the WebCodecs decode path.
*
* Ogg Opus is a containerized, paged format. To feed a WebCodecs `AudioDecoder` we must extract the
* individual Opus *packets* from the Ogg container the decoder takes packets (as `EncodedAudioChunk`s),
* not raw container bytes. This module is the client-side analogue of the C# `OggOpusParser`: it reads
* the page structure directly (the "OggS" capture pattern + the 27-byte page header + segment table) and
* reassembles packets across the lacing, tracking the granule position that gives each packet its time.
*
* It is deliberately *streaming*: `push(bytes)` accepts arbitrary network chunks (a packet, a page, or a
* fraction of either) and returns whatever WHOLE packets have become available, holding partial state
* across calls. This matches how `StreamAudioWithEarlyPlayback` feeds bytes in adaptive 1664 KB chunks.
*
* Lacing rules (RFC 3533 §6): a page's segment table lists N segment lengths (0255). A packet is the
* concatenation of consecutive segments up to and including the first segment whose length is < 255. A
* segment of exactly 255 means "this packet continues into the next segment" and if it is the page's
* LAST segment, the packet continues into the next page (the next page's header-type has the
* continuation flag set). The granule position on a page is the end-granule of the LAST packet that
* *completes* on that page.
*
* The two leading setup packets (OpusHead, OpusTags) are NOT audio and are skipped they configure the
* decoder (the sidecar carries them as the codec description), they are never decoded as audio packets.
*/
const OGG_CAPTURE = [0x4f, 0x67, 0x67, 0x53]; // "OggS"
const OGG_PAGE_HEADER_SIZE = 27;
const GRANULE_OFFSET = 6; // 64-bit granule position within the page header
const HEADER_TYPE_OFFSET = 5; // bit 0x01 = continued packet, 0x02 = BOS, 0x04 = EOS
const SEGMENT_COUNT_OFFSET = 26; // number of segment-table entries
const CONTINUATION_FLAG = 0x01;
const OPUS_HEAD_SIG = [0x4f, 0x70, 0x75, 0x73, 0x48, 0x65, 0x61, 0x64]; // "OpusHead"
const OPUS_TAGS_SIG = [0x4f, 0x70, 0x75, 0x73, 0x54, 0x61, 0x67, 0x73]; // "OpusTags"
/** A demuxed Opus audio packet plus the timing context needed to schedule and trim it. */
export interface OpusPacket {
/** Raw Opus packet bytes (one Opus frame's worth — fed straight to the AudioDecoder). */
data: Uint8Array;
/**
* The end-granule of the page this packet completed on, or null if the page carried no usable
* granule (mid-stream pages between completion points share the next completing page's granule
* we attach the granule only to the packet that completes on a granule-bearing page). A 48 kHz
* sample count; presentation time = (granule - preSkip) / 48000.
*/
pageGranule: number | null;
/** True when this packet completed on the stream's final (EOS) page — drives end-trim. */
isLastPage: boolean;
}
/** Read a little-endian uint64 as a JS number (exact to 2^53 — far beyond any real granule). */
function readUint64LE(buf: Uint8Array, offset: number): number {
let lo = 0;
let hi = 0;
for (let i = 0; i < 4; i++) lo += buf[offset + i] * 2 ** (8 * i);
for (let i = 0; i < 4; i++) hi += buf[offset + 4 + i] * 2 ** (8 * i);
return hi * 0x100000000 + lo;
}
function startsWith(buf: Uint8Array, sig: number[]): boolean {
if (buf.length < sig.length) return false;
for (let i = 0; i < sig.length; i++) if (buf[i] !== sig[i]) return false;
return true;
}
export class OggDemuxer {
// Unconsumed raw bytes carried across push() calls (a page may straddle a network-chunk boundary).
private pending: Uint8Array = new Uint8Array(0);
// Bytes of a packet that spans pages (255-length last segment + continuation flag next page).
private partialPacket: Uint8Array[] = [];
// Once both setup packets are seen, every subsequent packet is audio.
private setupPacketsSeen = 0;
/**
* Feed raw stream bytes (any size). Returns all WHOLE Opus AUDIO packets that became decodable,
* in order. Setup packets (OpusHead/OpusTags) are consumed and skipped. Incomplete trailing bytes
* are retained for the next push.
*/
push(bytes: Uint8Array): OpusPacket[] {
this.pending = this.concat(this.pending, bytes);
return this.drainPages();
}
/**
* Reset to a fresh stream. Used on seek/range-continuation: the new 206 body begins on a page
* boundary, so all partial-packet and pending state must be dropped. setupPacketsSeen is reset to
* 2 (already configured) for a continuation a mid-stream slice carries no setup pages, only audio
* pages so the demuxer treats the first page's packets as audio immediately.
*/
reset(isContinuation: boolean): void {
this.pending = new Uint8Array(0);
this.partialPacket = [];
this.setupPacketsSeen = isContinuation ? 2 : 0;
}
private drainPages(): OpusPacket[] {
const packets: OpusPacket[] = [];
for (;;) {
const page = this.tryReadPage();
if (!page) break;
this.parsePage(page, packets);
}
return packets;
}
/**
* Try to slice one complete Ogg page off the front of `pending`. Returns null (and leaves `pending`
* intact) when a whole page is not yet buffered. Resynchronises by scanning for "OggS" if `pending`
* does not start on a page boundary (defensive the encoder writes contiguous pages, but a
* continuation stream could in theory begin mid-garbage; the seek offset is always a page start).
*/
private tryReadPage(): { header: Uint8Array; segTable: Uint8Array; payload: Uint8Array; total: number } | null {
const buf = this.pending;
if (buf.length < OGG_PAGE_HEADER_SIZE) return null;
// Resync: ensure we are positioned at a capture pattern.
if (!startsWith(buf, OGG_CAPTURE)) {
const sync = this.findCapture(buf, 0);
if (sync < 0) {
// No capture pattern at all — keep only the last 3 bytes (a capture could straddle).
this.pending = buf.subarray(Math.max(0, buf.length - 3));
return null;
}
this.pending = buf.subarray(sync);
return this.tryReadPage();
}
const segCount = buf[SEGMENT_COUNT_OFFSET];
const segTableEnd = OGG_PAGE_HEADER_SIZE + segCount;
if (buf.length < segTableEnd) return null; // segment table not fully buffered yet
const segTable = buf.subarray(OGG_PAGE_HEADER_SIZE, segTableEnd);
let payloadSize = 0;
for (let i = 0; i < segCount; i++) payloadSize += segTable[i];
const total = segTableEnd + payloadSize;
if (buf.length < total) return null; // payload not fully buffered yet
const header = buf.subarray(0, OGG_PAGE_HEADER_SIZE);
const payload = buf.subarray(segTableEnd, total);
// Advance past this page.
this.pending = buf.subarray(total);
return { header, segTable, payload, total };
}
private parsePage(
page: { header: Uint8Array; segTable: Uint8Array; payload: Uint8Array; total: number },
out: OpusPacket[]
): void {
const { header, segTable, payload } = page;
const headerType = header[HEADER_TYPE_OFFSET];
const continued = (headerType & CONTINUATION_FLAG) !== 0;
const isEos = (headerType & 0x04) !== 0;
const granule = readUint64LE(header, GRANULE_OFFSET);
// 0xFFFFFFFFFFFFFFFF (-1) means "no packet completed on this page" — no usable timestamp.
// We check the raw bytes rather than comparing `granule === -1` (or the equivalent JS number):
// the full 64-bit sentinel exceeds 2^53 and cannot be represented exactly as an IEEE-754 double,
// so the parsed value from readUint64LE would not equal the sentinel. The byte check is exact.
const hasGranule = !(header[GRANULE_OFFSET] === 0xff && header[GRANULE_OFFSET + 1] === 0xff &&
header[GRANULE_OFFSET + 2] === 0xff && header[GRANULE_OFFSET + 3] === 0xff &&
header[GRANULE_OFFSET + 4] === 0xff && header[GRANULE_OFFSET + 5] === 0xff &&
header[GRANULE_OFFSET + 6] === 0xff && header[GRANULE_OFFSET + 7] === 0xff);
// If this page does NOT begin with a continuation, any half-built packet from a prior page is
// orphaned (should not happen in a well-formed stream, but never carry garbage forward).
if (!continued) this.partialPacket = [];
// Walk the segment table, reassembling packets. A packet ends at the first segment < 255.
const completedPackets: Uint8Array[] = [];
let segStart = 0;
let cursor = 0;
for (let i = 0; i < segTable.length; i++) {
const len = segTable[i];
cursor += len;
if (len < 255) {
// Packet boundary: segments [segStart, cursor) form (the tail of) a packet.
const slice = payload.subarray(segStart, cursor);
if (this.partialPacket.length > 0) {
this.partialPacket.push(slice);
completedPackets.push(this.flattenPartial());
this.partialPacket = [];
} else {
completedPackets.push(slice);
}
segStart = cursor;
}
// len === 255 with i === last segment -> packet spans into the next page (handled below).
}
// Any trailing 255-run that did not terminate is a packet continuing into the next page.
if (segStart < cursor) {
this.partialPacket.push(payload.subarray(segStart, cursor));
}
// Classify completed packets: the first two whole packets in the whole stream are the setup
// packets (OpusHead, OpusTags) and are skipped. Everything after is audio. The page granule is
// attached to the LAST completing audio packet on a granule-bearing page (the granule is that
// page's end-granule per RFC 7845).
for (let p = 0; p < completedPackets.length; p++) {
const pkt = completedPackets[p];
if (this.setupPacketsSeen < 2) {
// Only count packets that are actually the Opus setup headers; guard against a stray
// first audio packet being mistaken for setup on a continuation (reset handles that).
if (this.setupPacketsSeen === 0 && startsWith(pkt, OPUS_HEAD_SIG)) {
this.setupPacketsSeen = 1;
continue;
}
if (this.setupPacketsSeen === 1 && startsWith(pkt, OPUS_TAGS_SIG)) {
this.setupPacketsSeen = 2;
continue;
}
// Not a recognised setup packet while we still expected one — treat as audio (a
// continuation slice that began mid-stream). Fall through.
}
const isLastCompleting = p === completedPackets.length - 1;
out.push({
data: pkt,
pageGranule: hasGranule && isLastCompleting ? granule : null,
isLastPage: isEos
});
}
}
private flattenPartial(): Uint8Array {
if (this.partialPacket.length === 1) return this.partialPacket[0];
let len = 0;
for (const s of this.partialPacket) len += s.length;
const out = new Uint8Array(len);
let o = 0;
for (const s of this.partialPacket) {
out.set(s, o);
o += s.length;
}
return out;
}
private findCapture(buf: Uint8Array, from: number): number {
for (let i = from; i + 4 <= buf.length; i++) {
if (buf[i] === OGG_CAPTURE[0] && buf[i + 1] === OGG_CAPTURE[1] &&
buf[i + 2] === OGG_CAPTURE[2] && buf[i + 3] === OGG_CAPTURE[3]) {
return i;
}
}
return -1;
}
private concat(a: Uint8Array, b: Uint8Array): Uint8Array {
if (a.length === 0) return b;
if (b.length === 0) return a;
const out = new Uint8Array(a.length + b.length);
out.set(a, 0);
out.set(b, a.length);
return out;
}
}
/**
* Extract the raw OpusHead identification-header *packet* from the sidecar's setup-header bytes (which
* are the verbatim Ogg PAGES wrapping OpusHead + OpusTags). WebCodecs' `AudioDecoderConfig.description`
* for Opus is the OpusHead packet (RFC 7845 §5.1), not the Ogg page so we demux the setup pages and
* return the first packet's bytes. Returns null if no OpusHead packet is found.
*/
export function extractOpusHead(setupHeaderBytes: Uint8Array): Uint8Array | null {
// Walk pages manually (the setup region is small — at most two pages) and return the first packet
// that starts with the OpusHead signature.
let offset = 0;
while (offset + OGG_PAGE_HEADER_SIZE <= setupHeaderBytes.length) {
if (!(setupHeaderBytes[offset] === OGG_CAPTURE[0] && setupHeaderBytes[offset + 1] === OGG_CAPTURE[1] &&
setupHeaderBytes[offset + 2] === OGG_CAPTURE[2] && setupHeaderBytes[offset + 3] === OGG_CAPTURE[3])) {
return null;
}
const segCount = setupHeaderBytes[offset + SEGMENT_COUNT_OFFSET];
const segTableEnd = offset + OGG_PAGE_HEADER_SIZE + segCount;
if (segTableEnd > setupHeaderBytes.length) return null;
let payloadSize = 0;
for (let i = 0; i < segCount; i++) payloadSize += setupHeaderBytes[segTableEnd - segCount + i];
const payloadStart = segTableEnd;
const payloadEnd = payloadStart + payloadSize;
if (payloadEnd > setupHeaderBytes.length) return null;
const payload = setupHeaderBytes.subarray(payloadStart, payloadEnd);
if (startsWith(payload, OPUS_HEAD_SIG)) {
// The OpusHead packet is the whole first-page payload (it always fits one segment / page).
return payload;
}
offset = payloadEnd;
}
return null;
}
/** Channel count from an OpusHead packet (RFC 7845 §5.1: byte 9, after the 8-byte magic + version). */
export function opusHeadChannelCount(opusHead: Uint8Array): number {
if (opusHead.length < 10) return 2; // safe nominal
return opusHead[9];
}
@@ -0,0 +1,46 @@
/**
* OpusCapability - runtime detection of WebCodecs Ogg-Opus decode support.
*
* The Opus decode path is a WebCodecs `AudioDecoder` streaming pipeline (OpusStreamDecoder), NOT
* `decodeAudioData`. So the capability gate must test the path actually used: whether the browser has
* `AudioDecoder` AND supports the `codec: 'opus'` config. `AudioDecoder` is available on Chrome/Edge,
* Firefox 130+, and Safari 16.4+; older Safari and older Firefox lack it, and those listeners fall back
* to the universal lossless WAV path (§3.4 / OQ2 / AC7 no listener ever gets silence over a codec gap).
*
* This module is the detection *seam* only it answers "can this browser stream-decode Opus via
* WebCodecs?". The player (StreamingAudioPlayerService.ResolveStreamFormatAsync) consumes the answer to
* choose the delivery format; this module never touches the player or the stream request. The result is
* cached after the first probe (capability does not change within a session).
*/
let cachedSupport: Promise<boolean> | null = null;
/**
* Resolve whether this browser can stream-decode Ogg Opus via WebCodecs. Cached after the first call.
* Never rejects any failure (no AudioDecoder, unsupported config, thrown probe) resolves to `false`
* (treat as unsupported, fall back to lossless) so an interop error can never silence playback.
*/
export function canDecodeOggOpus(): Promise<boolean> {
if (cachedSupport === null) {
cachedSupport = probe();
}
return cachedSupport;
}
async function probe(): Promise<boolean> {
try {
if (typeof AudioDecoder === 'undefined' || typeof AudioDecoder.isConfigSupported !== 'function') {
return false;
}
// 48 kHz stereo is the canonical fullband Opus shape this site produces. isConfigSupported does
// not need the OpusHead `description` to report codec support, so we probe without it.
const result = await AudioDecoder.isConfigSupported({
codec: 'opus',
sampleRate: 48000,
numberOfChannels: 2
});
return result.supported === true;
} catch {
return false;
}
}
+196
View File
@@ -0,0 +1,196 @@
/**
* OpusSidecar - parser for the per-track Opus seek/setup sidecar artifact.
*
* The sidecar is built once at transcode time (wave 18.1, C# `OpusSidecar` /
* `OggOpusSeekIndex`) and fetched once on track load (wired by wave 18.5). It carries
* everything the client needs to seek a VBR Opus stream accurately and to decode any
* mid-stream slice:
* - the verbatim OpusHead + OpusTags setup pages (prepended to every post-seek slice),
* - the precomputed granule->byte seek index (the exact time->byte transfer function),
* - the pre_skip and totals needed for presentation-time math and seek clamping.
*
* This module is the byte-for-byte counterpart to the C# serializer. It is pure: it parses
* a blob into an `OpusSeekData` accelerator with no I/O. Wave 18.5 owns the HTTP fetch and
* injects the parsed result into `OpusFormatDecoder.setSidecar`.
*
* Binary layout (all little-endian), matching DeepDrftContent.Processors.Opus:
* [uint32 setupHeaderLength]
* [setupHeaderLength bytes -> OpusHead + OpusTags pages]
* [seek-index blob]:
* header (24 bytes):
* uint64 totalByteLength
* double totalDurationSeconds (pre-skip-corrected)
* uint32 pointCount
* uint16 preSkip
* uint16 reserved
* pointCount x 16-byte points:
* uint64 granulePosition (48 kHz sample count)
* uint64 byteOffset (page-start offset in the Opus file)
*/
/** Opus granule positions are always 48 kHz sample counts, regardless of input rate. */
export const OPUS_SAMPLE_RATE = 48000;
/** Size of the seek-index blob header: totalBytes(8) + duration(8) + count(4) + preSkip(2) + reserved(2). */
const SEEK_INDEX_HEADER_SIZE = 24;
/** Size of one serialized seek point: granulepos(8) + byteOffset(8). */
const SEEK_POINT_SIZE = 16;
/** One (granule, byteOffset) seek-index entry. Both are page-start-accurate. */
export interface OpusSeekPoint {
/** Page end granule position — a 48 kHz sample count. */
granulePosition: number;
/** Byte offset of the page start in the Opus file. */
byteOffset: number;
}
/**
* Parsed sidecar: the `seekData` accelerator the `OpusFormatDecoder` holds for the stream's
* lifetime. Holds the setup bytes (for `wrapSegment` carry) and the index (for `calculateByteOffset`).
*/
export interface OpusSeekData {
kind: 'opus-sidecar';
/** Verbatim OpusHead + OpusTags pages, prepended to every decodable segment. */
setupHeaderBytes: Uint8Array;
/** Ordered (granule, byteOffset) entries, ascending by granule. */
points: OpusSeekPoint[];
/** Pre-skip-corrected total stream duration in seconds. */
totalDurationSeconds: number;
/** Total Opus file byte length, for clamping a seek past the end. */
totalByteLength: number;
/** pre_skip from OpusHead (RFC 7845 §5.1); samples to discard before presentation. */
preSkip: number;
}
/**
* Parse a sidecar blob produced by the C# `OpusSidecar.ToBytes`. Returns null on any structural
* inconsistency (short blob, length prefix overrun, declared point count that does not fit)
* the format is exact, so a malformed blob is corruption, not a recoverable shape.
*
* Accepts a `Uint8Array`, an `ArrayBuffer`, or a typed-array view; copies nothing it can borrow.
*/
export function parseSidecar(input: Uint8Array | ArrayBuffer | ArrayBufferView): OpusSeekData | null {
const bytes = toUint8Array(input);
// DataView over the same backing buffer; honour the view's byteOffset so a borrowed view parses.
const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
if (bytes.byteLength < 4) return null;
const setupLength = view.getUint32(0, true);
const indexStart = 4 + setupLength;
// Need the setup region plus at least the index header.
if (bytes.byteLength < indexStart + SEEK_INDEX_HEADER_SIZE) return null;
// subarray is zero-copy; setup bytes are retained for wrapSegment for the stream's lifetime.
const setupHeaderBytes = bytes.subarray(4, indexStart);
// Seek-index blob header (relative to the DataView, which is bytes-relative).
const totalByteLength = readUint64(view, indexStart);
const totalDurationSeconds = view.getFloat64(indexStart + 8, true);
const pointCount = view.getUint32(indexStart + 16, true);
const preSkip = view.getUint16(indexStart + 20, true);
// bytes 22-23: reserved — ignored on read, for forward-compatibility (matches C#).
const pointsStart = indexStart + SEEK_INDEX_HEADER_SIZE;
const expectedEnd = pointsStart + pointCount * SEEK_POINT_SIZE;
if (bytes.byteLength < expectedEnd) return null;
const points: OpusSeekPoint[] = new Array(pointCount);
let cursor = pointsStart;
for (let i = 0; i < pointCount; i++) {
const granulePosition = readUint64(view, cursor);
const byteOffset = readUint64(view, cursor + 8);
points[i] = { granulePosition, byteOffset };
cursor += SEEK_POINT_SIZE;
}
return {
kind: 'opus-sidecar',
setupHeaderBytes,
points,
totalDurationSeconds,
totalByteLength,
preSkip
};
}
/**
* Pre-skip-corrected presentation time for a granule position: max(0, (granule - preSkip) / 48000).
* Matches the C# `OggOpusSeekIndex.PresentationTimeSeconds` so client and server agree on the
* seek transfer function.
*/
export function presentationTimeSeconds(granulePosition: number, preSkip: number): number {
return Math.max(0, (granulePosition - preSkip) / OPUS_SAMPLE_RATE);
}
/**
* Result of resolving a seek time to a page-start byte offset.
* `byteOffset` is the Range request origin; `landingTimeSeconds` is the actual presentation time of that
* page (t_page positionSeconds). The caller uses the delta `positionSeconds - landingTimeSeconds` to
* trim the decoded leading frames so playback lands at the requested position, not at t_page (AC9).
*/
export interface OpusSeekResolution {
/** Page-start byte offset to use as the Range request origin (Ogg-sync-aligned). */
byteOffset: number;
/**
* Presentation time of the resolved index page (seconds). Always positionSeconds. The decoder
* must trim `(positionSeconds - landingTimeSeconds) * OPUS_SAMPLE_RATE` leading frames so the
* audible start and the reported clock both land at positionSeconds, not at landingTimeSeconds.
*/
landingTimeSeconds: number;
}
/**
* Resolve a seek time (seconds) to a file-absolute, page-start byte offset via the precomputed index
* the accurate VBR-safe transfer function (§3.4a A/C). Binary-searches for the largest entry whose
* presentation time is <= `positionSeconds`. Returns both the page-start byte offset AND the actual
* landing time of that page, so callers can trim leading decoded frames to land precisely at
* `positionSeconds` (AC9 fine re-sync). NOT interpolation, NOT byteRate math.
*
* With an empty index it degrades to the start of audio (offset == setup-header length, landing == 0).
*
* This is the single source of truth for Opus seek-offset math, shared by the seek-beyond-buffer path
* (AudioPlayer) and any byte-offset resolver. The Range fetch from this offset lands the decoder
* Ogg-sync-aligned because every indexed offset is a real page start.
*/
export function resolveOpusByteOffset(sidecar: OpusSeekData, positionSeconds: number): OpusSeekResolution {
const points = sidecar.points;
if (points.length === 0) {
return { byteOffset: sidecar.setupHeaderBytes.length, landingTimeSeconds: 0 };
}
let lo = 0;
let hi = points.length - 1;
let best = 0;
while (lo <= hi) {
const mid = (lo + hi) >> 1;
const t = presentationTimeSeconds(points[mid].granulePosition, sidecar.preSkip);
if (t <= positionSeconds) {
best = mid;
lo = mid + 1;
} else {
hi = mid - 1;
}
}
return {
byteOffset: points[best].byteOffset,
landingTimeSeconds: presentationTimeSeconds(points[best].granulePosition, sidecar.preSkip)
};
}
function toUint8Array(input: Uint8Array | ArrayBuffer | ArrayBufferView): Uint8Array {
if (input instanceof Uint8Array) return input;
if (input instanceof ArrayBuffer) return new Uint8Array(input);
return new Uint8Array(input.buffer, input.byteOffset, input.byteLength);
}
/**
* Read a little-endian uint64 as a JS number. Opus byte offsets and granule positions are exact
* to 2^53 (~8 PB / ~5,700 years of audio at 48 kHz), far beyond any real file no BigInt needed,
* matching the FLAC seektable's same 2^53 assumption.
*/
function readUint64(view: DataView, offset: number): number {
const lo = view.getUint32(offset, true);
const hi = view.getUint32(offset + 4, true);
return hi * 0x100000000 + lo;
}
@@ -0,0 +1,524 @@
/**
* Opus WebCodecs decode-path tests the browser-independent pieces.
*
* The WebCodecs decode/playback/seek itself can only run in a real browser (verified by Daniel), so
* these tests cover the pure logic that surrounds it and that determines correctness:
* - OggSidecar parse: byte-for-byte round-trip against the C# wire format.
* - resolveOpusByteOffset: the seek transfer function (binary search over the precomputed index).
* - OggDemuxer: Ogg page -> Opus packet extraction (segment-table lacing, packets spanning pages,
* granule tracking, OpusHead/OpusTags setup-packet skipping, continuation reset).
* - extractOpusHead / opusHeadChannelCount: pulling the WebCodecs `description` out of the sidecar.
*
* There is no TS test runner configured in this repo (no package.json, no jest/vitest). This is a
* self-contained, zero-dependency test: a tiny inline assert harness, no `node:` imports beyond Buffer
* (Node global). It is EXCLUDED from the production tsc build (tsconfig `exclude: Interop/**\/*.test.ts`)
* so it never ships in wwwroot/js. To run it (Node 22+ strips TS types natively no tsc, no deps), the
* `.js` import specifiers must resolve to the COMPILED modules, so run a copy from the compiled output:
*
* # 1. produce the compiled modules (the normal build already does this):
* dotnet build DeepDrftPublic/DeepDrftPublic.csproj
* # 2. run this test next to the compiled .js siblings (Node strips the types at load):
* cp DeepDrftPublic/Interop/audio/OpusStreamDecoder.test.ts DeepDrftPublic/wwwroot/js/audio/
* node DeepDrftPublic/wwwroot/js/audio/OpusStreamDecoder.test.ts
*
* A thrown error / non-zero exit signals failure; "ALL <n> TESTS PASSED" signals success.
*
* The sidecar bytes built in `makeSidecar` reproduce the C# wire format byte-for-byte
* (DeepDrftContent.Processors.Opus.OpusSidecar.ToBytes / OggOpusSeekIndex.ToBytes):
* [uint32 setupHeaderLength][setup bytes]
* [uint64 totalByteLength][double totalDuration][uint32 count][uint16 preSkip][uint16 reserved]
* count x [uint64 granulePosition][uint64 byteOffset] all little-endian.
*/
import { parseSidecar, presentationTimeSeconds, resolveOpusByteOffset, OPUS_SAMPLE_RATE } from './OpusSidecar.js';
import type { OpusSeekData, OpusSeekResolution } from './OpusSidecar.js';
import { OggDemuxer, extractOpusHead, opusHeadChannelCount } from './OggDemuxer.js';
import { OpusStreamDecoder } from './OpusStreamDecoder.js';
// --- tiny inline harness (no dependencies) ---------------------------------------------------
let passed = 0;
const failures: string[] = [];
function test(name: string, fn: () => void): void {
try {
fn();
passed++;
} catch (e) {
failures.push(`FAIL: ${name}\n ${(e as Error).message}`);
}
}
function assertEqual(actual: unknown, expected: unknown, msg?: string): void {
if (actual !== expected) {
throw new Error(`${msg ?? 'assertEqual'}: expected ${String(expected)}, got ${String(actual)}`);
}
}
function assertArray(actual: ArrayLike<number>, expected: number[], msg?: string): void {
const a = Array.from(actual);
if (a.length !== expected.length || a.some((v, i) => v !== expected[i])) {
throw new Error(`${msg ?? 'assertArray'}: expected [${expected}], got [${a}]`);
}
}
function assertNull(actual: unknown, msg?: string): void {
if (actual !== null) throw new Error(`${msg ?? 'assertNull'}: expected null, got ${String(actual)}`);
}
function assertNotNull<T>(actual: T | null, msg?: string): T {
if (actual === null) throw new Error(`${msg ?? 'assertNotNull'}: got null`);
return actual;
}
interface SidecarSpec {
setupHeader: number[];
totalByteLength: number;
totalDuration: number;
preSkip: number;
points: Array<{ granule: number; byteOffset: number }>;
}
/** Serialize a sidecar blob exactly as the C# OpusSidecar/OggOpusSeekIndex writers do. */
function makeSidecar(spec: SidecarSpec): Uint8Array {
const SEEK_INDEX_HEADER_SIZE = 24;
const SEEK_POINT_SIZE = 16;
const setupLen = spec.setupHeader.length;
const total = 4 + setupLen + SEEK_INDEX_HEADER_SIZE + spec.points.length * SEEK_POINT_SIZE;
const bytes = new Uint8Array(total);
const view = new DataView(bytes.buffer);
view.setUint32(0, setupLen, true);
bytes.set(spec.setupHeader, 4);
let p = 4 + setupLen;
writeUint64(view, p, spec.totalByteLength);
view.setFloat64(p + 8, spec.totalDuration, true);
view.setUint32(p + 16, spec.points.length, true);
view.setUint16(p + 20, spec.preSkip, true);
// bytes 22-23 reserved (zero)
p += SEEK_INDEX_HEADER_SIZE;
for (const pt of spec.points) {
writeUint64(view, p, pt.granule);
writeUint64(view, p + 8, pt.byteOffset);
p += SEEK_POINT_SIZE;
}
return bytes;
}
function writeUint64(view: DataView, offset: number, value: number): void {
view.setUint32(offset, value >>> 0, true);
view.setUint32(offset + 4, Math.floor(value / 0x100000000), true);
}
// --- parseSidecar: byte-for-byte round-trip against the C# layout -----------------------------
test('parseSidecar round-trips the C# binary layout exactly', () => {
const setup = [0x4f, 0x70, 0x75, 0x73, 0x48, 0x65, 0x61, 0x64]; // "OpusHead" stand-in
const spec: SidecarSpec = {
setupHeader: setup,
totalByteLength: 1_234_567,
totalDuration: 212.5,
preSkip: 312,
points: [
{ granule: 312, byteOffset: 4096 }, // first point: granule == preSkip -> t=0
{ granule: 312 + 24000, byteOffset: 9000 }, // +0.5 s
{ granule: 312 + 48000, byteOffset: 14000 }, // +1.0 s
],
};
const parsed: OpusSeekData = assertNotNull(parseSidecar(makeSidecar(spec)));
assertEqual(parsed.kind, 'opus-sidecar', 'kind');
assertArray(parsed.setupHeaderBytes, setup, 'setup header bytes');
assertEqual(parsed.totalByteLength, spec.totalByteLength, 'totalByteLength');
assertEqual(parsed.totalDurationSeconds, spec.totalDuration, 'totalDuration');
assertEqual(parsed.preSkip, spec.preSkip, 'preSkip');
assertEqual(parsed.points.length, 3, 'point count');
assertEqual(parsed.points[1].granulePosition, 312 + 24000, 'point[1].granule');
assertEqual(parsed.points[1].byteOffset, 9000, 'point[1].byteOffset');
});
test('parseSidecar honours a borrowed view byteOffset (sidecar not at buffer start)', () => {
const blob = makeSidecar({
setupHeader: [1, 2, 3, 4],
totalByteLength: 100,
totalDuration: 1.0,
preSkip: 0,
points: [{ granule: 0, byteOffset: 8 }],
});
const padded = new Uint8Array(blob.length + 7);
padded.set(blob, 7);
const parsed = assertNotNull(parseSidecar(padded.subarray(7)));
assertArray(parsed.setupHeaderBytes, [1, 2, 3, 4], 'borrowed setup bytes');
assertEqual(parsed.points[0].byteOffset, 8, 'borrowed point offset');
});
test('parseSidecar returns null on a truncated blob', () => {
const blob = makeSidecar({
setupHeader: [0],
totalByteLength: 1,
totalDuration: 0,
preSkip: 0,
points: [{ granule: 0, byteOffset: 0 }],
});
assertNull(parseSidecar(blob.subarray(0, 3)), 'short of length prefix');
assertNull(parseSidecar(blob.subarray(0, blob.length - 4)), 'declared count overruns');
});
test('presentationTimeSeconds applies preSkip and clamps at zero (RFC 7845)', () => {
assertEqual(presentationTimeSeconds(312, 312), 0, 'granule == preSkip');
assertEqual(presentationTimeSeconds(0, 312), 0, 'below preSkip clamps');
assertEqual(presentationTimeSeconds(312 + OPUS_SAMPLE_RATE, 312), 1.0, '+48000 -> 1 s');
});
// --- resolveOpusByteOffset: binary search over the precomputed index (exact, not interpolation) -
function sidecarFrom(spec: SidecarSpec): OpusSeekData {
return assertNotNull(parseSidecar(makeSidecar(spec)), 'sidecar should parse');
}
test('resolveOpusByteOffset returns the page-start of the largest entry with time <= t', () => {
const points = [0, 1, 2, 3].map(i => ({
granule: 1000 + i * (OPUS_SAMPLE_RATE / 2),
byteOffset: 4096 + i * 5000,
}));
const sc = sidecarFrom({
setupHeader: [9, 9, 9, 9], totalByteLength: 999_999, totalDuration: 1.5, preSkip: 1000, points,
});
assertEqual(resolveOpusByteOffset(sc, 0.0).byteOffset, 4096, 't=0 -> first point');
assertEqual(resolveOpusByteOffset(sc, 0.4).byteOffset, 4096, 'just before bucket 1');
assertEqual(resolveOpusByteOffset(sc, 0.5).byteOffset, 9096, 'exactly bucket 1');
assertEqual(resolveOpusByteOffset(sc, 0.9).byteOffset, 9096, 'within bucket 1');
assertEqual(resolveOpusByteOffset(sc, 1.0).byteOffset, 14096, 'exactly bucket 2');
assertEqual(resolveOpusByteOffset(sc, 99).byteOffset, 19096, 'past end -> last point');
});
test('resolveOpusByteOffset never interpolates between points', () => {
const sc = sidecarFrom({
setupHeader: [0], totalByteLength: 10_000, totalDuration: 1.0, preSkip: 0,
points: [{ granule: 0, byteOffset: 100 }, { granule: OPUS_SAMPLE_RATE, byteOffset: 9000 }],
});
assertEqual(resolveOpusByteOffset(sc, 0.5).byteOffset, 100, 'midpoint snaps to lower page start');
});
test('resolveOpusByteOffset degrades to start of audio with an empty index', () => {
const sc = sidecarFrom({
setupHeader: [1, 2, 3, 4, 5], totalByteLength: 0, totalDuration: 0, preSkip: 0, points: [],
});
// start of audio == setup header length (server emits [setup pages][audio pages]).
assertEqual(resolveOpusByteOffset(sc, 10).byteOffset, 5, 'empty index degrades to audio start');
});
// --- resolveOpusByteOffset: landingTimeSeconds (AC9 fine re-sync, §3.4a step 4) -----------------
test('resolveOpusByteOffset landingTimeSeconds equals the resolved page time, not the requested time', () => {
// Index: two points at t=0 s and t=0.5 s.
const preSkip = 312;
const sc = sidecarFrom({
setupHeader: [0], totalByteLength: 50_000, totalDuration: 1.5, preSkip,
points: [
{ granule: preSkip, byteOffset: 4096 }, // t=0
{ granule: preSkip + OPUS_SAMPLE_RATE / 2, byteOffset: 9000 }, // t=0.5 s
],
});
// Seeking to 0.3 s lands on the t=0 page; landing should be 0, not 0.3.
const r03: OpusSeekResolution = resolveOpusByteOffset(sc, 0.3);
assertEqual(r03.byteOffset, 4096, 'seek 0.3 -> first page offset');
assertEqual(r03.landingTimeSeconds, 0, 'landing at t=0 (page time, not target)');
// Seeking to exactly 0.5 s lands on the second page; landing == requested time.
const r05: OpusSeekResolution = resolveOpusByteOffset(sc, 0.5);
assertEqual(r05.byteOffset, 9000, 'seek 0.5 -> second page offset');
assertEqual(r05.landingTimeSeconds, 0.5, 'landing == requested when exact page boundary');
});
test('resolveOpusByteOffset empty index returns landingTimeSeconds = 0', () => {
const sc = sidecarFrom({
setupHeader: [0, 1, 2], totalByteLength: 1000, totalDuration: 1.0, preSkip: 0, points: [],
});
const r = resolveOpusByteOffset(sc, 5.0);
assertEqual(r.landingTimeSeconds, 0, 'empty index: landing is stream start (0 s)');
});
// --- Lead-trim frame math (AC9 fine re-sync) ---------------------------------------------------
// The trim frame count is purely arithmetic: (target - landing) * 48000, rounded, clamped to ≥0.
// This is the exact formula in OpusStreamDecoder.reinitializeForRangeContinuation so we test it
// independently of the browser-bound WebCodecs decode.
function leadTrimFrames(landingTimeSeconds: number, targetTimeSeconds: number): number {
return Math.max(0, Math.round((targetTimeSeconds - landingTimeSeconds) * OPUS_SAMPLE_RATE));
}
test('lead-trim frame count is (target - landing) * 48000, rounded', () => {
// Page at t=0, seek to 0.3 s: trim 0.3 * 48000 = 14400 frames.
assertEqual(leadTrimFrames(0, 0.3), 14400, 'trim for 0.3 s offset');
// Page at t=0.5 s, seek to 0.7 s: trim 0.2 * 48000 = 9600 frames.
assertEqual(leadTrimFrames(0.5, 0.7), 9600, 'trim for 0.2 s offset');
// Exact page boundary: no trim needed.
assertEqual(leadTrimFrames(0.5, 0.5), 0, 'no trim when target == landing');
// Guard against floating-point rounding producing a tiny negative: clamp to 0.
assertEqual(leadTrimFrames(0.5000001, 0.5), 0, 'negative rounds to zero (guard)');
});
// --- OggDemuxer: page -> packet extraction ----------------------------------------------------
//
// Builds minimal Ogg pages by hand (no codec) so the lacing logic is exercised deterministically.
interface PageSpec {
granule: number; // -1 (0xFFFF...) means "no granule"
continued?: boolean; // header-type bit 0x01
eos?: boolean; // header-type bit 0x04
/** Packet payloads to lace into this page (each split into 255-byte segments per Ogg rules). */
packets?: Uint8Array[];
/** Raw segment lengths + payload, for hand-crafting page-spanning packets. */
rawSegments?: number[];
rawPayload?: Uint8Array;
}
function buildPage(spec: PageSpec): Uint8Array {
let segTable: number[];
let payload: Uint8Array;
if (spec.rawSegments && spec.rawPayload) {
segTable = spec.rawSegments;
payload = spec.rawPayload;
} else {
segTable = [];
const chunks: number[] = [];
for (const pkt of spec.packets ?? []) {
let remaining = pkt.length;
let o = 0;
// Lace: emit 255-byte segments until the final (< 255) segment terminates the packet.
for (;;) {
const seg = Math.min(255, remaining);
segTable.push(seg);
for (let i = 0; i < seg; i++) chunks.push(pkt[o + i]);
o += seg;
remaining -= seg;
if (seg < 255) break; // terminating segment
}
}
payload = new Uint8Array(chunks);
}
const header = new Uint8Array(OGG_HDR + segTable.length + payload.length);
header.set([0x4f, 0x67, 0x67, 0x53], 0); // "OggS"
header[4] = 0; // version
header[5] = (spec.continued ? 0x01 : 0) | (spec.eos ? 0x04 : 0);
// granule (LE uint64)
if (spec.granule < 0) {
for (let i = 0; i < 8; i++) header[6 + i] = 0xff;
} else {
let g = spec.granule;
for (let i = 0; i < 8; i++) { header[6 + i] = g & 0xff; g = Math.floor(g / 256); }
}
header[26] = segTable.length;
header.set(segTable, OGG_HDR);
header.set(payload, OGG_HDR + segTable.length);
return header;
}
const OGG_HDR = 27;
function opusHeadPacket(channels: number, preSkip: number): Uint8Array {
// "OpusHead"(8) version(1) channels(1) preSkip(2 LE) inputRate(4) gain(2) mapping(1) = 19 bytes
const p = new Uint8Array(19);
p.set([0x4f, 0x70, 0x75, 0x73, 0x48, 0x65, 0x61, 0x64], 0);
p[8] = 1;
p[9] = channels;
p[10] = preSkip & 0xff;
p[11] = (preSkip >> 8) & 0xff;
return p;
}
function opusTagsPacket(): Uint8Array {
const p = new Uint8Array(16);
p.set([0x4f, 0x70, 0x75, 0x73, 0x54, 0x61, 0x67, 0x73], 0); // "OpusTags"
return p;
}
test('OggDemuxer skips OpusHead/OpusTags and returns audio packets with the page granule', () => {
const head = buildPage({ granule: 0, packets: [opusHeadPacket(2, 312)] });
const tags = buildPage({ granule: 0, packets: [opusTagsPacket()] });
const audio = buildPage({ granule: 24000, packets: [new Uint8Array([0xaa, 0xbb]), new Uint8Array([0xcc])] });
const d = new OggDemuxer();
const packets = d.push(concat([head, tags, audio]));
assertEqual(packets.length, 2, 'two audio packets, setup skipped');
assertArray(packets[0].data, [0xaa, 0xbb], 'first audio packet bytes');
assertEqual(packets[0].pageGranule, null, 'non-final packet carries no granule');
assertArray(packets[1].data, [0xcc], 'second audio packet bytes');
assertEqual(packets[1].pageGranule, 24000, 'final completing packet carries the page granule');
assertEqual(packets[1].isLastPage, false, 'not EOS');
});
test('OggDemuxer flags the EOS page', () => {
const head = buildPage({ granule: 0, packets: [opusHeadPacket(1, 100)] });
const tags = buildPage({ granule: 0, packets: [opusTagsPacket()] });
const audio = buildPage({ granule: 48000, eos: true, packets: [new Uint8Array([0x01])] });
const d = new OggDemuxer();
const packets = d.push(concat([head, tags, audio]));
assertEqual(packets.length, 1, 'one audio packet');
assertEqual(packets[0].isLastPage, true, 'EOS flagged');
});
test('OggDemuxer reassembles a packet that spans two pages (255 last segment + continuation)', () => {
const head = buildPage({ granule: 0, packets: [opusHeadPacket(2, 0)] });
const tags = buildPage({ granule: 0, packets: [opusTagsPacket()] });
// First audio page: one 255-byte segment that does NOT terminate (packet continues).
const part1 = new Uint8Array(255).fill(0x11);
const pageA = buildPage({ granule: -1, rawSegments: [255], rawPayload: part1 });
// Second page (continued): a 10-byte terminating segment completes the packet.
const part2 = new Uint8Array(10).fill(0x22);
const pageB = buildPage({ granule: 24000, continued: true, rawSegments: [10], rawPayload: part2 });
const d = new OggDemuxer();
const packets = d.push(concat([head, tags, pageA, pageB]));
assertEqual(packets.length, 1, 'one reassembled packet');
assertEqual(packets[0].data.length, 265, 'packet is 255 + 10 bytes');
assertEqual(packets[0].data[0], 0x11, 'first byte from page A');
assertEqual(packets[0].data[264], 0x22, 'last byte from page B');
assertEqual(packets[0].pageGranule, 24000, 'granule from the completing page');
});
test('OggDemuxer handles bytes split across push() calls (page straddles a network chunk)', () => {
const head = buildPage({ granule: 0, packets: [opusHeadPacket(2, 0)] });
const tags = buildPage({ granule: 0, packets: [opusTagsPacket()] });
const audio = buildPage({ granule: 960, packets: [new Uint8Array([0x07, 0x08, 0x09])] });
const full = concat([head, tags, audio]);
const d = new OggDemuxer();
const cut = full.length - 2; // split mid-audio-page
const first = d.push(full.subarray(0, cut));
assertEqual(first.length, 0, 'no whole audio packet yet');
const second = d.push(full.subarray(cut));
assertEqual(second.length, 1, 'audio packet completes on the second push');
assertArray(second[0].data, [0x07, 0x08, 0x09], 'reassembled across pushes');
});
test('OggDemuxer.reset(continuation) treats the first page as audio (no setup expected)', () => {
const audio = buildPage({ granule: 96000, packets: [new Uint8Array([0x42])] });
const d = new OggDemuxer();
d.reset(true);
const packets = d.push(audio);
assertEqual(packets.length, 1, 'continuation: first page is audio');
assertArray(packets[0].data, [0x42], 'audio packet bytes');
});
// --- extractOpusHead / opusHeadChannelCount: WebCodecs description from the sidecar -----------
test('extractOpusHead returns the OpusHead packet from the setup pages', () => {
const head = buildPage({ granule: 0, packets: [opusHeadPacket(2, 312)] });
const tags = buildPage({ granule: 0, packets: [opusTagsPacket()] });
const setup = concat([head, tags]);
const opusHead = assertNotNull(extractOpusHead(setup), 'OpusHead extracted');
assertArray(opusHead.subarray(0, 8), [0x4f, 0x70, 0x75, 0x73, 0x48, 0x65, 0x61, 0x64], 'OpusHead magic');
assertEqual(opusHeadChannelCount(opusHead), 2, 'channel count');
});
test('extractOpusHead returns null when no OpusHead page is present', () => {
const tags = buildPage({ granule: 0, packets: [opusTagsPacket()] });
assertNull(extractOpusHead(tags), 'no OpusHead');
});
// --- OpusStreamDecoder.totalDuration: available from the sidecar BEFORE the first push ----------
//
// Defect 1 (dead Opus seekbar): the C# layer locks the UI Duration on the first chunk whose result
// carries a value, and AudioPlayer.processOpusChunk now surfaces `decoder.totalDuration` on that first
// chunk rather than gating it on the (async, possibly-empty-on-chunk-1) decoded buffers. The load-bearing
// guarantee that makes this correct is that `totalDuration` is known from the sidecar IMMEDIATELY — i.e.
// before any push and without WebCodecs. These tests pin that contract; the WebCodecs decode itself stays
// browser-verified. The constructor only stashes the context manager (totalDuration never touches it), so a
// null-shaped stub is safe and no AudioDecoder is constructed.
const stubContextManager = {} as unknown as ConstructorParameters<typeof OpusStreamDecoder>[0];
test('OpusStreamDecoder.totalDuration is the sidecar duration, available before any push', () => {
const sidecar = sidecarFrom({
setupHeader: [0x4f, 0x70, 0x75, 0x73, 0x48, 0x65, 0x61, 0x64],
totalByteLength: 500_000, totalDuration: 212.5, preSkip: 312,
points: [{ granule: 312, byteOffset: 4096 }],
});
const decoder = new OpusStreamDecoder(stubContextManager, sidecar);
// No push, no configure — the value the first chunk reports to C# must already be present.
assertEqual(decoder.totalDuration, 212.5, 'totalDuration from sidecar, pre-push');
});
test('OpusStreamDecoder.totalDuration is null when the sidecar carries no positive duration', () => {
const sidecar = sidecarFrom({
setupHeader: [0], totalByteLength: 0, totalDuration: 0, preSkip: 0, points: [],
});
const decoder = new OpusStreamDecoder(stubContextManager, sidecar);
// A zero/absent sidecar duration must report null (not 0) so the chunk result carries no spurious
// value — the WAV-header path, not a bogus Opus duration, then drives the UI.
assertEqual(decoder.totalDuration, null, 'no positive duration -> null');
});
// --- Phase 21.2b: Opus decode-ahead back-pressure (the stash-while-full half) ------------------
//
// When the shared scheduler is full, push() must NOT demux/decode ahead — it stashes the raw bytes
// and returns nothing, so the WebCodecs decode queue and decodedQueue stay near-empty (OQ7). The
// stash-while-full branch returns BEFORE ensureConfigured(), so it is testable without WebCodecs
// (no AudioDecoder is constructed). The drain-on-resume path needs the real WebCodecs decoder and
// stays browser-verified; here we pin the bound itself and the lifecycle resets.
// Access the private stash for white-box assertions (same idiom the scheduler tests use).
function stashLength(decoder: OpusStreamDecoder): number {
return (decoder as unknown as { pendingBytes: Uint8Array[] }).pendingBytes.length;
}
// The stash-while-full branch returns synchronously at the top of push() (before any real await),
// so the stash is observable immediately without awaiting the returned promise — keeping these
// tests inside the synchronous inline harness (which does not await test bodies).
test('push stashes bytes and decodes nothing while the scheduler is full (no decode-ahead)', () => {
const sidecar = sidecarFrom({
setupHeader: [0x4f, 0x70, 0x75, 0x73, 0x48, 0x65, 0x61, 0x64],
totalByteLength: 500_000, totalDuration: 100, preSkip: 312,
points: [{ granule: 312, byteOffset: 4096 }],
});
// Scheduler reports "full" → push must short-circuit before touching WebCodecs.
const decoder = new OpusStreamDecoder(stubContextManager, sidecar, () => true);
void decoder.push(new Uint8Array([1, 2, 3]));
void decoder.push(new Uint8Array([4, 5]));
assertEqual(stashLength(decoder), 2, 'both chunks stashed in arrival order');
assertEqual(decoder.ready, false, 'decoder not even configured while throttled');
});
test('reinitializeForRangeContinuation drops the pre-seek stash (C6 — no stale feed across reset)', () => {
const sidecar = sidecarFrom({
setupHeader: [0x4f, 0x70, 0x75, 0x73, 0x48, 0x65, 0x61, 0x64],
totalByteLength: 500_000, totalDuration: 100, preSkip: 312,
points: [{ granule: 312, byteOffset: 4096 }],
});
const decoder = new OpusStreamDecoder(stubContextManager, sidecar, () => true);
void decoder.push(new Uint8Array([1, 2, 3])); // stash one chunk while full
assertEqual(stashLength(decoder), 1, 'one chunk stashed pre-seek');
decoder.reinitializeForRangeContinuation(0, 5); // a seek
assertEqual(stashLength(decoder), 0, 'pre-seek stash dropped on range-continuation');
});
test('dispose clears the stash', () => {
const sidecar = sidecarFrom({
setupHeader: [0x4f, 0x70, 0x75, 0x73, 0x48, 0x65, 0x61, 0x64],
totalByteLength: 500_000, totalDuration: 100, preSkip: 312,
points: [{ granule: 312, byteOffset: 4096 }],
});
const decoder = new OpusStreamDecoder(stubContextManager, sidecar, () => true);
void decoder.push(new Uint8Array([9]));
assertEqual(stashLength(decoder), 1, 'stashed');
decoder.dispose();
assertEqual(stashLength(decoder), 0, 'stash cleared on dispose');
});
function concat(arrs: Uint8Array[]): Uint8Array {
let len = 0;
for (const a of arrs) len += a.length;
const out = new Uint8Array(len);
let o = 0;
for (const a of arrs) { out.set(a, o); o += a.length; }
return out;
}
// --- report ----------------------------------------------------------------------------------
if (failures.length > 0) {
console.error(failures.join('\n'));
throw new Error(`${failures.length} test(s) failed, ${passed} passed`);
}
console.log(`ALL ${passed} TESTS PASSED`);
@@ -0,0 +1,395 @@
/**
* OpusStreamDecoder - the WebCodecs streaming Opus decode pipeline.
*
* This replaces the fundamentally-broken per-segment `decodeAudioData` Opus model. Instead of cutting
* the Ogg stream into page-runs and decoding each as a standalone file (which re-applies pre-skip and
* starts from cold codec state at every boundary), it feeds a single stateful WebCodecs `AudioDecoder`
* the demuxed Opus packets in order correct pre-skip-once handling and full inter-frame continuity.
*
* Pipeline: OggDemuxer (pages -> Opus packets + granule) -> AudioDecoder (codec 'opus', configured
* from the OpusHead in the sidecar) -> AudioData (48 kHz PCM) -> AudioBuffer -> PlaybackScheduler.
*
* Pre-skip (encoder delay): handled ONCE, by the decoder. WebCodecs decodes Opus with the OpusHead
* passed as `AudioDecoderConfig.description`; the OpusHead carries `pre_skip`, and the WebCodecs Opus
* decoder discards those leading samples itself. We do NOT re-trim per packet doing so on top of the
* decoder's own trim would double-count. This is the spec-intended path (W3C WebCodecs Opus registration).
*
* End-trim: the sidecar's `totalDurationSeconds` is the exact pre-skip-corrected stream length. We cap
* cumulative emitted audio at that length so the final partial frame's padding does not leak past the
* true end. (Granule-position end-trim from the EOS page is the alternative; capping on the known total
* is equivalent and simpler, and the sidecar total is authoritative.)
*
* Sample rate: Opus always decodes at 48 kHz (RFC 7845). We force the AudioContext to 48 kHz at init so
* the decoded AudioData needs no resampling before scheduling the same `recreateWithSampleRate` seam
* the WAV path uses for non-44.1 sources.
*
* BROWSER-VERIFIED. The actual decode/playback/trim correctness is verified in Daniel's browser
* (WebCodecs cannot run in Node/headless here). The Ogg demux, packet timing, and end-trim *math* are
* unit-tested; the WebCodecs glue (configure/decode/flush/AudioData->AudioBuffer) is browser-verified.
*/
import { AudioContextManager } from './AudioContextManager.js';
import { decodePressure } from './decodePressure.js';
import { IStreamingDecoder } from './IStreamingDecoder.js';
import { OggDemuxer, OpusPacket, extractOpusHead, opusHeadChannelCount } from './OggDemuxer.js';
import { OpusSeekData, OPUS_SAMPLE_RATE } from './OpusSidecar.js';
/** Opus packet duration ceiling is 120 ms; at 48 kHz that is 5760 frames. Used only for chunk timestamps. */
const MAX_PACKET_FRAMES = 5760;
export class OpusStreamDecoder implements IStreamingDecoder {
private readonly contextManager: AudioContextManager;
private readonly sidecar: OpusSeekData;
// Phase 21.2b back-pressure hook: returns true when the shared scheduler is full (forward fill
// over high-water). While full, push() stashes raw bytes WITHOUT demuxing/decoding so the
// WebCodecs decode queue and decodedQueue stay near-empty behind a throttled socket (OQ7).
// Null = no back-pressure (e.g. unit tests), in which case the decoder feeds eagerly as before.
private readonly isSchedulerFull: (() => boolean) | null;
// Raw bytes received while the scheduler was full, held undemuxed until it drains. The C# read
// loop also pauses above high-water, so this stash is bounded to at most the in-flight chunks
// between the loop reading the productionPaused flag and actually stopping — a handful of KB,
// not a decode-ahead. Drained (demuxed + decoded) on the next push once below high-water.
private pendingBytes: Uint8Array[] = [];
private demuxer = new OggDemuxer();
private decoder: AudioDecoder | null = null;
private channelCount = 2;
private configured = false;
// OpusHead bytes used as the AudioDecoder `description`, captured once at first configure and reused
// verbatim on a range-continuation reconfigure (avoids re-extracting / a non-null assertion).
private opusHeadDescription: Uint8Array | null = null;
// Decoded AudioData awaiting conversion, filled by the AudioDecoder output callback.
private decodedQueue: AudioData[] = [];
private fatalError = false;
// Frames to discard from the head of the first post-seek decoded output (AC9 fine re-sync).
// Set by reinitializeForRangeContinuation to (targetTimeSeconds - landingTimeSeconds) * 48000,
// consumed frame-by-frame in audioDataToBuffer until exhausted (then zero for the rest of the stream).
private leadTrimFrames = 0;
// Monotonic packet timestamp (microseconds) handed to each EncodedAudioChunk. WebCodecs requires
// strictly increasing timestamps; the true value is irrelevant to us (we schedule by accumulation),
// so a synthetic 48 kHz-derived counter suffices and stays exact.
private nextTimestampUs = 0;
// Cumulative frames already emitted as AudioBuffers, for end-trim against the known total length.
private emittedFrames = 0;
private readonly totalFrames: number;
constructor(
contextManager: AudioContextManager,
sidecar: OpusSeekData,
isSchedulerFull: (() => boolean) | null = null) {
this.contextManager = contextManager;
this.sidecar = sidecar;
this.isSchedulerFull = isSchedulerFull;
this.totalFrames = sidecar.totalDurationSeconds > 0
? Math.round(sidecar.totalDurationSeconds * OPUS_SAMPLE_RATE)
: Number.POSITIVE_INFINITY;
}
get hasFatalError(): boolean {
return this.fatalError;
}
get ready(): boolean {
return this.configured;
}
get totalDuration(): number | null {
return this.sidecar.totalDurationSeconds > 0 ? this.sidecar.totalDurationSeconds : null;
}
/**
* Lazily build + configure the WebCodecs decoder from the sidecar's OpusHead. Idempotent. Forces the
* AudioContext to 48 kHz so decoded AudioData schedules without resampling. Returns false on a config
* the browser cannot support (caller should never reach here the capability gate runs first but
* we fail safe rather than throw into the stream loop).
*/
private async ensureConfigured(): Promise<boolean> {
if (this.configured) return true;
if (typeof AudioDecoder === 'undefined') {
this.fatalError = true;
return false;
}
const opusHead = extractOpusHead(this.sidecar.setupHeaderBytes);
if (!opusHead) {
this.fatalError = true;
return false;
}
this.channelCount = opusHeadChannelCount(opusHead);
// Copy the OpusHead into a standalone buffer — the sidecar subarray is a view we keep.
this.opusHeadDescription = opusHead.slice();
// Opus decodes at 48 kHz; align the context so no resample is needed. AudioPlayer.initializeStreaming
// already aligned it to 48 kHz up front (the format is resolved before any bytes flow), so in the
// common path this is an early-return no-op — the live graph is NOT close()'d and rebuilt mid-decode.
// Kept as the defensive backstop for any path that reaches a configured decoder on a non-48 kHz
// context (the same recreate seam the WAV path uses for non-44.1 sources).
if (this.contextManager.sampleRate !== OPUS_SAMPLE_RATE) {
await this.contextManager.recreateWithSampleRate(OPUS_SAMPLE_RATE);
}
this.decoder = new AudioDecoder({
output: (data) => this.decodedQueue.push(data),
error: (err) => {
console.error('Opus AudioDecoder error:', err.message);
this.fatalError = true;
}
});
this.decoder.configure(this.buildConfig());
this.configured = true;
return true;
}
private buildConfig(): AudioDecoderConfig {
return {
codec: 'opus',
sampleRate: OPUS_SAMPLE_RATE,
numberOfChannels: this.channelCount,
description: this.opusHeadDescription ?? undefined
};
}
async push(chunk: Uint8Array): Promise<AudioBuffer[]> {
if (this.fatalError) return [];
// 21.2b back-pressure: while the scheduler is full, do NOT demux/decode ahead. Stash the
// raw bytes in arrival order and return nothing — the WebCodecs decode queue and
// decodedQueue stay near-empty (OQ7). The bytes are demuxed/decoded on a later push once
// the scheduler has drained below low-water, in exactly the order received (Ogg demux is
// order-sensitive). configure() is deferred too — no need to spin up the decoder while
// throttled. The C# loop also stops reading above high-water, so the stash stays small.
if (this.isSchedulerFull?.()) {
this.pendingBytes.push(chunk);
return [];
}
if (!(await this.ensureConfigured())) return [];
// Drained below high-water: replay any stashed bytes first (preserving stream order), then
// the new chunk, through the demuxer as one contiguous feed.
const out: AudioBuffer[] = [];
if (this.pendingBytes.length > 0) {
const stashed = this.pendingBytes;
this.pendingBytes = [];
for (const bytes of stashed) {
this.decodePackets(this.demuxer.push(bytes));
}
}
this.decodePackets(this.demuxer.push(chunk));
// Wait until the WebCodecs decoder has processed the queued packets before draining.
await this.yieldToDecoder();
out.push(...this.drainDecoded());
return out;
}
async complete(): Promise<AudioBuffer[]> {
if (this.fatalError) {
return this.drainDecoded();
}
// End-of-stream may arrive while still throttled with bytes stashed (e.g. a short track
// that finished sending before the scheduler drained). Configure if needed and replay the
// stash so the tail is decoded before flush — otherwise the final seconds would be lost.
//
// OQ7/AC1-Opus precision note: the stash here is drained in full without a water-mark
// check. This is intentionally correct: the stream has ended — you cannot back-pressure a
// finished stream — and the remainder is tail-only (bounded by whatever the throttled C#
// loop left in flight, which is at most one push() worth of bytes). Adding a water-mark
// gate to complete() would silently drop the track's tail and is therefore wrong.
if (this.pendingBytes.length > 0) {
if (await this.ensureConfigured()) {
const stashed = this.pendingBytes;
this.pendingBytes = [];
for (const bytes of stashed) {
this.decodePackets(this.demuxer.push(bytes));
}
} else {
this.pendingBytes = [];
}
}
if (!this.decoder || this.decoder.state !== 'configured') {
return this.drainDecoded();
}
try {
await this.decoder.flush();
} catch (err) {
// A flush can reject if the decoder was reset/closed concurrently (track switch); the loop's
// own cancellation handles that — surface nothing, just drain what we have.
console.warn('Opus decoder flush interrupted:', (err as Error).message);
}
return this.drainDecoded();
}
/**
* Reinitialize for a Range-continuation stream after seek-beyond-buffer.
*
* @param landingTimeSeconds The actual page-start presentation time resolved from the seek index
* (t_page targetTimeSeconds). This is the time at which the decoder
* will begin emitting audio after reconfigure.
* @param targetTimeSeconds The user-requested seek position. The difference
* `(target - landing) * OPUS_SAMPLE_RATE` frames are trimmed from the
* head of the decoded output so playback lands precisely at the target
* (AC9 fine re-sync, §3.4a step 4).
*
* Pre-skip note: the reconfigure re-applies the WebCodecs Opus decoder's own pre-skip trim. The
* W3C spec is non-normative on the exact sample count and browsers vary (~312 samples at 48 kHz in
* practice). `leadTrimFrames` is computed from the sidecar's pre-skip-corrected presentation times
* (via `presentationTimeSeconds`), so it does NOT double-count the per-reconfigure pre-skip; the
* decoder handles that internally. If browser testing reveals a residual offset, adjust the
* `leadTrimFrames` calculation here this is the single point of control.
*/
reinitializeForRangeContinuation(landingTimeSeconds: number, targetTimeSeconds: number): void {
// New 206 body starts on a page boundary with no setup pages; the codec config is unchanged but
// inter-frame state must restart cleanly. AudioDecoder.reset() drops queued work and returns the
// decoder to 'unconfigured', so we reconfigure with the cached config. The demuxer goes into
// continuation mode (treat the first page's packets as audio — no setup pages in a 206 body).
this.demuxer.reset(true);
this.decodedQueue = [];
// Drop any bytes stashed by back-pressure: they belong to the PRE-seek stream position and
// must never be replayed against the post-seek (range-continuation) demux state (C6 — no
// stale feed racing the reset).
this.pendingBytes = [];
this.emittedFrames = 0; // post-seek buffers are positioned by the scheduler's playbackOffset
// Arm the lead trim: skip enough decoded frames to land at targetTimeSeconds, not at
// landingTimeSeconds (the page start). Clamp to ≥0 to guard against floating-point rounding.
this.leadTrimFrames = Math.max(0, Math.round((targetTimeSeconds - landingTimeSeconds) * OPUS_SAMPLE_RATE));
if (this.decoder && this.decoder.state === 'configured') {
this.decoder.reset();
this.decoder.configure(this.buildConfig());
}
}
dispose(): void {
for (const d of this.decodedQueue) {
try { d.close(); } catch { /* already closed */ }
}
this.decodedQueue = [];
this.pendingBytes = [];
if (this.decoder && this.decoder.state !== 'closed') {
try { this.decoder.close(); } catch { /* already closed */ }
}
this.decoder = null;
this.configured = false;
}
private decodePackets(packets: OpusPacket[]): void {
if (!this.decoder || this.decoder.state !== 'configured') return;
for (const pkt of packets) {
if (pkt.data.length === 0) continue;
// Every Opus packet is independently a "key" frame at the container level for WebCodecs's
// purposes — Opus has no key/delta distinction; 'key' is the correct type for all packets.
const chunk = new EncodedAudioChunk({
type: 'key',
timestamp: this.nextTimestampUs,
data: pkt.data
});
// Advance the synthetic clock by a packet's max duration; exact value is immaterial to us.
this.nextTimestampUs += Math.round((MAX_PACKET_FRAMES / OPUS_SAMPLE_RATE) * 1_000_000);
try {
this.decoder.decode(chunk);
} catch (err) {
console.error('Opus decode() threw:', (err as Error).message);
this.fatalError = true;
return;
}
}
}
/**
* Convert every queued AudioData into an AudioBuffer at the context sample rate, applying
* end-trim against the known total frame count and lead-trim for post-seek fine re-sync.
*/
private drainDecoded(): AudioBuffer[] {
const out: AudioBuffer[] = [];
const ctx = this.contextManager.getContext();
while (this.decodedQueue.length > 0) {
const data = this.decodedQueue.shift()!;
try {
const buffer = this.audioDataToBuffer(ctx, data);
if (buffer) out.push(buffer);
} finally {
try { data.close(); } catch { /* already closed */ }
}
}
return out;
}
/**
* Copy an AudioData's PCM into a new AudioBuffer, applying:
* 1. Lead-trim (post-seek fine re-sync): skip `leadTrimFrames` from the front so the audible
* start lands at the requested seek position, not at the preceding page boundary (AC9).
* 2. End-trim: cap cumulative output at `totalFrames` so the final partial frame's padding
* does not leak past the true stream end.
* Returns null when either trim leaves zero usable frames.
*/
private audioDataToBuffer(ctx: BaseAudioContext, data: AudioData): AudioBuffer | null {
const frames = data.numberOfFrames;
const channels = data.numberOfChannels;
// Lead-trim: consume frames from the front for post-seek fine re-sync (AC9).
let skip = 0;
if (this.leadTrimFrames > 0) {
skip = Math.min(this.leadTrimFrames, frames);
this.leadTrimFrames -= skip;
}
const available = frames - skip;
if (available <= 0) return null;
// End-trim: cap cumulative output at totalFrames.
let keep = available;
if (Number.isFinite(this.totalFrames)) {
const room = this.totalFrames - this.emittedFrames;
if (room <= 0) return null;
if (room < available) keep = room;
}
if (keep <= 0) return null;
const buffer = ctx.createBuffer(channels, keep, data.sampleRate);
// Allocate only for the frames we actually copy; frameOffset skips the lead-trim region.
const plane = new Float32Array(keep);
for (let ch = 0; ch < channels; ch++) {
data.copyTo(plane, { planeIndex: ch, frameOffset: skip, frameCount: keep, format: 'f32-planar' });
buffer.copyToChannel(plane, ch);
}
this.emittedFrames += keep;
return buffer;
}
/**
* Wait until the AudioDecoder's internal work queue drains (decodeQueueSize 0), so output
* callbacks have fired before we drain decodedQueue. Bounded to MAX_YIELD_ITERS × 4 ms to guard
* against a stuck decoder; any outputs collected before the cap are still returned. `complete()`
* uses decoder.flush() as its final barrier instead (flush() is the authoritative end-of-stream
* drain).
*/
private yieldToDecoder(): Promise<void> {
const MAX_YIELD_ITERS = 50; // 50 × 4 ms = 200 ms ceiling
return new Promise<void>((resolve) => {
let iters = 0;
const poll = () => {
if (!this.decoder || this.decoder.decodeQueueSize === 0 || iters >= MAX_YIELD_ITERS) {
// Hitting the 200 ms ceiling with the decode queue still non-empty means the WebCodecs
// decoder is falling behind realtime — the decode-starvation symptom that worsens with
// HW accel off (software WebGL render contending for the main thread). Report it as
// decode pressure so the visualizer throttles and yields the main thread back to decode.
if (this.decoder && iters >= MAX_YIELD_ITERS && this.decoder.decodeQueueSize > 0) {
decodePressure.report();
}
resolve();
return;
}
iters++;
setTimeout(poll, 4);
};
poll();
});
}
}
@@ -0,0 +1,834 @@
/**
* PlaybackScheduler partial-eviction tests (Phase 21.1) the anchor/index bookkeeping.
*
* The crux of 21.1 is that getCurrentPosition / playFromPosition / the schedule loop stay
* exact against a buffer array that no longer begins at absolute time 0 after front eviction.
* That math is pure given a clock and buffer durations, so it is testable in Node without a
* browser by injecting fakes for AudioContextManager and AudioBuffer (the scheduler only ever
* reads contextManager.currentTime, getGainNode(), getContext().createBufferSource(), and
* buffer.duration).
*
* Same harness convention as OpusStreamDecoder.test.ts: no test runner in this repo, run a
* copy from the COMPILED output so the `.js` import specifier resolves:
*
* dotnet build DeepDrftPublic/DeepDrftPublic.csproj
* cp DeepDrftPublic/Interop/audio/PlaybackScheduler.test.ts DeepDrftPublic/wwwroot/js/audio/
* node DeepDrftPublic/wwwroot/js/audio/PlaybackScheduler.test.ts
*
* A thrown error / non-zero exit signals failure; "ALL <n> TESTS PASSED" signals success.
* Excluded from the production tsc build via tsconfig `exclude: Interop/ ** /*.test.ts`.
*/
import { PlaybackScheduler } from './PlaybackScheduler.js';
import type { AudioContextManager } from './AudioContextManager.js';
// --- tiny inline harness (no dependencies) ---------------------------------------------------
let passed = 0;
const failures: string[] = [];
function test(name: string, fn: () => void): void {
try {
fn();
passed++;
} catch (e) {
failures.push(`FAIL: ${name}\n ${(e as Error).message}`);
}
}
function assertClose(actual: number, expected: number, msg?: string, eps = 1e-9): void {
if (Math.abs(actual - expected) > eps) {
throw new Error(`${msg ?? 'assertClose'}: expected ${expected}, got ${actual}`);
}
}
function assertEqual(actual: unknown, expected: unknown, msg?: string): void {
if (actual !== expected) {
throw new Error(`${msg ?? 'assertEqual'}: expected ${String(expected)}, got ${String(actual)}`);
}
}
// --- fakes -----------------------------------------------------------------------------------
/** A buffer source that records start/stop and fires onended on demand. */
class FakeSource {
public buffer: unknown = null;
public onended: (() => void) | null = null;
public started = false;
public stopped = false;
connect(): void { /* no-op */ }
start(): void { this.started = true; }
stop(): void {
this.stopped = true;
// The real Web Audio fires onended when a source is stopped; the scheduler relies on
// that for cleanup. Mirror it so handleSourceEnded paths are exercised.
this.onended?.();
}
}
/** Controllable clock + the minimal AudioContext surface the scheduler touches. */
class FakeContextManager {
public now = 0;
public sources: FakeSource[] = [];
get currentTime(): number { return this.now; }
getGainNode(): unknown { return {}; }
getContext(): unknown {
const self = this;
return {
createBufferSource(): FakeSource {
const s = new FakeSource();
self.sources.push(s);
return s;
}
};
}
}
/** A decoded buffer is, for the scheduler's purposes, just a duration. */
function buf(duration: number): AudioBuffer {
return { duration } as AudioBuffer;
}
/**
* A decoded buffer carrying realistic byte-footprint fields (length + numberOfChannels) for the
* OQ3 byte-ceiling test. Models 48 kHz stereo float PCM: length = duration × 48000 frames, 2 ch.
*/
function bufBytes(duration: number): AudioBuffer {
return { duration, length: Math.round(duration * 48000), numberOfChannels: 2 } as AudioBuffer;
}
function makeScheduler(cm: FakeContextManager): PlaybackScheduler {
// The scheduler only uses the subset FakeContextManager implements.
return new PlaybackScheduler(cm as unknown as AudioContextManager);
}
/**
* Drive the schedule cursor to the end of the buffer array WITHOUT running playback to
* completion, then clear the live-source set so neither nextBufferIndex nor a live source
* pins eviction. This isolates the back-retain threshold math from the live-frontier guards
* (which are exercised by their own tests).
*
* The lookahead in scheduleBuffersFrom only schedules ~500ms ahead per call; pushing the clock
* far back makes "lookahead" small so a single scheduleNewBuffers() call schedules everything
* remaining. We then drop the (white-box) live-source list and reset the schedule cursor to the
* end, leaving the array intact for a direct evictPlayedBuffers() call at a chosen position.
*/
function advanceCursorToEnd(s: PlaybackScheduler, cm: FakeContextManager): void {
const priv = s as unknown as { nextScheduleTime: number; nextBufferIndex: number; scheduledSources: unknown[] };
// Make the existing schedule anchor look "now" so the lookahead window is tiny, then let
// the scheduler lay down every remaining buffer in one pass.
priv.nextScheduleTime = cm.now;
s.scheduleNewBuffers();
// Repeat until the cursor reaches the end (lookahead may break early on long arrays).
let guard = 0;
while ((priv.nextBufferIndex as number) < s.getBufferCount() && guard++ < 1000) {
priv.nextScheduleTime = cm.now;
s.scheduleNewBuffers();
}
// Unpin the front: discard live sources without firing the onended cascade.
cm.sources.forEach(x => { x.onended = null; x.stopped = true; });
priv.scheduledSources.length = 0;
}
// --- tests -----------------------------------------------------------------------------------
// Position correctness after eviction: query current position after the front of the buffer
// array has been evicted; it must still equal wall-clock track time.
test('position stays exact after a front eviction', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
s.setBackRetainSeconds(0); // retain nothing behind the playhead — evict aggressively
// Ten 1s buffers, track [0,10).
for (let i = 0; i < 10; i++) s.addBuffer(buf(1));
cm.now = 0;
s.playFromPosition(0); // schedules a 500ms lookahead worth of sources from index 0
advanceCursorToEnd(s, cm);
cm.now = 3.0;
const dropped = s.evictPlayedBuffers();
if (dropped <= 0) throw new Error('expected front buffers to be evicted at t=3 with 0s retain');
// Absolute position must read 3.0 regardless of how many front buffers were dropped.
assertClose(s.getCurrentPosition(), 3.0, 'position after eviction');
// And buffers[0] no longer being the track start is reflected in the advanced offset.
if (s.getPlaybackOffset() <= 0) {
throw new Error('expected playbackOffset to advance past 0 after eviction');
}
});
// Eviction threshold respected: buffers older than back-retain are released; those within are
// kept. With back-retain = 2s at position 5, end<=3 is droppable, end in (3,..] is retained.
// Driven deterministically: advance the schedule cursor to the end (so nextBufferIndex does
// not pin eviction), clear live sources, then call eviction directly at a known position.
test('back-retain bound governs what is evicted', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
s.setBackRetainSeconds(2);
for (let i = 0; i < 10; i++) s.addBuffer(buf(1)); // track [0,10)
cm.now = 0;
s.playFromPosition(0);
advanceCursorToEnd(s, cm); // nextBufferIndex == 10, no live sources
cm.now = 5.0; // playhead at absolute t=5
const evicted = s.evictPlayedBuffers();
// currentPosition is 5.0; backRetain 2 => evictBefore = 3. Buffers ending at 1,2,3 are
// droppable (3 buffers); the buffer ending at 4 must be retained.
assertEqual(evicted, 3, 'evicted count under 2s back-retain at t=5');
assertEqual(s.getBufferCount(), 7, 'seven buffers retained');
assertClose(s.getPlaybackOffset(), 3.0, 'offset == dropped duration');
assertClose(s.getCurrentPosition(), 5.0, 'position unchanged by eviction');
});
// Resume-after-pause with an evicted front: playFromPosition resumes at the correct absolute
// time against the shortened array.
test('resume after pause lands at correct absolute time post-eviction', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
s.setBackRetainSeconds(1);
for (let i = 0; i < 10; i++) s.addBuffer(buf(1)); // [0,10)
cm.now = 0;
s.playFromPosition(0);
advanceCursorToEnd(s, cm);
cm.now = 4.0;
s.evictPlayedBuffers(); // back-retain 1 at t=4 => drops buffers ending <=3 (3 buffers)
// Pause at t=4: returns absolute position 4.0.
const paused = s.pause();
assertClose(paused, 4.0, 'pause returns absolute position');
// Front was evicted, so offset advanced. The buffer-relative anchor must net to absolute 4.
assertClose(s.getCurrentPosition(), 4.0, 'position holds at 4 while paused');
// Resume the way AudioPlayer.play does: buffer-relative = absolute - offset.
cm.now = 4.0;
const bufferRelative = paused - s.getPlaybackOffset();
if (bufferRelative < 0) throw new Error('buffer-relative resume position went negative');
s.playFromPosition(bufferRelative);
cm.now = 4.0;
assertClose(s.getCurrentPosition(), 4.0, 'resume restored absolute position');
});
// Seek-back into still-retained buffers works: with back-retain holding recent audio, a short
// backward seek stays in-buffer (queryable/playable), no clamp to the new front.
test('short seek-back into retained region resolves in-buffer', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
s.setBackRetainSeconds(3);
for (let i = 0; i < 10; i++) s.addBuffer(buf(1)); // [0,10)
cm.now = 0;
s.playFromPosition(0);
advanceCursorToEnd(s, cm);
cm.now = 6.0;
s.evictPlayedBuffers(); // back-retain 3 at t=6 => evictBefore=3, drops buffers ending <=3
const offset = s.getPlaybackOffset();
// back-retain 3 at t=6 => evictBefore=3, so buffers ending <=3 dropped, offset==3.
assertClose(offset, 3.0, 'offset after eviction with 3s retain');
// The retained region is [offset, totalEnd) == [3, 10). A seek back to t=4 is inside it.
const seekTarget = 4.0;
const bufferRelative = seekTarget - offset; // 1.0 into the retained array
if (bufferRelative < 0) throw new Error('seek-back target fell below retained front (should be in-buffer)');
cm.now = 6.0;
s.playFromPosition(bufferRelative);
cm.now = 6.0;
assertClose(s.getCurrentPosition(), seekTarget, 'seek-back resolved to absolute target');
});
// Eviction never crosses the live frontier: a buffer still referenced by an unstopped source
// must not be dropped even if the clock says it is "behind".
test('eviction does not drop buffers under live sources or past the schedule cursor', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
s.setBackRetainSeconds(0);
for (let i = 0; i < 10; i++) s.addBuffer(buf(1));
cm.now = 0;
s.playFromPosition(0); // schedules ~first 500ms+ of sources; they remain live (not ended)
// Jump the clock far ahead WITHOUT ending the live sources.
cm.now = 9.0;
const before = s.getBufferCount();
const dropped = s.evictPlayedBuffers();
// Nothing past the schedule cursor or under a live source may be dropped. The scheduled
// (live) sources pin the front, so eviction is bounded — it must not strip the whole array.
if (s.getBufferCount() < 0) throw new Error('buffer count went negative');
assertEqual(s.getBufferCount(), before - dropped, 'count matches dropped');
// The live sources start at index 0, so firstLiveIndex pins eviction at 0 — nothing drops.
assertEqual(dropped, 0, 'no eviction while front sources are live');
});
// handleSourceEnded cascade: eviction fires from the real production trigger (onended), not
// via a direct evictPlayedBuffers() call. Confirms the anchor/index invariants hold end-to-end
// through the scheduler's own event handling while playback is still active with a live source.
//
// Setup: 0.3s buffers so the 500ms lookahead window fits exactly two sources after
// playFromPosition(0). Buffer 0 ends at ~0.31s, buffer 1 ends at ~0.61s — both are scheduled.
// Clock is then advanced to t=0.6 so buffer 0's end (0.31) < evictBefore (0.6) while the live
// source on buffer 1 pins firstLiveIndex=1, blocking further eviction. This is the mid-array
// pinning scenario that later waves (21.2/21.3) build on.
test('eviction via handleSourceEnded: position exact, live bufferIndex decremented, frontier respected', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
// Retain nothing behind the playhead — evict aggressively so the cascade fires.
s.setBackRetainSeconds(0);
// Eight 0.3s buffers. scheduleBuffersFrom with lookaheadTarget=0.5s at t=0:
// after buf 0: nextScheduleTime≈0.31, lookahead=0.31 < 0.5 → continues
// after buf 1: nextScheduleTime≈0.61, lookahead=0.61 > 0.5 → breaks
// → exactly two sources are live after playFromPosition.
for (let i = 0; i < 8; i++) s.addBuffer(buf(0.3));
cm.now = 0;
s.playFromPosition(0);
// Reach inside to see which sources were scheduled and what bufferIndex they hold.
const priv = s as unknown as {
scheduledSources: Array<{ source: FakeSource; bufferIndex: number; startTime: number; endTime: number }>;
nextBufferIndex: number;
};
// Confirm two sources are live — the setup guarantee.
if (priv.scheduledSources.length < 2) {
throw new Error(`Expected ≥2 scheduled sources after playFromPosition, got ${priv.scheduledSources.length}`);
}
// Identify the first and second scheduled sources by bufferIndex order.
const sorted = [...priv.scheduledSources].sort((a, b) => a.bufferIndex - b.bufferIndex);
const firstScheduled = sorted[0]; // bufferIndex 0
const secondScheduled = sorted[1]; // bufferIndex 1
const secondBufferIndexBefore = secondScheduled.bufferIndex; // must be 1
// Record the second FakeSource so we can assert it was not stopped by eviction.
const secondFakeSource = secondScheduled.source as unknown as FakeSource;
// Advance clock to 0.6s. Buffer 0 ends at ~0.31s → evictBefore=0.6, end=0.31 ≤ 0.6 →
// droppable. Buffer 1 ends at ~0.61s → its live source pins firstLiveIndex=1 → NOT dropped.
cm.now = 0.6;
// Confirm playback is still active before firing the cascade.
assertEqual(s.isActive(), true, 'isActive must be true before cascade');
// Fire the cascade via the production trigger: stop the first source, which calls onended,
// which calls handleSourceEnded, which calls evictPlayedBuffers internally.
(firstScheduled.source as unknown as FakeSource).stop();
// (a) Absolute position must remain exactly 0.6.
assertClose(s.getCurrentPosition(), 0.6, 'position after handleSourceEnded cascade');
// (b) The second live source's bufferIndex must have been decremented by 1 (the one evicted
// front buffer), shifting it from absolute index 1 to absolute index 0.
const expectedSecondIndex = secondBufferIndexBefore - 1;
assertEqual(secondScheduled.bufferIndex, expectedSecondIndex, 'live source bufferIndex decremented');
// (c) Eviction stopped at firstLiveIndex=1, not nextBufferIndex — the second buffer was
// NOT dropped. Verify the second source was not stopped (it remained live throughout).
assertEqual(secondFakeSource.stopped, false, 'live second source not stopped by eviction');
// And the scheduler still has buffers (the array was not wiped past the frontier).
if (s.getBufferCount() === 0) {
throw new Error('eviction wiped all buffers — should have stopped at firstLiveIndex');
}
});
// === Phase 21.2 back-pressure: the forward water-mark signal =================================
//
// The signal is pure given the clock + buffer durations + the playhead position, so it is
// testable in Node with the same fakes. We drive forward lookahead by adding buffers (fill) and
// advancing the clock (drain), and assert the hysteresis latch and the OQ3 byte ceiling.
/**
* Fill the scheduler with `count` 1 s buffers, start playback at t=0, and advance the schedule
* cursor to the end so nextBufferIndex does not pin anything. Leaves all `count` buffers decoded
* and the playhead at the clock position the caller sets afterwards.
*/
function fillAndStart(s: PlaybackScheduler, cm: FakeContextManager, count: number): void {
for (let i = 0; i < count; i++) s.addBuffer(buf(1));
cm.now = 0;
s.playFromPosition(0);
advanceCursorToEnd(s, cm);
}
// High-water reached → production pauses; the signal reflects the forward lookahead.
test('evaluateProductionPause latches true when forward lookahead reaches high-water', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
s.setForwardWindow(10, 5, 0); // high 10s, low 5s, byte cap disabled
fillAndStart(s, cm, 40); // 40s decoded, track [0,40)
cm.now = 0; // playhead at 0 → forward lookahead = 40s ≥ 10s high-water
assertEqual(s.getForwardLookaheadSeconds(), 40, 'lookahead is full decoded tail at t=0');
assertEqual(s.evaluateProductionPause(), true, 'pauses at/above high-water');
});
// Below high-water but above low-water while NOT yet paused → stays unpaused (no premature pause).
test('evaluateProductionPause stays false in the hysteresis band before the high-water crossing', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
s.setForwardWindow(10, 5, 0);
fillAndStart(s, cm, 8); // 8s decoded
cm.now = 0; // lookahead 8s: between low(5) and high(10), never latched → unpaused
assertEqual(s.evaluateProductionPause(), false, 'no pause until high-water is actually reached');
});
// Hysteresis: once paused at high-water, stays paused through the band until lookahead drains
// below low-water, then resumes. Drain is modeled by advancing the clock (playhead moves forward,
// shrinking forward lookahead).
test('evaluateProductionPause holds through the band and resumes only below low-water', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
s.setForwardWindow(10, 5, 0);
fillAndStart(s, cm, 40); // track [0,40)
cm.now = 0;
assertEqual(s.evaluateProductionPause(), true, 'latched at high-water (40s ahead)');
// Playhead at 32 → lookahead 8s: in the band (5..10) → must STAY paused (hysteresis).
cm.now = 32;
assertEqual(s.getForwardLookaheadSeconds(), 8, 'lookahead drained to 8s');
assertEqual(s.evaluateProductionPause(), true, 'still paused inside the band');
// Playhead at 36 → lookahead 4s ≤ low-water 5 → resume.
cm.now = 36;
assertEqual(s.getForwardLookaheadSeconds(), 4, 'lookahead below low-water');
assertEqual(s.evaluateProductionPause(), false, 'resumes below low-water');
// Refill back over high-water re-latches (the next chunk would re-pause).
for (let i = 0; i < 20; i++) s.addBuffer(buf(1)); // +20s decoded ahead
assertEqual(s.evaluateProductionPause(), true, 're-latches when fill exceeds high-water again');
});
// OQ3 hard byte ceiling pauses production independent of the time window, and releases as soon as
// the footprint is back under the cap (no separate low-water band on the byte guard).
test('OQ3 byte ceiling pauses regardless of the time window', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
// Each 1s buffer here is 48000 frames × 2 ch × 4 bytes = 384000 bytes. Cap at ~1.5 MB ≈ 4 buffers.
const perBuffer = 48000 * 2 * 4;
s.setForwardWindow(1000, 500, perBuffer * 4); // time window huge so only the byte cap can fire
for (let i = 0; i < 6; i++) s.addBuffer(bufBytes(1)); // 6 buffers > 4-buffer cap
cm.now = 0;
s.playFromPosition(0);
advanceCursorToEnd(s, cm);
cm.now = 0;
if (s.getDecodedByteEstimate() <= perBuffer * 4) {
throw new Error('test setup: byte estimate should exceed the cap');
}
assertEqual(s.evaluateProductionPause(), true, 'byte ceiling pauses even with a huge time window');
});
// clear() / clearForSeek() release the latch so a fresh stream/seek starts unthrottled (C2).
test('clear and clearForSeek release the back-pressure latch (C2 latency parity)', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
s.setForwardWindow(10, 5, 0);
fillAndStart(s, cm, 40);
cm.now = 0;
assertEqual(s.evaluateProductionPause(), true, 'latched');
s.clear();
// After clear there are no buffers, lookahead is 0, and the latch is reset → unpaused.
assertEqual(s.evaluateProductionPause(), false, 'clear resets the latch and empties fill');
fillAndStart(s, cm, 40);
cm.now = 0;
assertEqual(s.evaluateProductionPause(), true, 'latched again after refill');
s.clearForSeek();
assertEqual(s.evaluateProductionPause(), false, 'clearForSeek resets the latch');
});
// Production defaults (no setForwardWindow): the widened 60s/30s cushion. The byte cap is the
// UNCHANGED hard OOM bound; these defaults only govern the time window. buf(1) carries no byte
// fields, so getDecodedByteEstimate is NaN and the byte guard never fires — the time window alone
// governs, which is exactly what we want to pin here.
test('default forward window throttles at 60s and resumes at 30s (no setForwardWindow)', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
// Deliberately no setForwardWindow() — exercise the PRODUCTION defaults (high 60s / low 30s).
for (let i = 0; i < 70; i++) s.addBuffer(buf(1)); // 70s decoded, track [0,70)
cm.now = 0;
s.playFromPosition(0);
advanceCursorToEnd(s, cm);
cm.now = 0; // forward lookahead = 70s ≥ 60s high-water
assertEqual(s.evaluateProductionPause(), true, 'pauses at the 60s default high-water');
cm.now = 35; // lookahead 35s: inside the 30..60 band → stays paused (hysteresis)
assertEqual(s.evaluateProductionPause(), true, 'holds through the widened band');
cm.now = 45; // lookahead 25s ≤ 30s low-water → resume
assertEqual(s.evaluateProductionPause(), false, 'resumes at the 30s default low-water');
});
// Lookahead correctness in the underrun state + the prime block hypothesis directly refuted: when the
// playhead has drained the queue mid-stream, forward lookahead must read ~0 (not a stale-high value)
// so production is NOT throttled while decoded audio is genuinely low.
test('forward lookahead is exact during an underrun park and never trips a false pause', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
// 5s decoded, playback started, stream NOT complete.
for (let i = 0; i < 5; i++) s.addBuffer(buf(1));
cm.now = 0;
s.playFromPosition(0);
// Playhead advances past the decoded tail and the queue drains → mid-stream underrun park.
cm.now = 6;
drainAllSources(s, cm);
assertEqual(s.isActive(), false, 'parked in underrun');
// At the park the playhead sits at the decoded tail: forward lookahead is 0, so production must
// NOT be throttled (the "paused while decoded audio is low" hypothesis must not hold here).
assertClose(s.getForwardLookaheadSeconds(), 0, 'lookahead is 0 at the underrun tail');
assertEqual(s.evaluateProductionPause(), false, 'low decoded audio does not pause production');
// Refill arriving during the park grows the lead monotonically; lookahead reflects exactly it,
// measured against the FROZEN playhead — not a stale pre-underrun position.
s.addBuffer(buf(1));
s.addBuffer(buf(1));
assertClose(s.getForwardLookaheadSeconds(), 2, 'lookahead equals the freshly-accumulated lead');
assertEqual(s.evaluateProductionPause(), false, 'still unthrottled well below the 60s high-water');
});
// === False end-of-playback guard (Opus-startup misfire) ======================================
//
// The scheduler must distinguish a GENUINE end-of-track (stream complete AND queue drained) from a
// transient startup/underrun gap (queue drained while bytes are still streaming — Opus decodes via
// WebCodecs asynchronously, so the first buffers can lag the playback-start minimum). The end
// callback fires only in the first case. These tests drive the real handleSourceEnded cascade via
// FakeSource.stop() and assert onPlaybackEnded fires exactly when it should.
/** Drive the schedule cursor + live sources to a fully-drained queue at the buffer tail. */
function drainAllSources(s: PlaybackScheduler, cm: FakeContextManager): void {
const priv = s as unknown as { scheduledSources: Array<{ source: FakeSource }> };
let guard = 0;
while (priv.scheduledSources.length > 0 && guard++ < 10000) {
// Stop the head source; its onended → handleSourceEnded removes it and schedules the next.
priv.scheduledSources[0].source.stop();
}
}
// A drained queue MID-STREAM (streamComplete false) must NOT fire onPlaybackEnded — it parks in
// underrun instead. This is the exact Opus-startup false-end.
test('drained queue while still streaming does not fire onPlaybackEnded (no false end)', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
let ended = 0;
s.onPlaybackEnded = () => { ended++; };
// A short run of buffers, playback started, but the stream is NOT marked complete.
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
cm.now = 0;
s.playFromPosition(0);
// Advance the clock past the buffered tail and drain every scheduled source.
cm.now = 1.0;
drainAllSources(s, cm);
assertEqual(ended, 0, 'no end callback fired mid-stream');
assertEqual(s.isActive(), false, 'scheduler parked (inactive) on underrun');
});
// After a mid-stream underrun, newly decoded buffers must RESUME playback (scheduleNewBuffers
// re-anchors and re-activates) — not stay stuck, and still not fire a false end.
test('underrun resumes when new buffers arrive', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
let ended = 0;
s.onPlaybackEnded = () => { ended++; };
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
cm.now = 0;
s.playFromPosition(0);
cm.now = 1.0;
drainAllSources(s, cm); // underrun
assertEqual(s.isActive(), false, 'inactive after underrun');
// Decode catches up: enough buffers arrive to clear the 1s rebuffer lead (4 × 0.3 = 1.2s).
for (let i = 0; i < 4; i++) s.addBuffer(buf(0.3));
s.scheduleNewBuffers();
assertEqual(s.isActive(), true, 'resumed active after refill');
assertEqual(ended, 0, 'still no false end after resume');
const priv = s as unknown as { scheduledSources: unknown[] };
if (priv.scheduledSources.length === 0) {
throw new Error('expected new sources scheduled on resume');
}
});
// === Rebuffer hysteresis (Opus-startup thrash fix) ===========================================
//
// After a mid-stream underrun the scheduler must NOT resume on the first arriving buffer (which,
// for ~20 ms Opus packets, plays one buffer, drains, and re-parks — the audible start/stop thrash).
// It re-accumulates a healthy decoded LEAD (DEFAULT_MIN_PLAYBACK_LEAD_SECONDS = 1s) first. The
// streamComplete override is the escape hatch so a genuine short tail still plays out, never parking
// forever. These drive the real handleSourceEnded/scheduleNewBuffers/setStreamComplete paths.
// Below the rebuffer lead: a thin refill must keep the scheduler parked (no resume, no false end);
// once the accumulated lead crosses the threshold, it resumes.
test('underrun does not resume below the rebuffer lead, resumes once it is met', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
let ended = 0;
s.onPlaybackEnded = () => { ended++; };
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
cm.now = 0;
s.playFromPosition(0);
cm.now = 1.0;
drainAllSources(s, cm); // underrun
assertEqual(s.isActive(), false, 'parked in underrun');
// Only 0.6s of fresh lead arrives — below the 1s rebuffer threshold. Must stay parked.
for (let i = 0; i < 2; i++) s.addBuffer(buf(0.3));
s.scheduleNewBuffers();
assertEqual(s.isActive(), false, 'still parked — lead below the rebuffer threshold');
assertEqual(ended, 0, 'no false end while re-accumulating lead');
const priv = s as unknown as { scheduledSources: unknown[] };
assertEqual(priv.scheduledSources.length, 0, 'nothing scheduled below the threshold');
// More lead arrives, crossing the threshold (0.6 + 0.6 = 1.2s ≥ 1s) → now resume.
for (let i = 0; i < 2; i++) s.addBuffer(buf(0.3));
s.scheduleNewBuffers();
assertEqual(s.isActive(), true, 'resumes once the lead crosses the threshold');
assertEqual(ended, 0, 'still no false end after resume');
});
// Genuine-end tail SHORTER than the rebuffer lead: while parked, a small tail arrives AND the stream
// completes. The threshold is overridden so the tail plays out and the genuine end fires exactly
// once — the scheduler must never park forever waiting for a lead that will never come.
test('streamComplete tail below the rebuffer lead still plays out and fires end once', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
let ended = 0;
s.onPlaybackEnded = () => { ended++; };
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
cm.now = 0;
s.playFromPosition(0);
cm.now = 1.0;
drainAllSources(s, cm); // underrun
assertEqual(s.isActive(), false, 'parked in underrun');
// A short final tail (0.6s, below the 1s threshold) arrives; the hysteresis keeps it parked.
for (let i = 0; i < 2; i++) s.addBuffer(buf(0.3));
s.scheduleNewBuffers();
assertEqual(s.isActive(), false, 'parked — tail below threshold, stream not yet complete');
assertEqual(ended, 0, 'no end before completion');
// The stream completes: the threshold no longer applies → the tail schedules and plays out.
s.setStreamComplete(true);
assertEqual(s.isActive(), true, 'resumed to play out the final tail on completion');
assertEqual(ended, 0, 'end not fired until the tail drains');
// Drain the tail → genuine end fires exactly once.
cm.now = 2.0;
drainAllSources(s, cm);
assertEqual(ended, 1, 'genuine end fires exactly once after the tail drains');
assertEqual(s.isActive(), false, 'inactive after genuine end');
});
// GENUINE end: stream complete AND queue drains → onPlaybackEnded fires exactly once.
test('genuine end (streamComplete + drained) fires onPlaybackEnded exactly once', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
let ended = 0;
s.onPlaybackEnded = () => { ended++; };
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
cm.now = 0;
s.playFromPosition(0);
s.setStreamComplete(true); // all bytes in, no more buffers coming
cm.now = 1.0;
drainAllSources(s, cm);
assertEqual(ended, 1, 'end fired once on genuine completion');
assertEqual(s.isActive(), false, 'inactive after genuine end');
});
// setStreamComplete arriving AFTER the queue has already drained mid-stream (the tail produced no
// new buffers) must finalise immediately — the genuine-end signal that landed late.
test('setStreamComplete after an already-drained queue finalises immediately', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
let ended = 0;
s.onPlaybackEnded = () => { ended++; };
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
cm.now = 0;
s.playFromPosition(0);
cm.now = 1.0;
drainAllSources(s, cm); // underrun, no end yet
assertEqual(ended, 0, 'no end before completion signal');
s.setStreamComplete(true); // signal arrives now → finalise
assertEqual(ended, 1, 'end fired when completion signalled post-drain');
});
// clearForSeek must reset streamComplete so a post-seek refill cannot inherit a stale "complete"
// and fire a premature end before its own bytes arrive.
test('clearForSeek resets streamComplete (no inherited end on refill)', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
let ended = 0;
s.onPlaybackEnded = () => { ended++; };
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
cm.now = 0;
s.playFromPosition(0);
s.setStreamComplete(true);
s.clearForSeek();
s.setPlaybackOffset(5);
// Post-seek continuation: fresh buffers, playback resumes, stream NOT yet complete.
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
cm.now = 5;
s.playFromPosition(0);
cm.now = 6.0;
drainAllSources(s, cm);
assertEqual(ended, 0, 'no end fired — stale streamComplete was cleared by clearForSeek');
});
// pause() during underrun: setStreamComplete must NOT fire end while the user is paused.
// This is the narrow window the fix to pause() closes: without the underrun_ clear, a paused
// scheduler that was mid-underrun satisfies the setStreamComplete immediate-finalise guard
// (complete && underrun_ && drained) and fires TrackEnded / queue-advance while paused.
test('pause during underrun: setStreamComplete does not fire end while paused', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
let ended = 0;
s.onPlaybackEnded = () => { ended++; };
// A short run of buffers, drain them mid-stream → scheduler parks in underrun.
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
cm.now = 0;
s.playFromPosition(0);
cm.now = 1.0;
drainAllSources(s, cm); // queue drained, streamComplete still false → underrun
assertEqual(s.isActive(), false, 'parked in underrun after drain');
assertEqual(ended, 0, 'no end before pause');
// User pauses while the scheduler is parked in underrun.
s.pause();
// Stream completes with no further buffers (the tail produced nothing new).
// With the fix, pause() cleared underrun_ so this must NOT finalise immediately.
s.setStreamComplete(true);
assertEqual(ended, 0, 'no end fired while paused — setStreamComplete must not fire during pause');
assertEqual(s.isActive(), false, 'scheduler stays inactive after setStreamComplete during pause');
});
// underrun → resume → genuine end fires exactly once: the full composition from a mid-stream gap
// through resumed playback to completion. Confirms no double-fire and no stuck scheduler.
test('underrun → resume → genuine end fires exactly once', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
let ended = 0;
s.onPlaybackEnded = () => { ended++; };
// Drain initial buffers into underrun.
for (let i = 0; i < 3; i++) s.addBuffer(buf(0.3));
cm.now = 0;
s.playFromPosition(0);
cm.now = 1.0;
drainAllSources(s, cm);
assertEqual(s.isActive(), false, 'underrun after initial drain');
assertEqual(ended, 0, 'no end count during underrun');
// Decode catches up: enough buffers arrive to clear the 1s rebuffer lead (4 × 0.3 = 1.2s).
for (let i = 0; i < 4; i++) s.addBuffer(buf(0.3));
s.scheduleNewBuffers();
assertEqual(s.isActive(), true, 'resumed active after refill');
assertEqual(ended, 0, 'still no end after resume');
// Mark the stream complete, then drain the resumed sources to genuine end.
s.setStreamComplete(true);
cm.now = 2.0;
drainAllSources(s, cm);
assertEqual(ended, 1, 'end fires exactly once after genuine completion');
assertEqual(s.isActive(), false, 'inactive after genuine end');
});
// === Complete-without-start (force-start fallback) ==========================================
//
// The C# producer calls StartStreamingPlayback after MarkStreamCompleteAsync when
// _streamingPlaybackStarted is still false (total audio below the start threshold). The JS-side
// effect is playFromPosition(0) called with streamComplete already true. This section covers the
// scheduler-side guarantee: sub-threshold buffers + streamComplete already set + forced
// playFromPosition drains and fires end exactly once, never zero, never twice.
//
// The C# transition itself is not exercisable here (requires StreamingAudioPlayerService +
// AudioInteropService), so the test covers the scheduler drain-and-end-once contract directly.
// Forced start after completion: sub-threshold total audio, streamComplete set BEFORE
// playFromPosition(0), sources drain and onPlaybackEnded fires exactly once.
test('forced start on complete stream: sub-threshold buffers drain and fire end exactly once', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
let ended = 0;
s.onPlaybackEnded = () => { ended++; };
// Sub-threshold buffers (0.4s total, below the 1s rebuffer lead). Never started.
for (let i = 0; i < 2; i++) s.addBuffer(buf(0.2));
// Stream marks complete BEFORE playback starts — the C# completion-path ordering:
// MarkStreamCompleteAsync fires first, then StartStreamingPlayback is called because
// _streamingPlaybackStarted is false. setStreamComplete with underrun_=false returns
// early (sets the flag but does not schedule/finalize — that is correct, nothing to drain yet).
s.setStreamComplete(true);
assertEqual(ended, 0, 'no end fired at setStreamComplete — playback not yet started');
assertEqual(s.isActive(), false, 'scheduler inactive before forced start');
// Forced start: C# calls startStreamingPlayback() → playFromPosition(0).
// With streamComplete already true and buffers present, this schedules all buffers.
cm.now = 0;
s.playFromPosition(0);
const priv = s as unknown as { scheduledSources: unknown[] };
if (priv.scheduledSources.length === 0) {
throw new Error('expected sources scheduled after forced playFromPosition');
}
assertEqual(ended, 0, 'end not fired yet — sources must drain first');
assertEqual(s.isActive(), true, 'scheduler active while sources are scheduled');
// Drain sources → streamComplete is true → genuine end fires exactly once.
cm.now = 0.5;
drainAllSources(s, cm);
assertEqual(ended, 1, 'end fires exactly once after forced-start drain');
assertEqual(s.isActive(), false, 'scheduler inactive after genuine end');
});
// No double-fire: calling setStreamComplete again after end has already fired is a no-op.
test('setStreamComplete after forced-start drain is a no-op (no double end)', () => {
const cm = new FakeContextManager();
const s = makeScheduler(cm);
let ended = 0;
s.onPlaybackEnded = () => { ended++; };
for (let i = 0; i < 2; i++) s.addBuffer(buf(0.2));
s.setStreamComplete(true);
cm.now = 0;
s.playFromPosition(0);
cm.now = 0.5;
drainAllSources(s, cm);
assertEqual(ended, 1, 'end fired once after forced-start drain');
// A redundant setStreamComplete (e.g. called again from a stale C# path) must not re-fire.
s.setStreamComplete(true);
assertEqual(ended, 1, 'still exactly one end after redundant setStreamComplete');
});
// --- run -------------------------------------------------------------------------------------
if (failures.length > 0) {
console.error(failures.join('\n'));
console.error(`\n${failures.length} FAILED, ${passed} passed`);
process.exit(1);
} else {
console.log(`ALL ${passed} TESTS PASSED`);
}
+449 -17
View File
@@ -2,10 +2,94 @@
* PlaybackScheduler - Manages AudioBuffer storage and playback scheduling.
*
* Single Responsibility: Store decoded buffers and schedule them for playback.
* Supports pause/resume/seek by retaining all buffers.
*
* Memory model (Phase 21.1 partial eviction)
* --------------------------------------------
* The scheduler is the single shared sink both decode paths feed (WAV/MP3/FLAC via
* `IFormatDecoder`, Opus via the WebCodecs `IStreamingDecoder`); eviction lives here once
* and serves both with zero format branches.
*
* THE INDEX/TIME-ANCHOR INVARIANT (the crux of 21.1):
* `playbackOffset` is the absolute track time at which `buffers[0]` begins. Every
* position query and scheduling decision is expressed as `playbackOffset` + a sum of
* `buffers[i].duration` from index 0. Originally `buffers[0]` was always the track start,
* so `playbackOffset` was 0 except after a seek-beyond-buffer. After partial eviction
* `buffers[0]` is no longer the track start so eviction MUST add the dropped buffers'
* total duration to `playbackOffset`. That one move keeps `getCurrentPosition`,
* `playFromPosition`, the `getTotalDuration`-based clamp/bounds, and the schedule loop all
* exact against a buffer array that no longer starts at absolute time 0.
*
* The second half of the invariant is the array indices. `nextBufferIndex` and every live
* `scheduledSources[].bufferIndex` are absolute positions into `buffers`; splicing `k`
* buffers off the front shifts every surviving index down by `k`, so both must be
* decremented by `k`. Eviction therefore never crosses the live frontier: it will not drop
* a buffer at/after `nextBufferIndex`, nor one still referenced by a scheduled source.
*/
import { AudioContextManager } from './AudioContextManager.js';
import { decodePressure } from './decodePressure.js';
/**
* Provisional back-retain default. The window-size POLICY (OQ1/OQ3) is not decided yet, so
* this is intentionally a tunable seam (see setBackRetainSeconds), not a baked-in number
* 21.2 feeds real water-marks in later. The default keeps a few seconds of already-played
* audio so a short seek-back stays in-buffer (UC3) without a network refetch.
*/
const DEFAULT_BACK_RETAIN_SECONDS = 10;
/**
* Forward back-pressure water-marks (Phase 21.2 the bound on the *unplayed* region).
*
* The single back-pressure signal is the scheduler's decoded forward lookahead: how many
* seconds of decoded audio sit AHEAD of the playhead (OQ7). Production (the C# read loop and,
* for Opus, the demux/decode feed) pauses above the high-water mark and resumes below the
* low-water mark classic hysteresis so the two producers do not chatter on/off per chunk.
*
* Time-based defaults the cushion, NOT the memory bound:
* - HIGH (60 s): the most decoded lookahead we hold ahead of the playhead before throttling.
* Comfortably above the playback-start minimum (`AudioPlayer.minBuffersForPlayback = 6`
* buffers, each typically 0.06 1 s depending on format/chunk size), so C2 holds first
* audio never waits on a throttle (the high-water is reached only well after playback runs).
* - LOW (30 s): resume producing here. Kept generous so the forward fill never drains to the
* ~500 ms scheduler lookahead under network/decode jitter (AC3 no starvation).
*
* Why 60/30 and not the old 30/15: the time window is a CUSHION knob, not the memory guarantee
* the OQ3 byte ceiling below is the hard OOM bound. The old 30 s was sized for WAV's byte density
* and needlessly starved the cushion for the async WebCodecs Opus path, whose decoded float
* footprint is tiny (48 kHz stereo 0.37 MB/s, so 60 s 23 MB a fraction of the 96 MB cap)
* yet whose per-packet decode jitter (HW-accel-off software decode, main-thread AudioData copies)
* needs a deeper buffer to stay ahead of the playhead. Doubling the window lets Opus use the memory
* headroom the byte cap already permits. The byte cap is UNCHANGED, so a high-footprint stream
* still pauses at exactly the same footprint as before the OOM fix does not regress.
*
* OQ3 hard memory ceiling: an absolute byte cap on total decoded float held, independent of the
* time window. This is the guard-rail that makes "1 GB never OOMs" a guarantee rather than a
* tuning hope production pauses on `lookahead >= high OR bytes > cap`, whichever fires first, so
* the footprint can never exceed the cap regardless of the time window. The decoded f32 footprint
* scales with sample rate × channels (not source codec), so for high-sample-rate / multichannel
* audio the byte cap fires before 60 s (bounding memory exactly as the old 30 s window's byte
* estimate did); for sparse 48 kHz stereo Opus the time window fires first, at ~23 MB. Estimated
* as channels × frames × 4 (f32).
*/
const DEFAULT_FORWARD_HIGH_WATER_SECONDS = 60;
const DEFAULT_FORWARD_LOW_WATER_SECONDS = 30;
const DEFAULT_MAX_DECODED_BYTES = 96 * 1024 * 1024; // ~96 MB of decoded float PCM — the HARD OOM bound
const BYTES_PER_FLOAT_SAMPLE = 4;
/**
* Rebuffer hysteresis lead the minimum SECONDS of decoded-but-unscheduled audio that must
* accumulate ahead of the schedule cursor before playback may (re)start after a mid-stream underrun.
*
* Why seconds, not a buffer count: the per-buffer duration differs wildly by format. A WAV/lossless
* segment is a sizeable slab (~0.10.4 s); a single Opus WebCodecs packet is ~20 ms. The old resume
* path re-anchored on the FIRST arriving buffer, so for Opus it scheduled ~20 ms, drained it, parked,
* resumed on the next ~20 ms, and so on the audible start/stop thrash during the WebCodecs decode
* ramp. Gating on a fixed LEAD in seconds gives a resume the same cushion a fresh start has,
* independent of format. 1 s is the same order as the lossless playback-start lead (~6 segments) and
* sits far below the 60 s forward high-water, so back-pressure never throttles production while the
* scheduler is still re-accumulating this lead. Tunable; not magic.
*/
const DEFAULT_MIN_PLAYBACK_LEAD_SECONDS = 1.0;
interface ScheduledSource {
source: AudioBufferSourceNode;
@@ -26,11 +110,53 @@ export class PlaybackScheduler {
private nextScheduleTime: number = 0; // AudioContext time for next buffer
private isActive_: boolean = false; // Prevents scheduling during pause/stop
// Offset for seek-beyond-buffer scenarios
// When seeking to position T beyond buffers, we clear buffers and set playbackOffset = T
// The new stream starts at T, so buffer positions are relative to T
// Offset for seek-beyond-buffer scenarios AND partial eviction.
// This is the absolute track time at which buffers[0] begins. It is set on
// seek-beyond-buffer (the new stream starts at T) and ADVANCED by eviction (when the
// front k buffers are dropped, their total duration is added here so buffers[0] still
// names the correct absolute time). See the index/time-anchor invariant in the header.
private playbackOffset: number = 0;
// Back-retain bound (seconds of already-played audio kept un-evicted). Provisional seam;
// 21.2 will drive this from the window policy. Not a hardcoded eviction decision.
private backRetainSeconds: number = DEFAULT_BACK_RETAIN_SECONDS;
// Forward back-pressure water-marks + the OQ3 hard byte ceiling (Phase 21.2). This is the
// single shared window policy (OQ6): both producers call evaluateProductionPause() and honor it
// in their own way — the C# read loop stops ReadAsync, the Opus feed stops demux/decode.
private forwardHighWaterSeconds: number = DEFAULT_FORWARD_HIGH_WATER_SECONDS;
private forwardLowWaterSeconds: number = DEFAULT_FORWARD_LOW_WATER_SECONDS;
private maxDecodedBytes: number = DEFAULT_MAX_DECODED_BYTES;
// Rebuffer hysteresis lead (seconds). The minimum decoded-but-unscheduled audio that must sit
// ahead of the schedule cursor before playback may (re)start — at a fresh start AND after a
// mid-stream underrun. Without it the underrun resume re-anchored on the first arriving buffer
// and thrashed on the Opus decode ramp. See DEFAULT_MIN_PLAYBACK_LEAD_SECONDS.
private minPlaybackLeadSeconds: number = DEFAULT_MIN_PLAYBACK_LEAD_SECONDS;
// Hysteresis latch for the production pause. Once forward fill crosses the high-water mark we
// stay paused until it drains below the low-water mark, so the two producers do not flap
// on/off around a single threshold (and a paused producer does not resume for one chunk only
// to re-pause immediately). False until first crossing; flips on the band edges.
// Mutated by evaluateProductionPause() — named to signal the state-advance on each call.
private productionPaused_: boolean = false;
// True once the producer (C# read loop / Opus feed) has signalled that ALL bytes are in and
// every decodable buffer has been added. This is the discriminator between a genuine
// end-of-track and a transient gap. End-of-playback fires ONLY when this is true AND the
// scheduled queue has drained — a drained queue while this is false is a startup/underrun gap,
// not the end (Opus decodes via WebCodecs asynchronously, so the first AudioBuffer can lag the
// playback-start minimum, briefly leaving zero scheduled sources before real playback). Reset
// by clear/clearForSeek/resetToStart; set by setStreamComplete.
private streamComplete: boolean = false;
// True while playback is logically running but the decoded queue ran dry mid-stream (underrun).
// We stop the scheduler (isActive_ = false) so no source schedules against a stale anchor, but
// remember we must re-anchor and resume the moment new buffers arrive — distinct from a paused/
// stopped player, which clears this. Without it, scheduleNewBuffers would silently no-op on the
// !isActive_ guard and playback would never recover from a starvation gap.
private underrun_: boolean = false;
// Callbacks
public onPlaybackEnded: (() => void) | null = null;
@@ -45,6 +171,38 @@ export class PlaybackScheduler {
this.buffers.push(buffer);
}
/**
* Mark whether the byte stream is complete (all bytes received and all decodable buffers added).
* The end-of-playback callback fires only when this is true AND the scheduled queue has drained
* so a drained queue while the stream is still in flight (startup/underrun) is never mistaken for
* end-of-track. Set true by AudioPlayer on markStreamComplete / decoder isComplete; set false on a
* fresh stream or a range-continuation reinit. Setting it true while playback has already drained
* mid-stream finalises the track immediately (the genuine-end signal arrived after the queue
* emptied e.g. the very last buffers were the tail).
*/
setStreamComplete(complete: boolean): void {
this.streamComplete = complete;
// Only act when the genuine-end signal lands while we are parked in underrun (logically
// playing but starved); a drained queue with no playback in flight — never started, or
// already finished — is left untouched. Gated on underrun_, not isActive_, which is false
// during a parked underrun.
if (!complete || !this.underrun_) {
return;
}
// The rebuffer threshold no longer applies — a complete stream yields no further buffers:
// - tail buffers accumulated below the threshold while we were parked (the new hysteresis
// kept us parked) → schedule them out; scheduleNewBuffers' underrun branch now resumes
// because streamComplete overrides the lead gate, and handleSourceEnded fires the genuine
// end when they drain. Without this the buffers would never schedule and we would park
// forever (queue drained, isActive_ false, threshold never met).
// - no tail at all (cursor already at the decoded end) → this drained state IS the end.
if (this.nextBufferIndex < this.buffers.length) {
this.scheduleNewBuffers();
} else if (this.scheduledSources.length === 0) {
this.finishPlayback();
}
}
/**
* Get total duration of all stored buffers
*/
@@ -88,6 +246,169 @@ export class PlaybackScheduler {
return this.playbackOffset;
}
/**
* Configure the back-retain bound (seconds of already-played audio kept un-evicted).
* Provisional config seam 21.2 feeds the real window policy in here. Negative values
* are clamped to 0 (retain nothing behind the playhead).
*/
setBackRetainSeconds(seconds: number): void {
this.backRetainSeconds = Math.max(0, seconds);
}
/**
* Configure the forward back-pressure water-marks (seconds of decoded lookahead) and the OQ3
* hard byte ceiling. Provisional config seam 21.4 tunes the numbers. Low is clamped below
* high so the hysteresis band is always valid; non-positive byte cap disables the OQ3 guard.
*/
setForwardWindow(highWaterSeconds: number, lowWaterSeconds: number, maxDecodedBytes: number): void {
this.forwardHighWaterSeconds = Math.max(0, highWaterSeconds);
this.forwardLowWaterSeconds = Math.max(0, Math.min(lowWaterSeconds, this.forwardHighWaterSeconds));
this.maxDecodedBytes = maxDecodedBytes;
}
/**
* Seconds of decoded audio sitting AHEAD of the current playhead the forward fill. This is
* the single back-pressure signal (OQ7): the absolute end time of the last decoded buffer
* minus the current playback position. Never negative (clamped at 0 when the playhead has
* caught up to or passed the decoded tail).
*/
getForwardLookaheadSeconds(): number {
const decodedEnd = this.getTotalDuration() + this.playbackOffset;
return Math.max(0, decodedEnd - this.getCurrentPosition());
}
/**
* Estimated bytes of decoded float PCM currently retained (OQ3 input). Web Audio AudioBuffers
* are 32-bit float per sample per channel; frames = duration × sampleRate. Summed across the
* retained buffers only evicted buffers are already reclaimed, so this tracks the live
* footprint, not the whole track.
*/
getDecodedByteEstimate(): number {
let bytes = 0;
for (const b of this.buffers) {
bytes += b.length * b.numberOfChannels * BYTES_PER_FLOAT_SAMPLE;
}
return bytes;
}
/**
* The single shared production-pause decision (Phase 21.2, OQ6/OQ7). Both producers the C#
* read loop (21.2a) and the Opus demux/decode feed (21.2b) call this and stop producing
* while it returns true. Hysteresis: pause when forward lookahead exceeds the high-water mark
* OR the decoded byte estimate exceeds the OQ3 ceiling; resume only once forward lookahead has
* drained below the low-water mark AND the byte estimate is back under the ceiling. The
* byte-ceiling test has no separate low-water band it is the hard guard rail, so it releases
* as soon as eviction brings the footprint back under the cap.
*
* Named `evaluateProductionPause` (not `isProductionPaused`) because each call may ADVANCE the
* hysteresis latch (`productionPaused_`), making it a state-advancing evaluation, not a pure
* read. `AudioPlayer.isProductionPaused()` is the pure-predicate wrapper exposed to callers
* outside the scheduler.
*/
evaluateProductionPause(): boolean {
const lookahead = this.getForwardLookaheadSeconds();
const overByteCeiling = this.maxDecodedBytes > 0 && this.getDecodedByteEstimate() > this.maxDecodedBytes;
if (this.productionPaused_) {
// Stay paused until BOTH the time window has drained below low-water AND the byte
// footprint is back under the ceiling.
if (lookahead <= this.forwardLowWaterSeconds && !overByteCeiling) {
this.productionPaused_ = false;
}
} else if (lookahead >= this.forwardHighWaterSeconds || overByteCeiling) {
this.productionPaused_ = true;
}
return this.productionPaused_;
}
/**
* Drop already-played buffers from the front of the array, reclaiming their decoded float
* memory, and advance the time anchor so all position/index bookkeeping stays exact.
*
* Eviction frontier: any buffer whose absolute END time is at or older than
* (currentPosition - backRetainSeconds) is droppable. We evict a contiguous run from the
* front only buffers are appended in playback order, so the front is always the oldest.
*
* Two hard safety bounds keep the live frontier intact (the second half of the
* index/time-anchor invariant):
* 1. Never evict at/after `nextBufferIndex` those are not yet scheduled; dropping them
* would lose unplayed audio and corrupt the schedule cursor.
* 2. Never evict a buffer still referenced by a live scheduled source its
* AudioBufferSourceNode is mid-flight and `handleSourceEnded` still tracks it.
*
* Returns the number of buffers evicted (0 if nothing was droppable).
*
* This is the SHARED eviction both decode paths get for free no format branch. It does
* not fetch, decode, or back-pressure (those are 21.2/21.3); with producers unchanged it
* makes the *played* region provably memory-bounded on both paths.
*/
evictPlayedBuffers(): number {
if (this.buffers.length === 0) {
return 0;
}
// Absolute time before which a fully-ended buffer may be dropped.
const evictBefore = this.getCurrentPosition() - this.backRetainSeconds;
// Lowest index still referenced by a live scheduled source (or buffers.length if none).
// Eviction must not cross this — those sources are playing now.
let firstLiveIndex = this.buffers.length;
for (const scheduled of this.scheduledSources) {
if (scheduled.bufferIndex < firstLiveIndex) {
firstLiveIndex = scheduled.bufferIndex;
}
}
// Hard ceiling on how many front buffers we may drop: not past the schedule cursor,
// and not past the oldest live source.
const maxEvictable = Math.min(this.nextBufferIndex, firstLiveIndex);
// Walk the front, accumulating absolute end times, counting droppable buffers.
let evictCount = 0;
let accumulatedEnd = this.playbackOffset;
for (let i = 0; i < maxEvictable; i++) {
accumulatedEnd += this.buffers[i].duration;
// Drop buffers whose END is at or behind the retain frontier (inclusive bound).
if (accumulatedEnd <= evictBefore) {
evictCount = i + 1;
} else {
break; // later buffers end even later — nothing more is droppable
}
}
if (evictCount === 0) {
return 0;
}
// Sum the dropped duration BEFORE splicing, then advance the time anchor by it so
// buffers[0] still names the correct absolute start time. This is the move that keeps
// every position/scheduling query exact against a front-evicted array.
let droppedDuration = 0;
for (let i = 0; i < evictCount; i++) {
droppedDuration += this.buffers[i].duration;
}
this.buffers.splice(0, evictCount);
// Advance the absolute time anchor (offset) by the dropped duration AND drop the
// buffer-relative anchor position by the same amount. These two move in lockstep:
// getCurrentPosition() is (playbackAnchorPosition + playbackOffset + elapsed), so
// adjusting only one would make the reported position jump by droppedDuration.
// Moving both by +d / -d leaves the ABSOLUTE position unchanged while keeping
// playbackAnchorPosition buffer-relative (the convention playFromPosition/pause use).
this.playbackOffset += droppedDuration;
this.playbackAnchorPosition -= droppedDuration;
// Every surviving absolute index shifts down by evictCount.
this.nextBufferIndex -= evictCount;
for (const scheduled of this.scheduledSources) {
scheduled.bufferIndex -= evictCount;
}
return evictCount;
}
/**
* Start or resume playback from a specific position
*/
@@ -111,18 +432,25 @@ export class PlaybackScheduler {
}
if (startBufferIndex >= this.buffers.length) {
// Position landed at or past the end of all buffers. Previously this
// returned silently, leaving the player stuck "playing" with no source
// scheduled — a pause near the end followed by play never recovered.
// Treat this as end-of-track so listeners (UI / end callback) fire.
this.isActive_ = false;
this.playbackAnchorTime = 0;
this.playbackAnchorPosition = 0;
this.onPlaybackEnded?.();
// Position landed at or past the end of all currently-decoded buffers. This is
// end-of-track ONLY if the stream is complete; otherwise it is a startup/underrun
// gap (decode hasn't caught up to the playhead yet) and firing onPlaybackEnded here
// would be a FALSE end — exactly the Opus-startup misfire. When complete, finish;
// when still streaming, park in underrun so scheduleNewBuffers resumes on the next
// decoded buffer rather than the player being stuck "playing" with nothing scheduled.
if (this.streamComplete) {
this.finishPlayback();
} else {
this.underrun_ = true;
this.playbackAnchorPosition = position;
this.nextBufferIndex = startBufferIndex;
this.isActive_ = false; // no source to schedule yet; resume() re-anchors on refill
}
return;
}
// Set timing anchors
this.underrun_ = false;
this.playbackAnchorPosition = position;
this.playbackAnchorTime = this.contextManager.currentTime;
this.nextScheduleTime = this.contextManager.currentTime + 0.01; // Small lookahead
@@ -141,6 +469,34 @@ export class PlaybackScheduler {
return; // No new buffers
}
// Resume from a mid-stream underrun: the queue had drained ahead of decode and we parked
// (isActive_ = false, underrun_ = true) instead of firing a false end. Newly decoded
// buffers are now available at nextBufferIndex, so re-anchor the clock at the resume point
// and re-enable scheduling. We re-anchor (rather than reusing the stale nextScheduleTime
// captured before the gap) so the resumed audio is contiguous from "now" — a stale anchor
// would schedule the next source in the past and the browser would drop or rush it.
if (this.underrun_) {
// Rebuffer hysteresis: do NOT resume on the first arriving buffer. With an empty scheduled
// tail, resuming on a single buffer plays it (~20 ms for Opus) and immediately re-drains,
// re-parking — the audible start/stop thrash on the Opus WebCodecs decode ramp. Stay parked
// and keep accumulating until a healthy lead has rebuilt, so the resumed playback has the
// same cushion a fresh start does. While parked the playhead is frozen, so each arriving
// buffer grows the lead monotonically toward the threshold (no starvation/deadlock).
//
// streamComplete overrides the gate: a finished stream produces no further buffers, so a
// tail shorter than the lead MUST still play out (here and via setStreamComplete) rather
// than park forever. handleSourceEnded fires the genuine end once that tail drains.
if (!this.streamComplete && !this.hasMinimumPlaybackLead()) {
return; // still re-accumulating the rebuffer lead — remain parked
}
this.underrun_ = false;
this.isActive_ = true;
this.playbackAnchorTime = this.contextManager.currentTime;
this.nextScheduleTime = this.contextManager.currentTime + 0.01;
this.scheduleBuffersFrom(this.nextBufferIndex, 0);
return;
}
// Use isActive_ as the sentinel for "playback is running", not nextScheduleTime === 0.
// AudioContext.currentTime can legitimately be 0 at context creation, which would cause
// nextScheduleTime === 0 to incorrectly reset a value already set by playFromPosition.
@@ -214,26 +570,69 @@ export class PlaybackScheduler {
this.scheduledSources.splice(index, 1);
}
// A source just finished, so its buffer is now behind the playhead — the natural
// point to reclaim played memory. Eviction is self-contained (no fetch/back-pressure)
// and runs before re-scheduling so index bookkeeping is settled first. This is the
// 21.1 trigger that keeps the PLAYED region bounded with producers unchanged.
this.evictPlayedBuffers();
// Schedule more buffers if available
if (this.nextBufferIndex < this.buffers.length) {
this.scheduleBuffersFrom(this.nextBufferIndex, 0);
}
// Check if all playback has finished
// The scheduled queue drained AND the cursor caught up to every decoded buffer. Whether
// this is the end depends on the stream:
// - streamComplete: genuine end-of-track — finish and fire onPlaybackEnded.
// - still streaming: a mid-stream UNDERRUN (decode fell behind the playhead — the Opus
// WebCodecs startup gap, or a network stall). Firing onPlaybackEnded here is the false
// end this guards against. Park in underrun; scheduleNewBuffers resumes on the next
// decoded buffer.
if (this.scheduledSources.length === 0 && this.nextBufferIndex >= this.buffers.length) {
this.isActive_ = false;
this.playbackAnchorTime = 0;
this.playbackAnchorPosition = 0;
this.onPlaybackEnded?.();
if (this.streamComplete) {
this.finishPlayback();
} else {
this.underrun_ = true;
// Mid-stream underrun: the scheduled queue drained and decode has not caught up. Report it
// as decode pressure so the visualizer throttles — a sustained run of these is exactly the
// HW-accel-off starvation the auto-throttle protects against. The hysteresis in the signal
// ignores a lone startup-ramp underrun; only a sustained run engages the throttle.
decodePressure.report();
// Hold the playhead at the decoded tail so getCurrentPosition stays exact during
// the gap. isActive_ goes false so no stale-anchor scheduling occurs; resume
// re-anchors at currentTime when buffers arrive.
this.playbackAnchorPosition = this.getCurrentPosition() - this.playbackOffset;
this.playbackAnchorTime = 0;
this.isActive_ = false;
}
}
}
/**
* Finalise playback: stop the clock, reset anchors, and fire the end-of-playback callback. The
* single genuine-end path, reached only when the stream is complete AND the queue has fully
* drained (handleSourceEnded / setStreamComplete) or playback resumed past a complete stream's
* end (playFromPosition). Never called for a transient startup/underrun gap.
*/
private finishPlayback(): void {
this.isActive_ = false;
this.underrun_ = false;
this.playbackAnchorTime = 0;
this.playbackAnchorPosition = 0;
this.onPlaybackEnded?.();
}
/**
* Pause playback - saves position and stops sources
*/
pause(): number {
const position = this.getCurrentPosition();
this.isActive_ = false; // Prevent handleSourceEnded from scheduling more
// Clear the underrun flag: if the queue drained mid-stream and the user pauses before new
// buffers arrive, a subsequent setStreamComplete must not fire finishPlayback while still
// paused. On resume, playFromPosition re-parks underrun if the decoded tail still hasn't
// caught up, so no genuine end is lost by clearing it here.
this.underrun_ = false;
this.stopAllSources();
// getCurrentPosition() returns absolute time (anchor + playbackOffset); the anchor
// is buffer-relative, so strip the offset back out before storing it.
@@ -262,6 +661,8 @@ export class PlaybackScheduler {
*/
resetToStart(): void {
this.isActive_ = false;
this.underrun_ = false;
this.streamComplete = false;
this.stopAllSources();
this.playbackAnchorPosition = 0;
this.playbackAnchorTime = 0;
@@ -274,6 +675,8 @@ export class PlaybackScheduler {
*/
clear(): void {
this.isActive_ = false;
this.underrun_ = false;
this.streamComplete = false;
this.stopAllSources();
this.buffers = [];
this.playbackAnchorPosition = 0;
@@ -281,6 +684,9 @@ export class PlaybackScheduler {
this.nextBufferIndex = 0;
this.nextScheduleTime = 0;
this.playbackOffset = 0;
// Release the back-pressure latch — a fresh stream must start unthrottled so its first
// chunks decode immediately (C2: no throttle-induced first-audio stall).
this.productionPaused_ = false;
}
/**
@@ -288,6 +694,11 @@ export class PlaybackScheduler {
*/
clearForSeek(): void {
this.isActive_ = false;
this.underrun_ = false;
// The range continuation is a fresh byte stream — it is NOT complete until its own
// markStreamComplete. Reset so a stale "complete" from the pre-seek stream cannot make the
// post-seek refill fire a premature end before its bytes arrive.
this.streamComplete = false;
this.stopAllSources();
this.buffers = [];
this.playbackAnchorPosition = 0;
@@ -295,6 +706,9 @@ export class PlaybackScheduler {
this.nextBufferIndex = 0;
this.nextScheduleTime = 0;
// Note: playbackOffset is NOT reset - it will be set by the caller
// Release the back-pressure latch — the post-seek continuation must refill from the new
// offset without inheriting the pre-seek paused state.
this.productionPaused_ = false;
}
/**
@@ -311,6 +725,24 @@ export class PlaybackScheduler {
return this.buffers.length >= minCount;
}
/**
* True once at least `minPlaybackLeadSeconds` of decoded-but-unscheduled audio sits ahead of the
* schedule cursor the rebuffer-hysteresis gate for both a fresh playback start (cursor at 0, so
* this measures the whole decoded head) and an underrun resume (cursor at the drained tail, so this
* measures only the freshly-accumulated lead). Sums only up to the threshold and short-circuits, so
* it is bounded (~one threshold's worth of buffers) regardless of how much is buffered ahead.
*/
hasMinimumPlaybackLead(): boolean {
let lead = 0;
for (let i = this.nextBufferIndex; i < this.buffers.length; i++) {
lead += this.buffers[i].duration;
if (lead >= this.minPlaybackLeadSeconds) {
return true;
}
}
return false;
}
/**
* Check if playback is active
*/
+49 -1
View File
@@ -61,6 +61,16 @@ export class StreamDecoder {
// at 4 GB by the 32-bit RIFF size field, so overflow is not a practical concern.
private totalRawBytes: number = 0;
private processedBytes: number = 0;
// Absolute count of raw bytes already DROPPED off the front of rawChunks (the memory bound).
// processedBytes is an absolute cursor into the whole logical byte stream; rawChunks no longer
// begins at stream byte 0 once consumed chunks are compacted away, so extractAlignedData walks
// from discardedBytes (the absolute position of rawChunks[0]) rather than 0. totalRawBytes and
// every offset stay absolute and unchanged — only the array's front moves. Without this, a long
// WAV (e.g. a 92-min mix ≈ 970 MB raw) accumulates its ENTIRE decoded-from body in rawChunks
// because consumed chunks were never released; Phase 21.2 bounds only the DECODED scheduler
// queue, not this raw queue — so software (HW-accel-off) playback crashed the tab on memory.
private discardedBytes: number = 0;
private totalStreamLength: number = 0;
private streamComplete: boolean = false;
private headerError: string | null = null;
@@ -94,6 +104,7 @@ export class StreamDecoder {
this.rawChunks = [];
this.totalRawBytes = 0;
this.processedBytes = 0;
this.discardedBytes = 0;
this.totalStreamLength = totalStreamLength;
this.streamComplete = false;
this.headerBytesReceived = 0;
@@ -228,6 +239,36 @@ export class StreamDecoder {
this.totalRawBytes += data.length;
}
/**
* Drop fully-consumed raw chunks off the front of rawChunks, reclaiming their bytes. A chunk is
* droppable only when its ENTIRE span lies at or before processedBytes (the decode cursor); a
* chunk that straddles the cursor still has unconsumed tail bytes a later segment will read, so
* the walk stops there. discardedBytes tracks the absolute start of rawChunks[0] so
* extractAlignedData keeps reading the correct bytes after compaction. Splicing once at the end
* (not per chunk) keeps this O(n) in the dropped count.
*
* This is the raw-side analogue of PlaybackScheduler.evictPlayedBuffers (the decoded side): both
* keep their queue bounded to roughly the live window, so a long stream never balloons memory.
*/
private releaseConsumedChunks(): void {
let dropCount = 0;
let frontPos = this.discardedBytes;
for (const chunk of this.rawChunks) {
// Drop only when the whole chunk is behind the cursor (end <= processedBytes). A chunk
// ending exactly at processedBytes has every byte consumed and is safe to drop.
if (frontPos + chunk.length <= this.processedBytes) {
frontPos += chunk.length;
dropCount++;
} else {
break; // this chunk straddles the cursor (or is ahead) — stop.
}
}
if (dropCount > 0) {
this.rawChunks.splice(0, dropCount);
this.discardedBytes = frontPos;
}
}
/**
* Try to decode the next segment of audio.
*
@@ -276,6 +317,9 @@ export class StreamDecoder {
// Advance only after a successful decode so a thrown timeout/decode
// failure does not silently drop the segment.
this.processedBytes += alignedSize;
// Release fully-consumed raw chunks now that the cursor has moved past them. This is the
// memory bound: without it rawChunks retains the whole stream body (the OOM on long WAVs).
this.releaseConsumedChunks();
return { buffer, duration: buffer.duration };
} catch (error) {
// Re-throw typed errors so the outer drain loop in processChunk /
@@ -339,7 +383,9 @@ export class StreamDecoder {
let extractedOffset = 0;
let remaining = size;
let streamPosition = this.processedBytes;
let currentPos = 0;
// rawChunks[0] now begins at absolute stream byte `discardedBytes` (front-compaction has
// dropped everything before it), so the walk starts there, not at 0.
let currentPos = this.discardedBytes;
for (const chunk of this.rawChunks) {
if (remaining <= 0) break;
@@ -473,6 +519,7 @@ export class StreamDecoder {
this.rawChunks = [];
this.totalRawBytes = 0;
this.processedBytes = 0;
this.discardedBytes = 0;
this.totalStreamLength = 0;
this.streamComplete = false;
this.headerBytesReceived = 0;
@@ -501,6 +548,7 @@ export class StreamDecoder {
this.rawChunks = [];
this.totalRawBytes = 0;
this.processedBytes = 0;
this.discardedBytes = 0;
this.streamComplete = false;
this.headerBytesReceived = 0;
this.headerSearchChunks = [];
@@ -0,0 +1,157 @@
/**
* decodePressure hysteresis tests the Part-1 auto-throttle signal logic.
*
* These cover the four named behaviours that make the visualizer-throttle safe: it engages only on
* SUSTAINED pressure, releases only after SUSTAINED recovery, never flaps on/off, and is a complete
* no-op when decode is healthy. The clock is injected so every transition is asserted at an exact
* timestamp no real timers, fully deterministic.
*
* Run (no test runner configured; Node 22+ strips TS types natively see OpusStreamDecoder.test.ts):
* dotnet build DeepDrftPublic/DeepDrftPublic.csproj
* cp DeepDrftPublic/Interop/audio/decodePressure.test.ts DeepDrftPublic/wwwroot/js/audio/
* node DeepDrftPublic/wwwroot/js/audio/decodePressure.test.ts
*
* A thrown error / non-zero exit signals failure; "ALL <n> TESTS PASSED" signals success.
*/
import {
DecodePressureSignal,
ENGAGE_EVENTS,
ENGAGE_WINDOW_MS,
RELEASE_QUIET_MS,
MIN_ENGAGED_MS,
} from './decodePressure.js';
// --- tiny inline harness (no dependencies) ---------------------------------------------------
let passed = 0;
const failures: string[] = [];
function test(name: string, fn: () => void): void {
try {
fn();
passed++;
} catch (e) {
failures.push(`FAIL: ${name}\n ${(e as Error).message}`);
}
}
function assertTrue(actual: boolean, msg?: string): void {
if (actual !== true) throw new Error(`${msg ?? 'assertTrue'}: expected true, got ${String(actual)}`);
}
function assertFalse(actual: boolean, msg?: string): void {
if (actual !== false) throw new Error(`${msg ?? 'assertFalse'}: expected false, got ${String(actual)}`);
}
/** A signal driven by a hand-advanced clock, so every transition is asserted at an exact time. */
function makeSignal() {
let now = 1000; // start at a non-zero base so "no prior stress" (-Infinity) is unambiguous
const sig = new DecodePressureSignal(() => now);
return {
sig,
at(ms: number) { now = ms; },
advance(ms: number) { now += ms; },
now() { return now; },
};
}
// --- no engage when healthy ------------------------------------------------------------------
test('healthy stream never engages (no reports at all)', () => {
const { sig, advance } = makeSignal();
for (let i = 0; i < 10; i++) {
advance(1000);
assertFalse(sig.isUnderPressure(), 'healthy must never be under pressure');
}
});
test('a single transient stress does not engage', () => {
const { sig, advance } = makeSignal();
sig.report();
assertFalse(sig.isUnderPressure(), 'one event is not sustained');
advance(500);
assertFalse(sig.isUnderPressure(), 'still not sustained');
});
test('fewer than ENGAGE_EVENTS within the window does not engage', () => {
const { sig, advance } = makeSignal();
for (let i = 0; i < ENGAGE_EVENTS - 1; i++) {
sig.report();
advance(10);
}
assertFalse(sig.isUnderPressure(), 'one short of the threshold must not engage');
});
test('stress spread wider than the window never accumulates enough to engage', () => {
const { sig, advance } = makeSignal();
// One report per full window: the prune drops each before the next, so the live count never
// reaches ENGAGE_EVENTS even after many reports.
for (let i = 0; i < ENGAGE_EVENTS * 3; i++) {
sig.report();
assertFalse(sig.isUnderPressure(), 'spread-out stress is not sustained');
advance(ENGAGE_WINDOW_MS);
}
});
// --- engages on sustained pressure -----------------------------------------------------------
test('ENGAGE_EVENTS within the window engages', () => {
const { sig, advance } = makeSignal();
for (let i = 0; i < ENGAGE_EVENTS; i++) {
sig.report();
advance(10); // all comfortably inside ENGAGE_WINDOW_MS
}
assertTrue(sig.isUnderPressure(), 'sustained pressure must engage');
});
// --- releases after recovery -----------------------------------------------------------------
test('releases after sustained quiet past the min dwell', () => {
const { sig, advance } = makeSignal();
for (let i = 0; i < ENGAGE_EVENTS; i++) { sig.report(); advance(10); }
assertTrue(sig.isUnderPressure(), 'engaged');
// Quiet long enough to satisfy BOTH the min engaged dwell and the release-quiet window.
advance(Math.max(MIN_ENGAGED_MS, RELEASE_QUIET_MS) + 1);
assertFalse(sig.isUnderPressure(), 'sustained recovery must release');
});
test('re-engages after a release when a fresh burst arrives', () => {
const { sig, advance } = makeSignal();
for (let i = 0; i < ENGAGE_EVENTS; i++) { sig.report(); advance(10); }
assertTrue(sig.isUnderPressure(), 'engaged first time');
advance(Math.max(MIN_ENGAGED_MS, RELEASE_QUIET_MS) + 1);
assertFalse(sig.isUnderPressure(), 'released');
for (let i = 0; i < ENGAGE_EVENTS; i++) { sig.report(); advance(10); }
assertTrue(sig.isUnderPressure(), 'a fresh sustained burst re-engages');
});
// --- no flap ---------------------------------------------------------------------------------
test('stays engaged during a brief quiet shorter than the release window', () => {
const { sig, advance } = makeSignal();
for (let i = 0; i < ENGAGE_EVENTS; i++) { sig.report(); advance(10); }
assertTrue(sig.isUnderPressure(), 'engaged');
// A gap shorter than RELEASE_QUIET_MS must NOT release — that is the anti-flap guarantee.
advance(RELEASE_QUIET_MS - 100);
assertTrue(sig.isUnderPressure(), 'a brief quiet must not drop the throttle');
});
test('continued stress holds the throttle engaged indefinitely', () => {
const { sig, advance } = makeSignal();
for (let i = 0; i < ENGAGE_EVENTS; i++) { sig.report(); advance(10); }
assertTrue(sig.isUnderPressure(), 'engaged');
// Keep reporting at a cadence under the release window; it must never release.
for (let i = 0; i < 20; i++) {
advance(RELEASE_QUIET_MS - 100);
sig.report();
assertTrue(sig.isUnderPressure(), 'ongoing stress keeps it engaged');
}
});
// --- report ----------------------------------------------------------------------------------
if (failures.length > 0) {
console.error(failures.join('\n'));
throw new Error(`${failures.length} test(s) failed, ${passed} passed`);
}
console.log(`ALL ${passed} TESTS PASSED`);
@@ -0,0 +1,93 @@
/**
* Shared decode-pressure signal the seam that lets the audio decode pipeline protect itself
* from the WebGL visualizer under CPU contention.
*
* THE PROBLEM (browser-confirmed): with hardware acceleration OFF the WaveformVisualizer's WebGL2
* lava-lamp software-renders on the main thread. WebCodecs Opus decode also runs on the main thread,
* so a 60 fps software render starves decode it falls behind realtime playback underruns. Turning
* the visualizer off makes decode keep up perfectly. With HW accel ON the render is on the GPU and
* there is no contention; WAV/lossless decodes synchronously and never pressures decode either.
*
* THE SEAM: this module is a singleton shared by two otherwise-independent browser module graphs
* the audio pipeline (`js/audio/*`, the PRODUCER) and the visualizer (`js/visualizer/*`, the
* CONSUMER) because an ES module is instantiated once per URL. The producer reports decode stress;
* the consumer reads {@link DecodePressureSignal.isUnderPressure} each frame and throttles its render
* cadence so the main thread yields time back to decode. No routing through C#, no constructor growth.
*
* HYSTERESIS (no flap): the signal engages only on SUSTAINED stress ( ENGAGE_EVENTS reports within
* ENGAGE_WINDOW_MS) and releases only after SUSTAINED recovery (no stress for RELEASE_QUIET_MS, and
* never before a MIN_ENGAGED_MS dwell). A lone startup-ramp blip never engages; once engaged the
* throttle cannot toggle off frame-to-frame.
*
* HEALTHY-CASE NO-OP: when decode keeps up nothing ever calls report(), so {@link isUnderPressure}
* stays false forever and the consumer runs at full quality. This protection only activates under
* genuine, sustained decode starvation.
*/
/** Stress reports required within {@link ENGAGE_WINDOW_MS} to engage the throttle. */
export const ENGAGE_EVENTS = 5;
/** Sliding window (ms) over which {@link ENGAGE_EVENTS} stress reports count toward engaging. */
export const ENGAGE_WINDOW_MS = 2500;
/** Stress-free dwell (ms) required before the throttle releases. */
export const RELEASE_QUIET_MS = 1500;
/** Minimum engaged dwell (ms) before release is even considered — the anti-flap floor. */
export const MIN_ENGAGED_MS = 1000;
type Clock = () => number;
export class DecodePressureSignal {
// Timestamps of recent stress reports, pruned to the engage window. Length ≥ ENGAGE_EVENTS is the
// "sustained pressure" condition. Bounded by the window, so this never grows unbounded.
private stressTimestamps: number[] = [];
private lastStressMs = Number.NEGATIVE_INFINITY;
private engaged = false;
private engagedAtMs = 0;
// Clock injectable purely for deterministic unit tests; production uses performance.now().
constructor(private readonly now: Clock = () => performance.now()) {}
/**
* Report one unit of decode stress decode falling behind realtime. Called by the producer at
* each genuine lag event: the WebCodecs decode queue staying non-empty past its yield ceiling
* (OpusStreamDecoder) and the scheduler parking on a mid-stream underrun (PlaybackScheduler).
*/
report(): void {
const t = this.now();
this.lastStressMs = t;
this.stressTimestamps.push(t);
this.prune(t);
}
/**
* Whether decode is under sustained pressure right now. Pure read for the caller, but it ADVANCES
* the hysteresis latch (engage on sustained stress, release on sustained quiet past the min dwell)
* so the transition is evaluated lazily on the clock, identical whether called once or per frame.
*/
isUnderPressure(): boolean {
const t = this.now();
this.prune(t);
if (this.engaged) {
const engagedFor = t - this.engagedAtMs;
const quietFor = t - this.lastStressMs;
if (engagedFor >= MIN_ENGAGED_MS && quietFor >= RELEASE_QUIET_MS) {
this.engaged = false;
}
} else if (this.stressTimestamps.length >= ENGAGE_EVENTS) {
this.engaged = true;
this.engagedAtMs = t;
}
return this.engaged;
}
/** Drop stress timestamps older than the engage window so the count reflects only the live window. */
private prune(t: number): void {
const cutoff = t - ENGAGE_WINDOW_MS;
while (this.stressTimestamps.length > 0 && this.stressTimestamps[0] < cutoff) {
this.stressTimestamps.shift();
}
}
}
/** The process-wide signal both the audio pipeline and the visualizer share. */
export const decodePressure = new DecodePressureSignal();
+43 -1
View File
@@ -3,6 +3,7 @@
*/
import { AudioPlayer, AudioResult, StreamingResult, AudioState } from './AudioPlayer.js';
import { canDecodeOggOpus } from './OpusCapability.js';
// Player instances by ID
const audioPlayers = new Map<string, AudioPlayer>();
@@ -31,12 +32,26 @@ const DeepDrftAudio = {
}
},
initializeStreaming: (playerId: string, totalStreamLength: number, contentType: string): AudioResult => {
initializeStreaming: async (playerId: string, totalStreamLength: number, contentType: string): Promise<AudioResult> => {
const player = audioPlayers.get(playerId);
if (!player) return { success: false, error: 'Player not found' };
return player.initializeStreaming(totalStreamLength, contentType);
},
// Opus injection seam (wave 18.4). Wave 18.5 fetches the per-track sidecar (setup header +
// seek index) over HTTP and hands the raw bytes here BEFORE initializeStreaming on an Opus
// stream. This module never fetches the sidecar — it only parses + stores it on the player.
setOpusSidecar: (playerId: string, sidecarBytes: Uint8Array): AudioResult => {
const player = audioPlayers.get(playerId);
if (!player) return { success: false, error: 'Player not found' };
return player.setOpusSidecar(sidecarBytes);
},
// Capability seam. Resolves whether this browser can stream-decode Ogg Opus via WebCodecs
// (AudioDecoder + codec:'opus'; Safari < 16.4 / older Firefox cannot). The player consumes this
// to choose lossless when unsupported; this module only reports the capability.
canDecodeOggOpus: (): Promise<boolean> => canDecodeOggOpus(),
processStreamingChunk: async (playerId: string, chunk: Uint8Array): Promise<StreamingResult> => {
const player = audioPlayers.get(playerId);
if (!player) return { success: false, error: 'Player not found' };
@@ -102,12 +117,39 @@ const DeepDrftAudio = {
return player?.calculateByteOffset(positionSeconds) ?? 0;
},
// "Load at timestamp" seam (Phase 18 wave 18.6 format switch). Resolve the file-absolute byte offset
// to begin a stream at `position` with no playback/buffer state — the C# load-from-position path calls
// this after initializeStreaming (Opus: sidecar resolves immediately; WAV: after a header probe) and
// then streams from the returned offset via the seek/refill loop. seekBeyondBuffer:true + byteOffset.
resolveStreamOffset: (playerId: string, position: number): AudioResult => {
const player = audioPlayers.get(playerId);
if (!player) return { success: false, error: 'Player not found' };
return player.resolveStreamOffset(position);
},
// Phase 21.2a back-pressure poll: the C# read loop calls this WHILE throttled to learn when
// the scheduler has drained below low-water and reading may resume. A missing player reads as
// "not paused" so a torn-down player never wedges a loop that is already exiting.
isProductionPaused: (playerId: string): boolean => {
const player = audioPlayers.get(playerId);
return player?.isProductionPaused() ?? false;
},
reinitializeFromOffset: (playerId: string, totalStreamLength: number, seekPosition: number): AudioResult => {
const player = audioPlayers.get(playerId);
if (!player) return { success: false, error: 'Player not found' };
return player.reinitializeFromOffset(totalStreamLength, seekPosition);
},
// Phase 21.3 / AC6: recover into a clean paused-but-loaded state after a window-miss refill
// (seek-back past the retained tail) failed its Range fetch or reinit. Prevents the starved
// scheduler from firing a silent false end; leaves the track loaded so a retry is possible.
recoverFromFailedRefill: (playerId: string, seekPosition: number): AudioResult => {
const player = audioPlayers.get(playerId);
if (!player) return { success: false, error: 'Player not found' };
return player.recoverFromFailedRefill(seekPosition);
},
setVolume: (playerId: string, volume: number): AudioResult => {
const player = audioPlayers.get(playerId);
if (!player) return { success: false, error: 'Player not found' };
+82
View File
@@ -0,0 +1,82 @@
/**
* Minimal ambient WebCodecs declarations.
*
* TypeScript 5.9's bundled lib.dom.d.ts does NOT yet ship the WebCodecs audio types
* (`AudioDecoder`, `EncodedAudioChunk`, `AudioData`, `AudioDecoderConfig`), and this repo has no
* package.json / node_modules to pull in `@types/dom-webcodecs`. Rather than add a dependency
* toolchain for one feature, this declares exactly the slice of the WebCodecs surface the Opus
* streaming decoder uses nothing more. The shapes follow the W3C WebCodecs spec.
*
* These are runtime-optional: `AudioDecoder` is absent on Safari < 16.4 and older Firefox, so every
* use site guards on `typeof AudioDecoder !== 'undefined'` before touching it (the capability gate).
*/
interface AudioDecoderConfig {
codec: string;
sampleRate: number;
numberOfChannels: number;
/** Codec-specific setup bytes. For Opus this is the OpusHead identification header. */
description?: BufferSource;
}
interface AudioDecoderSupport {
supported: boolean;
config: AudioDecoderConfig;
}
type AudioSampleFormat = 'u8' | 's16' | 's24' | 's32' | 'f32' | 'u8-planar' | 's16-planar' | 's24-planar' | 's32-planar' | 'f32-planar';
interface AudioDataCopyToOptions {
planeIndex: number;
frameOffset?: number;
frameCount?: number;
format?: AudioSampleFormat;
}
interface AudioData {
readonly format: AudioSampleFormat | null;
readonly sampleRate: number;
readonly numberOfFrames: number;
readonly numberOfChannels: number;
readonly duration: number;
/** Presentation timestamp in microseconds. */
readonly timestamp: number;
allocationSize(options: AudioDataCopyToOptions): number;
copyTo(destination: BufferSource, options: AudioDataCopyToOptions): void;
close(): void;
}
interface EncodedAudioChunkInit {
type: 'key' | 'delta';
/** Presentation timestamp in microseconds. */
timestamp: number;
duration?: number;
data: BufferSource;
}
declare class EncodedAudioChunk {
constructor(init: EncodedAudioChunkInit);
readonly type: 'key' | 'delta';
readonly timestamp: number;
readonly duration: number | null;
readonly byteLength: number;
}
interface AudioDecoderInit {
output: (data: AudioData) => void;
error: (error: DOMException) => void;
}
type CodecState = 'unconfigured' | 'configured' | 'closed';
declare class AudioDecoder {
constructor(init: AudioDecoderInit);
readonly state: CodecState;
readonly decodeQueueSize: number;
configure(config: AudioDecoderConfig): void;
decode(chunk: EncodedAudioChunk): void;
flush(): Promise<void>;
reset(): void;
close(): void;
static isConfigSupported(config: AudioDecoderConfig): Promise<AudioDecoderSupport>;
}
@@ -0,0 +1,33 @@
/**
* Listener-settings interop (Phase 18 wave 18.6). A safe, eval-free cookie helper for persisting
* public-site preferences (streaming quality, and any future setting added under PublicSiteSettings).
* The 365-day durable-truth seam dark mode uses same mechanism, no eval.
*
* Exposed on window.DeepDrftSettings; imported once in App.razor.
*/
const DeepDrftSettings = {
/**
* Write a cookie with the given name, value, and lifetime. Equivalent to the browser's
* document.cookie assignment but without building JS via string interpolation or eval.
* Path is always "/"; SameSite is always "Lax" matches the dark-mode cookie semantics.
*/
setCookie: (name: string, value: string, days: number): void => {
const expires = new Date();
expires.setTime(expires.getTime() + days * 24 * 60 * 60 * 1000);
document.cookie =
`${encodeURIComponent(name)}=${encodeURIComponent(value)}` +
`; expires=${expires.toUTCString()}` +
`; path=/; SameSite=Lax`;
},
};
declare global {
interface Window {
DeepDrftSettings: typeof DeepDrftSettings;
}
}
window.DeepDrftSettings = DeepDrftSettings;
export { DeepDrftSettings };
@@ -44,6 +44,13 @@
* position while !isPlaying). The loop stops only on tab-hidden (visibilitychange) and dispose.
*/
import { decodePressure } from '../audio/decodePressure.js';
// Re-exported so the Blazor bridge (WaveformVisualizer.razor.cs) reaches the HW-accel probe through
// the same module reference it already imports for create() — one JS import surface, no second handle.
// The probe itself (and its unit-tested pure classifier) lives in hwAccel.ts.
export { detectHardwareAcceleration } from './hwAccel.js';
// ── Tuning anchors (see spec §B). These are the load-bearing constants. ──────────
/**
@@ -148,6 +155,16 @@ const RIBBON_HALF_WIDTH_FRAC = 0.92;
*/
const MAX_DPR = 2;
/**
* Minimum milliseconds between drawn frames WHILE decode is under sustained pressure (Part 1
* auto-protect audio). 1000/15 66.7 ms caps the loop at ~15 fps, cutting the main-thread WebGL
* software-render + physics cost by ~75% so the synchronous WebCodecs Opus decode (which shares the
* main thread when HW accel is off) gets the time it needs to keep up. The decodePressure signal is
* false in the common case (HW accel on, or lossless), so this cap never applies and the loop draws
* every frame at full quality. Tunable; the exact fps that clears starvation is browser-confirmed.
*/
const PRESSURE_THROTTLE_FRAME_MS = 1000 / 15;
// ════════════════════════════════════════════════════════════════════════════════════
// R2 — the wax-blob lava physics (CPU step + uniform upload). The lava is now a real
// Lagrangian particle system integrated each frame on the JS side and rendered as
@@ -1679,6 +1696,10 @@ export function create(canvas: HTMLCanvasElement): WaveformVisualizerHandle {
let rafId: number | null = null;
let disposed = false;
const startTimeMs = performance.now();
// Wall-clock of the last DRAWN continuous-loop frame, for the decode-pressure throttle (Part 1).
// While decodePressure.isUnderPressure() the loop draws at most once per PRESSURE_THROTTLE_FRAME_MS
// so the main thread yields time back to a starved decode; unthrottled it draws every frame.
let lastDrawMs = performance.now();
// Wall-clock anchor for the physics dt (separate from the playhead decay clock).
let lastPhysicsMs = performance.now();
@@ -1923,9 +1944,30 @@ export function create(canvas: HTMLCanvasElement): WaveformVisualizerHandle {
rafId = null;
return;
}
// Auto-protect audio under decode pressure (Part 1). When the WebCodecs Opus decode pipeline
// reports SUSTAINED lag (decodePressure.isUnderPressure()), throttle the draw cadence to
// ~PRESSURE_THROTTLE_FRAME_MS so this loop's main-thread GL + physics cost yields time back to
// decode; we still reschedule every frame so full cadence resumes the instant decode recovers.
// A no-op when decode is healthy — isUnderPressure() stays false, the gate is always open, and
// every frame draws exactly as before. Skipping a draw also skips the physics step (it runs
// inside draw()), and its dt is clamped to PHYSICS_MAX_DT, so a throttled gap never lurches the
// lava. redrawOnce() (idle/control-tweak stills) is intentionally NOT throttled — those are rare
// one-shots, not the continuous loop.
const nowMs = performance.now();
if (!decodePressure.isUnderPressure() || nowMs - lastDrawMs >= PRESSURE_THROTTLE_FRAME_MS) {
lastDrawMs = nowMs;
drawFrame();
}
rafId = requestAnimationFrame(frame);
}
/** One drawn continuous-loop frame: the GL draw plus the gated FPS/lava diagnostic tally. */
function drawFrame(): void {
draw();
// FPS tally: count this callback, and once per elapsed second emit the rate.
// FPS tally: count this drawn frame, and once per elapsed second emit the rate.
// performance.now() is cheap (no GPU stall, unlike gl.getError); the gated log
// fires at most once/sec, so this adds no meaningful per-frame cost.
if (DEBUG) {
@@ -1968,10 +2010,6 @@ export function create(canvas: HTMLCanvasElement): WaveformVisualizerHandle {
fpsWindowStartMs = nowMs;
}
}
// Reschedule unconditionally — the loop runs continuously now (lava reframe Part C); it is
// stopped only by dispose() or the tab going hidden, never by audio pausing.
rafId = requestAnimationFrame(frame);
}
// ── Tab-visibility gating (lava reframe Part C power-saving). ────────────────────
@@ -0,0 +1,137 @@
/**
* hwAccel classifier tests the pure software-renderer signature matching and the
* uncertainty/failure policy that drives the lava default-off decision.
*
* These cover the code-PROVABLE half of the feature: given a renderer string (or its absence, or a
* total WebGL failure), is the browser classified "accelerated" (lava on) or not (lava off)? The
* impure probe (detectHardwareAcceleration real getContext) is browser-confirmed, not unit-tested.
*
* Same harness convention as decodePressure.test.ts no test runner in this repo; Node 22+ strips TS
* types natively. Run a copy from the COMPILED output so the `./hwAccel.js` import specifier resolves:
*
* dotnet build DeepDrftPublic/DeepDrftPublic.csproj
* cp DeepDrftPublic/Interop/visualizer/hwAccel.test.ts DeepDrftPublic/wwwroot/js/visualizer/
* node DeepDrftPublic/wwwroot/js/visualizer/hwAccel.test.ts
*
* A thrown error / non-zero exit signals failure; "ALL <n> TESTS PASSED" signals success.
* Excluded from the production tsc build via tsconfig `exclude: Interop/ ** /*.test.ts`.
*/
import {
classifyHardwareAcceleration,
isSoftwareRenderer,
SOFTWARE_RENDERER_SIGNATURES,
} from './hwAccel.js';
// --- tiny inline harness (no dependencies) ---------------------------------------------------
let passed = 0;
const failures: string[] = [];
function test(name: string, fn: () => void): void {
try {
fn();
passed++;
} catch (e) {
failures.push(`FAIL: ${name}\n ${(e as Error).message}`);
}
}
function assertTrue(actual: boolean, msg?: string): void {
if (actual !== true) throw new Error(`${msg ?? 'assertTrue'}: expected true, got ${String(actual)}`);
}
function assertFalse(actual: boolean, msg?: string): void {
if (actual !== false) throw new Error(`${msg ?? 'assertFalse'}: expected false, got ${String(actual)}`);
}
// --- isSoftwareRenderer: positive matches -----------------------------------------------------
// Real-world software renderer strings, as reported by UNMASKED_RENDERER_WEBGL on accel-off configs.
const SOFTWARE_STRINGS = [
'Google SwiftShader',
'ANGLE (Google, Vulkan 1.3.0 (SwiftShader Device (LLVM 10.0.0) (0x0000C0DE)), SwiftShader driver)',
'llvmpipe (LLVM 12.0.0, 256 bits)',
'Gallium 0.4 on llvmpipe (LLVM 17.0.6, 256 bits)',
'softpipe',
'Microsoft Basic Render Driver',
'Mesa OffScreen',
'Software Rasterizer',
];
for (const s of SOFTWARE_STRINGS) {
test(`isSoftwareRenderer matches software string: "${s}"`, () => {
assertTrue(isSoftwareRenderer(s), `"${s}" should match a software signature`);
});
}
// --- isSoftwareRenderer: hardware (GPU) strings must NOT match ---------------------------------
const HARDWARE_STRINGS = [
'ANGLE (NVIDIA, NVIDIA GeForce RTX 3080 Direct3D11 vs_5_0 ps_5_0, D3D11)',
'ANGLE (Intel, Intel(R) Iris(R) Xe Graphics Direct3D11 vs_5_0 ps_5_0, D3D11)',
'ANGLE (AMD, AMD Radeon RX 6800 XT Direct3D11 vs_5_0 ps_5_0, D3D11)',
'Apple GPU',
'Mali-G78',
'Adreno (TM) 650',
];
for (const s of HARDWARE_STRINGS) {
test(`isSoftwareRenderer rejects hardware string: "${s}"`, () => {
assertFalse(isSoftwareRenderer(s), `"${s}" should NOT match any software signature`);
});
}
// --- case-insensitivity -----------------------------------------------------------------------
test('isSoftwareRenderer is case-insensitive', () => {
assertTrue(isSoftwareRenderer('SWIFTSHADER'), 'upper-case must still match');
assertTrue(isSoftwareRenderer('LlVmPiPe'), 'mixed-case must still match');
});
test('every declared signature self-matches (sanity on the list)', () => {
for (const sig of SOFTWARE_RENDERER_SIGNATURES) {
assertTrue(isSoftwareRenderer(sig), `signature "${sig}" must match itself`);
}
});
// --- classifyHardwareAcceleration: the full policy --------------------------------------------
test('positive software match → NOT accelerated (lava off)', () => {
assertFalse(
classifyHardwareAcceleration(true, 'Google SwiftShader'),
'a working context with a software renderer must classify as not accelerated',
);
});
test('real GPU renderer → accelerated (lava on)', () => {
assertTrue(
classifyHardwareAcceleration(true, 'ANGLE (NVIDIA GeForce RTX 3080, D3D11)'),
'a working context with a GPU renderer must classify as accelerated',
);
});
// Uncertainty / default-on case: context works but the renderer string is masked or absent.
test('masked renderer (null) with a working context → accelerated (default on)', () => {
assertTrue(
classifyHardwareAcceleration(true, null),
'an unknown renderer must favor the HW-accel majority',
);
});
test('empty/whitespace renderer with a working context → accelerated (default on)', () => {
assertTrue(classifyHardwareAcceleration(true, ''), 'empty string is unknown, not software');
assertTrue(classifyHardwareAcceleration(true, ' '), 'whitespace is unknown, not software');
});
// Total-WebGL-failure case: no context at all → lava can't run → not accelerated.
test('no WebGL context at all → NOT accelerated (lava off), regardless of renderer arg', () => {
assertFalse(classifyHardwareAcceleration(false, null), 'no context → lava off');
assertFalse(
classifyHardwareAcceleration(false, 'ANGLE (NVIDIA GeForce RTX 3080, D3D11)'),
'no context dominates even a GPU-looking string',
);
});
// --- report ----------------------------------------------------------------------------------
if (failures.length > 0) {
console.error(failures.join('\n'));
throw new Error(`${failures.length} test(s) failed, ${passed} passed`);
}
console.log(`ALL ${passed} TESTS PASSED`);
@@ -0,0 +1,108 @@
/**
* Hardware-acceleration probe for the lava-lamp visualizer.
*
* WHY: with hardware acceleration OFF the WebGL2 lava field software-renders on the main thread and
* starves WebCodecs Opus decode playback struggles. The decodePressure auto-throttle alone is not
* enough even throttled, software-rendered lava is too expensive. So when there is no HW-accel
* support we default the LAVA subsystem OFF (the expensive part) while keeping the WAVEFORM ON. With
* HW accel present (the common case) nothing changes lava defaults on, full quality.
*
* The probe creates a throwaway WebGL context, reads the unmasked renderer string via
* WEBGL_debug_renderer_info, and matches it against known software-renderer signatures.
*
* UNCERTAINTY POLICY (favor the HW-accel majority): lava is disabled ONLY on a positive
* software-renderer match or a total failure to obtain any WebGL context (lava can't run at all). If
* the renderer string is unavailable/masked (some privacy configs strip
* WEBGL_debug_renderer_info) but a context otherwise succeeds, we default to "accelerated" we do not
* disable lava on absence of evidence, only on positive evidence of software rendering.
*
* LIMIT (browser-confirmed, not code-provable): UNMASKED_RENDERER_WEBGL can be masked, and a given
* browser running with HW accel OFF may report a string none of these signatures match in which
* case this probe reports "accelerated" and lava stays on. The signature list below is the only
* tunable; if a real software-renderer string slips through, add it here.
*/
/**
* Case-insensitive substrings that positively identify a software (non-GPU) WebGL renderer. Matching
* any one of these means the browser is software-rendering WebGL lava off. Order is irrelevant.
*/
export const SOFTWARE_RENDERER_SIGNATURES: readonly string[] = [
'swiftshader', // Chrome's software GL fallback (also "Google SwiftShader")
'llvmpipe', // Mesa software rasterizer (Linux)
'softpipe', // Mesa software rasterizer (older/gallium)
'microsoft basic render', // Windows "Microsoft Basic Render Driver"
'mesa offscreen', // Mesa headless/offscreen software path
'software', // generic catch-all ("... Software ...")
];
/**
* Pure predicate: does this renderer string positively identify a software renderer? Case-insensitive
* substring match against {@link SOFTWARE_RENDERER_SIGNATURES}. Empty/whitespace is NOT a match a
* masked/absent string is "unknown", not "software" (see {@link classifyHardwareAcceleration}).
*/
export function isSoftwareRenderer(renderer: string): boolean {
const r = renderer.toLowerCase();
return SOFTWARE_RENDERER_SIGNATURES.some((sig) => r.includes(sig));
}
/**
* Pure classifier mapping probe observations to "is hardware accelerated?". Split out from the
* DOM-touching {@link detectHardwareAcceleration} so the policy is unit-testable without a browser.
*
* no WebGL context at all false (lava can't run total failure)
* renderer masked/absent true (favor the HW-accel majority absence of evidence)
* positive software match false (positive evidence of software rendering)
* otherwise true (a real GPU renderer string)
*/
export function classifyHardwareAcceleration(hasWebglContext: boolean, renderer: string | null): boolean {
if (!hasWebglContext) return false;
if (renderer === null || renderer.trim() === '') return true;
return !isSoftwareRenderer(renderer);
}
/** Read the unmasked renderer string, or null when the debug extension is unavailable/masked. */
function readUnmaskedRenderer(gl: WebGLRenderingContext | WebGL2RenderingContext): string | null {
const ext = gl.getExtension('WEBGL_debug_renderer_info');
if (!ext) return null;
const renderer = gl.getParameter(ext.UNMASKED_RENDERER_WEBGL);
return typeof renderer === 'string' ? renderer : null;
}
// Probe once per page — the renderer is a constant for the lifetime of the document. Cached so the
// scoped C# control-state's one-time default-set never pays for a second throwaway context.
let cached: boolean | undefined;
/**
* Probe the browser for WebGL hardware acceleration. Returns true when the lava subsystem should
* default ON (HW accel present or renderer unknown), false when it should default OFF (positive
* software-renderer match or no WebGL context at all). Cached after the first call; never throws.
*/
export function detectHardwareAcceleration(): boolean {
if (cached !== undefined) return cached;
cached = probe();
return cached;
}
function probe(): boolean {
try {
const canvas = document.createElement('canvas');
const gl = (canvas.getContext('webgl2') ?? canvas.getContext('webgl')) as
| WebGLRenderingContext
| WebGL2RenderingContext
| null;
if (!gl) return classifyHardwareAcceleration(false, null);
const result = classifyHardwareAcceleration(true, readUnmaskedRenderer(gl));
// Release the throwaway context — WebGL contexts are a scarce per-page resource (~16 in
// Chrome before force-eviction). The renderer string is already captured in `result` above
// so this is safe to call before returning. Inner try/catch ensures a rogue loseContext
// implementation (or a browser that surfaces it incorrectly) cannot silently swallow the
// result or re-throw out of probe() and trigger the defensive `return true` fallback.
try { gl.getExtension('WEBGL_lose_context')?.loseContext(); } catch { /* defensive */ }
return result;
} catch {
// getContext/createElement do not throw in practice; this guard is purely defensive. An
// unexpected probe failure should NOT regress the HW-accel majority, so default to
// accelerated (lava on) — only the clean "no context" path above disables lava.
return true;
}
}
@@ -0,0 +1,25 @@
using DeepDrftPublic.Client.Common;
using DeepDrftPublic.Client.Services;
namespace DeepDrftPublic.Services;
/// <summary>
/// Server-side prerender reader for public-site listener settings (Phase 18 wave 18.6), the sibling of
/// <see cref="DarkModeService"/>. Reads each preference's cookie via <see cref="IHttpContextAccessor"/>
/// during prerender and seeds the scoped <see cref="PublicSiteSettings"/>, which <c>MainLayout</c> then
/// rounds through <c>PersistentComponentState</c> into WASM — so the first paint already reflects the
/// listener's choice with no wrong-default flash (the streaming-quality analogue of the wrong-theme fix).
/// Inherits the shared cookie names + parsers from <see cref="SettingsServiceBase"/> so the server read and
/// the client write agree on one wire format.
/// </summary>
public class SettingsService(PublicSiteSettings settings, IHttpContextAccessor httpAccessor) : SettingsServiceBase
{
public void CheckSettings()
{
var cookies = httpAccessor.HttpContext?.Request.Cookies;
if (cookies is null) return;
cookies.TryGetValue(StreamQualityCookieName, out var streamQuality);
settings.StreamQuality = ParseStreamQuality(streamQuality);
}
}
+5
View File
@@ -11,5 +11,10 @@ public static class Startup
builder.Services
.AddHttpContextAccessor()
.AddScoped<DarkModeService>();
// Server prerender read for public-site listener settings (Phase 18 wave 18.6), sibling to
// DarkModeService. PublicSiteSettings itself is registered in the client Startup (shared by SSR and
// WASM); this seeds it from the streamQuality cookie during prerender.
builder.Services.AddScoped<SettingsService>();
}
}
+1 -2
View File
@@ -1,8 +1,7 @@
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning"
"Default": "Warning"
}
},
"AllowedHosts": "*",
+2 -1
View File
@@ -21,6 +21,7 @@
"node_modules",
"bin/**/*",
"obj/**/*",
"publish/**/*"
"publish/**/*",
"Interop/**/*.test.ts"
]
}
@@ -463,6 +463,46 @@ h2, h3, h4, h5, h6,
flex: 1 1 auto;
}
/* Public-site Settings menu (Phase 18 wave 18.6). The MudMenu body renders inside .mud-popover, which
already re-points --mud-palette-surface to the theme-aware --deepdrft-popover-surface (see above), so the
panel inherits the correct surface + text in both themes with no dark override. These rules are layout
only: padding, the section heading, and each settings item's label/control stacking. Scoped via the
global stylesheet (not CSS isolation) because the menu body portals out of the component's DOM scope. */
.dd-settings-panel {
padding: 0.75rem 1rem;
min-width: 240px;
max-width: 320px;
color: var(--deepdrft-page-text);
}
.dd-settings-heading {
font-family: "DM Sans", sans-serif;
font-size: 0.7rem;
font-weight: 600;
letter-spacing: 0.12em;
text-transform: uppercase;
opacity: 0.7;
margin-bottom: 0.5rem;
}
.dd-settings-item + .dd-settings-item {
margin-top: 0.75rem;
padding-top: 0.75rem;
border-top: 1px solid var(--deepdrft-popover-surface);
}
.dd-settings-item-label {
font-weight: 500;
margin-bottom: 0.25rem;
}
/* The honest capability note under the quality control (OQ2 / AC7). */
.dd-setting-note {
font-size: 0.75rem;
opacity: 0.75;
margin-top: 0.25rem;
}
.deepdrft-share-embed-field .mud-input-slot {
font-family: var(--deepdrft-font-mono) !important;
font-size: 0.75rem;
+16
View File
@@ -0,0 +1,16 @@
using DeepDrftAPI.Services.Opus;
namespace DeepDrftTests;
/// <summary>
/// Test double for <see cref="IOpusTranscodeQueue"/>. The background Opus derive is out of scope for the
/// upload/dual-database tests — those assert SQL + source-vault behaviour, which is unchanged by Phase 18
/// (Opus is strictly additive). Enqueuing is a no-op so the upload path under test never spins up a real
/// transcode. Records the enqueued keys in case a test wants to assert the scheduling contract.
/// </summary>
public sealed class NoOpOpusTranscodeQueue : IOpusTranscodeQueue
{
public List<string> Enqueued { get; } = [];
public void Enqueue(string entryKey) => Enqueued.Add(entryKey);
}
+437
View File
@@ -0,0 +1,437 @@
using System.Buffers.Binary;
using System.Text;
using DeepDrftContent.Processors.Opus;
namespace DeepDrftTests;
/// <summary>
/// Coverage for the Phase 18.1 seek-index + setup-header extraction (§3.4a). These exercise the pure
/// Ogg-Opus walker and the sidecar codec over hand-built Ogg streams — no ffmpeg dependency — so the
/// granule→byte mapping, pre-skip correction (RFC 7845 §4.3), page-boundary snapping, 0.5 s bucketing,
/// t=0 anchor, clamp totals, and setup-header capture are asserted deterministically. The byte layout
/// mirrors a real Opus stream: an OpusHead page, an OpusTags page, then audio pages each carrying an
/// end granule position at 48 kHz.
/// </summary>
[TestFixture]
public class OggOpusParserTests
{
// libopus default pre-skip: 312 samples at 48 kHz (≈ 6.5 ms). FFmpeg may use 3840 (~80 ms).
// Using 312 here as a realistic non-zero value that is small enough not to affect the test
// granules (which start at 48000), while still exercising the pre-skip subtraction path.
private const ushort TestPreSkip = 312;
[Test]
public void Walk_CapturesSetupHeader_AsLeadingOpusHeadAndOpusTagsPagesVerbatim()
{
var head = OggPage(granule: 0, OpusHeadPacket());
var tags = OggPage(granule: 0, OpusTagsPacket());
var audio = OggPage(granule: 48000, AudioPacket(64));
var stream = Concat(head, tags, audio);
var walk = OggOpusParser.Walk(stream);
Assert.That(walk, Is.Not.Null, "A well-formed Ogg Opus stream must walk");
var expectedSetup = Concat(head, tags);
Assert.That(walk!.SetupHeaderBytes, Is.EqualTo(expectedSetup),
"Setup header must be the OpusHead + OpusTags pages, byte-for-byte, and stop before the first audio page");
}
[Test]
public void Walk_FirstSeekPoint_IsAnchoredAtTimeZero_PointingAtFirstAudioPage()
{
// Use a non-zero pre-skip to verify that the first seek point is explicitly anchored at
// corrected time = 0, not at the raw granule time.
var head = OggPage(granule: 0, OpusHeadPacket(preSkip: TestPreSkip));
var tags = OggPage(granule: 0, OpusTagsPacket());
var audio = OggPage(granule: 48000, AudioPacket(64));
var stream = Concat(head, tags, audio);
var firstAudioOffset = (ulong)(head.Length + tags.Length);
var walk = OggOpusParser.Walk(stream);
Assert.That(walk, Is.Not.Null);
var first = walk!.SeekIndex.Points[0];
// The first point's byte offset must be the first audio page start (exact page boundary).
Assert.That(first.ByteOffset, Is.EqualTo(firstAudioOffset),
"The first seek point must land on the first audio page's start offset (an exact page boundary)");
// The first point's stored granule is clamped to preSkip so corrected presentation time = 0.
// This guarantees a binary search for t=0 always resolves to the first audio page.
Assert.That(walk.SeekIndex.PreSkip, Is.EqualTo(TestPreSkip),
"PreSkip must be parsed from OpusHead and carried into the seek index");
Assert.That(walk.SeekIndex.PresentationTimeSeconds(first), Is.EqualTo(0.0).Within(1e-12),
"First seek point must have corrected presentation time = 0 so a seek to t=0 always resolves");
}
[Test]
public void Walk_PreSkip_IsSubtractedFromGranuleInTimeCalculations()
{
// Pre-skip of 3840 samples (≈ 80 ms, the libopus typical value used by ffmpeg).
// Without the fix, pageTime = 48000 / 48000 = 1.0 s; with fix, (48000 - 3840) / 48000 = 0.92 s.
const ushort preSkip = 3840;
var head = OggPage(granule: 0, OpusHeadPacket(preSkip: preSkip));
var tags = OggPage(granule: 0, OpusTagsPacket());
// First audio page at granule 48000 (1.0 s raw; 0.92 s corrected)
var a1 = OggPage(granule: 48000, AudioPacket(64));
// Second audio page at granule 96000 (2.0 s raw; 1.92 s corrected)
var a2 = OggPage(granule: 96000, AudioPacket(64));
// Third audio page at granule 144000 (3.0 s raw; 2.92 s corrected)
var a3 = OggPage(granule: 144000, AudioPacket(64));
var walk = OggOpusParser.Walk(Concat(head, tags, a1, a2, a3));
Assert.That(walk, Is.Not.Null);
var index = walk!.SeekIndex;
Assert.That(index.PreSkip, Is.EqualTo(preSkip), "PreSkip must be parsed from OpusHead");
// TotalDurationSeconds must be pre-skip-corrected: (144000 - 3840) / 48000 = 2.92 s
var expectedDuration = (144000.0 - preSkip) / 48000.0;
Assert.That(index.TotalDurationSeconds, Is.EqualTo(expectedDuration).Within(1e-9),
"TotalDurationSeconds must subtract preSkip (RFC 7845 §4.3), not use raw lastGranule / 48000");
// The second indexed point (first real bucket) must have corrected time, not raw time.
// With correctedTime(a2) = 1.92 s and bucket = 0.5 s, it should fall in the 1.5 s bucket.
if (index.Points.Count > 1)
{
var secondPoint = index.Points[1];
var corrected = index.PresentationTimeSeconds(secondPoint);
Assert.That(corrected, Is.GreaterThan(0.0),
"Non-first indexed points must have positive corrected presentation times");
Assert.That(secondPoint.RawTimeSeconds, Is.GreaterThan(corrected),
"Raw time must be greater than corrected time when pre-skip > 0");
}
}
[Test]
public void Walk_SeekToZero_ResolvesToFirstAudioPageOffset_WithNonZeroPreSkip()
{
// This is the AC9 / Critical-2 regression test: a seek to t=0 must resolve to the first audio
// page's byte offset, not produce "no entry found". With the old code (no t=0 anchor and no
// pre-skip correction), the first indexed point had correctedTime ≈ 0.92 s (for preSkip=3840),
// so a binary search for t=0 would find no entry with time ≤ 0 and fail.
const ushort preSkip = 3840;
var head = OggPage(granule: 0, OpusHeadPacket(preSkip: preSkip));
var tags = OggPage(granule: 0, OpusTagsPacket());
var a1 = OggPage(granule: 48000, AudioPacket(64));
var a2 = OggPage(granule: 96000, AudioPacket(64));
var stream = Concat(head, tags, a1, a2);
var firstAudioByteOffset = (ulong)(head.Length + tags.Length);
var walk = OggOpusParser.Walk(stream);
Assert.That(walk, Is.Not.Null);
var index = walk!.SeekIndex;
var firstPoint = index.Points[0];
// Simulate the binary search: find the largest entry with PresentationTimeSeconds ≤ 0.
// With the fix, the first point has corrected time = 0.0, so it IS found.
Assert.That(index.PresentationTimeSeconds(firstPoint), Is.EqualTo(0.0).Within(1e-12),
"First point corrected time must be exactly 0.0 so binary search for t=0 resolves it");
Assert.That(firstPoint.ByteOffset, Is.EqualTo(firstAudioByteOffset),
"The t=0 anchor must point at the first audio page's byte offset, not the stream start");
}
[Test]
public void Walk_EverySeekOffset_LandsOnARealPageBoundary()
{
// Ten audio pages, ~0.25 s each (12000 samples). Bucketing at 0.5 s means roughly every other
// page is indexed; every indexed offset must still be the start of some OggS page.
var head = OggPage(granule: 0, OpusHeadPacket());
var tags = OggPage(granule: 0, OpusTagsPacket());
var pages = new List<byte[]> { head, tags };
ulong granule = 0;
for (var i = 0; i < 10; i++)
{
granule += 12000; // 0.25 s at 48 kHz
pages.Add(OggPage(granule, AudioPacket(50 + i)));
}
var stream = Concat(pages.ToArray());
var pageOffsets = CollectPageStartOffsets(stream);
var walk = OggOpusParser.Walk(stream);
Assert.That(walk, Is.Not.Null);
foreach (var point in walk!.SeekIndex.Points)
{
Assert.That(pageOffsets, Does.Contain(point.ByteOffset),
$"Seek offset {point.ByteOffset} must be a real OggS page start");
}
}
[Test]
public void Walk_Bucketing_EmitsRoughlyOneEntryPerHalfSecond()
{
// Twenty audio pages of 0.25 s each = 5 s total (zero pre-skip). At 0.5 s buckets:
// first point (anchored at t=0) + one per 0.5 s boundary = 1 + 10 = 11 entries expected.
var head = OggPage(granule: 0, OpusHeadPacket());
var tags = OggPage(granule: 0, OpusTagsPacket());
var pages = new List<byte[]> { head, tags };
ulong granule = 0;
for (var i = 0; i < 20; i++)
{
granule += 12000; // 0.25 s
pages.Add(OggPage(granule, AudioPacket(40)));
}
var stream = Concat(pages.ToArray());
var walk = OggOpusParser.Walk(stream);
Assert.That(walk, Is.Not.Null);
// 5 s of audio with 0.5 s buckets: 1 anchor + 10 bucket crossings = 11 entries.
// Accept 1012 for floating-point boundary tolerance, but must be far below 20 (one per page).
Assert.That(walk!.SeekIndex.Points.Count, Is.InRange(10, 12),
"Bucketing must coalesce ~0.25 s pages into ~0.5 s index entries, not one per page");
}
[Test]
public void Walk_PointsAreStrictlyAscending_InBothGranuleAndOffset()
{
var head = OggPage(granule: 0, OpusHeadPacket());
var tags = OggPage(granule: 0, OpusTagsPacket());
var pages = new List<byte[]> { head, tags };
ulong granule = 0;
for (var i = 0; i < 12; i++)
{
granule += 24000; // 0.5 s — one index entry per page
pages.Add(OggPage(granule, AudioPacket(30)));
}
var walk = OggOpusParser.Walk(Concat(pages.ToArray()));
Assert.That(walk, Is.Not.Null);
var points = walk!.SeekIndex.Points;
for (var i = 1; i < points.Count; i++)
{
Assert.That(points[i].GranulePosition, Is.GreaterThan(points[i - 1].GranulePosition));
Assert.That(points[i].ByteOffset, Is.GreaterThan(points[i - 1].ByteOffset));
}
}
[Test]
public void Walk_ClampValues_ReflectPreSkipCorrectedDurationAndTotalByteLength()
{
const ushort preSkip = 312;
var head = OggPage(granule: 0, OpusHeadPacket(preSkip: preSkip));
var tags = OggPage(granule: 0, OpusTagsPacket());
var a1 = OggPage(granule: 48000, AudioPacket(64)); // 1.0 s raw; ~0.9935 s corrected
var a2 = OggPage(granule: 144000, AudioPacket(64)); // 3.0 s raw; ~2.9935 s corrected (final)
var stream = Concat(head, tags, a1, a2);
var walk = OggOpusParser.Walk(stream);
Assert.That(walk, Is.Not.Null);
Assert.That(walk!.SeekIndex.TotalByteLength, Is.EqualTo((ulong)stream.Length),
"Total byte length must equal the full stream length for end-of-stream clamping");
var expectedDuration = (144000.0 - preSkip) / 48000.0;
Assert.That(walk.SeekIndex.TotalDurationSeconds, Is.EqualTo(expectedDuration).Within(1e-9),
"TotalDurationSeconds must be pre-skip-corrected: (lastGranule - preSkip) / 48000");
Assert.That(walk.SeekIndex.PreSkip, Is.EqualTo(preSkip),
"PreSkip must round-trip through the seek index");
}
[Test]
public void Walk_MalformedStream_ReturnsNull_RatherThanThrowing()
{
var notOgg = Encoding.ASCII.GetBytes("this is not an ogg stream at all");
Assert.That(OggOpusParser.Walk(notOgg), Is.Null);
// OpusHead present but no audio pages → no seek points → null (nothing to index).
var headOnly = Concat(OggPage(0, OpusHeadPacket()), OggPage(0, OpusTagsPacket()));
Assert.That(OggOpusParser.Walk(headOnly), Is.Null);
}
[Test]
public void SeekIndex_RoundTrips_ThroughBinaryEncoding()
{
var points = new[]
{
new OpusSeekPoint(312, 200), // first point anchored at preSkip
new OpusSeekPoint(72000, 512),
new OpusSeekPoint(96000, 900),
};
var index = new OggOpusSeekIndex(points, TotalDurationSeconds: 2.0, TotalByteLength: 1024,
PreSkip: 312);
var restored = OggOpusSeekIndex.FromBytes(index.ToBytes());
Assert.That(restored, Is.Not.Null);
Assert.That(restored!.TotalByteLength, Is.EqualTo(1024UL));
Assert.That(restored.TotalDurationSeconds, Is.EqualTo(2.0));
Assert.That(restored.PreSkip, Is.EqualTo((ushort)312),
"PreSkip must survive the binary round-trip");
Assert.That(restored.Points, Is.EqualTo(points));
}
[Test]
public void SeekIndex_PresentationTimeSeconds_SubtractsPreSkip()
{
const ushort preSkip = 3840;
var point = new OpusSeekPoint(GranulePosition: 48000, ByteOffset: 200);
var index = new OggOpusSeekIndex(
new[] { point }, TotalDurationSeconds: 0.92, TotalByteLength: 500, PreSkip: preSkip);
var corrected = index.PresentationTimeSeconds(point);
var expected = (48000.0 - preSkip) / 48000.0; // ≈ 0.92 s
Assert.That(corrected, Is.EqualTo(expected).Within(1e-9),
"PresentationTimeSeconds must return (granule - preSkip) / 48000, not raw granule / 48000");
}
[Test]
public void SeekIndex_PresentationTimeSeconds_ClampsToZeroForFirstAnchorPoint()
{
const ushort preSkip = 3840;
// First anchor point: granule stored as preSkip, so corrected time = 0.
var firstPoint = new OpusSeekPoint(GranulePosition: preSkip, ByteOffset: 150);
var index = new OggOpusSeekIndex(
new[] { firstPoint }, TotalDurationSeconds: 2.0, TotalByteLength: 500, PreSkip: preSkip);
Assert.That(index.PresentationTimeSeconds(firstPoint), Is.EqualTo(0.0).Within(1e-12),
"The t=0 anchor point (granule == preSkip) must yield corrected time = 0.0 exactly");
}
[Test]
public void Sidecar_RoundTrips_PreservingSetupHeaderAndIndex()
{
var setup = Encoding.ASCII.GetBytes("OpusHead-and-OpusTags-bytes-go-here");
var index = new OggOpusSeekIndex(
new[] { new OpusSeekPoint(312, 200), new OpusSeekPoint(96000, 700) },
TotalDurationSeconds: 2.0, TotalByteLength: 800, PreSkip: 312);
var sidecar = new OpusSidecar(setup, index);
var restored = OpusSidecar.FromBytes(sidecar.ToBytes());
Assert.That(restored, Is.Not.Null);
Assert.That(restored!.SetupHeaderBytes, Is.EqualTo(setup),
"The sidecar must preserve the setup header so the client can prepend it to mid-stream slices");
Assert.That(restored.SeekIndex.Points, Is.EqualTo(index.Points));
Assert.That(restored.SeekIndex.TotalByteLength, Is.EqualTo(800UL));
Assert.That(restored.SeekIndex.PreSkip, Is.EqualTo((ushort)312),
"PreSkip must survive the sidecar binary round-trip");
}
[Test]
public void Sidecar_FromBytes_RejectsTruncatedBlob()
{
Assert.That(OpusSidecar.FromBytes(new byte[2]), Is.Null, "A blob shorter than the length prefix is corruption");
// A length prefix that overruns the buffer must be rejected, not over-read.
var bad = new byte[8];
BinaryPrimitives.WriteUInt32LittleEndian(bad, 9999);
Assert.That(OpusSidecar.FromBytes(bad), Is.Null);
}
// ---- Ogg stream construction helpers (minimal, single-packet pages) ----
// Builds one Ogg page wrapping a single packet payload with the given end granule position. The page
// header layout matches the spec the parser reads: capture "OggS", version, header-type, granulepos,
// serial, sequence, checksum (zeroed — the parser does not verify CRC), page-segments, segment table.
private static byte[] OggPage(ulong granule, byte[] packet)
{
// Lacing: a packet of length L is split into 255-byte segments plus a final < 255 segment.
var segments = new List<byte>();
var remaining = packet.Length;
while (remaining >= 255)
{
segments.Add(255);
remaining -= 255;
}
segments.Add((byte)remaining);
var header = new byte[OggOpusConstants.OggPageHeaderSize + segments.Count];
OggOpusConstants.CapturePattern.CopyTo(header);
header[4] = 0; // version
header[5] = 0; // header-type flags
BinaryPrimitives.WriteUInt64LittleEndian(header.AsSpan(OggOpusConstants.GranulePositionOffset, 8), granule);
BinaryPrimitives.WriteUInt32LittleEndian(header.AsSpan(14, 4), 0xDEAD); // serial
BinaryPrimitives.WriteUInt32LittleEndian(header.AsSpan(18, 4), 0); // sequence (unused by parser)
BinaryPrimitives.WriteUInt32LittleEndian(header.AsSpan(22, 4), 0); // checksum (unverified)
header[OggOpusConstants.PageSegmentCountOffset] = (byte)segments.Count;
for (var i = 0; i < segments.Count; i++)
header[OggOpusConstants.OggPageHeaderSize + i] = segments[i];
return Concat(header, packet);
}
private static byte[] OpusHeadPacket(ushort preSkip = 0)
{
// "OpusHead" + RFC 7845 §5.1 identification header:
// [0] version = 1, [1] channel count = 2,
// [2-3] pre_skip (little-endian uint16), [4-7] input sample rate = 0xBB80 = 48000,
// [8-9] output gain = 0, [10] channel mapping family = 0.
var tail = new byte[11];
tail[0] = 1; // version
tail[1] = 2; // channels
BinaryPrimitives.WriteUInt16LittleEndian(tail.AsSpan(2, 2), preSkip); // pre_skip
BinaryPrimitives.WriteUInt32LittleEndian(tail.AsSpan(4, 4), 48000); // input sample rate
tail[10] = 0; // channel mapping family
return Concat(OggOpusConstants.OpusHeadSignature.ToArray(), tail);
}
private static byte[] OpusTagsPacket()
{
// "OpusTags" + a tiny vendor string region (length-prefixed) + zero user comments.
var vendor = Encoding.ASCII.GetBytes("test");
var packet = new List<byte>();
packet.AddRange(OggOpusConstants.OpusTagsSignature.ToArray());
packet.AddRange(BitConverter.GetBytes((uint)vendor.Length));
packet.AddRange(vendor);
packet.AddRange(BitConverter.GetBytes(0u)); // user comment count
return packet.ToArray();
}
private static byte[] AudioPacket(int size)
{
var packet = new byte[size];
for (var i = 0; i < size; i++)
packet[i] = (byte)(i & 0xFF);
return packet;
}
private static List<ulong> CollectPageStartOffsets(byte[] stream)
{
var offsets = new List<ulong>();
var span = stream.AsSpan();
var offset = 0;
while (offset + OggOpusConstants.OggPageHeaderSize <= span.Length)
{
var page = span.Slice(offset);
if (!page[..4].SequenceEqual(OggOpusConstants.CapturePattern))
break;
var segmentCount = page[OggOpusConstants.PageSegmentCountOffset];
var payload = 0;
for (var i = 0; i < segmentCount; i++)
payload += page[OggOpusConstants.OggPageHeaderSize + i];
offsets.Add((ulong)offset);
offset += OggOpusConstants.OggPageHeaderSize + segmentCount + payload;
}
return offsets;
}
private static byte[] Concat(params byte[][] parts)
{
var total = parts.Sum(p => p.Length);
var result = new byte[total];
var cursor = 0;
foreach (var part in parts)
{
part.CopyTo(result, cursor);
cursor += part.Length;
}
return result;
}
}
+251
View File
@@ -0,0 +1,251 @@
using System.Text;
using Data.Data.Repositories;
using Data.Managers;
using DeepDrftAPI.Services;
using DeepDrftContent;
using DeepDrftContent.Constants;
using DeepDrftContent.FileDatabase.Models;
using DeepDrftContent.Processors;
using DeepDrftContent.Processors.Opus;
using DeepDrftData;
using DeepDrftData.Data;
using DeepDrftData.Repositories;
using DeepDrftModels.DTOs;
using DeepDrftModels.Entities;
using DeepDrftModels.Enums;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using FileDb = DeepDrftContent.FileDatabase.Services.FileDatabase;
namespace DeepDrftTests;
/// <summary>
/// Tests for the Phase 18.5 Backfill-Opus scheduling contract
/// (<see cref="UnifiedTrackService.BackfillOpusAsync"/> and <see cref="UnifiedTrackService.EnqueueOpusAsync"/>).
/// These assert the enqueue decision — which tracks get a background derive scheduled — over a real
/// <see cref="FileDb"/>, a real <see cref="TrackFormatResolver"/>, and an in-memory SQL store, with a
/// recording <see cref="NoOpOpusTranscodeQueue"/> standing in for the background worker (the actual transcode
/// is not exercised here — it needs ffmpeg and is out of scope for the scheduling contract).
///
/// The decision under test: a track is enqueued iff it lacks a COMPLETE Opus artifact (both the Opus audio
/// bytes and the seek/setup sidecar). A track with both is skipped; a half-derived track (audio without
/// sidecar) is treated as incomplete and re-enqueued so a backfill heals it.
/// </summary>
[TestFixture]
public class OpusBackfillTests
{
private string _testDir = string.Empty;
private DeepDrftContext _context = null!;
[SetUp]
public void SetUp()
{
_testDir = Path.Combine(Path.GetTempPath(), "OpusBackfillTests", Guid.NewGuid().ToString());
Directory.CreateDirectory(_testDir);
var options = new DbContextOptionsBuilder<DeepDrftContext>()
.UseInMemoryDatabase(databaseName: Guid.NewGuid().ToString())
.Options;
_context = new DeepDrftContext(options);
}
[TearDown]
public void TearDown()
{
_context.Dispose();
try { Directory.Delete(_testDir, recursive: true); }
catch { /* Best-effort cleanup — ignore failures */ }
}
private TrackManager CreateManager()
{
var repository = new TrackRepository(
_context, NullLogger<Repository<DeepDrftContext, TrackEntity>>.Instance);
return new TrackManager(
repository, NullLogger<Manager<TrackEntity, TrackDto, TrackRepository, TrackConverter>>.Instance);
}
private sealed record Harness(
UnifiedTrackService Service,
NoOpOpusTranscodeQueue Queue,
TrackContentService Content,
FileDb FileDatabase,
ITrackService Sql);
private async Task<Harness> BuildAsync()
{
var fileDatabase = await FileDb.FromAsync(_testDir);
Assert.That(fileDatabase, Is.Not.Null);
var content = new TrackContentService(
fileDatabase!, new AudioProcessorRouter(
new AudioProcessor(), new Mp3AudioProcessor(), new FlacAudioProcessor()));
var waveforms = new WaveformProfileService(
fileDatabase!, new AudioProcessor(), new RmsLoudnessAlgorithm(),
Options.Create(new WaveformProfileOptions()), NullLogger<WaveformProfileService>.Instance);
var resolver = new TrackFormatResolver(
fileDatabase!, content, NullLogger<TrackFormatResolver>.Instance);
var queue = new NoOpOpusTranscodeQueue();
var sql = CreateManager();
var service = new UnifiedTrackService(
content, sql, fileDatabase!, waveforms, queue, resolver,
NullLogger<UnifiedTrackService>.Instance);
await fileDatabase!.CreateVaultAsync(VaultConstants.TrackOpus, MediaVaultType.Audio);
return new Harness(service, queue, content, fileDatabase!, sql);
}
// Seeds a track: stores a real source WAV in the tracks vault and a SQL row pointing at the same EntryKey.
// Returns the EntryKey so the test can selectively add Opus artifacts to a subset.
private async Task<string> SeedTrackAsync(Harness h, string title)
{
var wavPath = Path.Combine(_testDir, Guid.NewGuid().ToString("N") + ".wav");
await File.WriteAllBytesAsync(wavPath, BuildMinimalPcmWav(2.0));
var unpersisted = await h.Content.AddTrackAsync(wavPath, title, "Artist");
Assert.That(unpersisted, Is.Not.Null);
var dto = new TrackDto { EntryKey = unpersisted!.EntryKey, TrackName = title };
var created = await h.Sql.Create(dto);
Assert.That(created.Success, Is.True, created.Messages.FirstOrDefault()?.Message);
return unpersisted.EntryKey;
}
private async Task StoreOpusAudioAsync(Harness h, string entryKey)
{
var opus = new AudioBinary(new AudioBinaryParams("opus"u8.ToArray(), 4, ".opus", 2.0, 320));
Assert.That(
await h.FileDatabase.RegisterResourceAsync(
VaultConstants.TrackOpus, OpusTranscodeService.OpusAudioKey(entryKey), opus),
Is.True);
}
private async Task StoreSidecarAsync(Harness h, string entryKey)
{
var sidecar = new MediaBinary(new MediaBinaryParams("idx"u8.ToArray(), 3, ".opusidx"));
Assert.That(
await h.FileDatabase.RegisterResourceAsync(
VaultConstants.TrackOpus, OpusTranscodeService.OpusSidecarKey(entryKey), sidecar),
Is.True);
}
[Test]
public async Task BackfillOpus_EnqueuesOnlyTracksWithoutCompleteOpus()
{
var h = await BuildAsync();
// Three tracks: one fully derived (audio + sidecar), one bare (no Opus), one half-derived (audio only).
var complete = await SeedTrackAsync(h, "Complete");
await StoreOpusAudioAsync(h, complete);
await StoreSidecarAsync(h, complete);
var bare = await SeedTrackAsync(h, "Bare");
var halfDerived = await SeedTrackAsync(h, "HalfDerived");
await StoreOpusAudioAsync(h, halfDerived); // audio but no sidecar → unseekable → treated as incomplete
var result = await h.Service.BackfillOpusAsync(CancellationToken.None);
Assert.That(result.Success, Is.True, result.Messages.FirstOrDefault()?.Message);
Assert.Multiple(() =>
{
Assert.That(result.Value.Enqueued, Is.EqualTo(2), "the bare and half-derived tracks must be enqueued");
Assert.That(result.Value.Skipped, Is.EqualTo(1), "the fully-derived track must be skipped");
Assert.That(h.Queue.Enqueued, Does.Contain(bare));
Assert.That(h.Queue.Enqueued, Does.Contain(halfDerived));
Assert.That(h.Queue.Enqueued, Does.Not.Contain(complete), "a complete Opus artifact is not re-enqueued");
});
}
[Test]
public async Task BackfillOpus_WhenAllTracksHaveOpus_EnqueuesNothing()
{
var h = await BuildAsync();
var a = await SeedTrackAsync(h, "A");
await StoreOpusAudioAsync(h, a);
await StoreSidecarAsync(h, a);
var result = await h.Service.BackfillOpusAsync(CancellationToken.None);
Assert.That(result.Success, Is.True);
Assert.Multiple(() =>
{
Assert.That(result.Value.Enqueued, Is.Zero);
Assert.That(result.Value.Skipped, Is.EqualTo(1));
Assert.That(h.Queue.Enqueued, Is.Empty, "an all-derived catalogue schedules no transcodes");
});
}
[Test]
public async Task EnqueueOpus_KnownTrack_Enqueues()
{
var h = await BuildAsync();
var entryKey = await SeedTrackAsync(h, "Solo");
var result = await h.Service.EnqueueOpusAsync(entryKey, CancellationToken.None);
Assert.That(result.Success, Is.True);
Assert.That(h.Queue.Enqueued, Does.Contain(entryKey));
}
[Test]
public async Task EnqueueOpus_UnknownTrack_FailsWithNotFound_AndEnqueuesNothing()
{
var h = await BuildAsync();
var result = await h.Service.EnqueueOpusAsync("no-such-track", CancellationToken.None);
Assert.Multiple(() =>
{
Assert.That(result.Success, Is.False);
Assert.That(result.Messages.FirstOrDefault()?.Message, Is.EqualTo(UnifiedTrackService.TrackNotFoundMessage));
Assert.That(h.Queue.Enqueued, Is.Empty, "an unknown track must not schedule a transcode");
});
}
// Standard-PCM mono 16-bit 44.1 kHz WAV, full-scale square wave. Same layout as the other suites.
private static byte[] BuildMinimalPcmWav(double durationSeconds)
{
const int sampleRate = 44100;
const ushort channels = 1;
const ushort bitsPerSample = 16;
const ushort blockAlign = channels * (bitsPerSample / 8);
const uint byteRate = sampleRate * blockAlign;
var frames = (int)(sampleRate * durationSeconds);
var data = new byte[frames * blockAlign];
for (var i = 0; i < frames; i++)
{
var sample = (i % 2 == 0) ? short.MaxValue : short.MinValue;
data[i * 2] = (byte)(sample & 0xFF);
data[i * 2 + 1] = (byte)((sample >> 8) & 0xFF);
}
using var ms = new MemoryStream();
using var w = new BinaryWriter(ms, Encoding.ASCII, leaveOpen: true);
w.Write(Encoding.ASCII.GetBytes("RIFF"));
w.Write((uint)(36 + data.Length));
w.Write(Encoding.ASCII.GetBytes("WAVE"));
w.Write(Encoding.ASCII.GetBytes("fmt "));
w.Write(16u);
w.Write((ushort)1); // PCM
w.Write(channels);
w.Write((uint)sampleRate);
w.Write(byteRate);
w.Write(blockAlign);
w.Write(bitsPerSample);
w.Write(Encoding.ASCII.GetBytes("data"));
w.Write((uint)data.Length);
w.Write(data);
w.Flush();
return ms.ToArray();
}
}
+180
View File
@@ -0,0 +1,180 @@
using System.Net;
using DeepDrftModels.Enums;
using DeepDrftPublic.Client.Clients;
using DeepDrftPublic.Client.Services;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.JSInterop;
using Microsoft.JSInterop.Infrastructure;
namespace DeepDrftTests;
/// <summary>
/// Unit tests for the Phase 18.5 player-side format-selection seam
/// (<see cref="StreamingAudioPlayerService.ResolveStreamFormatAsync"/>): the default policy (OQ2) of "Opus
/// when the browser can decode Ogg Opus AND a sidecar exists, else lossless", the capability gate (AC7), and
/// the sidecar-absent → lossless fallback (C2). The seam is the single, overridable hook 18.6 will use to
/// inject the listener's quality preference; these tests pin the capability-gated default it falls through to.
///
/// The seam touches two collaborators: <see cref="AudioInteropService"/> (over a fake <see cref="IJSRuntime"/>
/// — <c>canDecodeOggOpus</c> + <c>setOpusSidecar</c>) and <see cref="TrackMediaClient"/> (over a stub HTTP
/// handler — the one-time sidecar fetch). Both are real instances wired over the fakes; only the network/JS
/// boundary is faked, so the selection logic under test is exercised exactly as it runs in the browser.
/// </summary>
[TestFixture]
public class OpusFormatSelectionTests
{
// A scriptable JS runtime: canDecodeOggOpus returns a configured bool; setOpusSidecar returns a
// configured success/failure; every other invocation returns default. Records the calls so a test can
// assert the set-before-init contract was honoured (the sidecar was actually handed to the player).
private sealed class FakeJsRuntime : IJSRuntime
{
private readonly bool _canDecode;
private readonly bool _sidecarParseSucceeds;
public FakeJsRuntime(bool canDecode, bool sidecarParseSucceeds)
{
_canDecode = canDecode;
_sidecarParseSucceeds = sidecarParseSucceeds;
}
public int SetSidecarCallCount { get; private set; }
public ValueTask<TValue> InvokeAsync<TValue>(string identifier, object?[]? args)
{
if (identifier == "DeepDrftAudio.canDecodeOggOpus")
return ValueTask.FromResult((TValue)(object)_canDecode);
if (identifier == "DeepDrftAudio.setOpusSidecar")
{
SetSidecarCallCount++;
var result = new AudioOperationResult
{
Success = _sidecarParseSucceeds,
Error = _sidecarParseSucceeds ? null : "Invalid Opus sidecar blob",
};
return ValueTask.FromResult((TValue)(object)result);
}
return ValueTask.FromResult<TValue>(default!);
}
public ValueTask<TValue> InvokeAsync<TValue>(string identifier, CancellationToken cancellationToken, object?[]? args)
=> InvokeAsync<TValue>(identifier, args);
}
// Returns a configured status (with a body) for GET api/track/{id}/opus/seekdata; any other request 404s.
private sealed class StubSidecarHandler : HttpMessageHandler
{
private readonly HttpStatusCode _status;
private readonly byte[] _body;
public StubSidecarHandler(HttpStatusCode status, byte[]? body = null)
{
_status = status;
_body = body ?? [];
}
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var response = new HttpResponseMessage(_status);
if (_status == HttpStatusCode.OK)
response.Content = new ByteArrayContent(_body);
return Task.FromResult(response);
}
}
private sealed class SingleClientFactory : IHttpClientFactory
{
private readonly HttpMessageHandler _handler;
public SingleClientFactory(HttpMessageHandler handler) => _handler = handler;
public HttpClient CreateClient(string name) =>
new(_handler, disposeHandler: false) { BaseAddress = new Uri("https://content.test/") };
}
// Exposes the protected seam for direct assertion. The 18.6 override will replace this same method.
private sealed class TestablePlayer : StreamingAudioPlayerService
{
public TestablePlayer(AudioInteropService interop, TrackMediaClient media)
: base(interop, media, NullLogger<StreamingAudioPlayerService>.Instance) { }
public Task<AudioFormat> ResolveFormatForTest(string entryKey) =>
ResolveStreamFormatAsync(entryKey, CancellationToken.None);
}
private static TestablePlayer BuildPlayer(
bool canDecode, bool sidecarParseSucceeds, HttpStatusCode sidecarStatus, byte[]? sidecarBody)
{
var js = new FakeJsRuntime(canDecode, sidecarParseSucceeds);
var interop = new AudioInteropService(js);
var media = new TrackMediaClient(new SingleClientFactory(new StubSidecarHandler(sidecarStatus, sidecarBody)));
return new TestablePlayer(interop, media);
}
private static readonly byte[] SidecarBytes = "setup-header+seek-index"u8.ToArray();
// Capable browser + present sidecar → Opus. The happy path: the default policy picks the low-data format.
[Test]
public async Task ResolveStreamFormat_CapableBrowser_SidecarPresent_ChoosesOpus()
{
var player = BuildPlayer(canDecode: true, sidecarParseSucceeds: true,
HttpStatusCode.OK, SidecarBytes);
var format = await player.ResolveFormatForTest("track-1");
Assert.That(format, Is.EqualTo(AudioFormat.Opus));
}
// Capability gate (AC7): a browser that cannot decode Ogg Opus always gets lossless, and the sidecar is
// never even fetched/injected — Opus is off the table before any sidecar work.
[Test]
public async Task ResolveStreamFormat_IncapableBrowser_ChoosesLossless_AndDoesNotInjectSidecar()
{
var js = new FakeJsRuntime(canDecode: false, sidecarParseSucceeds: true);
var interop = new AudioInteropService(js);
var media = new TrackMediaClient(new SingleClientFactory(
new StubSidecarHandler(HttpStatusCode.OK, SidecarBytes)));
var player = new TestablePlayer(interop, media);
var format = await player.ResolveFormatForTest("track-1");
Assert.Multiple(() =>
{
Assert.That(format, Is.EqualTo(AudioFormat.Lossless), "incapable browser must fall back to lossless");
Assert.That(js.SetSidecarCallCount, Is.Zero, "no sidecar should be injected when Opus is gated out");
});
}
// C2 fallback: capable browser but no sidecar (legacy / not-yet-transcoded track, 404) → lossless.
[Test]
public async Task ResolveStreamFormat_CapableBrowser_NoSidecar_FallsBackToLossless()
{
var player = BuildPlayer(canDecode: true, sidecarParseSucceeds: true,
HttpStatusCode.NotFound, sidecarBody: null);
var format = await player.ResolveFormatForTest("track-1");
Assert.That(format, Is.EqualTo(AudioFormat.Lossless),
"a capable browser with no Opus sidecar must request lossless, not Opus");
}
// A present-but-unparseable sidecar (the JS decoder rejects the bytes) → lossless, so a malformed sidecar
// never breaks playback. The injection was attempted (set-before-init), but its failure degrades safely.
[Test]
public async Task ResolveStreamFormat_SidecarPresentButUnparseable_FallsBackToLossless()
{
var js = new FakeJsRuntime(canDecode: true, sidecarParseSucceeds: false);
var interop = new AudioInteropService(js);
var media = new TrackMediaClient(new SingleClientFactory(
new StubSidecarHandler(HttpStatusCode.OK, SidecarBytes)));
var player = new TestablePlayer(interop, media);
var format = await player.ResolveFormatForTest("track-1");
Assert.Multiple(() =>
{
Assert.That(format, Is.EqualTo(AudioFormat.Lossless), "an unparseable sidecar must degrade to lossless");
Assert.That(js.SetSidecarCallCount, Is.EqualTo(1), "the player attempted the set-before-init injection");
});
}
}
@@ -0,0 +1,209 @@
using System.Net;
using DeepDrftModels.Enums;
using DeepDrftPublic.Client.Clients;
using DeepDrftPublic.Client.Common;
using DeepDrftPublic.Client.Services;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.JSInterop;
namespace DeepDrftTests;
/// <summary>
/// Unit tests for the critical invariant on
/// <see cref="PreferenceAwareStreamingPlayerService.ResolveStreamFormatAsync"/> (Phase 18 wave 18.6):
/// <list type="bullet">
/// <item>A <see cref="StreamQuality.Lossless"/> preference returns <see cref="AudioFormat.Lossless"/>
/// immediately — without probing Opus capability or fetching the sidecar.</item>
/// <item>Any other preference (currently only <see cref="StreamQuality.LowData"/>) delegates to
/// <c>base.ResolveStreamFormatAsync</c>, so the AC7 capability gate and the C2 sidecar-absent →
/// lossless fallback from Phase 18.5 are inherited, not bypassed.</item>
/// </list>
/// The test infra (FakeJsRuntime, StubSidecarHandler, SingleClientFactory) mirrors
/// <see cref="OpusFormatSelectionTests"/> exactly so both test classes exercise the same seam with
/// the same fake boundary.
/// </summary>
[TestFixture]
public class PreferenceAwareStreamingPlayerServiceTests
{
// Scriptable JS runtime: tracks whether the Opus-capability probe was called so tests can assert
// the Lossless branch never touches it.
private sealed class FakeJsRuntime : IJSRuntime
{
private readonly bool _canDecode;
private readonly bool _sidecarParseSucceeds;
public FakeJsRuntime(bool canDecode = true, bool sidecarParseSucceeds = true)
{
_canDecode = canDecode;
_sidecarParseSucceeds = sidecarParseSucceeds;
}
public int CanDecodeCallCount { get; private set; }
public int SetSidecarCallCount { get; private set; }
public ValueTask<TValue> InvokeAsync<TValue>(string identifier, object?[]? args)
{
if (identifier == "DeepDrftAudio.canDecodeOggOpus")
{
CanDecodeCallCount++;
return ValueTask.FromResult((TValue)(object)_canDecode);
}
if (identifier == "DeepDrftAudio.setOpusSidecar")
{
SetSidecarCallCount++;
var result = new AudioOperationResult
{
Success = _sidecarParseSucceeds,
Error = _sidecarParseSucceeds ? null : "Invalid Opus sidecar blob",
};
return ValueTask.FromResult((TValue)(object)result);
}
return ValueTask.FromResult<TValue>(default!);
}
public ValueTask<TValue> InvokeAsync<TValue>(string identifier, CancellationToken cancellationToken, object?[]? args)
=> InvokeAsync<TValue>(identifier, args);
}
// Returns a configured status (with an optional body) for sidecar requests; anything else 404s.
private sealed class StubSidecarHandler : HttpMessageHandler
{
private readonly HttpStatusCode _status;
private readonly byte[] _body;
public StubSidecarHandler(HttpStatusCode status, byte[]? body = null)
{
_status = status;
_body = body ?? [];
}
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var response = new HttpResponseMessage(_status);
if (_status == HttpStatusCode.OK)
response.Content = new ByteArrayContent(_body);
return Task.FromResult(response);
}
}
private sealed class SingleClientFactory : IHttpClientFactory
{
private readonly HttpMessageHandler _handler;
public SingleClientFactory(HttpMessageHandler handler) => _handler = handler;
public HttpClient CreateClient(string name) =>
new(_handler, disposeHandler: false) { BaseAddress = new Uri("https://content.test/") };
}
// Exposes the protected seam for direct assertion.
private sealed class TestablePreferencePlayer : PreferenceAwareStreamingPlayerService
{
public TestablePreferencePlayer(
AudioInteropService interop,
TrackMediaClient media,
PublicSiteSettings settings)
: base(interop, media, NullLogger<StreamingAudioPlayerService>.Instance, settings) { }
public Task<AudioFormat> ResolveFormatForTest(string entryKey) =>
ResolveStreamFormatAsync(entryKey, CancellationToken.None);
}
private static readonly byte[] SidecarBytes = "setup-header+seek-index"u8.ToArray();
private static (TestablePreferencePlayer player, FakeJsRuntime js) Build(
StreamQuality quality,
bool canDecode = true,
bool sidecarParseSucceeds = true,
HttpStatusCode sidecarStatus = HttpStatusCode.OK,
byte[]? sidecarBody = null)
{
sidecarBody ??= SidecarBytes;
var js = new FakeJsRuntime(canDecode, sidecarParseSucceeds);
var interop = new AudioInteropService(js);
var media = new TrackMediaClient(new SingleClientFactory(new StubSidecarHandler(sidecarStatus, sidecarBody)));
var settings = new PublicSiteSettings { StreamQuality = quality };
return (new TestablePreferencePlayer(interop, media, settings), js);
}
// ── Lossless branch: must NOT probe capability or fetch the sidecar ──
// A Lossless preference short-circuits to AudioFormat.Lossless with zero JS calls.
[Test]
public async Task ResolveStreamFormat_LosslessPreference_ReturnsLosslessWithoutProbe()
{
var (player, js) = Build(StreamQuality.Lossless, canDecode: true);
var format = await player.ResolveFormatForTest("track-1");
Assert.Multiple(() =>
{
Assert.That(format, Is.EqualTo(AudioFormat.Lossless), "Lossless preference must yield lossless");
Assert.That(js.CanDecodeCallCount, Is.Zero, "Lossless branch must not probe Opus capability");
Assert.That(js.SetSidecarCallCount, Is.Zero, "Lossless branch must not inject a sidecar");
});
}
// The Lossless branch is independent of browser Opus capability — even an incapable browser
// must get lossless without a probe (the probe is unnecessary and should not fire).
[Test]
public async Task ResolveStreamFormat_LosslessPreference_IncapableBrowser_ReturnsLosslessWithoutProbe()
{
var (player, js) = Build(StreamQuality.Lossless, canDecode: false);
var format = await player.ResolveFormatForTest("track-2");
Assert.Multiple(() =>
{
Assert.That(format, Is.EqualTo(AudioFormat.Lossless));
Assert.That(js.CanDecodeCallCount, Is.Zero, "no probe on the Lossless path, even for incapable browsers");
});
}
// ── LowData branch: must delegate to base (capability gate + C2 fallback are inherited) ──
// Happy path: capable browser + present sidecar → Opus (base logic reached and succeeded).
[Test]
public async Task ResolveStreamFormat_LowDataPreference_CapableBrowser_SidecarPresent_ChoosesOpus()
{
var (player, js) = Build(StreamQuality.LowData, canDecode: true, sidecarParseSucceeds: true,
HttpStatusCode.OK, SidecarBytes);
var format = await player.ResolveFormatForTest("track-3");
Assert.Multiple(() =>
{
Assert.That(format, Is.EqualTo(AudioFormat.Opus), "capable browser + present sidecar → Opus");
Assert.That(js.CanDecodeCallCount, Is.EqualTo(1), "LowData path must invoke the capability probe");
});
}
// AC7 inherited: LowData + incapable browser → lossless (base capability gate fires).
[Test]
public async Task ResolveStreamFormat_LowDataPreference_IncapableBrowser_FallsBackToLossless()
{
var (player, js) = Build(StreamQuality.LowData, canDecode: false);
var format = await player.ResolveFormatForTest("track-4");
Assert.Multiple(() =>
{
Assert.That(format, Is.EqualTo(AudioFormat.Lossless), "AC7: incapable browser must get lossless");
Assert.That(js.CanDecodeCallCount, Is.EqualTo(1), "the capability probe was called (and returned false)");
Assert.That(js.SetSidecarCallCount, Is.Zero, "no sidecar injected when Opus is gated out");
});
}
// C2 inherited: LowData + capable browser + no sidecar → lossless (base C2 fallback fires).
[Test]
public async Task ResolveStreamFormat_LowDataPreference_CapableBrowser_NoSidecar_FallsBackToLossless()
{
var (player, _) = Build(StreamQuality.LowData, canDecode: true,
sidecarStatus: HttpStatusCode.NotFound, sidecarBody: null);
var format = await player.ResolveFormatForTest("track-5");
Assert.That(format, Is.EqualTo(AudioFormat.Lossless),
"C2: no sidecar → lossless even with a capable browser");
}
}
+1
View File
@@ -1252,5 +1252,6 @@ public class QueueServiceTests
public Task ClearError() => Task.CompletedTask;
public Task WarmAudioContext() => Task.CompletedTask;
public Task StageTrack(TrackDto track) => Task.CompletedTask;
public Task ReloadPreservingPositionAsync() => Task.CompletedTask;
}
}
+171
View File
@@ -0,0 +1,171 @@
using System.Text;
using DeepDrftAPI.Services;
using DeepDrftContent;
using DeepDrftContent.Processors;
using DeepDrftContent.Processors.Opus;
using DeepDrftData;
using DeepDrftModels.DTOs;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using Models.Common;
using NetBlocks.Models;
using FileDb = DeepDrftContent.FileDatabase.Services.FileDatabase;
namespace DeepDrftTests;
/// <summary>
/// Confirms the Phase 18.5 acceptance point that <see cref="UnifiedTrackService.ReplaceAudioAsync"/>
/// regenerates the Opus artifact: a replace must schedule a background Opus re-derive for the track, because
/// the stale Opus no longer matches the new source bytes (the same reason it regenerates the waveform datums
/// and re-derives duration). The enqueue was wired in 18.1; this test pins it so a future refactor cannot
/// silently drop it, leaving a track serving Opus that does not match its lossless source.
///
/// The vault + content + waveform collaborators are real (over a temp-dir <see cref="FileDb"/>); the SQL
/// service is a focused fake. The fake is used here rather than the in-memory EF store because the replace
/// path's duration write goes through <c>SetDuration</c> → <c>ExecuteUpdateAsync</c>, which the EF in-memory
/// provider does not support — the fake lets the orchestration run to the post-write enqueue under test.
/// </summary>
[TestFixture]
public class ReplaceAudioOpusRegenTests
{
private string _testDir = string.Empty;
[SetUp]
public void SetUp()
{
_testDir = Path.Combine(Path.GetTempPath(), "ReplaceAudioOpusRegenTests", Guid.NewGuid().ToString());
Directory.CreateDirectory(_testDir);
}
[TearDown]
public void TearDown()
{
try { Directory.Delete(_testDir, recursive: true); }
catch { /* Best-effort cleanup — ignore failures */ }
}
// A focused ITrackService fake: only GetById (returns the seeded track) and SetDuration (records the
// write and reports success) are meaningful — the two members the replace path calls. Everything else
// throws, documenting that the replace orchestration touches nothing else on the SQL boundary.
private sealed class FakeTrackService : ITrackService
{
private readonly TrackDto _track;
public double? LastDurationWritten { get; private set; }
public FakeTrackService(TrackDto track) => _track = track;
public Task<ResultContainer<TrackDto?>> GetById(long id) =>
Task.FromResult(ResultContainer<TrackDto?>.CreatePassResult(id == _track.Id ? _track : null));
public Task<ResultContainer<int>> SetDuration(long id, double durationSeconds, CancellationToken ct = default)
{
LastDurationWritten = durationSeconds;
return Task.FromResult(ResultContainer<int>.CreatePassResult(1));
}
// Unused by the replace path — fail loudly if the orchestration ever reaches them.
public Task<ResultContainer<TrackDto?>> GetByEntryKey(string entryKey) => throw new NotSupportedException();
public Task<ResultContainer<TrackDto?>> GetRandom(CancellationToken ct = default) => throw new NotSupportedException();
public Task<ResultContainer<List<TrackDto>>> GetAll() => throw new NotSupportedException();
public Task<ResultContainer<PagedResult<TrackDto>>> GetPaged(int p, int s, string? c, bool d, TrackFilter? f = null, CancellationToken ct = default) => throw new NotSupportedException();
public Task<ResultContainer<List<ReleaseDto>>> GetReleases(CancellationToken ct = default) => throw new NotSupportedException();
public Task<ResultContainer<List<GenreSummaryDto>>> GetDistinctGenres(CancellationToken ct = default) => throw new NotSupportedException();
public Task<ResultContainer<HomeStatsDto>> GetHomeStats(CancellationToken ct = default) => throw new NotSupportedException();
public Task<ResultContainer<List<TrackDto>>> GetTracksMissingDuration(CancellationToken ct = default) => throw new NotSupportedException();
public Task<ResultContainer<int>> UpdateDuration(long id, double d, CancellationToken ct = default) => throw new NotSupportedException();
public Task<ResultContainer<(ReleaseDto Release, bool WasCreated)>> FindOrCreateRelease(string t, string a, ReleaseDto r, CancellationToken ct = default) => throw new NotSupportedException();
public Task<ResultContainer<ReleaseDto?>> GetReleaseByTitleAndArtist(string t, string a, CancellationToken ct = default) => throw new NotSupportedException();
public Task<ResultContainer<TrackDto>> Create(TrackDto t) => throw new NotSupportedException();
public Task<ResultContainer<TrackDto>> Update(TrackDto t) => throw new NotSupportedException();
public Task<Result> Delete(long id) => throw new NotSupportedException();
public Task<Result> DeleteRelease(long id, CancellationToken ct = default) => throw new NotSupportedException();
public Task<ResultContainer<int>> CountLiveTracksByRelease(long releaseId, CancellationToken ct = default) => throw new NotSupportedException();
}
[Test]
public async Task ReplaceAudio_EnqueuesOpusRegen_ForTheReplacedTrack()
{
var fileDatabase = await FileDb.FromAsync(_testDir);
Assert.That(fileDatabase, Is.Not.Null);
var content = new TrackContentService(
fileDatabase!, new AudioProcessorRouter(
new AudioProcessor(), new Mp3AudioProcessor(), new FlacAudioProcessor()));
var waveforms = new WaveformProfileService(
fileDatabase!, new AudioProcessor(), new RmsLoudnessAlgorithm(),
Options.Create(new WaveformProfileOptions()), NullLogger<WaveformProfileService>.Instance);
var resolver = new TrackFormatResolver(
fileDatabase!, content, NullLogger<TrackFormatResolver>.Instance);
var queue = new NoOpOpusTranscodeQueue();
// Seed the source WAV in the vault and point a fake SQL row at the same EntryKey.
var originalPath = Path.Combine(_testDir, Guid.NewGuid().ToString("N") + ".wav");
await File.WriteAllBytesAsync(originalPath, BuildMinimalPcmWav(2.0));
var unpersisted = await content.AddTrackAsync(originalPath, "Original", "Artist");
Assert.That(unpersisted, Is.Not.Null);
const long trackId = 42;
var sql = new FakeTrackService(new TrackDto { Id = trackId, EntryKey = unpersisted!.EntryKey, TrackName = "Original" });
var service = new UnifiedTrackService(
content, sql, fileDatabase!, waveforms, queue, resolver,
NullLogger<UnifiedTrackService>.Instance);
// Replace the audio with a longer take.
var replacementPath = Path.Combine(_testDir, Guid.NewGuid().ToString("N") + ".wav");
await File.WriteAllBytesAsync(replacementPath, BuildMinimalPcmWav(6.0));
var result = await service.ReplaceAudioAsync(trackId, replacementPath, CancellationToken.None);
Assert.That(result.Success, Is.True, result.Messages.FirstOrDefault()?.Message);
Assert.Multiple(() =>
{
Assert.That(queue.Enqueued, Does.Contain(unpersisted.EntryKey),
"a replace must schedule an Opus re-derive so the artifact tracks the new source");
Assert.That(sql.LastDurationWritten, Is.GreaterThan(0),
"the replace must also write the new duration (the enqueue follows a successful duration write)");
});
}
// Standard-PCM mono 16-bit 44.1 kHz WAV, full-scale square wave. Same layout as the other suites.
private static byte[] BuildMinimalPcmWav(double durationSeconds)
{
const int sampleRate = 44100;
const ushort channels = 1;
const ushort bitsPerSample = 16;
const ushort blockAlign = channels * (bitsPerSample / 8);
const uint byteRate = sampleRate * blockAlign;
var frames = (int)(sampleRate * durationSeconds);
var data = new byte[frames * blockAlign];
for (var i = 0; i < frames; i++)
{
var sample = (i % 2 == 0) ? short.MaxValue : short.MinValue;
data[i * 2] = (byte)(sample & 0xFF);
data[i * 2 + 1] = (byte)((sample >> 8) & 0xFF);
}
using var ms = new MemoryStream();
using var w = new BinaryWriter(ms, Encoding.ASCII, leaveOpen: true);
w.Write(Encoding.ASCII.GetBytes("RIFF"));
w.Write((uint)(36 + data.Length));
w.Write(Encoding.ASCII.GetBytes("WAVE"));
w.Write(Encoding.ASCII.GetBytes("fmt "));
w.Write(16u);
w.Write((ushort)1); // PCM
w.Write(channels);
w.Write((uint)sampleRate);
w.Write(byteRate);
w.Write(blockAlign);
w.Write(bitsPerSample);
w.Write(Encoding.ASCII.GetBytes("data"));
w.Write((uint)data.Length);
w.Write(data);
w.Flush();
return ms.ToArray();
}
}
+427
View File
@@ -0,0 +1,427 @@
using System.Collections.Concurrent;
using System.Net;
using System.Net.Http.Headers;
using DeepDrftModels.DTOs;
using DeepDrftPublic.Client.Clients;
using DeepDrftPublic.Client.Services;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.JSInterop;
namespace DeepDrftTests;
/// <summary>
/// Phase 21 Direction B — the segmented forward read loop in <see cref="StreamingAudioPlayerService"/>.
/// Drives a real <c>SelectTrackStreaming</c> against a fake JS runtime and a scripted HTTP handler that
/// serves bounded 206 segments, then asserts the loop's contract:
/// <list type="bullet">
/// <item>forward playback fetches in bounded <c>bytes=start-end</c> segments (the network-memory bound);</item>
/// <item>the cursor advances contiguously across segment boundaries until the file total is reached (EOF);</item>
/// <item>the next segment is NOT fetched while the scheduler reports production paused (the fill gate);</item>
/// <item>a seek converges onto the SAME segment loop — reinit then continued segmentation, no forked path.</item>
/// </list>
/// True network/browser memory behaviour is Daniel's manual re-run; this pins the request sequencing and
/// gating the harness can observe.
/// </summary>
[TestFixture]
public class SegmentedStreamLoopTests
{
private const long SegmentSize = 4 * 1024 * 1024;
// Records every audio Range request and serves a bounded 206 slice. Audio bodies are zero-filled —
// the fake JS decoder does not inspect bytes; it scripts canStart/productionPaused directly.
private sealed class SegmentServer : HttpMessageHandler
{
private readonly long _total;
public List<(long From, long? To)> AudioRanges { get; } = new();
public SegmentServer(long total) => _total = total;
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var path = request.RequestUri!.AbsolutePath;
// Waveform profile + sidecar fetches are best-effort side calls — 404 them so the load
// path falls back cleanly and the test stays focused on the audio segment loop.
if (path.EndsWith("/waveform") || path.Contains("/opus/"))
{
return Task.FromResult(new HttpResponseMessage(HttpStatusCode.NotFound));
}
var rangeItem = request.Headers.Range!.Ranges.First();
var from = rangeItem.From ?? 0;
var to = rangeItem.To ?? (_total - 1);
if (to > _total - 1) to = _total - 1;
lock (AudioRanges) AudioRanges.Add((rangeItem.From ?? 0, rangeItem.To));
var body = new byte[Math.Max(0, to - from + 1)];
var response = new HttpResponseMessage(HttpStatusCode.PartialContent)
{
Content = new ByteArrayContent(body),
};
response.Content.Headers.ContentRange = new ContentRangeHeaderValue(from, to, _total);
response.Content.Headers.ContentType = new MediaTypeHeaderValue("audio/wav");
return Task.FromResult(response);
}
}
// Serves the first segment normally, then truncates subsequent segment bodies to a short slice
// (Content-Range reports the correct total, but the HTTP body ends early — simulating a
// connection drop mid-segment while cursor < totalLength).
private sealed class TruncatingAfterFirstSegmentServer : HttpMessageHandler
{
private readonly long _total;
private readonly long _truncatedBodyBytes; // bytes to actually deliver for non-first segments
private int _audioRequestCount;
public TruncatingAfterFirstSegmentServer(long total, long truncatedBodyBytes)
{
_total = total;
_truncatedBodyBytes = truncatedBodyBytes;
}
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var path = request.RequestUri!.AbsolutePath;
if (path.EndsWith("/waveform") || path.Contains("/opus/"))
return Task.FromResult(new HttpResponseMessage(HttpStatusCode.NotFound));
var rangeItem = request.Headers.Range!.Ranges.First();
var from = rangeItem.From ?? 0;
var to = rangeItem.To ?? (_total - 1);
if (to > _total - 1) to = _total - 1;
var requestIndex = Interlocked.Increment(ref _audioRequestCount);
// First segment (requestIndex == 1): serve fully. Subsequent segments: truncate.
var fullSliceLength = to - from + 1;
var bodyLength = requestIndex == 1 ? fullSliceLength : Math.Min(_truncatedBodyBytes, fullSliceLength);
var body = new byte[bodyLength];
var response = new HttpResponseMessage(HttpStatusCode.PartialContent)
{
Content = new ByteArrayContent(body),
};
// Content-Range always reports the true full total — the truncation is in the body, not the header.
response.Content.Headers.ContentRange = new ContentRangeHeaderValue(from, to, _total);
response.Content.Headers.ContentType = new MediaTypeHeaderValue("audio/wav");
return Task.FromResult(response);
}
}
// Serves the first segment normally, then returns HTTP 500 for all subsequent requests —
// simulating a mid-stream fetch failure after playback is underway.
private sealed class FailingAfterFirstSegmentServer : HttpMessageHandler
{
private readonly long _total;
private int _audioRequestCount;
public FailingAfterFirstSegmentServer(long total) => _total = total;
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var path = request.RequestUri!.AbsolutePath;
if (path.EndsWith("/waveform") || path.Contains("/opus/"))
return Task.FromResult(new HttpResponseMessage(HttpStatusCode.NotFound));
var requestIndex = Interlocked.Increment(ref _audioRequestCount);
if (requestIndex > 1)
return Task.FromResult(new HttpResponseMessage(HttpStatusCode.InternalServerError));
var rangeItem = request.Headers.Range!.Ranges.First();
var from = rangeItem.From ?? 0;
var to = rangeItem.To ?? (_total - 1);
if (to > _total - 1) to = _total - 1;
var body = new byte[to - from + 1];
var response = new HttpResponseMessage(HttpStatusCode.PartialContent)
{
Content = new ByteArrayContent(body),
};
response.Content.Headers.ContentRange = new ContentRangeHeaderValue(from, to, _total);
response.Content.Headers.ContentType = new MediaTypeHeaderValue("audio/wav");
return Task.FromResult(response);
}
}
private sealed class SingleClientFactory : IHttpClientFactory
{
private readonly HttpMessageHandler _handler;
public SingleClientFactory(HttpMessageHandler handler) => _handler = handler;
public HttpClient CreateClient(string name) =>
new(_handler, disposeHandler: false) { BaseAddress = new Uri("https://content.test/") };
}
/// <summary>
/// Scriptable JS runtime. processStreamingChunk reports canStart=true immediately (so playback
/// starts on the first chunk) and a productionPaused value pulled from a queue the test controls;
/// isProductionPaused (the inter-segment poll) reads a separate queue so a test can hold the gate
/// closed for N polls then release it. Records reinitializeFromOffset / markStreamComplete calls.
/// </summary>
private sealed class FakeJsRuntime : IJSRuntime
{
private readonly Func<bool> _chunkProductionPaused;
private readonly Func<bool> _isProductionPaused;
private readonly long? _seekByteOffset;
public FakeJsRuntime(
Func<bool>? chunkProductionPaused = null,
Func<bool>? isProductionPaused = null,
long? seekByteOffset = null)
{
_chunkProductionPaused = chunkProductionPaused ?? (() => false);
_isProductionPaused = isProductionPaused ?? (() => false);
_seekByteOffset = seekByteOffset;
}
public int ReinitCallCount { get; private set; }
public int MarkCompleteCallCount { get; private set; }
public int IsProductionPausedCallCount { get; private set; }
public int RecoverCallCount { get; private set; }
public ValueTask<TValue> InvokeAsync<TValue>(string identifier, object?[]? args)
{
switch (identifier)
{
case "DeepDrftAudio.isReady":
return Ok<TValue>(true);
case "DeepDrftAudio.canDecodeOggOpus":
return Ok<TValue>(false); // force the lossless path — no sidecar dance
case "DeepDrftAudio.isProductionPaused":
IsProductionPausedCallCount++;
return Ok<TValue>(_isProductionPaused());
case "DeepDrftAudio.processStreamingChunk":
return (ValueTask<TValue>)(object)ValueTask.FromResult(new StreamingResult
{
Success = true,
CanStartStreaming = true,
HeaderParsed = true,
BufferCount = 8,
Duration = 600,
ProductionPaused = _chunkProductionPaused(),
});
case "DeepDrftAudio.seek":
// When a seek offset is scripted, report seek-beyond-buffer so Seek() routes into
// SeekBeyondBuffer → the shared segment loop with a continuation reinit.
return (ValueTask<TValue>)(object)ValueTask.FromResult(_seekByteOffset is { } off
? new SeekResult { Success = true, SeekBeyondBuffer = true, ByteOffset = off }
: new SeekResult { Success = true });
case "DeepDrftAudio.reinitializeFromOffset":
ReinitCallCount++;
return Result<TValue>(true);
case "DeepDrftAudio.markStreamComplete":
MarkCompleteCallCount++;
return (ValueTask<TValue>)(object)ValueTask.FromResult(new StreamingResult { Success = true });
case "DeepDrftAudio.recoverFromFailedRefill":
RecoverCallCount++;
return Result<TValue>(true);
default:
// createPlayer / setOnProgressCallback / setOnEndCallback / setVolume /
// ensureAudioContextReady / initializeStreaming / startStreamingPlayback /
// stop / unload / disposePlayer → generic success.
if (typeof(TValue) == typeof(AudioOperationResult))
return Result<TValue>(true);
return Ok<TValue>(default!);
}
}
public ValueTask<TValue> InvokeAsync<TValue>(string identifier, CancellationToken cancellationToken, object?[]? args)
=> InvokeAsync<TValue>(identifier, args);
private static ValueTask<TValue> Ok<TValue>(object? value) => ValueTask.FromResult((TValue)value!);
private static ValueTask<TValue> Result<TValue>(bool success) =>
ValueTask.FromResult((TValue)(object)new AudioOperationResult { Success = success });
}
private static TrackDto Track() => new() { EntryKey = "mix-1", TrackName = "Long Mix" };
private static StreamingAudioPlayerService BuildPlayer(HttpMessageHandler server, FakeJsRuntime js)
{
var interop = new AudioInteropService(js);
var media = new TrackMediaClient(new SingleClientFactory(server));
return new StreamingAudioPlayerService(interop, media, NullLogger<StreamingAudioPlayerService>.Instance);
}
[Test]
public async Task ForwardPlayback_FetchesBoundedSegments_AdvancingCursorToEof()
{
// 10 MB file → 3 segments (4 MB, 4 MB, 2 MB tail). No back-pressure: drains straight through.
var total = 10L * 1024 * 1024;
var server = new SegmentServer(total);
var js = new FakeJsRuntime();
var player = BuildPlayer(server, js);
await player.SelectTrackStreaming(Track());
Assert.Multiple(() =>
{
Assert.That(server.AudioRanges, Has.Count.EqualTo(3),
"a 10 MB file at a 4 MB segment size is fetched as three bounded segments");
// Contiguous, bounded segments advancing the cursor to EOF.
Assert.That(server.AudioRanges[0], Is.EqualTo((0L, (long?)(SegmentSize - 1))));
Assert.That(server.AudioRanges[1], Is.EqualTo((SegmentSize, (long?)(2 * SegmentSize - 1))));
Assert.That(server.AudioRanges[2], Is.EqualTo((2 * SegmentSize, (long?)(3 * SegmentSize - 1))));
Assert.That(js.MarkCompleteCallCount, Is.EqualTo(1), "stream-complete fires once at true EOF, not per segment");
});
}
[Test]
public async Task ForwardPlayback_DoesNotFetchNextSegment_WhileProductionPaused()
{
var total = 10L * 1024 * 1024;
var server = new SegmentServer(total);
// Chunk results report paused=true (so the loop enters the inter-segment gate), and the poll
// stays paused for the first two checks, then releases — so the next segment is delayed, not
// skipped. The gate must hold the SECOND fetch until the poll clears.
var pollChecks = 0;
var js = new FakeJsRuntime(
chunkProductionPaused: () => true,
isProductionPaused: () => ++pollChecks < 3); // paused for polls 1,2; clear on poll 3
var player = BuildPlayer(server, js);
await player.SelectTrackStreaming(Track());
Assert.Multiple(() =>
{
Assert.That(js.IsProductionPausedCallCount, Is.GreaterThanOrEqualTo(3),
"the inter-segment gate must poll the fill signal while paused, not fetch immediately");
Assert.That(server.AudioRanges, Has.Count.EqualTo(3),
"once the gate clears, segmentation resumes and still reaches EOF — paused delays, never skips");
});
}
[Test]
public async Task SmallFile_FetchedInOneShortSegment_NoSecondFetch()
{
// 2 MB file < one segment: the first bounded request returns a short slice → EOF, no second fetch.
var total = 2L * 1024 * 1024;
var server = new SegmentServer(total);
var js = new FakeJsRuntime();
var player = BuildPlayer(server, js);
await player.SelectTrackStreaming(Track());
Assert.Multiple(() =>
{
Assert.That(server.AudioRanges, Has.Count.EqualTo(1), "a sub-segment file needs exactly one fetch");
Assert.That(server.AudioRanges[0], Is.EqualTo((0L, (long?)(SegmentSize - 1))),
"the request is still the bounded segment shape; the server returns the short available tail");
Assert.That(js.MarkCompleteCallCount, Is.EqualTo(1));
});
}
[Test]
public async Task ForwardLoad_NeverReinitializesDecoder()
{
var total = 20L * 1024 * 1024;
var server = new SegmentServer(total);
var js = new FakeJsRuntime();
var player = BuildPlayer(server, js);
await player.SelectTrackStreaming(Track());
Assert.Multiple(() =>
{
Assert.That(server.AudioRanges, Has.Count.EqualTo(5), "20 MB / 4 MB = five contiguous forward segments");
Assert.That(js.ReinitCallCount, Is.Zero,
"a forward load from byte 0 never reinitializes the decoder — reinit is the seek-only continuation step");
});
}
[Test]
public async Task MidStream_TruncatedSegment_RoutesToRecovery_NotSilentEof()
{
// 10 MB file → 3 segments. The second segment's body is short (1 MB instead of 4 MB) while
// cursor < totalLength — simulates a connection drop mid-segment. The loop must NOT treat this
// as EOF (must not call MarkStreamComplete) and must route to recovery (scheduler halted) so
// the buffered tail cannot drain into a silent false end.
var total = 10L * 1024 * 1024;
var truncatedBodyBytes = 1L * 1024 * 1024; // 1 MB short body for segment 2
var server = new TruncatingAfterFirstSegmentServer(total, truncatedBodyBytes);
var js = new FakeJsRuntime();
var player = BuildPlayer(server, js);
await player.SelectTrackStreaming(Track());
Assert.Multiple(() =>
{
Assert.That(js.MarkCompleteCallCount, Is.Zero,
"a truncated non-final segment must NOT be reported as a clean EOF — MarkStreamComplete must not fire");
Assert.That(js.RecoverCallCount, Is.EqualTo(1),
"a truncated segment while cursor < totalLength is a failure: scheduler must be halted via recovery");
Assert.That(player.IsLoaded, Is.True,
"recovery leaves the track loaded so the listener can retry — not torn down to unloaded");
Assert.That(player.IsStreamingMode, Is.True,
"recovery must restore IsStreamingMode=true so Seek() is not wedged (AC6 / Phase 21.3 retry contract)");
Assert.That(player.IsPaused, Is.True,
"recovery settles into a paused state, not playing");
Assert.That(player.ErrorMessage, Is.Not.Null.And.Not.Empty,
"recovery surfaces an error message to the UI");
});
}
[Test]
public async Task MidStream_SegmentFetchFailure_RoutesToRecovery_NotSilentFalseEnd()
{
// 10 MB file → 3 segments. The second segment fetch fails (HTTP 500), simulating a network
// error after playback is already underway. The loop must halt the JS scheduler via recovery
// rather than letting the buffered first-segment tail drain into a silent false end (AC6).
var total = 10L * 1024 * 1024;
var server = new FailingAfterFirstSegmentServer(total);
var js = new FakeJsRuntime();
var player = BuildPlayer(server, js);
await player.SelectTrackStreaming(Track());
Assert.Multiple(() =>
{
Assert.That(js.MarkCompleteCallCount, Is.Zero,
"a mid-stream fetch failure must not report clean EOF — MarkStreamComplete must not fire");
Assert.That(js.RecoverCallCount, Is.EqualTo(1),
"a mid-stream fetch failure must halt the scheduler via recovery, not leave it to drain");
Assert.That(player.IsLoaded, Is.True,
"recovery leaves the track loaded so the listener can retry — not torn down to unloaded");
Assert.That(player.IsStreamingMode, Is.True,
"recovery must restore IsStreamingMode=true so Seek() is not wedged (AC6 / Phase 21.3 retry contract)");
Assert.That(player.IsPaused, Is.True,
"recovery settles into a paused state, not playing");
Assert.That(player.ErrorMessage, Is.Not.Null.And.Not.Empty,
"recovery surfaces an error message to the UI");
});
}
[Test]
public async Task SeekBeyondBuffer_ReinitsOnceThenSegmentsForwardFromOffset()
{
// 20 MB file. After load, seek to a byte offset 12 MB in: the seek must reinit the decoder once,
// then continue the SAME bounded-segment loop from 12 MB to EOF (no forked fetch path — C1/C5).
var total = 20L * 1024 * 1024;
var seekOffset = 12L * 1024 * 1024;
var server = new SegmentServer(total);
var js = new FakeJsRuntime(seekByteOffset: seekOffset);
var player = BuildPlayer(server, js);
await player.SelectTrackStreaming(Track());
var afterLoad = server.AudioRanges.Count;
await player.Seek(300); // arbitrary time; the scripted seek returns the 12 MB byte offset
// Segments fetched by the seek/refill: everything after the initial-load segments.
var refillRanges = server.AudioRanges.Skip(afterLoad).ToList();
Assert.Multiple(() =>
{
Assert.That(js.ReinitCallCount, Is.EqualTo(1),
"a seek-beyond-buffer reinitializes the decoder exactly once for the header-less continuation");
Assert.That(refillRanges, Has.Count.EqualTo(2),
"from 12 MB to 20 MB at a 4 MB segment is two bounded segments (4 MB + 4 MB tail)");
Assert.That(refillRanges[0], Is.EqualTo((seekOffset, (long?)(seekOffset + SegmentSize - 1))),
"the first refill segment is bounded and starts at the resolved seek offset");
Assert.That(refillRanges[1], Is.EqualTo((seekOffset + SegmentSize, (long?)(seekOffset + 2 * SegmentSize - 1))),
"segmentation continues forward from the seek offset — the same loop, the same bounded shape");
});
}
}
+73
View File
@@ -0,0 +1,73 @@
using DeepDrftPublic.Client.Common;
using DeepDrftPublic.Client.Services;
namespace DeepDrftTests;
/// <summary>
/// Unit tests for the <see cref="SettingsServiceBase"/> parse/format contract (Phase 18 wave 18.6):
/// the <c>streamQuality</c> cookie round-trips through <c>FormatStreamQuality</c> → wire string →
/// <c>ParseStreamQuality</c>, and an absent, empty, or unrecognized cookie value defaults to
/// <see cref="StreamQuality.LowData"/> (the OQ2 default — a missing preference should never force
/// lossless unnecessarily on a low-bandwidth listener).
/// </summary>
[TestFixture]
public class SettingsServiceBaseTests
{
// Expose the protected static helpers via a concrete subclass — no other members needed.
private sealed class Exposed : SettingsServiceBase
{
public static StreamQuality Parse(string? v) => ParseStreamQuality(v);
public static string Format(StreamQuality q) => FormatStreamQuality(q);
}
// ── round-trip ──
// LowData serializes and deserializes intact.
[Test]
public void FormatThenParse_LowData_RoundTrips()
{
var wire = Exposed.Format(StreamQuality.LowData);
Assert.That(Exposed.Parse(wire), Is.EqualTo(StreamQuality.LowData));
}
// Lossless serializes and deserializes intact.
[Test]
public void FormatThenParse_Lossless_RoundTrips()
{
var wire = Exposed.Format(StreamQuality.Lossless);
Assert.That(Exposed.Parse(wire), Is.EqualTo(StreamQuality.Lossless));
}
// ── default for absent / garbled cookie ──
// A null cookie (no cookie present) defaults to LowData, not Lossless.
[Test]
public void Parse_Null_DefaultsToLowData()
=> Assert.That(Exposed.Parse(null), Is.EqualTo(StreamQuality.LowData));
// An empty string (cookie present but blank) defaults to LowData.
[Test]
public void Parse_Empty_DefaultsToLowData()
=> Assert.That(Exposed.Parse(string.Empty), Is.EqualTo(StreamQuality.LowData));
// A garbled/unknown name defaults to LowData rather than throwing.
// Note: Enum.TryParse accepts numeric strings as valid (e.g. "0" → LowData, "1" → Lossless),
// so only non-numeric unrecognized names are tested here.
[TestCase("garbage")]
[TestCase("lossless_extra")]
public void Parse_Unrecognized_DefaultsToLowData(string value)
=> Assert.That(Exposed.Parse(value), Is.EqualTo(StreamQuality.LowData));
// Case-insensitive parse: the enum parse is case-insensitive, so wire values survive case drift.
[TestCase("lossless")]
[TestCase("LOSSLESS")]
[TestCase("Lossless")]
public void Parse_LosslessCaseVariants_ParseCorrectly(string value)
=> Assert.That(Exposed.Parse(value), Is.EqualTo(StreamQuality.Lossless));
[TestCase("lowdata")]
[TestCase("LOWDATA")]
[TestCase("LowData")]
public void Parse_LowDataCaseVariants_ParseCorrectly(string value)
=> Assert.That(Exposed.Parse(value), Is.EqualTo(StreamQuality.LowData));
}
+289
View File
@@ -0,0 +1,289 @@
using System.Text;
using DeepDrftAPI.Controllers;
using DeepDrftContent.Constants;
using DeepDrftContent.FileDatabase.Models;
using DeepDrftContent.Processors;
using DeepDrftContent.Processors.Opus;
using DeepDrftModels.Enums;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using ContentTrackService = DeepDrftContent.TrackContentService;
using FileDb = DeepDrftContent.FileDatabase.Services.FileDatabase;
namespace DeepDrftTests;
/// <summary>
/// Delivery-layer tests for the Phase 18.3 <c>?format=</c> stream selector and the Opus seek/setup sidecar
/// endpoint on <see cref="TrackController"/>. These exercise the real <see cref="FileDb"/>, the real
/// <see cref="TrackContentService"/>, and the real <see cref="TrackFormatResolver"/> over temp-directory
/// vaults — the same integration posture as <see cref="TrackReplaceAudioTests"/>.
///
/// The SQL-only collaborators (<c>UnifiedTrackService</c>, <c>ITrackService</c>) are passed as null: the
/// actions under test (<see cref="TrackController.GetTrack"/>, <see cref="TrackController.GetOpusSeekData"/>)
/// only touch the FileDatabase + resolver path, never the SQL services, so standing up a database is not
/// required to assert the delivery contract.
///
/// The Range→206 contract is asserted at the load-bearing seam: ASP.NET performs the actual byte-slicing
/// for any <see cref="FileResult"/> whose <see cref="FileResult"/>.EnableRangeProcessing is true. The lossless
/// path proves this via the disk-stream <see cref="FileStreamResult"/>; the resolved Opus path via the
/// in-memory <see cref="FileContentResult"/> — both must report range processing enabled, the explicit fix
/// the 18.2 reviewer flagged for the byte[] path.
/// </summary>
[TestFixture]
public class TrackFormatDeliveryTests
{
private string _testDir = string.Empty;
[SetUp]
public void SetUp()
{
_testDir = Path.Combine(Path.GetTempPath(), "TrackFormatDeliveryTests", Guid.NewGuid().ToString());
Directory.CreateDirectory(_testDir);
}
[TearDown]
public void TearDown()
{
try { Directory.Delete(_testDir, recursive: true); }
catch { /* Best-effort cleanup — ignore failures */ }
}
// --- Format resolution at the endpoint ---
[Test]
public async Task GetTrack_FormatOpus_WhenOpusArtifactPresent_ServesOpusBytesAndOggContentType()
{
var (controller, entryKey) = await SeedAsync(withOpus: true, withSidecar: false);
var result = await controller.GetTrack(entryKey, format: "opus");
var file = result as FileContentResult;
Assert.That(file, Is.Not.Null, "Opus delivery serves an in-memory byte[] (FileContentResult)");
Assert.Multiple(() =>
{
Assert.That(file!.ContentType, Is.EqualTo("audio/ogg"), "Opus bytes must carry the audio/ogg content-type");
Assert.That(file.FileContents, Is.EqualTo(OpusBytes), "The served bytes must be the Opus artifact, not the source");
Assert.That(file.EnableRangeProcessing, Is.True, "Range processing must be enabled on the resolved Opus byte[] path");
});
}
// --- The C2 fallback ---
[Test]
public async Task GetTrack_FormatOpus_WhenNoOpusArtifact_FallsBackToLosslessBytesAndContentType()
{
// No Opus artifact stored — the resolver degrades to lossless (C2): the listener still gets audio,
// never a 404 or silence, and the content-type reports the lossless format actually served.
var (controller, entryKey) = await SeedAsync(withOpus: false, withSidecar: false);
var result = await controller.GetTrack(entryKey, format: "opus");
var file = result as FileContentResult;
Assert.That(file, Is.Not.Null, "The fallback still serves resolved bytes via the byte[] path");
Assert.Multiple(() =>
{
Assert.That(file!.ContentType, Is.EqualTo("audio/wav"), "Fallback content-type must be the lossless source's MIME");
Assert.That(file.FileContents, Is.EqualTo(_sourceWav), "Fallback must serve the lossless source bytes");
Assert.That(file.EnableRangeProcessing, Is.True, "Range processing stays enabled on the fallback path too");
});
}
[Test]
public async Task GetTrack_NoFormatParam_ServesLosslessDiskStream_ByteIdenticalToPrePhase18()
{
// The no-format path must be byte-identical to today: the zero-copy disk-stream FileStreamResult,
// NOT the resolver's in-memory byte[] path (which would force the whole source into memory).
var (controller, entryKey) = await SeedAsync(withOpus: true, withSidecar: false);
var result = await controller.GetTrack(entryKey, format: null);
var file = result as FileStreamResult;
Assert.That(file, Is.Not.Null, "Lossless delivery streams from disk (FileStreamResult), not a byte[]");
Assert.Multiple(() =>
{
Assert.That(file!.ContentType, Is.EqualTo("audio/wav"));
Assert.That(file.EnableRangeProcessing, Is.True, "Range→206 must work on the lossless disk-stream path");
});
}
[Test]
public async Task GetTrack_FormatLossless_TakesTheLosslessDiskStreamPath()
{
// An explicit format=lossless must behave exactly like no param — the disk-stream path, never Opus.
var (controller, entryKey) = await SeedAsync(withOpus: true, withSidecar: false);
var result = await controller.GetTrack(entryKey, format: "lossless");
Assert.That(result, Is.InstanceOf<FileStreamResult>(),
"format=lossless must take the disk-stream path even when an Opus artifact exists");
}
[Test]
public async Task GetTrack_FormatOpus_WhenTrackDoesNotExist_Returns404()
{
var controller = BuildController(await FreshDbAsync());
var result = await controller.GetTrack("no-such-track", format: "opus");
Assert.That(result, Is.InstanceOf<NotFoundResult>(),
"When even the lossless source is missing, the Opus request 404s (no audio at all)");
}
// --- Sidecar 200 / 404 ---
[Test]
public async Task GetOpusSeekData_WhenSidecarPresent_Returns200WithRawBytes()
{
var (controller, entryKey) = await SeedAsync(withOpus: true, withSidecar: true);
var result = await controller.GetOpusSeekData(entryKey);
var file = result as FileContentResult;
Assert.That(file, Is.Not.Null, "A stored sidecar is served as raw bytes");
Assert.Multiple(() =>
{
Assert.That(file!.ContentType, Is.EqualTo("application/octet-stream"));
Assert.That(file.FileContents, Is.EqualTo(SidecarBytes), "The served bytes must be the stored sidecar blob");
});
}
[Test]
public async Task GetOpusSeekData_WhenNoSidecar_Returns404()
{
var (controller, entryKey) = await SeedAsync(withOpus: true, withSidecar: false);
var result = await controller.GetOpusSeekData(entryKey);
Assert.That(result, Is.InstanceOf<NotFoundResult>(),
"No sidecar → 404, so the client degrades to lossless rather than treating it as an error");
}
// --- Fixtures + helpers ---
private static readonly byte[] OpusBytes = Encoding.ASCII.GetBytes("OggS-fake-opus-payload-for-delivery-test");
private static readonly byte[] SidecarBytes = Encoding.ASCII.GetBytes("setup-header+seek-index-sidecar-blob");
private byte[] _sourceWav = [];
private async Task<FileDb> FreshDbAsync()
{
var db = await FileDb.FromAsync(_testDir);
Assert.That(db, Is.Not.Null);
return db!;
}
// Seeds a track's lossless source in the tracks vault and, optionally, its Opus artifact and sidecar in
// the track-opus vault, then returns a controller wired over those real vaults plus the entry key.
private async Task<(TrackController Controller, string EntryKey)> SeedAsync(bool withOpus, bool withSidecar)
{
var db = await FreshDbAsync();
var content = new ContentTrackService(db, new AudioProcessorRouter(
new AudioProcessor(), new Mp3AudioProcessor(), new FlacAudioProcessor()));
var wavPath = Path.Combine(_testDir, Guid.NewGuid().ToString("N") + ".wav");
_sourceWav = BuildMinimalPcmWav(2.0);
await File.WriteAllBytesAsync(wavPath, _sourceWav);
var seeded = await content.AddTrackAsync(wavPath, "Track", "Artist");
Assert.That(seeded, Is.Not.Null);
var entryKey = seeded!.EntryKey;
// GetAudioBinaryAsync re-reads what AddTrackAsync stored, so the bytes we assert the fallback against
// are the exact stored source bytes (the processor may normalize the input WAV before storing).
var storedSource = await content.GetAudioBinaryAsync(entryKey);
Assert.That(storedSource, Is.Not.Null);
_sourceWav = storedSource!.Buffer;
await db.CreateVaultAsync(VaultConstants.TrackOpus, MediaVaultType.Audio);
if (withOpus)
{
var opus = new AudioBinary(new AudioBinaryParams(OpusBytes, OpusBytes.Length, ".opus", 2.0, 320));
Assert.That(
await db.RegisterResourceAsync(VaultConstants.TrackOpus, OpusTranscodeService.OpusAudioKey(entryKey), opus),
Is.True);
}
if (withSidecar)
{
var sidecar = new MediaBinary(new MediaBinaryParams(SidecarBytes, SidecarBytes.Length, ".opusidx"));
Assert.That(
await db.RegisterResourceAsync(VaultConstants.TrackOpus, OpusTranscodeService.OpusSidecarKey(entryKey), sidecar),
Is.True);
}
return (BuildController(db, content), entryKey);
}
private static TrackController BuildController(FileDb db, ContentTrackService? content = null)
{
content ??= new ContentTrackService(db, new AudioProcessorRouter(
new AudioProcessor(), new Mp3AudioProcessor(), new FlacAudioProcessor()));
var waveforms = new WaveformProfileService(
db, new AudioProcessor(), new RmsLoudnessAlgorithm(),
Options.Create(new WaveformProfileOptions()), NullLogger<WaveformProfileService>.Instance);
var resolver = new TrackFormatResolver(db, content, NullLogger<TrackFormatResolver>.Instance);
// SQL-only collaborators are null: the delivery actions under test never touch them.
var controller = new TrackController(
content,
db,
unifiedService: null!,
sqlTrackService: null!,
waveforms,
resolver,
stagingDirectory: null!,
NullLogger<TrackController>.Instance)
{
ControllerContext = new ControllerContext { HttpContext = new DefaultHttpContext() }
};
return controller;
}
// Standard-PCM mono 16-bit 44.1 kHz WAV, full-scale square wave. Same layout as the other suites.
private static byte[] BuildMinimalPcmWav(double durationSeconds)
{
const int sampleRate = 44100;
const ushort channels = 1;
const ushort bitsPerSample = 16;
const ushort blockAlign = channels * (bitsPerSample / 8);
const uint byteRate = sampleRate * blockAlign;
var frames = (int)(sampleRate * durationSeconds);
var data = new byte[frames * blockAlign];
for (var i = 0; i < frames; i++)
{
var sample = (i % 2 == 0) ? short.MaxValue : short.MinValue;
data[i * 2] = (byte)(sample & 0xFF);
data[i * 2 + 1] = (byte)((sample >> 8) & 0xFF);
}
using var ms = new MemoryStream();
using var w = new BinaryWriter(ms, Encoding.ASCII, leaveOpen: true);
w.Write(Encoding.ASCII.GetBytes("RIFF"));
w.Write((uint)(36 + data.Length));
w.Write(Encoding.ASCII.GetBytes("WAVE"));
w.Write(Encoding.ASCII.GetBytes("fmt "));
w.Write(16u);
w.Write((ushort)1); // PCM
w.Write(channels);
w.Write((uint)sampleRate);
w.Write(byteRate);
w.Write(blockAlign);
w.Write(bitsPerSample);
w.Write(Encoding.ASCII.GetBytes("data"));
w.Write((uint)data.Length);
w.Write(data);
w.Flush();
return ms.ToArray();
}
}
+180
View File
@@ -0,0 +1,180 @@
using DeepDrftContent;
using DeepDrftContent.Constants;
using DeepDrftContent.FileDatabase.Models;
using DeepDrftContent.Processors;
using DeepDrftContent.Processors.Opus;
using DeepDrftModels.Enums;
using Microsoft.Extensions.Logging.Abstractions;
using FileDb = DeepDrftContent.FileDatabase.Services.FileDatabase;
namespace DeepDrftTests;
/// <summary>
/// Integration tests for the Phase 18.2 format resolution + sidecar lookup seam
/// (<see cref="TrackFormatResolver"/>) over a real <see cref="FileDb"/>. They exercise the four
/// resolution branches the brief specifies — lossless, Opus hit, the C2 Opus→lossless fallback, and
/// the unknown-track miss — plus sidecar hit/miss. Artifacts are seeded into the vaults exactly as
/// 18.1's <see cref="OpusTranscodeService"/> stores them (Opus audio under the bare EntryKey, sidecar
/// under the <c>-sidecar</c>-suffixed key, the source in the <c>tracks</c> vault), so the test is faithful
/// to the real storage convention rather than a stand-in.
/// </summary>
[TestFixture]
public class TrackFormatResolverTests
{
private string _testDir = string.Empty;
[SetUp]
public void SetUp()
{
_testDir = Path.Combine(Path.GetTempPath(), "TrackFormatResolverTests", Guid.NewGuid().ToString());
Directory.CreateDirectory(_testDir);
}
[TearDown]
public void TearDown()
{
try { Directory.Delete(_testDir, recursive: true); }
catch { /* Best-effort cleanup — ignore failures */ }
}
private static TrackFormatResolver CreateResolver(FileDb fileDatabase)
{
// The resolver only calls GetAudioBinaryAsync (a vault read), which never touches the router —
// but TrackContentService requires one, so supply a real router with real processors.
var router = new AudioProcessorRouter(
new AudioProcessor(), new Mp3AudioProcessor(), new FlacAudioProcessor());
var contentService = new TrackContentService(fileDatabase, router);
return new TrackFormatResolver(
fileDatabase, contentService, NullLogger<TrackFormatResolver>.Instance);
}
// Seeds a source artifact in the tracks vault with the given extension, mirroring how the upload path
// stores the original bytes (WAV/MP3/FLAC). Returns the bytes for downstream identity assertions.
private static async Task<byte[]> SeedSourceAsync(FileDb db, string entryKey, string extension)
{
await db.CreateVaultAsync(VaultConstants.Tracks, MediaVaultType.Audio);
var bytes = new byte[] { 1, 2, 3, 4, 5 };
var audio = new AudioBinary(new AudioBinaryParams(bytes, bytes.Length, extension, 12.0, 1411));
var ok = await db.RegisterResourceAsync(VaultConstants.Tracks, entryKey, audio);
Assert.That(ok, Is.True, "source seed must succeed");
return bytes;
}
// Seeds the Opus audio + sidecar in the track-opus vault exactly as OpusTranscodeService does:
// audio under OpusAudioKey (the bare EntryKey) with the .opus extension, sidecar under OpusSidecarKey.
private static async Task<(byte[] opus, byte[] sidecar)> SeedOpusAsync(FileDb db, string entryKey)
{
await db.CreateVaultAsync(VaultConstants.TrackOpus, MediaVaultType.Audio);
var opusBytes = new byte[] { 9, 9, 9 };
var opusAudio = new AudioBinary(new AudioBinaryParams(
opusBytes, opusBytes.Length, OggOpusConstants.OpusExtension, 12.0, 320));
var audioOk = await db.RegisterResourceAsync(
VaultConstants.TrackOpus, OpusTranscodeService.OpusAudioKey(entryKey), opusAudio);
Assert.That(audioOk, Is.True, "opus audio seed must succeed");
var sidecarBytes = new byte[] { 7, 7, 7, 7 };
var sidecar = new MediaBinary(new MediaBinaryParams(
sidecarBytes, sidecarBytes.Length, OggOpusConstants.SidecarExtension));
var sidecarOk = await db.RegisterResourceAsync(
VaultConstants.TrackOpus, OpusTranscodeService.OpusSidecarKey(entryKey), sidecar);
Assert.That(sidecarOk, Is.True, "sidecar seed must succeed");
return (opusBytes, sidecarBytes);
}
[Test]
public async Task ResolveAsync_Lossless_ReturnsSourceArtifactWithItsRealMime()
{
var db = (await FileDb.FromAsync(_testDir))!;
const string entryKey = "lossless-track";
// A FLAC source — proves the lossless branch does NOT assume WAV: the content-type tracks the
// stored extension's MIME, not a hard-coded audio/wav.
var bytes = await SeedSourceAsync(db, entryKey, ".flac");
var resolver = CreateResolver(db);
var resolved = await resolver.ResolveAsync(entryKey, AudioFormat.Lossless);
Assert.That(resolved, Is.Not.Null);
Assert.That(resolved!.ResolvedFormat, Is.EqualTo(AudioFormat.Lossless));
Assert.That(resolved.ContentType, Is.EqualTo("audio/flac"));
Assert.That(resolved.Audio.Buffer, Is.EqualTo(bytes));
Assert.That(resolved.DidFallBack(AudioFormat.Lossless), Is.False);
}
[Test]
public async Task ResolveAsync_OpusWhenArtifactExists_ReturnsOpusWithOggContentType()
{
var db = (await FileDb.FromAsync(_testDir))!;
const string entryKey = "opus-track";
await SeedSourceAsync(db, entryKey, ".wav");
var (opusBytes, _) = await SeedOpusAsync(db, entryKey);
var resolver = CreateResolver(db);
var resolved = await resolver.ResolveAsync(entryKey, AudioFormat.Opus);
Assert.That(resolved, Is.Not.Null);
Assert.That(resolved!.ResolvedFormat, Is.EqualTo(AudioFormat.Opus));
Assert.That(resolved.ContentType, Is.EqualTo("audio/ogg"));
Assert.That(resolved.Audio.Buffer, Is.EqualTo(opusBytes));
Assert.That(resolved.DidFallBack(AudioFormat.Opus), Is.False);
}
[Test]
public async Task ResolveAsync_OpusWhenNoArtifact_FallsBackToLosslessNeverNull()
{
var db = (await FileDb.FromAsync(_testDir))!;
const string entryKey = "no-opus-track";
// Source exists; no Opus artifact has been derived. The C2 rule: degrade to lossless, never 404.
var bytes = await SeedSourceAsync(db, entryKey, ".wav");
var resolver = CreateResolver(db);
var resolved = await resolver.ResolveAsync(entryKey, AudioFormat.Opus);
Assert.That(resolved, Is.Not.Null, "Opus absence must degrade to lossless, not null/404");
Assert.That(resolved!.ResolvedFormat, Is.EqualTo(AudioFormat.Lossless),
"the resolved format must reflect what was actually returned");
Assert.That(resolved.ContentType, Is.EqualTo("audio/wav"),
"a fallback returns the lossless content-type so the decoder picks the right decoder");
Assert.That(resolved.Audio.Buffer, Is.EqualTo(bytes));
Assert.That(resolved.DidFallBack(AudioFormat.Opus), Is.True);
}
[Test]
public async Task ResolveAsync_UnknownTrack_ReturnsNull()
{
var db = (await FileDb.FromAsync(_testDir))!;
var resolver = CreateResolver(db);
// No source at all — the one case the caller maps to 404. Holds for both requested formats:
// Opus falls back to lossless, finds nothing, and returns null too.
Assert.That(await resolver.ResolveAsync("ghost", AudioFormat.Lossless), Is.Null);
Assert.That(await resolver.ResolveAsync("ghost", AudioFormat.Opus), Is.Null);
}
[Test]
public async Task GetOpusSidecarAsync_WhenPresent_ReturnsBytes()
{
var db = (await FileDb.FromAsync(_testDir))!;
const string entryKey = "sidecar-track";
var (_, sidecarBytes) = await SeedOpusAsync(db, entryKey);
var resolver = CreateResolver(db);
var bytes = await resolver.GetOpusSidecarAsync(entryKey);
Assert.That(bytes, Is.Not.Null);
Assert.That(bytes, Is.EqualTo(sidecarBytes));
}
[Test]
public async Task GetOpusSidecarAsync_WhenAbsent_ReturnsNull()
{
var db = (await FileDb.FromAsync(_testDir))!;
var resolver = CreateResolver(db);
// No Opus artifacts derived for this track — the sidecar lookup misses to null, not an exception.
var bytes = await resolver.GetOpusSidecarAsync("no-sidecar-track");
Assert.That(bytes, Is.Null);
}
}
@@ -0,0 +1,96 @@
using System.Net;
using System.Net.Http.Headers;
using DeepDrftModels.Enums;
using DeepDrftPublic.Client.Clients;
namespace DeepDrftTests;
/// <summary>
/// Pins the Phase 21 Direction B bounded-Range request shape on <see cref="TrackMediaClient.GetTrackMedia"/>.
/// The network-memory bound rests on each forward fetch being a finite <c>bytes=start-end</c> slice (so the
/// browser buffers only one segment), and on the caller learning the file total from the 206
/// <c>Content-Range</c> header (the EOF boundary the segment cursor advances toward). Both are request/response
/// plumbing the harness can observe directly; the actual browser memory behaviour is Daniel's manual re-run.
/// </summary>
[TestFixture]
public class TrackMediaBoundedRangeTests
{
// Captures the outgoing Range header and returns a 206 with a Content-Range so TotalLength resolves.
private sealed class RangeCapturingHandler : HttpMessageHandler
{
private readonly long _total;
public RangeHeaderValue? CapturedRange { get; private set; }
public RangeCapturingHandler(long total) => _total = total;
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
CapturedRange = request.Headers.Range;
var from = request.Headers.Range?.Ranges.First().From ?? 0;
var to = request.Headers.Range?.Ranges.First().To ?? (_total - 1);
var body = new byte[to - from + 1];
var response = new HttpResponseMessage(HttpStatusCode.PartialContent)
{
Content = new ByteArrayContent(body),
};
response.Content.Headers.ContentRange = new ContentRangeHeaderValue(from, to, _total);
return Task.FromResult(response);
}
}
private sealed class SingleClientFactory : IHttpClientFactory
{
private readonly HttpMessageHandler _handler;
public SingleClientFactory(HttpMessageHandler handler) => _handler = handler;
public HttpClient CreateClient(string name) =>
new(_handler, disposeHandler: false) { BaseAddress = new Uri("https://content.test/") };
}
[Test]
public async Task GetTrackMedia_WithByteEnd_SendsBoundedRange()
{
var handler = new RangeCapturingHandler(total: 100_000_000);
var client = new TrackMediaClient(new SingleClientFactory(handler));
await client.GetTrackMedia("track-1", byteOffset: 0, byteEnd: 4 * 1024 * 1024 - 1, format: AudioFormat.Lossless);
var range = handler.CapturedRange!.Ranges.First();
Assert.Multiple(() =>
{
Assert.That(range.From, Is.EqualTo(0), "bounded request starts at the cursor");
Assert.That(range.To, Is.EqualTo(4 * 1024 * 1024 - 1),
"bounded request must carry an inclusive end so the server returns a finite slice (one segment)");
});
}
[Test]
public async Task GetTrackMedia_WithoutByteEnd_SendsOpenEndedRange()
{
var handler = new RangeCapturingHandler(total: 100_000_000);
var client = new TrackMediaClient(new SingleClientFactory(handler));
await client.GetTrackMedia("track-1", byteOffset: 1024, byteEnd: null, format: AudioFormat.Lossless);
var range = handler.CapturedRange!.Ranges.First();
Assert.Multiple(() =>
{
Assert.That(range.From, Is.EqualTo(1024), "open-ended request starts at the cursor");
Assert.That(range.To, Is.Null, "no byteEnd → open-ended bytes=start- (pre-Direction-B shape, kept working)");
});
}
[Test]
public async Task GetTrackMedia_206Response_SurfacesTotalLengthFromContentRange()
{
var handler = new RangeCapturingHandler(total: 970_000_000);
var client = new TrackMediaClient(new SingleClientFactory(handler));
var result = await client.GetTrackMedia("track-1", byteOffset: 0, byteEnd: 4 * 1024 * 1024 - 1);
Assert.That(result.Success, Is.True);
Assert.That(result.Value!.TotalLength, Is.EqualTo(970_000_000),
"the file total (the segment cursor's EOF boundary) must come from the 206 Content-Range");
}
}
@@ -0,0 +1,80 @@
using System.Net;
using DeepDrftModels.Enums;
using DeepDrftPublic.Client.Clients;
namespace DeepDrftTests;
/// <summary>
/// Pins the Phase 21.4 transport fix: every audio media fetch must carry the browser response-streaming
/// option so the body streams incrementally in WASM. Without it the browser buffers the whole payload
/// before the response stream yields a byte, and the 21.2 read-loop pause backpressures nothing.
///
/// The flag is set by <c>SetBrowserResponseStreamingEnabled(true)</c>, which records it in
/// <see cref="HttpRequestMessage.Options"/> under the framework key <c>"WebAssemblyEnableStreamingResponse"</c>.
/// A stub handler reads that option back during <c>SendAsync</c> — the same network-boundary fake the Opus
/// format-selection tests use. True network backpressure is browser-only and cannot be unit-profiled; this
/// asserts the request is *configured* for streaming, which is the part the harness can observe. Daniel's
/// 21.4 manual re-run confirms the actual bounded-memory behaviour.
///
/// Both the initial full-stream request (byteOffset 0) and the seek/refill Range request (byteOffset &gt; 0,
/// Phase 21.3) flow through <see cref="TrackMediaClient.GetTrackMedia"/>, so both are asserted here.
/// </summary>
[TestFixture]
public class TrackMediaStreamingEnabledTests
{
// The framework key SetBrowserResponseStreamingEnabled writes into HttpRequestMessage.Options.
private static readonly HttpRequestOptionsKey<bool> StreamingOptionKey = new("WebAssemblyEnableStreamingResponse");
// Captures the streaming option off each outgoing request, then returns a minimal 200 with a body so
// GetTrackMedia reaches its pass path (ReadAsStreamAsync over ByteArrayContent).
private sealed class CapturingHandler : HttpMessageHandler
{
public bool? StreamingEnabled { get; private set; }
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
StreamingEnabled = request.Options.TryGetValue(StreamingOptionKey, out var enabled) ? enabled : null;
var response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new ByteArrayContent("audio-bytes"u8.ToArray()),
};
return Task.FromResult(response);
}
}
private sealed class SingleClientFactory : IHttpClientFactory
{
private readonly HttpMessageHandler _handler;
public SingleClientFactory(HttpMessageHandler handler) => _handler = handler;
public HttpClient CreateClient(string name) =>
new(_handler, disposeHandler: false) { BaseAddress = new Uri("https://content.test/") };
}
[Test]
public async Task GetTrackMedia_InitialStream_EnablesBrowserResponseStreaming()
{
var handler = new CapturingHandler();
var client = new TrackMediaClient(new SingleClientFactory(handler));
var result = await client.GetTrackMedia("track-1", byteOffset: 0, format: AudioFormat.Lossless);
Assert.That(result.Success, Is.True, "the fetch should succeed against the stub");
Assert.That(handler.StreamingEnabled, Is.True,
"the initial media stream must enable browser response streaming or the read-loop pause backpressures nothing");
}
[Test]
public async Task GetTrackMedia_SeekOffsetStream_EnablesBrowserResponseStreaming()
{
var handler = new CapturingHandler();
var client = new TrackMediaClient(new SingleClientFactory(handler));
var result = await client.GetTrackMedia("track-1", byteOffset: 1_048_576, format: AudioFormat.Opus);
Assert.That(result.Success, Is.True, "the offset fetch should succeed against the stub");
Assert.That(handler.StreamingEnabled, Is.True,
"the seek/refill Range request must also enable streaming — 21.3 refill depends on the same backpressure");
}
}
@@ -4,6 +4,7 @@ using Data.Managers;
using DeepDrftAPI.Services;
using DeepDrftContent;
using DeepDrftContent.Processors;
using DeepDrftContent.Processors.Opus;
using DeepDrftData;
using DeepDrftData.Data;
using DeepDrftData.Repositories;
@@ -74,9 +75,13 @@ public class UploadDuplicateDetectionTests
var waveforms = new WaveformProfileService(
fileDatabase!, new AudioProcessor(), new RmsLoudnessAlgorithm(),
Options.Create(new WaveformProfileOptions()), NullLogger<WaveformProfileService>.Instance);
var resolver = new TrackFormatResolver(
fileDatabase!, content, NullLogger<TrackFormatResolver>.Instance);
return new UnifiedTrackService(
content, sqlTrackService, fileDatabase!, waveforms,
new NoOpOpusTranscodeQueue(),
resolver,
NullLogger<UnifiedTrackService>.Instance);
}
@@ -88,4 +88,93 @@ public class WaveformVisualizerControlStateTests
Assert.That(observedTheaterMode, Is.False);
}
// ── ApplyCapabilityDefault: the one-time HW-accel default-set ──
// No HW accel → lava defaults off, waveform stays on (the whole point of the feature).
[Test]
public void ApplyCapabilityDefault_NoHardwareAccel_LavaOffWaveformOn()
{
_state.ApplyCapabilityDefault(hardwareAccelerated: false);
Assert.Multiple(() =>
{
Assert.That(_state.LavaEnabled, Is.False);
Assert.That(_state.WaveformEnabled, Is.True);
});
}
// HW accel present → no change; lava keeps its shipped on-default (the common case is untouched).
[Test]
public void ApplyCapabilityDefault_HardwareAccelerated_LavaStaysOn()
{
_state.ApplyCapabilityDefault(hardwareAccelerated: true);
Assert.Multiple(() =>
{
Assert.That(_state.LavaEnabled, Is.True);
Assert.That(_state.WaveformEnabled, Is.True);
});
}
// No HW accel raises Changed once so the controls UI / bridge / theater observers all react.
[Test]
public void ApplyCapabilityDefault_NoHardwareAccel_RaisesChanged()
{
var raised = 0;
_state.Changed += () => raised++;
_state.ApplyCapabilityDefault(hardwareAccelerated: false);
Assert.That(raised, Is.EqualTo(1));
}
// HW accel present must not churn observers — no Changed when nothing changed.
[Test]
public void ApplyCapabilityDefault_HardwareAccelerated_DoesNotRaiseChanged()
{
var raised = 0;
_state.Changed += () => raised++;
_state.ApplyCapabilityDefault(hardwareAccelerated: true);
Assert.That(raised, Is.Zero);
}
// Guarded to once per session: a second call cannot re-clobber, even with a different verdict.
[Test]
public void ApplyCapabilityDefault_SecondCall_IsNoOp()
{
_state.ApplyCapabilityDefault(hardwareAccelerated: true); // first call wins (accelerated → on)
_state.ApplyCapabilityDefault(hardwareAccelerated: false); // must be ignored
Assert.That(_state.LavaEnabled, Is.True);
}
// Must not override an explicit in-session toggle: once applied, a user re-enabling lava on a
// software renderer sticks — a later (guarded) call never reverts it.
[Test]
public void ApplyCapabilityDefault_DoesNotOverrideExplicitUserToggle()
{
_state.ApplyCapabilityDefault(hardwareAccelerated: false); // default: lava off
_state.LavaEnabled = true; // user re-enables at their own risk
_state.ApplyCapabilityDefault(hardwareAccelerated: false); // guarded → no-op
Assert.That(_state.LavaEnabled, Is.True);
}
// No HW accel while Theater Mode is on with only lava active → coercion exits theater in the same
// cycle (lava off + waveform off would strand theater; here waveform stays on, but verify the
// coercion path is exercised when it WOULD strand).
[Test]
public void ApplyCapabilityDefault_NoHardwareAccel_CoercesStrandedTheater()
{
_state.WaveformEnabled = false; // only lava active
_state.TheaterMode = true;
_state.ApplyCapabilityDefault(hardwareAccelerated: false); // lava off → both off → strand
Assert.That(_state.TheaterMode, Is.False);
}
}
+4 -202
View File
@@ -443,213 +443,15 @@ not the same work; this phase does not satisfy or depend on that one.
---
## Phase 18 — Opus Low-Data Streaming (dual-format lossless + Opus delivery)
## Phase 18 — Opus Low-Data Streaming (Completed)
The concrete realization of the long-deferred **"Non-WAV formats"** intent (`CONTEXT.md §5`). Daniel's
direction (2026-06-23): **two delivery formats per track — the existing lossless WAV path, and a new
low-data Ogg Opus (fullband, 320 kbps) path — so the listener gets a choice, with Opus the
bandwidth-friendly default-candidate.** Lossless streaming becomes *optional*, not the only path. The
bespoke Web Audio decode→schedule graph is **retained by deliberate choice** — Opus feeds the same
`IFormatDecoder` seam, not an HTML `<media>` element or MSE (the decision shared with Phase 21 OQ5).
**Sequenced BEFORE Phase 21** — windowing must work across both formats. Surfaces: ingest/preprocessing
in `DeepDrftContent` (`AudioProcessor`/router/`WaveformProfileService`) + `DeepDrftAPI`
(`UnifiedTrackService.UploadAsync`, replace-audio); delivery/decode in `DeepDrftAPI` (stream endpoint +
`Range`) + `DeepDrftPublic` proxy + `DeepDrftPublic.Client` player stack + `DeepDrftPublic/Interop/audio`
TS decoders. Full design, the three directions with SOLID/road-not-taken rationale, the storage and
delivery options, the Opus decoder + seek math, acceptance criteria, open questions, and wave
decomposition: `product-notes/phase-18-opus-low-data-streaming.md`.
**Much further along than the backlog line implies (verified 2026-06-23).** The multi-format *substrate*
already exists on both sides: the producer-side `AudioProcessorRouter` routes `.wav`/`.mp3`/`.flac` and
`TrackContentService.AddTrackAsync` is format-agnostic (it **stores originals**, no transcode); the
decoder-side `AudioPlayer.createFormatDecoder` is a **wired** strategy registry dispatching on
`Content-Type` (WAV/MP3/FLAC decoders all present — correcting the Phase 21 spec's stale
"implemented-not-wired" note). **The actual gap is Daniel's specific ask:** (1) a **transcode-at-ingest**
step that *derives* an Opus 320 artifact per track (nothing derives Opus today), and (2) a **per-format
delivery selection** so one track serves as either WAV or Opus on request.
**Open questions RESOLVED (Daniel, 2026-06-23).** OQ1 selection UX → **global, via a new public-site
Settings menu** (not a bare app-bar control); OQ2 default → **Opus by default, capability-gated** (defer
network-awareness); OQ3 remembered → **persisted via the dark-mode seam** (cookie → prerender-read →
`PersistentComponentState` → client cookie service); OQ4 → **always-on Opus + Backfill-Opus**; OQ5 →
**Ogg Opus**; OQ6 transcode model → **background job after the file is available, with a visible
Post-Processing phase on the CMS upload meter.** OQ7 (seek-index granularity) → **0.5 s (half-second)
buckets** (~115 KB index for a 1-hour mix).
**Architectural spine — a derived artifact set + a delivery param + one new decoder + a precomputed
accurate seek index; leaf implementations only, zero changes to existing format code (the strong OCP
signal).** Transcode is a new processor sibling in `DeepDrftContent`, invoked post-store alongside
`WaveformProfileService` **as a background job** (a 1 GB WAV transcode must not block the upload; the source
is stored and the track plays lossless *first*, then Opus is derived) — mirroring the landed waveform-datum
pattern (derive at ingest, regenerate via a CMS bulk action + ApiKey endpoint). The Opus bytes are a
**derived artifact** stored like the high-res waveform datum (recommend a dedicated `track-opus` vault, the
`track-waveforms` precedent; final call staff-engineer's). Delivery adds a **`?format=opus|lossless` param**
(mirroring the existing `offset` param threading through `TrackProxyController`) resolved server-side to the
right artifact + content-type, with a **lossless fallback** when no Opus artifact exists (additive, never
404/silence). The player gains one `OpusFormatDecoder` (`IFormatDecoder`): Ogg-page-aligned segmenting
(`OggS` scan — the FLAC frame-sync analogue) and `OpusHead`/`OpusTags` setup-bytes carry (the FLAC
`streamInfoBytes` analogue). **Browser constraint flagged:** Ogg-Opus `decodeAudioData` is Safari-18.4+ only
(Chrome/FF long-standing), so the Opus default is **capability-gated** — fall back to the universal lossless
path on browsers that can't decode it.
**VBR-safe ACCURATE seeking (Daniel, 2026-06-23 — supersedes the earlier "approximate" hand-wave).** Raw
byte-offset seek and rough page interpolation are inadequate for VBR Opus — there is no linear time↔byte
relationship. The fix is an **accurate transfer function built at transcode time** (the one moment the
whole encoded stream is walked): a precomputed **seek index** mapping Ogg-page `granulepos` (48 kHz sample
counts → time) → exact byte offset (**0.5 s buckets** snapped to page starts — OQ7; ~7,200 entries ×
16 bytes ≈ ~115 KB for a 1-hour mix). The decode **setup header** (`OpusHead`/`OpusTags`, needed to decode any mid-stream slice) is made
available too. Recommended concrete design: **one sidecar artifact per track = `[setup header][seek
index]`, built at transcode, stored beside the Opus bytes, fetched once on track load**, parsed into
`OpusSeekData`. Client seek flow: `calculateByteOffset(t)` binary-searches the index for the exact page
offset → `Range: bytes=X-` fetch (landed Phase 4 primitive, unchanged) → prepend the cached setup header →
decode → fine re-sync to `t` within the bucket. **The listener lands at the correct time, not
approximately** (AC9), **without** the full PCM in memory — so it composes with Phase 21 windowed refill,
which calls the **same** index resolver. The earlier "approximate page-interpolation" language is rejected.
**Constraints/invariants:** keep the bespoke graph (no MSE); preprocessing is **additive** (WAV path
untouched, byte-for-byte; a track with no Opus artifact still plays losslessly); reuse the landed
`Range`/offset seek path; no format branches leak outside the new decoder + one selection arm + the
transcode/delivery seam; transcode failure must not block ingest; format selection is a delivery-time
decision resolving one `EntryKey` to one of two artifacts (one source, two views — **not** a second
`TrackEntity` row, which would fracture share/queue/play-count/release identity).
Sequenced as six waves. `18.1 → 18.2 → {18.3, 18.4} → 18.5`, with `18.6` (Settings menu) able to run in
parallel (it needs only 18.3's format mechanism before its toggle is live). **18.1 (ingest transcode +
seek-index + setup-header derived artifacts) is the cold-start prerequisite** — nothing downstream has
bytes to serve, decode, or seek against until those artifacts exist.
- **18.1 — Ingest transcode + seek-index + setup-header (cold-start; load-bearing).** New
`OpusTranscodeService`/processor in `DeepDrftContent`, invoked post-store from
`UnifiedTrackService.UploadAsync` alongside `WaveformProfileService` **as a background job** (OQ6);
produces Ogg Opus fullband 320; **walks the encoded stream once to build the granule→byte seek index and
extract the `OpusHead`/`OpusTags` setup header**; stores the Opus bytes **and** the combined seek/setup
**sidecar** as derived artifacts (recommend a `track-opus` vault). Failure-tolerant. **Independent of the
delivery/decoder waves — can begin immediately.**
- **18.2 — Storage + lookup contract.** The derived-artifact key/vault convention (Opus bytes + sidecar) +
server-side "given `EntryKey` + format, return the right `AudioBinary` + content-type (+ the sidecar),"
including the lossless fallback. **Depends on 18.1.**
- **18.3 — Delivery: `?format=opus|lossless` param + sidecar serving + proxy threading.** On the
`DeepDrftAPI` stream endpoint (resolves via 18.2), forwarded through `TrackProxyController` (mirror
`offset`), `Range` serving the chosen artifact; **plus serving the seek/setup sidecar**; player sends the
format param via `TrackMediaClient`. **Depends on 18.2; parallel-ok with 18.4.**
- **18.4 — `OpusFormatDecoder` + index-based seek resolver in the player stack.** New `IFormatDecoder`
(Ogg-page segmenting via `OggS` scan, `OpusHead`/`OpusTags` setup carry from the cached sidecar,
**`calculateByteOffset` that binary-searches the precomputed seek index** — NOT interpolation — with an
`OpusSeekData` accelerator holding the parsed index + setup bytes, and the one-time sidecar fetch+parse on
track load) + one arm in `createFormatDecoder` on `audio/ogg`/`audio/opus`; capability detection for the
lossless fallback. **Depends on 18.2; parallel-ok with 18.3.**
- **18.5 — Backfill + replace-audio + end-to-end validation (incl. seek accuracy).** "Backfill Opus" CMS
bulk action (third sibling to Generate-Profiles / Backfill-High-res), rebuilding Opus bytes + sidecar for
existing tracks; replace-audio Opus + sidecar regeneration; the AC1AC10 acceptance pass **including AC9
(an Opus seek lands at the correct time, not approximately)** and the Phase-21 handshake (Opus windowable
via the index resolver + sidecar setup header). **Depends on 18.118.4.**
- **18.6 — Public Settings menu + quality toggle (the listener selection UX).** New public-site
Settings-menu shell (app-bar trigger + MudBlazor menu + a settings-item abstraction + a
`PublicSiteSettings`/`ListenerSettings` object + the dark-mode-pattern persistence seam: `streamQuality`
cookie, a `DeepDrftPublic` prerender-read service, `PersistentComponentState` bridge, client cookie
service); the **quality toggle is its first occupant** (Low-data/Lossless, Opus default, capability-gated)
+ the CMS upload meter's **Post-Processing phase** (OQ6). Built design-for-adaptability so dark mode can
plug in later without restructuring (not migrated now). **Depends on 18.3** for the toggle; the menu shell
can be built ahead. *Splittable* (shell, then toggle) if Daniel wants the shell proven first.
**Dependency shape:** `18.1 → 18.2 → {18.3 ∥ 18.4} → 18.5`; `18.6 ∥` (needs 18.3 for the live toggle);
18.1 is the only cold-start wave. **Phase-level: 18 precedes Phase 21** (windowed refill consumes the Phase
18 seek-index resolver). **OQ1OQ7 RESOLVED (above); OQ7 (seek-index granularity) = 0.5 s buckets.** None
block 18.1.
See `COMPLETED.md` for Phase 18 — dual-format lossless + Opus delivery, including the as-built WebCodecs decoder divergence — which landed on `streaming-overhaul` (2026-06-23).
---
## Phase 21 — Windowed Streaming Buffer (bounded client memory for long streams)
## Phase 21 — Windowed Streaming Buffer (Completed)
Bound the **client memory** a playing track consumes to a small, configurable forward window —
**independent of total stream length** — so a 1 GB+ DJ MIX (Phase 9 `Mix` medium: a single long track)
plays without the whole decoded PCM accumulating in the browser. **Public listener site only**
(`DeepDrftPublic.Client` player stack + `DeepDrftPublic` TypeScript audio interop); no CMS, no API
endpoint, no schema change.
**Sequenced AFTER Phase 18 (Opus Low-Data Streaming) — Daniel, 2026-06-23.** Format support (the
derived Ogg Opus 320 low-data path, Phase 18) is a prerequisite that comes first; windowing must work
across **both** delivery formats. Phase 21's C5 invariant already anticipated this ("must not foreclose
MP3/FLAC"); **Opus is now the concrete VBR/paged driver** — windowing an Opus stream uses the decoder's
**accurate index-based** byte↔time mapping (`OpusFormatDecoder.calculateByteOffset`, a binary search in the
Phase 18 precomputed seek index — *not* the exact CBR-WAV `byteRate` math, and *not* approximate page
interpolation: VBR-safe and exact, per the Phase 18 seek-model resolution 2026-06-23). The windowed refill
controller calls the **same** index resolver an explicit seek does, and a window opening away from byte 0
still decodes via the Phase 18 sidecar setup header. Build the window machinery format-agnostically so it
inherits Opus for free.
The network path already streams in adaptive 1664 KB chunks. The accumulation is on the **decode
side**: `PlaybackScheduler` holds an `AudioBuffer[]` it **never evicts** ("Supports pause/resume/seek by
retaining all buffers" — its own doc comment). Decoded PCM is larger than the source (Web Audio is
32-bit float per sample/channel — a 16-bit stereo WAV roughly doubles once decoded), so a 1 GB WAV
becomes ~2 GB of retained float data. That is the OOM. The fix: hold only a sliding forward window plus a
small back-retain, discard already-played buffers, and refill on demand.
**Architectural spine — a sliding window keyed on playback position, built as a generalization of the
landed seek-beyond-buffer path.** The Phase 4 HTTP `Range: bytes=X-` → 206 primitive already does every
plumbing primitive the window needs (discard-buffers-keep-offset via `clearForSeek`/`setPlaybackOffset`;
fetch-from-offset via `TrackMediaClient`; decode-header-less-body via
`StreamDecoder.reinitializeForRangeContinuation`; time→byte via `IFormatDecoder.calculateByteOffset`),
just triggered manually and one-shot. The only genuinely new mechanisms are **partial eviction** on the
scheduler and **back-pressure** on the forward read loop (stop calling `ReadAsync` above a high-water
mark, resume below low-water). Recommended **Direction A** (sliding window on the existing single forward
stream); **Direction B** (discrete Range-fetched segments — the HLS/DASH/MSE-eviction analogue) held as
the documented fallback; **Direction C** (adopt MSE and let the browser manage the buffer) **rejected
(OQ5 = NO, Daniel 2026-06-23)** — the bespoke Web Audio graph is a deliberate long-term commitment, and
the compressed-delivery move that would have justified MSE is met instead by **Phase 18 (Opus) feeding
the same bespoke graph** through the `IFormatDecoder` seam. Direction A is therefore the permanent
destination, not a stopgap MSE would retire.
**Invariants that must hold (the §3.5 seam contract).** Reuse the Range path, don't fork it; playback-
start latency at parity; the `IFormatDecoder` abstraction untouched (windowing is format-agnostic, so
wiring MP3/FLAC later inherits it free); read-only playback (no new control); the single-instance JS
decoder stays single-writer (every refill routes through the existing cancellation/drain discipline). The
**Mix visualizer is provably unaffected** — it renders from the preprocessed per-track high-res datum
(Phase 10/12), never from live decoded PCM, so evicting played buffers cannot starve it. The 1 GB mix is
both the canonical case *and* the proof the eviction is safe.
**Interaction with deferred Phase 1 features (same seam):** windowing should land **before** preload
(1.3) — it makes preload of long tracks memory-safe by construction (a staged next-track decoder inherits
the bounded scheduler); it makes crossfade (1.4) between two long mixes affordable (the overlap doubles
the *window*, not the track); it adds a minor "don't evict the final window before the gapless boundary"
care point for 1.5. It **enlarges the error surface** (1.6): windowed refill issues mid-stream fetches
the listener didn't initiate, one of which can fail deep into a 1 GB mix — so the *cheap* half of 1.6
(clean refill-failure handling, no wedged player) is folded into this phase's acceptance criteria, not
left fully to 1.6.
Full design, the three directions with SOLID/road-not-taken rationale, use cases, acceptance criteria,
the open-question set, and the wave decomposition: `product-notes/phase-21-windowed-streaming-buffer.md`.
Sequenced as four waves. `21.1 → 21.2 → 21.3`, with `21.4` validating the whole. **21.1 is the cold-start
prerequisite and the load-bearing change** — independent of the open questions (window *sizes* are
parameters fed in later).
- **21.1 — Partial eviction in `PlaybackScheduler` (cold-start; load-bearing).** Drop already-played
buffers while keeping the position/index/time-anchor bookkeeping exact against a buffer array that no
longer begins at absolute time 0 (today `getCurrentPosition`/`playFromPosition`/the schedule loop all
assume `buffers[0]` is the track start). The hardest correctness work in the phase. No refill yet.
**Independent of the open questions — can begin immediately.**
- **21.2 — Back-pressure on the forward read loop.** Stop `ReadAsync` above the high-water mark, resume
below low-water; together with 21.1 this bounds *both* the played and unplayed regions (the AC1
guarantee). Routes resume/pause through the existing single-loop cancellation discipline. **Depends on
21.1.**
- **21.3 — Seek-back-past-window refill.** When a backward seek lands earlier than the retained tail,
refetch via the existing seek-beyond-buffer Range path pointed at the earlier offset; plus the minimal
clean refill-failure handling (the 1.6 adjacency). Mostly reuse of the landed seek path. **Depends on
21.1 + 21.2.**
- **21.4 — Validation against the 1 GB target (acceptance).** Memory profiling (bounded under 1 GB is the
headline), latency parity, edge-to-edge playback, the seek matrix, induced refill failure, visualizer-
running, rapid-seek concurrency. Largely measurement; breaks are tuning fixes in 21.1's anchor math or
21.2's water-marks. **Depends on 21.121.3.**
**Dependency shape:** `21.1 → 21.2 → 21.3 → 21.4`; 21.1 is the only cold-start wave. **Phase-level
prerequisite: Phase 18 (Opus) lands first** so windowing is built against both formats. **Open questions
for Daniel (spec §6):** window-size policy axis (time-based window + memory guard — recommended); seek-
back-past-window re-buffer acceptable (recommend yes, symmetric to forward); a hard total in-flight
memory cap as a guard rail (recommend yes); window everything vs. only long tracks (recommend everything
— one path, short tracks never hit a refill). **OQ5 (adopt MSE) — RESOLVED NO (Daniel 2026-06-23): the
bespoke graph stays by deliberate choice; recorded considered-and-declined, kept visible per file
convention.** None block 21.1.
See `COMPLETED.md` for Phase 21 — bounded client memory for long streams, including the as-built Direction A→B pivot — which landed on `streaming-overhaul` (2026-06-24).
---
+2 -1
View File
@@ -41,7 +41,8 @@ apt-get install -y --no-install-recommends \
openssl \
jq \
wget \
ca-certificates
ca-certificates \
ffmpeg
echo " [ok] apt packages installed"
+8 -1
View File
@@ -41,6 +41,12 @@ if ! command -v nginx &>/dev/null; then
die "nginx not found on PATH. Install nginx before running this script."
fi
# ffmpeg (with libopus) is required at runtime by DeepDrftAPI for Opus transcode (Phase 18.1).
# On Ubuntu/Debian the stock ffmpeg apt package includes libopus; bootstrap.sh installs it.
if ! command -v ffmpeg &>/dev/null; then
die "ffmpeg not found on PATH. Install it before running this script: apt-get install -y ffmpeg"
fi
# rrsync ships in the rsync package; path varies by distro version
RRSYNC_BIN=""
if command -v rrsync &>/dev/null; then
@@ -51,7 +57,7 @@ else
die "rrsync not found. Install the rsync package: apt-get install -y rsync"
fi
ok "psql present, PostgreSQL active, nginx present"
ok "psql present, PostgreSQL active, nginx present, ffmpeg present"
ok "rrsync found at: ${RRSYNC_BIN}"
# ── Step 0b: Parameter collection ─────────────────────────────────────────────
@@ -433,6 +439,7 @@ Installed / configured:
- PostgreSQL role '${PG_ROLE}' and databases: ${DB_META}, ${DB_AUTH}
- SSH authorized_keys with forced-command + restrict
- nginx sites-available + sites-enabled, default site removed
- ffmpeg (with libopus) installed at $(command -v ffmpeg) — runtime prereq for Opus transcode
------------------------------------------------------------------------
Remaining manual steps:
@@ -1,27 +1,54 @@
# Phase 21 — Windowed Streaming Buffer (bounded client memory for long streams)
Product spec. Status: **design / framing — implementation-ready pending Daniel's open-question calls.**
Author: product-designer. Date: 2026-06-23. **No code has been written by this doc.**
Product spec. Status: **LANDED 2026-06-24 on `streaming-overhaul`.** See `COMPLETED.md` for the full
as-built record. Author: product-designer. Date: 2026-06-23 (reconciliation pass after Phase 18 landed).
> **AS-BUILT NOTE — Direction A→B pivot (2026-06-24).** This spec recommended **Direction A** (sliding
> window on one open-ended forward stream, pausing `ReadAsync`/the segment loop to backpressure the
> socket) and held **Direction B** (discrete bounded `Range: bytes=start-end` segments, §3.2) as the
> documented fallback. **21.4 browser validation proved Direction A insufficient for Blazor WASM:** the
> browser `fetch` API buffers the entire HTTP response body regardless of read pace — pausing reads
> bounded the *decode* but not the *network download*, so the whole ~970 MB body accumulated in browser
> memory even with the application decoding only a window of it. **We shipped Direction B.** The forward
> stream now issues sequential 4 MB bounded Range requests (`SegmentSizeBytes = 4 MB`), fetched via
> `RunSegmentedStreamAsync` in `StreamingAudioPlayerService`, each issued only after
> `PlaybackScheduler.evaluateProductionPause()` clears below low-water. Browser holds ~one 4 MB segment
> of raw bytes; 21.4 confirmed network-memory bounding in Daniel's browser run. The decode-side windowing
> (21.1/21.2) is unchanged and pairs with Direction B; seek/refill converge on the same segmented loop
> via `RecoverFromFailedRefill`. Direction A is recorded as tried-in-validation and found insufficient
> for the WASM `fetch` runtime. Sections §3.2–§3.3 below retain the original A vs. B vs. C analysis as
> the decision record; **Direction B is what shipped.**
Surface: **public listener site only** (`DeepDrftPublic.Client` player stack + `DeepDrftPublic`
TypeScript audio interop). No CMS (`DeepDrftManager`) change. No data-model or schema change. The one
server touch is **reuse, not new surface**: the existing `DeepDrftAPI` HTTP `Range: bytes=X-`
partial-content primitive (Phase 4, landed) is the load-bearing dependency; this phase adds no new API
endpoint.
> **Sequencing dependency (Daniel, 2026-06-23): Phase 18 (Opus Low-Data Streaming) comes BEFORE this
> phase.** Format support — specifically the derived **Ogg Opus fullband 320** low-data delivery path
> (`product-notes/phase-18-opus-low-data-streaming.md`) — is a prerequisite that sequences ahead of
> windowing. Phase 21's windowing must work across **both** delivery formats (lossless WAV and Opus).
> Its C5 invariant below already anticipated this ("must not foreclose MP3/FLAC"); **Opus is now the
> concrete VBR/containerized driver of C5.** Windowing an Opus stream uses the decoder's **accurate
> index-based** byte↔time mapping (`OpusFormatDecoder.calculateByteOffset` — a binary search in the Phase 18
> precomputed seek index), exactly the C5 case — *not* the exact CBR-WAV `byteRate` math, and *not*
> approximate Ogg-page interpolation. **Correction (Daniel, 2026-06-23):** an earlier draft described the
> Opus mapping as "approximate page interpolation"; the Phase 18 seek-model resolution rejected that — Opus
> seeking is **accurate**, backed by a precomputed seek index built at transcode time, so refill resolves to
> the *exact* page offset. The windowed refill controller calls the **same** index resolver an explicit seek
> does (Phase 18 §3.4a D); a window opening away from byte 0 still decodes via the Phase 18 sidecar setup
> header. Build the window machinery format-agnostically (§2 C3/C5) so it inherits Opus for free.
> **Phase 18 (Opus Low-Data Streaming) has LANDED (2026-06-23, `COMPLETED.md`). This spec is reconciled
> to the as-built reality.** Phase 18 changed the landscape in two ways that reshape this phase:
>
> 1. **There are now TWO decode paths feeding the one `PlaybackScheduler`, not one.** (a) The original
> **WAV/MP3/FLAC** path — `StreamDecoder``IFormatDecoder` (wrap-each-segment + `decodeAudioData`).
> (b) A new **Opus** path — `OggDemuxer``OpusStreamDecoder` (the `IStreamingDecoder` seam, a stateful
> **WebCodecs `AudioDecoder`** pipeline). The §3.1 unbounded-memory root cause (the scheduler's
> push-only `AudioBuffer[]`) applies to **both** — but the Opus path adds a *second* accumulation locus
> upstream of the scheduler (the WebCodecs decode queue + `decodedQueue: AudioData[]`), so windowing it
> is not the same mechanism as windowing WAV. See §3.1.
> 2. **The accurate index-driven Opus seek the original spec assumed Phase 21 would build is ALREADY
> LIVE.** Phase 18 ships `resolveOpusByteOffset` (binary-search the precomputed seek index in
> `OpusSeekData`) → Range fetch → `OpusStreamDecoder.reinitializeForRangeContinuation(landingTime,
> target)` with frame-accurate lead-trim. Opus seek is **accurate, not approximate** — and **already
> shipping**. Phase 21 does **not** build Opus seek; it **reuses** that live seek for window-miss
> refills.
>
> **Correction of stale spec language.** The original draft described Opus as a future-wired
> `OpusFormatDecoder.calculateByteOffset` joining the `IFormatDecoder` registry, with seek as "approximate
> vs accurate." All of that is now wrong against the landed code: Opus does **not** use `IFormatDecoder`
> (it diverged to the `IStreamingDecoder`/WebCodecs seam precisely because per-segment `decodeAudioData` is
> architecturally wrong for Opus — see `IStreamingDecoder.ts`), and its seek is accurate and shipping. The
> body below is rewritten to the two-path reality. **The headline is unchanged:** bound client memory to a
> sliding window regardless of stream length, for the canonical 1 GB mix, across both delivery formats.
---
@@ -31,18 +58,34 @@ Bound the **client memory** a playing track consumes to a small, configurable fo
**independent of total stream length** — so a 1 GB+ DJ MIX (Phase 9 `Mix` medium: a single long track)
plays without the whole decoded PCM accumulating in the browser.
**The defect, stated precisely.** The network path already streams in adaptive 1664 KB chunks
(`StreamingAudioPlayerService.StreamAudioWithEarlyPlayback`) — that part is fine. The accumulation is on
the **decode side**: `PlaybackScheduler` holds `private buffers: AudioBuffer[]` and **never evicts**
("Supports pause/resume/seek by **retaining all buffers**" — its own doc comment). Every 64 KB segment
the `StreamDecoder` decodes is pushed via `addBuffer()` and kept for the life of the track. Decoded PCM
is **larger than the compressed-or-raw source** in memory (Web Audio `AudioBuffer` is 32-bit float per
sample per channel — a 16-bit stereo WAV roughly **doubles** in size once decoded), so a 1 GB WAV becomes
~2 GB of retained `AudioBuffer` float data. That is the OOM.
**The defect, stated precisely — and it now has two faces, one shared.** The network path already
streams in adaptive 1664 KB chunks (`StreamingAudioPlayerService.StreamAudioWithEarlyPlayback`) — that
part is fine. The accumulation is on the **decode side**, and Phase 18 split the decode side into two
pipelines that both terminate at the same sink:
**One-line framing:** today the player decodes the whole track into memory and keeps it; Phase 21 makes
it keep only a sliding forward window and discard what has already played, refilling on demand from the
Range primitive it already uses for seek.
- **The shared sink (both paths) — the unbounded scheduler.** `PlaybackScheduler` holds
`private buffers: AudioBuffer[]` and **never evicts** ("Supports pause/resume/seek by **retaining all
buffers**" — its own doc comment). Both decode paths call `scheduler.addBuffer()` (via
`AudioPlayer.processFormatChunk` for WAV/MP3/FLAC and `processOpusChunk` for Opus); nothing is ever
removed. Decoded PCM is **larger than the source** in memory (Web Audio `AudioBuffer` is 32-bit float
per sample per channel — a 16-bit stereo WAV roughly **doubles** once decoded; Opus decodes to the same
48 kHz float PCM regardless of how few bytes the *compressed* stream was). So a 1 GB WAV becomes ~2 GB
of retained float, **and a low-data Opus mix becomes the same ~2 GB of decoded float once played**
the compressed transfer is small, but the *decoded* footprint is identical. The scheduler is the OOM for
both. **This is the §3.1 root cause, unchanged from the original spec — it just now afflicts two
producers.**
- **The Opus-only second locus — upstream decode-ahead.** The Opus path accumulates *before* the
scheduler too: the WebCodecs `AudioDecoder` work queue (`decodeQueueSize`), the `decodedQueue:
AudioData[]` awaiting conversion, and the `OggDemuxer`'s partial-page state. Bounding the scheduler
alone does not bound these — they fill from the same C# `ReadAsync` loop, so they need their own
back-pressure (on the *demuxer/decoder feed*), not only the read loop's. WAV has no equivalent
upstream queue (its `StreamDecoder` decodes synchronously into the scheduler), so this is genuinely
Opus-specific.
**One-line framing:** today the player decodes the whole track into memory and keeps it — true for both
formats; Phase 21 makes it keep only a sliding forward window and discard what has already played,
refilling on demand from the Range primitive both paths already use for seek (WAV via `IFormatDecoder`,
Opus via the live index-driven `resolveOpusByteOffset`).
---
@@ -60,36 +103,51 @@ docs. This phase **modifies that seam** — so the contract it must preserve is
- **C2 — Playback start latency unchanged.** Today playback starts as soon as a configurable minimum
buffer count is queued (header-derived duration, not full-file). The window model must keep first-audio
latency at parity — bounding memory must not reintroduce a fetch-then-play stall.
- **C3 — The format-decoder abstraction is untouched.** `IFormatDecoder` owns all format-specific
byte math; `AudioPlayer.createFormatDecoder` already dispatches on `Content-Type` (WAV/MP3/FLAC
decoders all wired today — verified 2026-06-23; an `OpusFormatDecoder` joins them in Phase 18).
Windowing lives in the
**format-agnostic** layer (`PlaybackScheduler` eviction + `StreamDecoder`/player refill
orchestration); it must add **no** format-specific branches. A future wired MP3/FLAC decoder inherits
windowing for free.
- **C3 — Neither decoder seam's contract is forked; windowing lives in the shared layer plus a thin
per-seam hook.** There are two decoder seams as of Phase 18: `IFormatDecoder` (WAV/MP3/FLAC, owns
format byte math; `AudioPlayer.createFormatDecoder` dispatches on `Content-Type`) and `IStreamingDecoder`
(Opus, the WebCodecs pipeline; selected in `initializeStreaming` when the content type is
`audio/ogg`/`audio/opus` and a sidecar is present). **The eviction half of windowing is fully shared**
it lives in `PlaybackScheduler`, which both seams feed identically via `addBuffer`, so eviction adds
**zero** format branches. **The back-pressure / decode-ahead half is necessarily seam-aware** — the WAV
path back-pressures the C# `ReadAsync` loop; the Opus path must additionally bound the WebCodecs
decode-ahead and the `decodedQueue` (§3.1). Express that as a **small uniform signal** ("the scheduler is
full, stop producing") that each decode path honors in its own way, rather than a windowing controller
that reaches into either decoder's internals. The goal the original C3 stated still holds — no
format-specific logic leaking into the *scheduler* — but the spec now acknowledges the producer side has
two shapes, not one.
- **C4 — Read-only playback only.** This is a memory-management change, not a UX change. No new
user-visible control, no change to seek/transport semantics beyond what the listener already
experiences. Seek must still feel identical.
- **C5 — Must window both delivery formats (WAV lossless AND Opus low-data).** Byte↔time mapping for
refill is exact and cheap for WAV (CBR: `byteRate` from the header). **Phase 18 (Opus) is sequenced
before this phase and is the concrete VBR driver here** — and its mapping is **also exact**, but by a
different mechanism: an Ogg Opus 320 stream has no linear time↔byte relationship, so
`OpusFormatDecoder.calculateByteOffset` resolves via a **precomputed seek index** (granule→byte, built at
transcode; Phase 18 §3.4a), a binary search that returns the exact page offset — **not** an approximate
page interpolation. (An earlier draft of this invariant said "approximate"; the Phase 18 seek-model
resolution, Daniel 2026-06-23, made Opus seeking accurate. Corrected here.) The window machinery must
express refill purely in terms of the decoder's existing `calculateByteOffset`, so the same code windows
WAV (via `byteRate`) and Opus (via the index) — **no WAV-special-cased offset math in the window layer**,
and no approximation for either. A window that opens away from byte 0 must also prepend the decoder's
retained/sidecar setup header (Phase 18 §3.4a B) — the format-agnostic refill path already routes
continuations through the decoder's header-carry, so this comes for free. (MP3/FLAC decoders are already
wired in the registry too — the registry dispatches on content-type today; an `OpusFormatDecoder` joins
them in Phase 18.)
- **C6 — No regression to the single-instance JS decoder concurrency guarantees.** The current code is
careful that only one streaming loop touches the single JS `StreamDecoder` at a time
(`DrainActiveStreamingTaskAsync`, the `_streamingCancellation` identity dance). Windowed refill
introduces *more* mid-stream fetches; it must route through the **same** drain/cancellation discipline,
not around it.
- **C5 — Window both decode paths without forking the scheduler/seam, reusing the live index-driven
seek for refill.** Both delivery formats must be windowed, and the byte↔time mapping each refill needs is
**already accurate and already shipping** for both:
- **WAV/MP3/FLAC**`IFormatDecoder.calculateByteOffset` (CBR `byteRate` for WAV; the MP3/FLAC seek
accelerators for those), reached through `StreamDecoder.calculateByteOffset` / `AudioPlayer.seekBeyondBuffer`.
- **Opus**`resolveOpusByteOffset(activeOpusSidecar, t)` (binary search the precomputed granule→byte
seek index in `OpusSeekData`), returning an exact page-start offset **and** a `landingTimeSeconds` for
the decoder's frame-accurate lead-trim. This is **accurate, not approximate, and landed in Phase 18.**
Phase 21 does **not** build either mapping. The window's refill trigger calls *whichever resolver the
active path already uses* — for Opus, the **same** `resolveOpusByteOffset` an explicit listener seek
calls (the live path in `AudioPlayer.seekBeyondBuffer`), so windowed refill is literally "a seek the
listener didn't initiate." A window opening away from byte 0 decodes correctly on the Opus path because
the setup header (`OpusHead`/`OpusTags`) is already cached from the sidecar and re-applied by
`reinitializeForRangeContinuation` (Phase 18 §3.4a B); the WAV path re-applies its retained header the
same way. **No new offset math, no approximation, no header re-fetch — all reused.** The invariant is
therefore *not* "make refill format-agnostic" (the two paths legitimately resolve offsets through
different code); it is **"reuse the live seek of each path verbatim; add only the eviction and the
refill *trigger*, never a second seek mechanism."**
- **C6 — No regression to the single-writer decoder concurrency guarantee — now covering both decoders.**
The C# loop is careful that only one streaming task feeds the active JS decoder at a time
(`DrainActiveStreamingTaskAsync`, the `_streamingCancellation` identity dance in
`StreamingAudioPlayerService`). This matters *more* for Opus: the WebCodecs `AudioDecoder` is stateful
and async — a `reset()`+`configure()` on a range-continuation (`reinitializeForRangeContinuation`) racing
a still-draining `push()` from a stale loop would corrupt inter-frame state, not merely deliver a wrong
buffer. Windowed refill introduces *more* mid-stream fetches against whichever decoder is active; every
one must route through the **same** drain/cancellation discipline, not around it. The discipline is
already decoder-agnostic at the C# layer (it cancels the loop, not the decoder), so this is a "keep using
it" invariant — but it is the rule most likely to be violated by a naive Opus refill, and is the hardest
failure to diagnose, so it is called out as a hard invariant for both paths.
- **C7 — The Mix visualizer's data source is independent and must stay that way.** The Phase 10/12
WebGL2 lava visualizer renders from a **preprocessed high-res waveform datum** fetched per-track
(`GET api/track/{entryKey}/waveform/high-res`), **not** from live decoded PCM. Confirmed: evicting
@@ -103,8 +161,10 @@ docs. This phase **modifies that seam** — so the contract it must preserve is
### 3.0 The mental model
A track's audio is a byte range `[0, fileLength)` on disk. At any moment the listener is at playback
position `P` (seconds → byte offset via the format decoder). The player should hold decoded
`AudioBuffer`s only for a bounded window roughly `[P - back, P + ahead]`:
position `P` (seconds → byte offset via the active path's resolver — `IFormatDecoder.calculateByteOffset`
for WAV/MP3/FLAC, `resolveOpusByteOffset` over the seek index for Opus). The player should hold decoded
`AudioBuffer`s only for a bounded window roughly `[P - back, P + ahead]` — and, on the Opus path, keep the
upstream WebCodecs decode queue near-empty too (§3.1):
- **forward fill (`ahead`)** — enough decoded lookahead that playback never starves (covers the existing
500 ms scheduler lookahead plus network jitter headroom);
@@ -119,42 +179,64 @@ position `P` (seconds → byte offset via the format decoder). The player should
This is a **ring/sliding-window buffer keyed on playback position**, driven by high/low-water marks —
the standard bounded-producer/bounded-consumer pattern, transplanted onto the decode→schedule seam.
### 3.1 Why this is a generalization of seek-beyond-buffer, not a new mechanism
### 3.1 Why refill is a generalization of seek-beyond-buffer, not a new mechanism — for both paths
The seek-beyond-buffer path already does **every primitive** the window needs, just triggered manually
and one-shot:
The seek-beyond-buffer path already does **every refill primitive** the window needs, just triggered
manually and one-shot. As of Phase 18 each primitive has a WAV branch and an Opus branch, both live:
| Window operation | Existing seek-beyond-buffer machinery it reuses |
|-------------------------------|-----------------------------------------------------------------------------------|
| Discard buffers, keep offset | `PlaybackScheduler.clearForSeek()` + `setPlaybackOffset()` (clears buffers, retains the absolute-time anchor) |
| Fetch from a byte offset | `TrackMediaClient.GetTrackMedia(key, byteOffset)``Range: bytes=X-` → 206 |
| Decode a header-less body | `StreamDecoder.reinitializeForRangeContinuation(remainingByteLength)` |
| Map time → byte offset | `StreamDecoder.calculateByteOffset()``IFormatDecoder.calculateByteOffset()` |
| Single-loop safety on refetch | `_streamingCancellation` swap + `DrainActiveStreamingTaskAsync()` |
| Window operation | WAV/MP3/FLAC machinery reused | Opus machinery reused (Phase 18, landed) |
|-------------------------------|--------------------------------------------------------------------|--------------------------------------------------------------------------------------|
| Discard buffers, keep offset | `PlaybackScheduler.clearForSeek()` + `setPlaybackOffset()` | *same* — the scheduler is shared |
| Fetch from a byte offset | `TrackMediaClient``Range: bytes=X-` → 206 | *same* (with `?format=opus`) — the Range path is shared |
| Map time → byte offset | `StreamDecoder.calculateByteOffset()``IFormatDecoder` | `resolveOpusByteOffset(activeOpusSidecar, t)` (index binary search → exact page) |
| Decode a header-less body | `StreamDecoder.reinitializeForRangeContinuation(len)` | `OpusStreamDecoder.reinitializeForRangeContinuation(landingTime, target)` (demux/codec reset + lead-trim) |
| Single-loop safety on refetch | `_streamingCancellation` swap + `DrainActiveStreamingTaskAsync()` | *same* — the C# discipline is decoder-agnostic |
The difference is **eviction does not exist yet** (the scheduler only ever `clear()`s wholesale) and
**refill is one-shot** (a seek, not a continuous low-water-triggered loop). So the new work is two
seams: a *partial-evict* on the scheduler, and a *position-driven refill controller* on the player. The
fetch/decode/offset plumbing is reused verbatim.
The genuinely-new work, by path:
- **Shared (both paths):** *partial eviction* on `PlaybackScheduler` (today it only ever `clear()`s
wholesale), and a *position-driven refill trigger* (a continuous low-water loop, not a one-shot seek).
- **WAV path:** *back-pressure on the C# `ReadAsync` loop* — stop reading the socket above the high-water
mark, resume below low-water. WAV's `StreamDecoder` decodes synchronously into the scheduler, so the
read loop is the *only* producer to throttle; pausing `ReadAsync` bounds it fully.
- **Opus path:** *the same C# back-pressure, plus bounding the WebCodecs decode-ahead.* Throttling
`ReadAsync` alone is **not sufficient** for Opus, because `OpusStreamDecoder.push()` is async and the
WebCodecs `AudioDecoder` keeps its own internal work queue (`decodeQueueSize`) plus a `decodedQueue:
AudioData[]` of decoded-but-not-yet-converted frames. The Opus producer must also stop *feeding the
decoder* (stop demuxing/decoding new packets) when the scheduler is full, and resume below low-water —
back-pressure on the **demuxer/decoder feed**, not only on the socket read. This is the one place the
two paths' windowing genuinely diverges.
Everything else — the fetch, the offset resolution, the header-carry continuation, the single-loop
cancellation safety — is **reused verbatim** on both paths. Phase 21 builds eviction + the refill trigger
+ the (per-path) back-pressure; it builds **no** new fetch, offset, or seek mechanism.
### 3.2 The three candidate directions
Per file convention the alternatives are recorded; the recommendation follows.
**Direction A — Sliding window on the existing single forward stream (recommended).**
Keep the current model where the C# loop reads one forward HTTP stream and pumps chunks into the JS
decoder. Add two things: (1) `PlaybackScheduler` gains *partial eviction* — drop buffers whose
Keep the current model where the C# loop reads one forward HTTP stream and pumps chunks into the active
JS decoder. Add three things: (1) `PlaybackScheduler` gains *partial eviction* — drop buffers whose
absolute-time end is older than `P - back`, adjusting its index bookkeeping so `getCurrentPosition()`
and scheduling stay correct against a buffer array that no longer starts at index 0; (2) a
*back-pressure* signalwhen forward decoded lookahead exceeds the high-water mark, the C# loop
**pauses reading** the HTTP stream (stops calling `ReadAsync`) until playback drains it below low-water,
then resumes. Memory is bounded by high-water + back-retain. Seek-back beyond the retained window falls
through to the **existing** seek-beyond-buffer path unchanged.
and scheduling stay correct against a buffer array that no longer starts at index 0 (**shared by both
paths**the scheduler is the common sink); (2) *back-pressure on the C# read loop* — when forward
decoded lookahead exceeds the high-water mark, the C# loop **pauses reading** the HTTP stream (stops
calling `ReadAsync`) until playback drains it below low-water, then resumes; (3) **for the Opus path
only, back-pressure on the WebCodecs decode-ahead** — the producer also stops demuxing/decoding new
packets when the scheduler is full, so the `AudioDecoder` work queue and `decodedQueue` do not balloon
behind a throttled socket. Memory is bounded by high-water + back-retain on both paths. Seek-back beyond
the retained window falls through to the **existing** seek-beyond-buffer path (the right one per format)
unchanged.
*Why recommended:* smallest change to the load-bearing seam; reuses the live forward stream (no extra
connections in the common case); eviction and back-pressure are the only genuinely new mechanisms, and
both are local (one to the scheduler, one to the read loop). Back-pressure via "stop reading the socket"
is exactly how TCP flow control already wants to behave — pausing `ReadAsync` lets the kernel window
close; we are not fighting the transport.
connections in the common case); eviction and back-pressure are the only genuinely new mechanisms, all
local (the scheduler; the read loop; for Opus, the demux/decode feed). Back-pressure via "stop reading
the socket" is exactly how TCP flow control already wants to behave — pausing `ReadAsync` lets the kernel
window close; we are not fighting the transport. The Opus decode-ahead bound is the one addition Phase 18
forces, and it is local to the Opus producer.
*Open question it raises (OQ6, new):* whether the two paths' back-pressure is driven by **one shared
window controller** that exposes a "scheduler full / drained" signal both producers poll, or by **two
parallel implementations** sharing only the eviction code. Recommend the **shared signal** — see §6 OQ6.
**Direction B — Discrete window segments, each its own Range fetch.**
Treat the file as fixed-size byte segments (e.g. 4 MB). Hold N decoded segments around `P`; fetch the
@@ -177,32 +259,41 @@ problem.
wants containerized formats (fragmented MP4/WebM, or MP3/AAC elementary streams). The entire bespoke
visualizer/spectrum graph is wired to the Web Audio `AudioContext`, not a `<media>` element. Adopting
MSE is a **rewrite of the playback substrate**, not a windowing change. It *looked* like the real
long-term answer once compressed delivery arrived — but Daniel has decided compressed delivery
(**Phase 18 Opus**) will feed the **same bespoke graph** via the `IFormatDecoder` seam, so the
compressed-delivery move that would have justified MSE happens *without* surrendering the graph. **The
bespoke graph is a deliberate long-term commitment; MSE is rejected.** Direction A is therefore the
permanent destination, not a stopgap that MSE will retire. Recorded as considered-and-declined.
long-term answer once compressed delivery arrived — but compressed delivery (**Phase 18 Opus, now
landed**) feeds the **same bespoke graph** via the WebCodecs `IStreamingDecoder` seam (parallel to the
WAV `IFormatDecoder` seam, both terminating at the shared `PlaybackScheduler`), so the compressed-delivery
move that would have justified MSE happened *without* surrendering the graph. Notably, Phase 18 chose a
**WebCodecs `AudioDecoder`** for Opus rather than `decodeAudioData` — which is itself the "use the platform
codec, keep the bespoke graph" move, but at the *decoder* granularity, not the *media-element* granularity
MSE would impose. **The bespoke graph is a deliberate long-term commitment; MSE is rejected.** Direction A
is therefore the permanent destination, not a stopgap that MSE will retire. Recorded as
considered-and-declined.
### 3.3 Recommended direction: A, with B held as the documented fallback
Direction A is the smallest coherent change that hits the headline (bounded memory under a 1 GB stream)
while honoring C1C7. It keeps the live forward stream, reuses the seek-beyond-buffer path for the only
genuinely random-access case (seek-back past the retained tail), and isolates the two new mechanisms.
**The final architecture and the exact eviction/back-pressure API are staff-engineer's call at
implementation** (per file convention); this spec fixes the *shape* and the invariants, not the method
signatures.
while honoring C1C7. It keeps the live forward stream, reuses each path's seek-beyond-buffer machinery
for the only genuinely random-access case (seek-back past the retained tail), and isolates the new
mechanisms (eviction shared; back-pressure per path). **The final architecture and the exact
eviction/back-pressure API are staff-engineer's call at implementation** (per file convention); this spec
fixes the *shape* and the invariants, not the method signatures.
### 3.4 SOLID / road-not-taken rationale
- **SRP, preserved.** Eviction is a `PlaybackScheduler` concern (it already owns buffer storage); refill
orchestration is a player-service/`StreamDecoder` concern (they already own the fetch loop); byte↔time
math stays in `IFormatDecoder`. No responsibility crosses a boundary it does not already own.
- **OCP, via C3/C5.** Windowing added in the format-agnostic layer means wiring MP3/FLAC later changes
zero window code. The window expresses refill through `calculateByteOffset` — the one seam the
decoders already implement.
- **The seam stays single-writer (C6).** Every new refetch routes through the existing
cancellation/drain discipline, so "only one loop touches the JS decoder" remains true. This is the
rule most likely to be violated by a naive implementation and is called out as a hard invariant.
- **SRP, preserved.** Eviction is a `PlaybackScheduler` concern (it already owns buffer storage, and is
the single shared sink both decode paths feed); refill orchestration is a player-service concern (it
already owns the C# fetch loop and the seek dispatch); byte↔time math stays where each path already keeps
it — `IFormatDecoder.calculateByteOffset` for WAV/MP3/FLAC, `resolveOpusByteOffset` (over `OpusSeekData`)
for Opus. No responsibility crosses a boundary it does not already own.
- **OCP, via the shared sink + the live per-path seek.** Eviction added at the scheduler changes zero
decoder code on either path. Refill reuses each path's *already-implemented* offset resolver — Phase 21
adds no offset math to either seam. The one place windowing is not purely additive is the Opus
decode-ahead bound (§3.1), which lives inside the Opus producer, not in the shared layer.
- **The seam stays single-writer (C6) — for both decoders.** Every new refetch routes through the existing
C# cancellation/drain discipline, so "only one loop feeds the active decoder" remains true for the WAV
`StreamDecoder` and the stateful Opus `AudioDecoder` alike. This is the rule most likely to be violated
by a naive Opus refill (a stale `push()` racing a `reset()`+`configure()`), and is called out as a hard
invariant.
- **Road not taken — eager full decode with a memory cap that just stops decoding.** Tempting (decode
until you hit a byte budget, then stop) but it breaks playback of long tracks past the cap entirely —
it bounds memory by *refusing to play the rest*, not by sliding. Rejected: it is a degradation, not a
@@ -213,8 +304,15 @@ signatures.
## 4. Use cases
- **UC1 — Play a 1 GB+ DJ MIX start to finish (the headline).** Memory stays bounded throughout; the
listener experiences continuous playback identical to a short track.
- **UC2 — Seek forward within a long track.** Already handled by seek-beyond-buffer; under windowing the
listener experiences continuous playback identical to a short track. **Holds in both formats** — the
lossless WAV mix (~2 GB decoded if unbounded) and the low-data Opus mix (small transfer, but the *same*
~2 GB decoded float once played, so it needs windowing just as much; see §1).
- **UC1-Opus — The same mix streamed as Opus, windowed.** The low-data win (Phase 18) shrinks the
*transfer*; Phase 21 shrinks the *decoded footprint*. The two compound: a metered-connection listener on
Opus gets both the small download and the bounded memory. Windowing the Opus path additionally bounds the
WebCodecs decode-ahead and `decodedQueue`, not only the scheduler (§3.1).
- **UC2 — Seek forward within a long track.** Already handled by seek-beyond-buffer (the right resolver per
format — `IFormatDecoder` for WAV, the live `resolveOpusByteOffset` for Opus); under windowing the
forward seek clears the window and refills at the target — no behavior change, now with eviction so the
pre-seek region does not linger.
- **UC3 — Seek back a few seconds.** Served from the back-retain window with **no** network refetch
@@ -248,8 +346,11 @@ This phase touches the **same decoder/scheduler seam** as the deferred Phase 1.3
- **1.5 Gapless (deferred).** Sample-accurate hand-off of the next track's first buffer at the current
track's last buffer. Windowing changes *which* buffers are retained but not the hand-off mechanism;
the only care point is that the current track's **final** window must not be evicted before the gapless
boundary is scheduled. A minor invariant for whoever builds 1.5, not a blocker. Note 1.5's existing
WAV-only caveat stands.
boundary is scheduled. A minor invariant for whoever builds 1.5, not a blocker. **Phase 18 note:** the
former "1.5 is WAV-only" caveat is superseded — Opus is live, and it has its own encoder pre-skip/priming
(handled once by the WebCodecs decoder, see `OpusStreamDecoder.ts`), so a gapless Opus hand-off must
respect the end-trim against the sidecar's authoritative total length. That is 1.5's problem to absorb,
not Phase 21's; flagged so 1.5 inherits it.
- **1.6 Track-skip on error (deferred).** *Windowing enlarges the error surface — call this out.* Today
a fetch failure happens at load (one fetch) or at a user seek (one fetch). Windowed refill issues
**mid-stream** fetches the listener did not initiate; one of those can fail at byte 700 M of a 1 GB
@@ -260,9 +361,12 @@ This phase touches the **same decoder/scheduler seam** as the deferred Phase 1.3
failure handling into Phase 21's acceptance criteria** (AC6) rather than leaving it entirely to 1.6 —
it is created by this phase.
- **1.7 Safari compatibility (deferred).** Windowing adds no new Safari-specific surface beyond what the
streaming path already has. The one adjacency: more frequent `AudioContext` activity during refill
should be checked against the older-Safari `webkitAudioContext` quirks when 1.7 is addressed — note it,
do not block on it.
streaming path already has. Two adjacencies, both Phase-18-introduced: (a) more frequent `AudioContext`
activity during refill should be checked against older-Safari `webkitAudioContext` quirks; (b) the Opus
path depends on **WebCodecs `AudioDecoder`**, whose Safari availability is narrower than `decodeAudioData`
Ogg-Opus support — Phase 18's capability gate already falls a non-WebCodecs browser back to the lossless
WAV path, so a Safari that can't run the Opus pipeline windows the *WAV* path (which has no decode-ahead
locus, only the scheduler), i.e. the simpler windowing case. Note it; do not block on it.
---
@@ -294,22 +398,50 @@ These are policy calls with user-visible or resource trade-offs — flagged rath
adopt MSE. The bespoke Web Audio decode→schedule graph stays — it is bespoke by deliberate choice, a
long-term commitment, not a stopgap.** Daniel's rationale: the player is intentionally a custom
graph, not an HTML `<media>` element; the compressed-delivery move that *would* have made MSE
tempting is being met instead by **Phase 18 (Opus low-data path)** feeding the **same bespoke graph**
through the `IFormatDecoder` seam — so compressed delivery arrives *without* surrendering the graph.
Consequence for this phase: Direction A (the hand-rolled sliding window) is the destination, not a
placeholder; invest in it as permanent machinery. It will window both the WAV and the Opus path
(the sequencing note at the top). Direction C is recorded as **considered and declined** per file
convention; kept visible so a future reader sees the road not taken and why.
`[RESOLVED — bespoke graph retained; MSE rejected]`
tempting was met instead by **Phase 18 (Opus low-data path, now landed)** feeding the **same bespoke
graph** through the WebCodecs `IStreamingDecoder` seam (parallel to the WAV `IFormatDecoder` seam) — so
compressed delivery arrived *without* surrendering the graph. Consequence for this phase: Direction A
(the hand-rolled sliding window) is the destination, not a placeholder; invest in it as permanent
machinery. It windows both the WAV and the Opus path (the header note). Direction C is recorded as
**considered and declined** per file convention; kept visible so a future reader sees the road not taken
and why. `[RESOLVED — bespoke graph retained; MSE rejected]`
- **OQ6 — One window controller for both decode paths, or two? (NEW — raised by the Phase 18 two-path
reality.)** Eviction is unambiguously shared (the scheduler is the one sink). Back-pressure is not: the
WAV path throttles the C# `ReadAsync` loop; the Opus path must *also* throttle the WebCodecs
decode-ahead (§3.1). Should there be **one window controller** exposing a uniform "scheduler full /
drained" signal that both producers honor in their own way (recommended — keeps the *policy* — window
sizes, water-marks, OQ1/OQ3 — in one place, with two thin per-path back-pressure hooks), or **two
parallel windowing implementations** sharing only the eviction code (simpler per-path, but duplicates the
water-mark logic and risks the two drifting)? Recommend the **shared controller + per-path hook**. This
is more an architecture call than a product call — flagged for staff-engineer at implementation, with the
recommendation as the default. `[staff-engineer call; recommendation: shared controller]`
- **OQ7 — How does the Opus WebCodecs decode-ahead bound interact with scheduler eviction? (NEW; technical,
for staff-engineer.)** The Opus producer has two queues to bound (the `AudioDecoder` work queue and
`decodedQueue: AudioData[]`) *plus* the shared scheduler. The clean rule is "stop feeding the decoder when
decoded-lookahead-in-the-scheduler exceeds high-water" — i.e. the **scheduler's** fill level is the
single back-pressure signal, and the upstream Opus queues are kept near-empty by simply not demuxing
ahead. The alternative (let the decoder run ahead into `decodedQueue` and bound *that* separately) adds a
second budget to tune and a second eviction point. Recommend the former: **one fill signal (scheduler
decoded-lookahead), drive both the read-loop pause and the demux/decode pause from it.** Confirm at
implementation that the WebCodecs decoder tolerates being starved of input mid-stream and resumes cleanly
(it should — it is fed packet-by-packet via `decode()`), and that `decodedQueue` is drained promptly so
it never holds more than one `push()` worth. `[staff-engineer call; recommendation: single
scheduler-fill signal]`
---
## 7. Acceptance criteria
- **AC1 (headline) — Bounded memory under a 1 GB stream.** Playing a 1 GB+ WAV mix start to finish, the
browser tab's retained decoded-audio memory stays bounded to the configured window (not growing toward
~2 GB). Verifiable via browser memory tooling: peak decoded-audio footprint is independent of track
length and tracks the window-size policy, not the file size.
- **AC1 (headline) — Bounded memory under a 1 GB stream, in BOTH formats.** Playing a 1 GB+ mix start to
finish — **as lossless WAV and as low-data Opus** — the browser tab's retained decoded-audio memory
stays bounded to the configured window (not growing toward ~2 GB). Verifiable via browser memory tooling:
peak decoded-audio footprint is independent of track length and tracks the window-size policy, not the
file size. The Opus case must be verified explicitly — its small *transfer* does not imply a small
*decoded* footprint (§1), so "Opus already streams small" is **not** sufficient.
- **AC1-Opus — The Opus upstream decode-ahead is bounded too (§3.1 / OQ7).** Under a long Opus stream, the
WebCodecs decode queue and `decodedQueue` do not grow unboundedly behind the scheduler — back-pressure
reaches the demux/decode feed, not only the scheduler. Verifiable: the upstream queues stay near-empty
(one `push()` worth) regardless of stream length.
- **AC2 — Playback-start latency at parity (C2).** First-audio latency for a track is unchanged from
pre-windowing (within noise). Windowing does not introduce a fetch-then-play stall.
- **AC3 — Continuous playback, no starvation.** A long mix plays edge to edge with no audible gaps,
@@ -324,41 +456,70 @@ These are policy calls with user-visible or resource trade-offs — flagged rath
"playing" with a starved scheduler). It must not silently hang.
- **AC7 — The Mix visualizer is unaffected (C7).** With the lava visualizer running on a long mix, the
visualizer renders identically (it reads the preprocessed datum, never the evicted buffers).
- **AC8 — Single-decoder concurrency invariant holds (C6).** Under rapid seek + refill activity, no
interleaved `ProcessStreamingChunk` calls corrupt the single JS decoder (the existing drain/cancel
discipline still governs every fetch).
- **AC8 — Single-writer decoder concurrency invariant holds (C6) — both decoders.** Under rapid seek +
refill activity, no interleaved `ProcessStreamingChunk` / `push` calls corrupt the active decoder the
existing drain/cancel discipline still governs every fetch. **For Opus this is stricter:** no stale
`push()` may land against the WebCodecs `AudioDecoder` across a `reinitializeForRangeContinuation`
reset+reconfigure (which would corrupt inter-frame state, not just a buffer). Verify under a rapid
seek-storm on an Opus mix specifically.
---
## 8. Wave decomposition
**Decomposition choice: split by *concern* (eviction → back-pressure → seek-back refill → validate), not
by *path* (WAV-track vs Opus-track).** Rationale: the eviction concern (21.1) is genuinely shared — the
scheduler is the one sink both paths feed — so a path-split would duplicate the hardest correctness work or
arbitrarily assign it to one track. The concern spine keeps that shared work as a single cold-start wave
and lets the *one* genuinely path-divergent concern (back-pressure, 21.2) carry an explicit two-track
split *inside* the wave rather than fracturing the whole phase. This also matches how the seek-back refill
(21.3) reuses each path's already-live seek — it is one concern (window-miss → refetch) with a per-path
resolver underneath, not two features. The spine is unchanged from the original spec; the mechanisms
inside 21.2 and 21.3 are made correct for both paths.
Dependency shape: `21.1 → 21.2 → 21.3`, with `21.4` validating the whole. 21.1 is the cold-start
prerequisite and the load-bearing change; the rest layer on it.
- **21.1 — Partial eviction in `PlaybackScheduler` (cold-start; the load-bearing change).** Give the
scheduler the ability to drop already-played buffers and keep its position/index bookkeeping correct
against a buffer array that no longer begins at absolute time 0 (today `getCurrentPosition`,
`playFromPosition`, and the scheduling loop all assume `buffers[0]` is the track start). This is the
hardest correctness work in the phase — the time-anchor math must stay exact through eviction. No
refill yet; with eviction alone and the forward read loop unchanged, this is provably memory-bounded
for the *played* region. **Independent of the §6 open questions** — it can begin immediately; the
window *sizes* (OQ1/OQ3) are parameters fed in later. Settled and cold-start.
- **21.2 — Back-pressure on the forward read loop (the bound on the *unplayed* region).** Make the C#
`StreamAudioWithEarlyPlayback` loop stop calling `ReadAsync` when forward decoded lookahead exceeds the
high-water mark, and resume below low-water. Together with 21.1, this bounds *both* the played and
unplayed sides — the full memory guarantee (AC1). Must route resume/pause through the existing
cancellation-safe single-loop discipline (C6). **Depends on 21.1** (eviction must exist so the drained
region is reclaimed, not merely un-read).
- **21.3Seek-back-past-window refill (close the random-access case).** Wire UC4 — when a backward
seek lands earlier than the retained tail, refetch via the existing seek-beyond-buffer Range path
pointed at the earlier offset, and the minimal AC6 refill-failure handling. Mostly **reuse** of the
landed seek path; the new work is the trigger (window-miss detection) and the clean-failure path.
**Depends on 21.1 + 21.2** (needs the window boundaries they define).
- **21.4 — Validation pass against the 1 GB target (acceptance).** Exercise AC1AC8 against a real 1 GB+
mix: memory profiling (AC1), latency parity (AC2), edge-to-edge playback (AC3), the seek matrix
(AC4/AC5), induced refill failure (AC6), visualizer-running (AC7), and rapid-seek concurrency (AC8).
Largely test/measurement; any break is likely a tuning fix in the 21.1 anchor math or the 21.2
water-marks. **Depends on 21.121.3.**
- **21.1 — Partial eviction in `PlaybackScheduler` (cold-start; the load-bearing change; SHARED by both
paths).** Give the scheduler the ability to drop already-played buffers and keep its position/index
bookkeeping correct against a buffer array that no longer begins at absolute time 0 (today
`getCurrentPosition`, `playFromPosition`, and the scheduling loop all assume `buffers[0]` is the track
start). This is the hardest correctness work in the phase — the time-anchor math must stay exact through
eviction. Because both decode paths feed the scheduler identically via `addBuffer`, **eviction is written
once and serves both** — no per-path branch. No refill yet; with eviction alone and the forward producers
unchanged, this is provably memory-bounded for the *played* region on both paths. **Independent of the §6
open questions** — it can begin immediately; the window *sizes* (OQ1/OQ3) are parameters fed in later.
Settled and cold-start.
- **21.2 — Back-pressure (the bound on the *unplayed* region) — two tracks, one signal.** Bound the
not-yet-played decoded audio by stopping production above a high-water mark and resuming below low-water,
driven by the scheduler's decoded-lookahead fill (OQ7). The fill *signal* is shared; the *throttle* has
two sites because Phase 18 gave the two paths different producers:
- **21.2aC# read-loop back-pressure (serves both paths).** Make `StreamAudioWithEarlyPlayback` stop
calling `ReadAsync` above high-water and resume below low-water. Routes resume/pause through the
existing cancellation-safe single-loop discipline (C6). For the WAV path this is *sufficient* (its
`StreamDecoder` decodes synchronously into the scheduler).
- **21.2b — Opus decode-ahead back-pressure (Opus path only).** Additionally stop demuxing/decoding new
packets when the same fill signal is over high-water, so the WebCodecs decode queue and `decodedQueue`
do not balloon behind a throttled socket (§3.1, OQ7). This is the one mechanism with no WAV analogue.
Confirm the WebCodecs decoder resumes cleanly after being starved of input mid-stream.
Together with 21.1 this bounds *both* the played and unplayed sides on *both* formats — the full memory
guarantee (AC1 + AC1-Opus). **Depends on 21.1** (eviction must exist so the drained region is reclaimed,
not merely un-read). Per OQ6, 21.2a and 21.2b ideally share one window controller exposing the fill
signal; the recommendation is the shared controller + two thin hooks.
- **21.3 — Seek-back-past-window refill (close the random-access case; one concern, per-path resolver).**
Wire UC4 — when a backward seek lands earlier than the retained tail, refetch via the existing
seek-beyond-buffer path pointed at the earlier offset, **using whichever resolver the active path already
ships** (`IFormatDecoder`/`StreamDecoder.calculateByteOffset` for WAV; the live
`resolveOpusByteOffset` + `OpusStreamDecoder.reinitializeForRangeContinuation` for Opus) — plus the
minimal AC6 refill-failure handling. Mostly **reuse** of the landed seek paths; the new work is the
trigger (window-miss detection) and the clean-failure path, both format-agnostic. **Depends on 21.1 +
21.2** (needs the window boundaries they define).
- **21.4 — Validation pass against the 1 GB target, BOTH formats (acceptance).** Exercise AC1AC8 against a
real 1 GB+ mix **streamed as WAV and as Opus**: memory profiling (AC1 both formats + AC1-Opus upstream
queues), latency parity (AC2), edge-to-edge playback (AC3), the seek matrix (AC4/AC5), induced refill
failure (AC6), visualizer-running (AC7), and rapid-seek concurrency (AC8 — including the Opus
seek-storm). Largely test/measurement; any break is likely a tuning fix in the 21.1 anchor math, the
21.2 water-marks, or the 21.2b Opus decode-ahead bound. **Depends on 21.121.3.**
---
@@ -366,18 +527,35 @@ prerequisite and the load-bearing change; the rest layer on it.
- Root `CLAUDE.md` "Streaming-first audio playback" / `CONTEXT.md §3.5` — the seam this phase modifies;
the §2 invariants here restate its contract. Both flag it as the most load-bearing path.
- `PLAN.md` Phase 4 (landed) / `COMPLETED.md` — the HTTP Range `bytes=X-` primitive this generalizes.
- **`COMPLETED.md` Phase 18 — Opus Low-Data Streaming (landed 2026-06-23) — read this first.** The
"as-built divergence" note records why Opus uses a **WebCodecs `AudioDecoder`** streaming pipeline
(`IStreamingDecoder`) rather than the spec'd-and-replaced per-segment `decodeAudioData`/`IFormatDecoder`
model. This is the two-path reality this phase reconciles to. `product-notes/phase-18-opus-low-data-streaming.md`
is the design memo (note: its §3.4 `OpusFormatDecoder` framing predates the WebCodecs divergence — the
*seek-index/sidecar* design in §3.4a is accurate and landed; the *decoder-shape* discussion was superseded
by `IStreamingDecoder`).
- `PLAN.md` Phase 4 (landed) / `COMPLETED.md` — the HTTP Range `bytes=X-` primitive this generalizes
(now serving both `?format=lossless` and `?format=opus`).
- `PLAN.md` Phase 1.3 / 1.4 / 1.5 / 1.6 / 1.7 — the deferred decoder/scheduler-seam features; §5 above
reconciles each.
reconciles each (1.5 and 1.7 updated for the Opus path).
- `PLAN.md` Phase 9 — defines the `Mix` medium (single long track), the canonical 1 GB case.
- `PLAN.md` Phase 10 / `product-notes/phase-10-mix-visualizer-lava-reframe.md` /
`product-notes/phase-12-waveform-visualizer-generalization.md` — establishes the preprocessed
per-track high-res waveform datum; the basis for C7 (visualizer does not read live PCM).
- `DeepDrftPublic/Interop/audio/PlaybackScheduler.ts` — owns the unbounded `buffers: AudioBuffer[]`;
21.1 lives here.
- `DeepDrftPublic/Interop/audio/StreamDecoder.ts``reinitializeForRangeContinuation`,
`calculateByteOffset`; the refill substrate.
- `DeepDrftPublic/Interop/audio/PlaybackScheduler.ts` — owns the unbounded `buffers: AudioBuffer[]`, the
**shared sink for both decode paths**; 21.1 (eviction) lives here.
- `DeepDrftPublic/Interop/audio/AudioPlayer.ts` — the dispatch: `processFormatChunk` (WAV/MP3/FLAC) vs
`processOpusChunk` (Opus), both calling `scheduler.addBuffer`; `seekBeyondBuffer`/`reinitializeFromOffset`
branch per path; the place the refill trigger (21.3) and the fill-signal wiring (21.2) hook.
- `DeepDrftPublic/Interop/audio/StreamDecoder.ts` + `IFormatDecoder.ts` — the WAV/MP3/FLAC refill substrate
(`reinitializeForRangeContinuation`, `calculateByteOffset`).
- `DeepDrftPublic/Interop/audio/IStreamingDecoder.ts` + `OpusStreamDecoder.ts` + `OggDemuxer.ts` +
`OpusSidecar.ts` — the **Opus** path: the WebCodecs decode pipeline, the `decodeQueueSize`/`decodedQueue`
upstream accumulation 21.2b must bound, and the live `resolveOpusByteOffset` /
`reinitializeForRangeContinuation(landingTime, target)` seek 21.3 reuses. **`IStreamingDecoder.ts` is the
seam the Opus windowing hooks into** (push/complete/reinitialize lifecycle).
- `DeepDrftPublic.Client/Services/StreamingAudioPlayerService.cs` — the C# forward read loop
(`StreamAudioWithEarlyPlayback`), the seek-beyond-buffer path (`SeekBeyondBuffer`), and the
cancellation/drain discipline (C6); 21.2/21.3 live here.
- `DeepDrftPublic.Client/Clients/TrackMediaClient.cs` — the Range-capable media fetch reused by refill.
(`StreamAudioWithEarlyPlayback`, feeding *both* decoders), the seek-beyond-buffer path (`SeekBeyondBuffer`),
and the cancellation/drain discipline (C6); 21.2a/21.3 live here.
- `DeepDrftPublic.Client/Clients/TrackMediaClient.cs` — the Range-capable media fetch (with the `?format=`
param) reused by refill on both paths.