diff --git a/CLAUDE.md b/CLAUDE.md index 313e521..a2782a3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -66,11 +66,28 @@ If step 1 succeeds and step 2 fails, audio is orphaned in the vault (no rollback The player is not fetch-then-play: -1. Client calls `GET api/track/{id}` on DeepDrftContent and receives WAV bytes as a stream (`HttpCompletionOption.ResponseHeadersRead`). -2. `StreamingAudioPlayerService` reads in adaptive 16–64 KB chunks, pushes each via `AudioInteropService.processStreamingChunk`. -3. TypeScript `StreamDecoder` parses WAV header, decodes chunks to `AudioBuffer`s. `PlaybackScheduler` schedules them on a Web Audio graph. +1. Client calls `GET api/track/{id}` on DeepDrftContent and receives the first bounded segment (`Range: bytes=0-{SegmentSizeBytes-1}`, 4 MB) via `HttpCompletionOption.ResponseHeadersRead`. +2. `StreamingAudioPlayerService` reads in adaptive 16–64 KB chunks within each segment, pushes each via `AudioInteropService.processStreamingChunk`. +3. TypeScript `StreamDecoder` parses the format header, decodes chunks to `AudioBuffer`s. `PlaybackScheduler` schedules them on a Web Audio graph. 4. Playback starts as soon as a min buffer is queued; UI duration from parsed header (not waiting for full file). -5. **Seek beyond buffer**: if seek target is past what's decoded, client issues `GET api/track/{id}` with `Range: bytes={byteOffset}-`. Server streams raw bytes from that file-absolute offset with a `206 Partial Content` response. Player retains the parsed WAV header and feeds the raw PCM continuation into the existing decode pipeline. +5. **Seek beyond buffer**: if seek target is past what's decoded, `StreamingAudioPlayerService` issues a new bounded segment starting at the seek byte offset. Server responds 206; player retains the parsed header and feeds the raw continuation into the existing decode pipeline. + +**Memory bounding (three complementary layers, all required):** +- **Raw-queue bound (`StreamDecoder`):** `releaseConsumedChunks()` front-compacts `rawChunks` after each aligned segment is decoded, using a `discardedBytes` absolute cursor so all offsets remain absolute even as the array's front moves. Without this, a long WAV (e.g. a 92-min mix ≈ 970 MB raw) accumulates its entire decoded-from body in `rawChunks` regardless of the decode-side bounds below. +- **Decoded-queue bound (`PlaybackScheduler`):** `evictPlayedBuffers()` discards already-played `AudioBuffer`s, capping the scheduler's forward fill to a 96 MB ceiling (Phase 21.2). Decoded PCM is larger than source (Web Audio uses 32-bit float — a 16-bit stereo WAV roughly doubles; Opus decodes to the same float footprint). +- **Network bound (segmented fetch, Phase 21 Direction B):** The forward stream is fetched as sequential bounded `bytes=cursor-{cursor+4MB-1}` Range requests via `RunSegmentedStreamAsync`; the next segment is fetched only after `DrainBackpressureAsync` confirms the scheduler is below low-water. Because each segment is fully consumed before the next is issued, the browser holds at most ~one segment of raw bytes. A per-chunk drain inside the segment loop (gated on `_streamingPlaybackStarted` so it cannot deadlock first-audio) additionally prevents high-density codecs (e.g. Opus, where a 4 MB segment is ~100 s of audio) from decoding the whole segment eagerly ahead of the playhead before the inter-segment gate runs. + +**Playback stability invariants (streaming-stabilization arc):** +- **Back-pressure water marks:** forward fill 60s high / 30s low (`PlaybackScheduler.DEFAULT_FORWARD_HIGH_WATER_SECONDS` / `...LOW_WATER_SECONDS`). Production pauses on `lookahead ≥ high OR decoded bytes > 96 MB ceiling`, whichever first. The time window is a jitter cushion for Opus's async decode ramp; the byte cap is the hard OOM guarantee and is unchanged. +- **Genuine end-of-playback:** `PlaybackScheduler` uses a `streamComplete` flag (set by `setStreamComplete`) combined with an `underrun_` park/resume state to distinguish a drained-but-still-streaming queue (startup/underrun gap → park, resume on refill) from a truly finished track (stream complete AND queue drained → `finishPlayback()`, the single genuine-end path). Prevents the false `onPlaybackEnded` that previously fired when the Opus WebCodecs queue momentarily drained during the decode ramp. +- **Rebuffer hysteresis:** Opus playback start and underrun-resume are gated on `hasMinimumPlaybackLead()` (1s decoded lead, `DEFAULT_MIN_PLAYBACK_LEAD_SECONDS`); WAV keeps `hasMinimumBuffers(6)`. `streamComplete` overrides the gate so a short tail still plays out. `StreamingAudioPlayerService.TryStartPlaybackAsync()` force-starts after `MarkStreamCompleteAsync` when the threshold was never crossed (ultra-short track protection). +- **Opus `AudioContext` pre-aligned to 48 kHz** in `AudioPlayer.initializeStreaming` before any bytes flow, eliminating the mid-decode context teardown that previously OOM'd the tab under software rendering. + +**Visualizer / decode contention (decodePressure + hwAccel):** +- `DeepDrftPublic/Interop/audio/decodePressure.ts` exports a shared `DecodePressureSignal` singleton. The audio pipeline (`OpusStreamDecoder` yield-cap events, `PlaybackScheduler` underrun-parking events) calls `report()` on sustained lag; the visualizer calls `isUnderPressure()` each rAF frame. The signal engages only after ≥ 5 reports within 2500 ms with a 1s minimum hold (hysteresis); a lone startup-ramp blip never engages. +- Under pressure, `WaveformVisualizer.ts` throttles its rAF loop to ~15 fps (`PRESSURE_THROTTLE_FRAME_MS = 1000/15`), cutting main-thread WebGL software-render + physics cost so WebCodecs decode recovers. A no-op under HW accel. +- `DeepDrftPublic/Interop/visualizer/hwAccel.ts` probes `WEBGL_debug_renderer_info.UNMASKED_RENDERER_WEBGL` for software-renderer signatures (SwiftShader, llvmpipe, softpipe, Microsoft Basic Render, etc.). Absence of the debug extension is treated as accelerated — lava is disabled only on positive evidence. On first interactive render, `WaveformVisualizerControlState.ApplyCapabilityDefault(bool)` applies a one-time scoped default: `LavaEnabled = false` (the expensive subsystem) when no HW accel is detected, `WaveformEnabled` stays on. Guarded by `_capabilityDefaultApplied`; never overrides an explicit user toggle. +- **Known limitation / deferred escalation:** HW-accel-off Opus playback is made usable by defaulting lava off and throttling under pressure. If starvation recurs (e.g. waveform-only path also proves too costly, or a software renderer slips the probe), the documented next step is moving Opus WebCodecs decode off the main thread (Web Worker / AudioWorklet) so it stops competing with main-thread rendering. Not a current implementation — a recorded fallback. Keep this seam clean — it is the most architecturally load-bearing part of the playback path. diff --git a/COMPLETED.md b/COMPLETED.md index 39b6ebe..52ef8fe 100644 --- a/COMPLETED.md +++ b/COMPLETED.md @@ -47,6 +47,83 @@ Newest entries at the top. Group by phase/wave header (mirroring `PLAN.md` / `CM --- +## Phase 21 — Windowed Streaming Buffer (landed 2026-06-24) + +**Landed:** 2026-06-24 on `streaming-overhaul`. + +- **What:** Bounded client memory for long audio streams. Playing a 1 GB+ DJ Mix (the Phase 9 `Mix` medium — a single long track) no longer accumulates the full decoded PCM in the browser; instead the player holds only a sliding forward window and discards what has already played. Four waves, all landed. Public listener site only (`DeepDrftPublic.Client` player stack + `DeepDrftPublic` TypeScript audio interop); no CMS, no API endpoint, no schema change. + +- **Why:** The `PlaybackScheduler` held an `AudioBuffer[]` it never evicted — both decode paths (`StreamDecoder`/`IFormatDecoder` for WAV/MP3/FLAC, and `OggDemuxer` → `OpusStreamDecoder` for Opus) pushed into it without limit. Decoded PCM is larger than the source (Web Audio is 32-bit float per sample/channel — a 16-bit stereo WAV roughly doubles once decoded; a low-data Opus mix decodes to the same float footprint regardless of how few compressed bytes arrived), so a 1 GB WAV or Opus mix could accumulate ~2 GB of retained float in the browser. The Opus path also had a second upstream accumulation locus: the WebCodecs `AudioDecoder` work queue and `decodedQueue: AudioData[]`. + +- **Shape (by wave):** + - **21.1 — Partial eviction in `PlaybackScheduler` (cold-start; load-bearing; shared by both paths).** Drop already-played buffers while keeping position/index/time-anchor bookkeeping exact against a buffer array that no longer begins at absolute time 0. Written once, serves both decode paths (they `addBuffer` identically). This is the hardest correctness work in the phase. No refill yet. + - **21.2 — Back-pressure (one fill signal, two throttle sites).** Bound the unplayed region by stopping production above a high-water mark and resuming below low-water, driven by `PlaybackScheduler.evaluateProductionPause()` — a single shared scheduler-fill signal (OQ6/OQ7 resolved: shared controller, per-path hook). **21.2a** — `StreamingAudioPlayerService` stops fetching the next segment above high-water, polling `evaluateProductionPause()` at 100 ms cadence until the fill drains below low-water (serves both paths). **21.2b** — the Opus demux/decode feed is additionally stopped when the same signal is set, so the WebCodecs decode queue and `decodedQueue` do not balloon behind a paused segment loop (Opus only; no WAV analogue). Together with 21.1 this bounds both the played and unplayed sides on both formats. + - **21.3 — Seek-back-past-window refill + clean failure recovery.** When a backward seek lands earlier than the retained tail, the existing seek-beyond-buffer path is reused pointed at the earlier offset (whichever resolver the active path ships: `IFormatDecoder`/`StreamDecoder` for WAV; the live `resolveOpusByteOffset` + `OpusStreamDecoder.reinitializeForRangeContinuation` for Opus). Minimal AC6 refill-failure handling added: `RecoverFromFailedRefill` clears the scheduler and surfaces a user-visible error rather than leaving the player wedged. + - **21.4 — Validation + Direction A→B pivot (the critical wave — see as-built divergence below).** Network-memory bounding confirmed in Daniel's browser run: the segmented approach delivers ~4 MB segments pacing with playback, with the browser holding ~one segment of raw bytes rather than the full artifact. + +- **As-built divergence — Direction A→B pivot (important).** The spec recommended **Direction A** (sliding window on one open-ended forward stream, relying on pausing `ReadAsync` / the segment loop to backpressure the socket) with **Direction B** (discrete bounded `Range: bytes=start-end` segments, each fetched only when the scheduler drains below low-water) held as the documented fallback "if Direction A's back-pressure proves leaky in practice." + + **21.4 browser validation proved it leaky.** On Blazor WASM, the browser `fetch` API buffers the entire HTTP response body regardless of how slowly the application reads it: pausing reads bounded the *decode* but not the *network download*, so the entire ~970 MB body accumulated in browser memory even though the application only decoded a window of it. Direction A's core assumption — that pausing `ReadAsync` closes the TCP flow-control window before the browser caches the whole body — does not hold in the WASM `fetch` runtime. + + **We pivoted to Direction B.** The forward stream now issues sequential bounded `bytes=cursor-(cursor+SegmentSizeBytes-1)` Range requests (`SegmentSizeBytes = 4 MB`), each fetched only after the scheduler drains below low-water — via `RunSegmentedStreamAsync` in `StreamingAudioPlayerService`. Because each 4 MB request is fully consumed before the next is issued, the browser holds at most ~one segment of raw bytes regardless of file size. The decode-side windowing (21.1/21.2) pairs with it: the segment loop's segment-gate replaces the raw `ReadAsync` pause as the production throttle (21.2a), and the Opus decode-ahead throttle (21.2b) hooks the same `evaluateProductionPause()` signal. Seek and refill converge on the same segmented loop. Direction B is recorded as the shipped approach; Direction A is recorded as tried-in-validation and found insufficient for the WASM `fetch` runtime. + +- **What Phase 21 does NOT include:** the full AC matrix re-run (Opus seek-storm, visualizer, rapid-seek concurrency under the new segmented loop) beyond the network-memory bounding confirmed in 21.4. Those acceptance criteria remain the validation baseline for any follow-on work touching this seam. + +- **Design memo:** `product-notes/phase-21-windowed-streaming-buffer.md` (note: the spec recommended Direction A; the as-built pivot to Direction B is annotated at the top of that doc and in §3.2/§3.3). + +### Phase 21 — Post-landing: raw-queue and per-chunk bounds (landed 2026-06-24) + +**Landed:** 2026-06-24 on `streaming-overhaul`. Fixes a Chrome tab crash (main-process OOM with hardware acceleration OFF) on long-track playback. Two unbounded memory queues remained after Phase 21 landed; this patch adds complementary bounds on both. + +1. **`StreamingAudioPlayerService` — per-chunk drain inside segments.** Phase 21 Direction B's inter-segment back-pressure gate is matched to WAV byte density (~24 s per 4 MB segment), but not to Opus: at 320 kbps a 4 MB segment is ~100 s of decodable audio. The inner chunk loop had no fill check, so the entire segment decoded eagerly ahead of the playhead — piling decoded f32 PCM into main-process RAM before the inter-segment gate ever ran. Fix: an extracted `DrainBackpressureAsync` helper (shared with the inter-segment gate) is called per chunk once the scheduler is over high-water, gated on `_streamingPlaybackStarted` so it cannot deadlock first-audio (the playhead doesn't advance until playback starts, so draining to low-water would block indefinitely before the first buffer plays). Also added load-generation diagnostic logging (`_loadGeneration`) to confirm single-load-per-play-action in-browser. + +2. **`StreamDecoder.ts` — `releaseConsumedChunks()` front-compaction of `rawChunks`.** `rawChunks` retained the entire decoded-from stream body because consumed chunks were never released — a 92-min WAV mix (≈ 970 MB raw) accumulated all of it in the browser. Phase 21.2's 96 MB decoded-side ceiling does not reach this raw queue. Fix: `releaseConsumedChunks()` walks `rawChunks` after each aligned segment is decoded and splices off all fully-consumed chunks (any chunk whose end ≤ `processedBytes`), advancing `discardedBytes` (an absolute cursor into the logical stream) so `extractAlignedData` can continue walking from the correct position even though `rawChunks[0]` no longer begins at byte 0. This is the raw-side analogue of `PlaybackScheduler.evictPlayedBuffers` (the decoded side). + +Together with the existing decoded-side 96 MB ceiling (Phase 21.2) and the segmented fetch (Phase 21 Direction B), these complete the three-layer memory bound: raw queue, decoded queue, and network. + +### Phase 21 — Post-landing: streaming stabilization (landed 2026-06-26) + +**Landed:** 2026-06-26 on `streaming-overhaul`. Resolves playback instability that surfaced specifically with Opus + hardware acceleration off: the Opus underrun/false-end cycle, visualizer starvation of WebCodecs decode, and an OOM path from mid-decode `AudioContext` teardown. All changes are TypeScript-only (`DeepDrftPublic/Interop/audio/` + `DeepDrftPublic/Interop/visualizer/`) plus the C# service that calls them; no API, no schema, no CMS change. + +1. **Back-pressure water marks widened to 60s high / 30s low.** `PlaybackScheduler.ts` `DEFAULT_FORWARD_HIGH_WATER_SECONDS` / `DEFAULT_FORWARD_LOW_WATER_SECONDS` doubled from 30/15. The 96 MB decoded-byte ceiling is unchanged and remains the hard OOM bound; production pauses on `lookahead ≥ high OR bytes > cap`, whichever first. The wider time window absorbs per-packet WebCodecs decode jitter (Opus 48 kHz stereo ≈ 0.37 MB/s → 60 s ≈ 23 MB — well within the cap) without false back-pressure on the Opus decode ramp. + +2. **End-of-playback gated on genuine completion (`streamComplete` + `underrun_`).** `PlaybackScheduler` now distinguishes a drained-but-still-streaming queue (startup/underrun gap) from a truly finished track (stream complete AND queue drained). `streamComplete` (set by `setStreamComplete`, called from `AudioPlayer.markStreamComplete`) is the discriminator. When the queue drains while `streamComplete` is false the scheduler parks in `underrun_` state (anchors frozen, `isActive_` false) and resumes from `scheduleNewBuffers` when buffers arrive. `finishPlayback()` is the single genuine-end path, reached only when both conditions hold simultaneously. Previously a drained Opus queue during the WebCodecs startup ramp fired `onPlaybackEnded`, nulling Duration and advancing the queue. + +3. **Rebuffer hysteresis — 1s decoded-lead gate.** `hasMinimumPlaybackLead()` gates Opus playback START and underrun RESUME (`DEFAULT_MIN_PLAYBACK_LEAD_SECONDS = 1.0`), preventing the decode-ramp thrash (resume on a 20 ms Opus packet → drain immediately → re-park → repeat). `streamComplete` overrides the gate so a short tail plays out rather than parking forever. WAV keeps its `hasMinimumBuffers(6)` buffer-count start gate unchanged — large synchronous WAV segments rarely underrun and that gate's character must not change. **Complete-without-start force-start:** `StreamingAudioPlayerService.TryStartPlaybackAsync()` is called after `MarkStreamCompleteAsync` when `_streamingPlaybackStarted` is still false, preventing an ultra-short Opus track (total audio below the 1s lead) from sitting loaded-but-not-playing. + +4. **Opus `AudioContext` pre-aligned to 48 kHz.** `AudioPlayer.initializeStreaming` calls `contextManager.recreateWithSampleRate(OPUS_SAMPLE_RATE)` when the content type resolves as Opus, before any bytes flow. The context is already at 48 kHz when the decoder's own lazy `recreateWithSampleRate` check runs, so it short-circuits (no-op). Eliminates the mid-decode context teardown-and-rebuild that previously double-decoded the stream and OOM'd the tab under HW accel off. + +5. **`decodePressure` singleton + `WaveformVisualizer` auto-throttle (~15 fps under pressure).** `DeepDrftPublic/Interop/audio/decodePressure.ts` exports a process-wide `DecodePressureSignal` singleton shared between the audio pipeline (producer: `OpusStreamDecoder` yield-cap events and `PlaybackScheduler` underrun-parking events) and the visualizer (consumer). Engages only on sustained stress (≥ 5 `report()` calls within a 2500 ms window) and holds for a 1000 ms minimum dwell before releasing, suppressing lone startup-ramp blips. `WaveformVisualizer.ts` checks `decodePressure.isUnderPressure()` each rAF frame; when engaged it skips frames so at most one renders per `PRESSURE_THROTTLE_FRAME_MS` (`1000/15` ms, ~15 fps), cutting main-thread WebGL software-render + physics cost so WebCodecs decode recovers. A no-op under HW accel — healthy decode never calls `report()` and `isUnderPressure()` stays false indefinitely. + +6. **Hardware-acceleration detection + `ApplyCapabilityDefault`.** `DeepDrftPublic/Interop/visualizer/hwAccel.ts` probes `WEBGL_debug_renderer_info.UNMASKED_RENDERER_WEBGL` for known software-renderer signatures (SwiftShader, llvmpipe, softpipe, Microsoft Basic Render Driver, Mesa Offscreen, "software"). Absence of the debug extension is treated as accelerated — the probe disables lava only on positive evidence of software rendering, not on uncertainty. `WaveformVisualizerControlState.ApplyCapabilityDefault(bool hardwareAccelerated)` is called once per session by the visualizer bridge on first interactive render: when `hardwareAccelerated` is false it sets `LavaEnabled = false` (the expensive main-thread subsystem) while leaving `WaveformEnabled` on, coerces Theater Mode, and raises `Changed` once so all observers see the default in a single cycle. Guarded by `_capabilityDefaultApplied` so it runs at most once per session and never overrides an explicit in-session user toggle. With HW accel present the call is a no-op. + +--- + +## Phase 18 — Opus Low-Data Streaming (landed 2026-06-23) + +_Note: Two distinct efforts share the "Phase 18" label — phase numbers are organisational, not sequential. See also **Phase 18 — Theme / Dark-Mode Remediation (landed 2026-06-19)** below._ + +**Landed:** 2026-06-23 on `streaming-overhaul`. + +- **What:** Dual-format audio delivery — the existing lossless WAV path plus a new low-data Ogg Opus (fullband, 320 kbps) path — giving listeners a quality choice. Opus is the default (capability-gated). Six waves, all landed. + +- **Why:** Lossless WAV works but imposes high bandwidth cost, particularly for long DJ Mixes. Opus at 320 kbps is indistinguishable for typical listening and an order of magnitude smaller, making the site usable on metered or constrained connections. The bespoke Web Audio decode→schedule graph is retained by deliberate choice; Opus feeds the same `IFormatDecoder` seam rather than an HTML `` element or MSE. + +- **Shape (by wave):** + - **18.1 — Ingest transcode + seek-index sidecar.** New `OpusTranscodeService`/processor in `DeepDrftContent`, invoked post-store from `UnifiedTrackService.UploadAsync` as a background job (failure-tolerant; the track plays lossless first). Produces the Ogg Opus 320 artifact and a combined seek/setup **sidecar** (0.5 s-bucketed granule→byte index + `OpusHead`/`OpusTags` setup header), both stored in a new `track-opus` vault. **ffmpeg/libopus** added to the deploy bootstrap as a runtime prereq on the API host. + - **18.2 — Storage + lookup contract.** Server-side "given `EntryKey` + format, return the right artifact + content-type, with lossless fallback when no Opus artifact exists." + - **18.3 — Delivery: `?format=opus|lossless` + sidecar serving + proxy threading.** `?format=` param on the stream endpoint (Range preserved); sidecar-serving endpoint; param forwarded through `TrackProxyController`; `TrackMediaClient` sends the format param. + - **18.4 — Client Opus decoder + index-based seek + capability detection.** New `OpusFormatDecoder` (`IFormatDecoder`): Ogg-page demux, setup-header carry from the cached sidecar, `calculateByteOffset` binary-searching the precomputed seek index (VBR-safe accurate seek — not interpolation). Capability gate falls back to the lossless path on browsers without Opus support. Testing-phase fixes also landed: a valid ffmpeg-generated capability-probe sample and correct duration reporting (fixed the seekbar + visualizer in Opus mode). + - **18.5 — End-to-end integration.** Player format selection (Opus-by-default, capability-gated); sidecar fetch/inject on track load; **Backfill-Opus** CMS bulk action (third sibling to Generate-Profiles / Backfill-High-res); replace-audio regenerates Opus + sidecar. + - **18.6 — Settings menu + quality toggle + CMS status.** New public-site **Settings menu** (app-bar trigger + settings-item abstraction + `ListenerSettings`, persisted via the dark-mode-pattern seam: `streamQuality` cookie → prerender-read → `PersistentComponentState` → client cookie service); streaming-quality toggle (Low-data/Lossless) as its first occupant. CMS **Opus-status badge** on the track grid. CMS upload meter **Post-Processing** phase visualizing background Opus transcode. + +- **As-built divergence — the Opus decoder (important).** The originally-spec'd "segment-and-decode-each-page via `decodeAudioData`" model was implemented and then **replaced**: browser testing showed it glitches at every segment boundary and broke seeking (Opus has pre-skip + inter-frame state, so independent per-segment decode is architecturally wrong). The landed decoder is a **WebCodecs `AudioDecoder` streaming pipeline** (Ogg demux → packets → stateful continuous decode), with index-driven seek and a WebCodecs-based capability gate. Daniel's call at the decision point — offered a decode-whole stopgap (Option A) vs. the proper WebCodecs streaming decoder (Option B), he chose B: "Let's go for B, why do you like half-solutions so much" (the same no-stopgap posture as the Mix visualizer WebGL2 decision, where he said "no stopgap — WebGL as step 1, no pussyfooting"). The segmented approach is recorded as tried-and-replaced. + +- **What Phase 18 does NOT include:** windowed / bounded-memory streaming. Phase 18 still fetches the whole (smaller) Opus artifact; bounding client memory to a sliding forward window is **Phase 21's job**. + +- **Design memo:** `product-notes/phase-18-opus-low-data-streaming.md`. +--- + ## Phase 20 — Theater Mode (landed 2026-06-20) **Landed:** 2026-06-20 on dev. Pending: final manual browser/GPU smoke-test on dev. @@ -76,6 +153,8 @@ Newest entries at the top. Group by phase/wave header (mirroring `PLAN.md` / `CM ## Phase 18 — Theme / Dark-Mode Remediation (landed 2026-06-19) +_Note: Two distinct efforts share the "Phase 18" label — phase numbers are organisational, not sequential. See also **Phase 18 — Opus Low-Data Streaming (landed 2026-06-23)** above._ + **Landed:** 2026-06-19 on dev (Wave 1 + Wave 2 + Wave 3). - **What:** A DRY token pass resolving six theming symptoms (five in dark mode, one in light) that all traced to three root causes: neutral page surfaces bound to constant brand tokens, the play chip bound to a constant light-grey, and no theme-aware popover-surface token. Resolved as one coherent pass via a shared token layer rather than per-component patches. diff --git a/DeepDrftAPI/Controllers/TrackController.cs b/DeepDrftAPI/Controllers/TrackController.cs index 89e7a9d..bbfc13c 100644 --- a/DeepDrftAPI/Controllers/TrackController.cs +++ b/DeepDrftAPI/Controllers/TrackController.cs @@ -5,6 +5,7 @@ using DeepDrftContent.Constants; using DeepDrftContent.FileDatabase.Services; using DeepDrftContent.FileDatabase.Models; using DeepDrftContent.Processors; +using DeepDrftContent.Processors.Opus; using DeepDrftData; using DeepDrftModels.DTOs; using DeepDrftModels.Enums; @@ -20,6 +21,7 @@ public class TrackController : ControllerBase private readonly UnifiedTrackService _unifiedService; private readonly ITrackService _sqlTrackService; private readonly WaveformProfileService _waveformProfileService; + private readonly TrackFormatResolver _formatResolver; private readonly UploadStagingDirectory _stagingDirectory; private readonly ILogger _logger; @@ -35,6 +37,7 @@ public class TrackController : ControllerBase UnifiedTrackService unifiedService, ITrackService sqlTrackService, WaveformProfileService waveformProfileService, + TrackFormatResolver formatResolver, UploadStagingDirectory stagingDirectory, ILogger logger) { @@ -43,6 +46,7 @@ public class TrackController : ControllerBase _unifiedService = unifiedService; _sqlTrackService = sqlTrackService; _waveformProfileService = waveformProfileService; + _formatResolver = formatResolver; _stagingDirectory = stagingDirectory; _logger = logger; } @@ -245,6 +249,40 @@ public class TrackController : ControllerBase return Ok(status); } + // GET api/track/opus-status ([ApiKeyAuthorize]) + // Admin Post-Processing view (18.6): returns every track with a flag for whether it carries a COMPLETE + // Opus artifact — both the Opus audio AND the seek/setup sidecar present (TrackFormatResolver.HasOpusAsync, + // the same completeness rule the 18.5 Backfill-Opus pass enqueues against; a half-derived track counts as + // missing). Mirrors GET waveform-status exactly: same ApiKey auth, same unpaged whole-catalogue shape, same + // literal-route placement before "{trackId}". The CMS reads it to show the Backfill-Opus "missing N" badge + // and to poll per-track Post-Processing status after an upload. + [ApiKeyAuthorize] + [HttpGet("opus-status")] + public async Task GetOpusStatus() + { + var tracks = await _sqlTrackService.GetAll(); + if (!tracks.Success || tracks.Value is null) + { + var error = tracks.Messages.FirstOrDefault()?.Message ?? "Unknown error"; + _logger.LogError("GetOpusStatus failed to load tracks: {Error}", error); + return StatusCode(500, "Failed to load tracks"); + } + + var status = new List(tracks.Value.Count); + foreach (var track in tracks.Value) + { + status.Add(new OpusStatusDto + { + TrackId = track.Id, + EntryKey = track.EntryKey, + TrackName = track.TrackName, + HasOpus = await _formatResolver.HasOpusAsync(track.EntryKey), + }); + } + + return Ok(status); + } + // POST api/track/duration/backfill ([ApiKeyAuthorize], no body) // One-time admin backfill: for every track whose SQL duration is still null, read the duration from // the vault audio and write it to SQL. Mirrors the waveform backfill posture. Idempotent — a re-run @@ -265,6 +303,27 @@ public class TrackController : ControllerBase return Ok(new { updated = result.Value.Updated, skipped = result.Value.Skipped }); } + // POST api/track/opus/backfill ([ApiKeyAuthorize], no body) + // Backfill-Opus (18.5, OQ4): enqueue a background Opus derive for every track lacking a complete Opus + // artifact (audio + sidecar). Mirrors the duration-backfill posture — enqueue-only and non-blocking, the + // transcodes run on the shared serial worker. Idempotent: a re-run only schedules tracks still missing + // Opus. Returns { enqueued, skipped }. Declared in the literal-route block (before "{trackId}") so the + // "opus/backfill" segment is never treated as a trackId; distinct shape from "{trackId}/opus" (per-track). + [ApiKeyAuthorize] + [HttpPost("opus/backfill")] + public async Task BackfillOpus(CancellationToken cancellationToken) + { + var result = await _unifiedService.BackfillOpusAsync(cancellationToken); + if (!result.Success) + { + var error = result.Messages.FirstOrDefault()?.Message ?? "Unknown error"; + _logger.LogError("BackfillOpus failed: {Error}", error); + return StatusCode(500, error); + } + + return Ok(new { enqueued = result.Value.Enqueued, skipped = result.Value.Skipped }); + } + // POST api/track/upload: raw audio in (multipart/form-data) + metadata → persisted TrackDto out. // Accepts .wav, .mp3, and .flac. Used by the CMS upload flow on DeepDrftManager; that host // proxies the upload here so it never touches the vault disk path or SQL directly. @@ -642,10 +701,27 @@ public class TrackController : ControllerBase // --- Parameterized routes --- + // GET api/track/{trackId}?format=opus|lossless (unauthenticated) + // Streams the track's audio bytes with HTTP Range support. The optional `format` selector (Phase 18.3) + // picks the delivery rendering: absent or unrecognized ⇒ Lossless (byte-identical to pre-Phase-18 — + // the existing zero-copy disk-stream path, untouched); `opus` ⇒ the derived Ogg Opus 320 artifact + // when present, falling back to lossless when it is not (C2 — never 404/silence). The Opus path serves + // the resolved in-memory bytes via File(..., enableRangeProcessing: true) so Range: bytes=X- still + // yields 206 (load-bearing for streaming + seek), matching the lossless disk-stream's range behavior. [HttpGet("{trackId}")] - public async Task GetTrack(string trackId) + public async Task GetTrack(string trackId, [FromQuery] string? format = null) { - _logger.LogInformation("GetTrack called with trackId: {TrackId}", trackId); + _logger.LogInformation("GetTrack called with trackId: {TrackId}, format: {Format}", trackId, format); + + // Only `opus` diverges from today's behavior; everything else (null, "lossless", garbage) takes the + // unchanged lossless disk-stream path below, preserving the large-file zero-copy streaming. Routing + // lossless through the resolver would force the whole source (up to ~1 GB) into memory per request — + // a regression the resolver's in-memory byte[] result is fine for Opus (small) but not for lossless. + if (Enum.TryParse(format, ignoreCase: true, out var requestedFormat) + && requestedFormat == AudioFormat.Opus) + { + return await GetTrackOpusAsync(trackId); + } try { @@ -700,6 +776,58 @@ public class TrackController : ControllerBase } } + // The ?format=opus arm of GetTrack. Resolves the Opus artifact (or the lossless fallback when none + // exists, C2) via TrackFormatResolver and serves the resolved bytes with explicit range processing. + // enableRangeProcessing:true is the load-bearing detail the 18.2 reviewer flagged: File(byte[], ...) + // does NOT get ASP.NET's automatic range handling unless asked, so without this flag a Range: bytes=X- + // would silently return the whole body as 200 instead of a 206 slice — breaking seek for the Opus path + // (and Phase 21 windowing). The resolver reports the *actually-served* format via ResolvedAudio, so the + // content-type matches the bytes (audio/ogg on a hit, the source MIME on a fallback) and the eventual + // client decoder dispatches correctly. + private async Task GetTrackOpusAsync(string trackId) + { + try + { + var resolved = await _formatResolver.ResolveAsync(trackId, AudioFormat.Opus); + if (resolved is null) + { + _logger.LogWarning("Track not found for Opus request: {TrackId}", trackId); + return NotFound(); + } + + _logger.LogInformation( + "Streaming track {TrackId} as {Format} ({Size} bytes, {ContentType})", + trackId, resolved.ResolvedFormat, resolved.Audio.Buffer.Length, resolved.ContentType); + + return File(resolved.Audio.Buffer, resolved.ContentType, enableRangeProcessing: true); + } + catch (Exception ex) + { + _logger.LogError(ex, "Error retrieving track as Opus: {TrackId}", trackId); + return StatusCode(500, "Internal server error"); + } + } + + // GET api/track/{trackId}/opus/seekdata (unauthenticated) + // Returns the Opus setup-header + granule→byte seek-index sidecar bytes (Phase 18.3). The client + // fetches this once on track load and parses it into OpusSeekData (18.4) before issuing any Opus seek. + // Raw octet-stream — the bytes are the OpusSidecar blob exactly as 18.1 stored them. 404 when no sidecar + // is stored (no Opus artifact yet, or an older derive predating the sidecar); the client then degrades + // to lossless, mirroring the C2 posture of the audio path. Same public auth posture as the audio stream. + // The "opus/seekdata" literal suffix keeps this distinct from the audio and waveform routes. + [HttpGet("{trackId}/opus/seekdata")] + public async Task GetOpusSeekData(string trackId) + { + var sidecar = await _formatResolver.GetOpusSidecarAsync(trackId); + if (sidecar is null) + { + _logger.LogInformation("No Opus sidecar for track: {TrackId}", trackId); + return NotFound(); + } + + return File(sidecar, "application/octet-stream"); + } + // GET api/track/{trackId}/waveform (unauthenticated) // Returns the stored waveform loudness profile for a track, base64-encoded. Public listener // data, same auth posture as GET api/track/{trackId} streaming. 404 when no profile is stored @@ -816,6 +944,32 @@ public class TrackController : ControllerBase return Ok(); } + // POST api/track/{trackId}/opus ([ApiKeyAuthorize]) + // Per-track Opus (re)derive trigger (18.5): schedule a single track's background transcode. Enqueue-only + // and non-blocking — the transcode runs on the shared serial worker; this returns as soon as it is + // scheduled. Re-runnable: overwrites any prior artifact in place. trackId is the EntryKey. 404 when the + // track id is unknown. The "opus" literal suffix keeps this distinct from the audio/waveform routes and + // from the parameterized PUT "{trackId}". Returns 202 Accepted — the work is queued, not done inline. + [ApiKeyAuthorize] + [HttpPost("{trackId}/opus")] + public async Task GenerateOpus(string trackId, CancellationToken cancellationToken) + { + var result = await _unifiedService.EnqueueOpusAsync(trackId, cancellationToken); + if (result.Success) + { + return Accepted(); + } + + var error = result.Messages.FirstOrDefault()?.Message ?? "Unknown error"; + if (string.Equals(error, UnifiedTrackService.TrackNotFoundMessage, StringComparison.Ordinal)) + { + return NotFound(); + } + + _logger.LogError("GenerateOpus failed for {TrackId}: {Error}", trackId, error); + return StatusCode(500, error); + } + [ApiKeyAuthorize] [HttpPut("{trackId}")] public async Task PutTrack(string trackId, [FromBody] AudioBinaryDto track) diff --git a/DeepDrftAPI/Program.cs b/DeepDrftAPI/Program.cs index 57ae729..6838e2a 100644 --- a/DeepDrftAPI/Program.cs +++ b/DeepDrftAPI/Program.cs @@ -4,6 +4,7 @@ using DeepDrftAPI; using DeepDrftAPI.Middleware; using DeepDrftAPI.Models; using DeepDrftAPI.Services; +using DeepDrftAPI.Services.Opus; using DeepDrftData; using DeepDrftData.Data; using DeepDrftData.Repositories; @@ -66,6 +67,13 @@ builder.Services .AddScoped(sp => sp.GetRequiredService()); builder.Services.AddScoped(); +// Background Opus transcode (Phase 18.1, OQ6). One singleton is both the enqueue seam +// (IOpusTranscodeQueue, injected into the scoped UnifiedTrackService) and the hosted drain loop +// (IHostedService). It resolves OpusTranscodeService — a singleton — so no scope is captured. +builder.Services.AddSingleton(); +builder.Services.AddSingleton(sp => sp.GetRequiredService()); +builder.Services.AddHostedService(sp => sp.GetRequiredService()); + // Phase 16 anonymous telemetry — append-only event logs + incremental play-counter rollup (all SQL). // EventManager is the IEventService boundary; EventRepository owns the EF writes and the // release-resolution + counter-bump transaction. diff --git a/DeepDrftAPI/Services/Opus/IOpusTranscodeQueue.cs b/DeepDrftAPI/Services/Opus/IOpusTranscodeQueue.cs new file mode 100644 index 0000000..b30f317 --- /dev/null +++ b/DeepDrftAPI/Services/Opus/IOpusTranscodeQueue.cs @@ -0,0 +1,18 @@ +namespace DeepDrftAPI.Services.Opus; + +/// +/// The enqueue seam for the background Opus transcode (OQ6 / §3.1a). +/// depends only on this thin interface — not on the worker — so adding the background derive to the +/// upload/replace paths costs one small dependency, not the whole transcode graph. Enqueuing is +/// non-blocking and best-effort: a freshly uploaded track is already persisted and playable losslessly +/// before anything is enqueued, and the transcode runs off the request thread. +/// +public interface IOpusTranscodeQueue +{ + /// + /// Schedules a background Opus derive for the track identified by . Returns + /// immediately. A dropped or failed enqueue must not affect the caller — the track remains + /// lossless-only and eligible for backfill. + /// + void Enqueue(string entryKey); +} diff --git a/DeepDrftAPI/Services/Opus/OpusTranscodeBackgroundService.cs b/DeepDrftAPI/Services/Opus/OpusTranscodeBackgroundService.cs new file mode 100644 index 0000000..2ff077f --- /dev/null +++ b/DeepDrftAPI/Services/Opus/OpusTranscodeBackgroundService.cs @@ -0,0 +1,72 @@ +using System.Threading.Channels; +using DeepDrftContent.Processors.Opus; + +namespace DeepDrftAPI.Services.Opus; + +/// +/// The background worker behind (OQ6 / §3.1a). An unbounded in-process +/// channel buffers EntryKeys enqueued by the upload and replace-audio paths; a single hosted loop drains +/// them one at a time and runs for each. Serial +/// by design — a transcode is CPU-heavy (§3.1), so running them concurrently would starve request +/// handling; one-at-a-time keeps the derive strictly off the hot path without saturating the host. +/// +/// This worker IS the queue (implements ) so enqueue and drain share one +/// channel with no extra indirection. It is registered as a singleton and surfaced under both the +/// interface and . +/// +public sealed class OpusTranscodeBackgroundService : BackgroundService, IOpusTranscodeQueue +{ + private readonly Channel _channel = + Channel.CreateUnbounded(new UnboundedChannelOptions { SingleReader = true }); + + private readonly OpusTranscodeService _transcodeService; + private readonly ILogger _logger; + + public OpusTranscodeBackgroundService( + OpusTranscodeService transcodeService, + ILogger logger) + { + _transcodeService = transcodeService; + _logger = logger; + } + + public void Enqueue(string entryKey) + { + if (string.IsNullOrWhiteSpace(entryKey)) + return; + + if (!_channel.Writer.TryWrite(entryKey)) + { + // Unbounded writer only rejects after Complete(), i.e. during shutdown. The track stays + // lossless-only and is eligible for backfill, so a dropped enqueue is non-fatal — log it. + _logger.LogWarning("Opus transcode: could not enqueue {EntryKey} (queue closed).", entryKey); + } + } + + protected override async Task ExecuteAsync(CancellationToken stoppingToken) + { + await foreach (var entryKey in _channel.Reader.ReadAllAsync(stoppingToken)) + { + try + { + await _transcodeService.TranscodeAndStoreAsync(entryKey, stoppingToken); + } + catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested) + { + break; // host shutting down + } + catch (Exception ex) + { + // TranscodeAndStoreAsync already swallows expected failures; this guards the loop against + // anything unexpected so one bad track never kills the worker. + _logger.LogError(ex, "Opus transcode: unhandled failure draining {EntryKey}; worker continues.", entryKey); + } + } + } + + public override Task StopAsync(CancellationToken cancellationToken) + { + _channel.Writer.TryComplete(); + return base.StopAsync(cancellationToken); + } +} diff --git a/DeepDrftAPI/Services/UnifiedTrackService.cs b/DeepDrftAPI/Services/UnifiedTrackService.cs index 4023955..7a86a73 100644 --- a/DeepDrftAPI/Services/UnifiedTrackService.cs +++ b/DeepDrftAPI/Services/UnifiedTrackService.cs @@ -1,6 +1,8 @@ +using DeepDrftAPI.Services.Opus; using DeepDrftContent; using DeepDrftContent.Constants; using DeepDrftContent.Processors; +using DeepDrftContent.Processors.Opus; using DeepDrftData; using DeepDrftModels.DTOs; using DeepDrftModels.Enums; @@ -39,6 +41,8 @@ public class UnifiedTrackService private readonly ITrackService _sqlTrackService; private readonly FileDb _fileDatabase; private readonly WaveformProfileService _waveformProfileService; + private readonly IOpusTranscodeQueue _opusTranscodeQueue; + private readonly TrackFormatResolver _formatResolver; private readonly ILogger _logger; public UnifiedTrackService( @@ -46,12 +50,16 @@ public class UnifiedTrackService ITrackService sqlTrackService, FileDb fileDatabase, WaveformProfileService waveformProfileService, + IOpusTranscodeQueue opusTranscodeQueue, + TrackFormatResolver formatResolver, ILogger logger) { _contentTrackContentService = contentTrackContentService; _sqlTrackService = sqlTrackService; _fileDatabase = fileDatabase; _waveformProfileService = waveformProfileService; + _opusTranscodeQueue = opusTranscodeQueue; + _formatResolver = formatResolver; _logger = logger; } @@ -219,6 +227,11 @@ public class UnifiedTrackService // frontend, so a failure here is logged and swallowed — never fails the upload. await TryStoreWaveformDatumsAsync(unpersisted.EntryKey, ct); + // Schedule the low-data Opus derive (OQ6 / §3.1a): the track is persisted and lossless-playable + // NOW; the transcode + seek-index build run on a background worker. Non-blocking and best-effort + // — the upload response never waits on it, and a transcode failure leaves the track lossless-only. + _opusTranscodeQueue.Enqueue(unpersisted.EntryKey); + return saveResult; } @@ -297,6 +310,11 @@ public class UnifiedTrackService return Result.CreateFailResult("Audio replaced but duration metadata could not be updated."); } + // The stale Opus artifact (if any) no longer matches the new source. Schedule a background + // regenerate — the transcode service overwrites the prior artifacts in place keyed by the same + // EntryKey. Best-effort, off the request thread, mirrors the waveform regen above. + _opusTranscodeQueue.Enqueue(entryKey); + return Result.CreatePassResult(); } @@ -379,6 +397,69 @@ public class UnifiedTrackService return ResultContainer<(int, int)>.CreatePassResult((updated, skipped)); } + /// + /// Backfill-Opus (18.5, OQ4): enqueue a background Opus derive for every non-deleted track that lacks a + /// complete Opus artifact (missing audio OR missing sidecar — a half-derived track is treated as missing + /// and re-derived). Mirrors the duration-backfill posture: enumerate SQL rows, check each against the + /// track-opus vault, schedule the misses. Enqueue-only and non-blocking — the actual transcodes run + /// on the shared background worker, serially (the same queue the upload/replace paths feed), so this + /// returns as soon as the misses are scheduled rather than waiting on CPU-heavy transcodes. Idempotent: + /// a re-run only enqueues tracks still missing Opus, and already-queued/in-flight derives simply overwrite + /// in place. Returns (enqueued, skipped) — skipped = tracks that already have a complete Opus artifact. + /// + public async Task> BackfillOpusAsync(CancellationToken ct) + { + var all = await _sqlTrackService.GetAll(); + if (!all.Success || all.Value is null) + { + var error = all.Messages.FirstOrDefault()?.Message ?? "Unknown error"; + _logger.LogError("BackfillOpusAsync: failed to load tracks: {Error}", error); + return ResultContainer<(int, int)>.CreateFailResult($"Could not load tracks: {error}"); + } + + var enqueued = 0; + var skipped = 0; + foreach (var track in all.Value) + { + ct.ThrowIfCancellationRequested(); + + if (await _formatResolver.HasOpusAsync(track.EntryKey)) + { + skipped++; + continue; + } + + _opusTranscodeQueue.Enqueue(track.EntryKey); + enqueued++; + } + + _logger.LogInformation("BackfillOpusAsync complete: {Enqueued} enqueued, {Skipped} already had Opus.", + enqueued, skipped); + return ResultContainer<(int, int)>.CreatePassResult((enqueued, skipped)); + } + + /// + /// Per-track Opus (re)derive trigger (18.5): schedule a background transcode for one track. Returns false + /// only when the track id is unknown; the enqueue itself is non-blocking and best-effort, like the bulk + /// backfill. Re-runnable — overwrites any prior artifact in place. + /// + public async Task EnqueueOpusAsync(string entryKey, CancellationToken ct) + { + var lookup = await _sqlTrackService.GetByEntryKey(entryKey); + if (!lookup.Success) + { + var error = lookup.Messages.FirstOrDefault()?.Message ?? "Unknown error"; + _logger.LogError("EnqueueOpusAsync: lookup failed for {EntryKey}: {Error}", entryKey, error); + return Result.CreateFailResult("Failed to load track."); + } + + if (lookup.Value is null) + return Result.CreateFailResult(TrackNotFoundMessage); + + _opusTranscodeQueue.Enqueue(entryKey); + return Result.CreatePassResult(); + } + /// /// Delete a track's SQL row, then its vault entry. SQL is the source of truth: a SQL delete /// failure fails the operation (and leaves the vault untouched), but a subsequent vault delete diff --git a/DeepDrftAPI/Startup.cs b/DeepDrftAPI/Startup.cs index f735fd1..2b66f0f 100644 --- a/DeepDrftAPI/Startup.cs +++ b/DeepDrftAPI/Startup.cs @@ -4,6 +4,7 @@ using DeepDrftContent.Constants; using DeepDrftContent.FileDatabase.Models; using DeepDrftContent.FileDatabase.Services; using DeepDrftContent.Processors; +using DeepDrftContent.Processors.Opus; using Microsoft.Extensions.Logging; using NetBlocks.Utilities.Environment; @@ -44,6 +45,7 @@ namespace DeepDrftAPI InitializeTrackVault(db).GetAwaiter().GetResult(); InitializeImageVault(db).GetAwaiter().GetResult(); InitializeTrackWaveformsVault(db).GetAwaiter().GetResult(); + InitializeTrackOpusVault(db).GetAwaiter().GetResult(); return db; }); @@ -65,6 +67,21 @@ namespace DeepDrftAPI Environment.SetEnvironmentVariable("ASPNETCORE_TEMP", stagingPath); builder.Services.AddSingleton(new UploadStagingDirectory(stagingPath)); + // Opus low-data transcode (Phase 18.1). The domain service lives in DeepDrftContent; the host + // owns only the engine config and the background worker. Bitrate/ffmpeg-path come from the + // OpusTranscode config section; StagingPath is forced to the same data-disk staging directory + // the upload path uses so large transcode temp files never land on the /tmp tmpfs. + builder.Services.Configure( + builder.Configuration.GetSection(nameof(OpusTranscodeOptions))); + builder.Services.PostConfigure(o => o.StagingPath = stagingPath); + builder.Services.AddSingleton(); + builder.Services.AddSingleton(); + + // Opus delivery format resolution + sidecar lookup (Phase 18.2). The seam 18.3 calls behind + // the ?format= stream param and the sidecar path. Stateless over the FileDatabase + content + // service singletons; the lossless branch reuses the existing read path unchanged (C2). + builder.Services.AddSingleton(); + return Task.CompletedTask; } @@ -107,5 +124,15 @@ namespace DeepDrftAPI await fileDatabase.CreateVaultAsync(VaultConstants.TrackWaveforms, MediaVaultType.Media); } } + + // Ensure the track-opus vault exists (Phase 18.1). Holds the derived low-data Ogg Opus artifacts + // — the Opus audio bytes and the setup-header + seek-index sidecar — keyed by the track's EntryKey. + private static async Task InitializeTrackOpusVault(FileDatabase fileDatabase) + { + if (!fileDatabase.HasVault(VaultConstants.TrackOpus)) + { + await fileDatabase.CreateVaultAsync(VaultConstants.TrackOpus, MediaVaultType.Audio); + } + } } } \ No newline at end of file diff --git a/DeepDrftAPI/appsettings.json b/DeepDrftAPI/appsettings.json index 6ea22dc..63ede5e 100644 --- a/DeepDrftAPI/appsettings.json +++ b/DeepDrftAPI/appsettings.json @@ -1,9 +1,7 @@ { "Logging": { "LogLevel": { - "Default": "Information", - "Microsoft.AspNetCore": "Warning", - "DeepDrftContent.Controllers.TrackController": "Information" + "Default": "Information" } }, "AllowedHosts": "*", diff --git a/DeepDrftContent/Constants/VaultConstants.cs b/DeepDrftContent/Constants/VaultConstants.cs index ba1d5fc..906b74e 100644 --- a/DeepDrftContent/Constants/VaultConstants.cs +++ b/DeepDrftContent/Constants/VaultConstants.cs @@ -28,4 +28,13 @@ public static class VaultConstants /// The datum resolution is duration-derived (≈333 samples/sec, see WaveformResolution). /// public const string TrackWaveforms = "track-waveforms"; + + /// + /// Vault name for the derived low-data Ogg Opus artifacts, keyed by the track's EntryKey (Phase 18, + /// S2). Holds two entries per track: the Opus audio bytes (.opus) and the combined setup-header + /// + granule→byte seek-index sidecar (.opusidx). Both are best-effort derived artifacts — + /// regenerable, and a track without them still plays losslessly. Distinct from the source tracks + /// vault so the source means exactly one thing (mirrors the track-waveforms precedent). + /// + public const string TrackOpus = "track-opus"; } \ No newline at end of file diff --git a/DeepDrftContent/FileDatabase/Models/MediaModels.cs b/DeepDrftContent/FileDatabase/Models/MediaModels.cs index 44cc961..703d7a5 100644 --- a/DeepDrftContent/FileDatabase/Models/MediaModels.cs +++ b/DeepDrftContent/FileDatabase/Models/MediaModels.cs @@ -206,6 +206,7 @@ public static class MimeTypeExtensions { ".flac", "audio/flac" }, { ".aac", "audio/aac" }, { ".ogg", "audio/ogg" }, + { ".opus", "audio/ogg" }, { ".m4a", "audio/mp4" } }; diff --git a/DeepDrftContent/Processors/Opus/FfmpegOpusEncoder.cs b/DeepDrftContent/Processors/Opus/FfmpegOpusEncoder.cs new file mode 100644 index 0000000..65af701 --- /dev/null +++ b/DeepDrftContent/Processors/Opus/FfmpegOpusEncoder.cs @@ -0,0 +1,140 @@ +using System.Diagnostics; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; + +namespace DeepDrftContent.Processors.Opus; + +/// +/// Encodes a source audio file (any format the source vault holds — WAV/MP3/FLAC) to Ogg Opus fullband +/// 320 kbps by shelling out to FFmpeg (libopus). FFmpeg is chosen over a managed encoder because it +/// muxes a correct Ogg container with accurate granule positions across every input format — the page +/// structure the seek-index walk depends on — which a raw libopus binding does not provide. The external +/// ffmpeg binary is therefore a host runtime prerequisite (flagged in the wave handoff). +/// +public sealed class FfmpegOpusEncoder +{ + private readonly OpusTranscodeOptions _options; + private readonly ILogger _logger; + + public FfmpegOpusEncoder(IOptions options, ILogger logger) + { + _options = options.Value; + _logger = logger; + } + + /// + /// Transcodes to an Ogg Opus file at . + /// Returns true on a clean exit with a non-empty output. Returns false (logged) on a non-zero exit, + /// a timeout, a missing ffmpeg binary, or any process failure — a transcode failure must never throw + /// to the caller (C6); the background worker treats false as "leave the track lossless-only". + /// + public async Task EncodeAsync(string sourcePath, string destinationPath, CancellationToken ct) + { + var ffmpeg = string.IsNullOrWhiteSpace(_options.FfmpegPath) ? "ffmpeg" : _options.FfmpegPath; + + // -vn drops any cover-art video stream; -map a:0 takes the first audio stream; -ar 48000 forces + // fullband (Opus internally resamples to 48 kHz anyway, but stating it keeps granulepos math + // unambiguous); libopus VBR at the target bitrate; -f ogg for an explicit Ogg container; -y + // overwrites the (pre-created, empty) destination temp file. + var args = new[] + { + "-hide_banner", "-nostdin", "-loglevel", "error", + "-i", sourcePath, + "-vn", "-map", "a:0", + "-c:a", "libopus", "-b:a", $"{_options.BitrateKbps}k", + "-ar", "48000", + "-f", "ogg", + "-y", destinationPath, + }; + + var psi = new ProcessStartInfo(ffmpeg) + { + RedirectStandardError = true, + RedirectStandardOutput = true, + UseShellExecute = false, + CreateNoWindow = true, + }; + foreach (var arg in args) + psi.ArgumentList.Add(arg); + + using var process = new Process { StartInfo = psi }; + + try + { + if (!process.Start()) + { + _logger.LogError("Opus transcode: ffmpeg failed to start ({Ffmpeg}).", ffmpeg); + return false; + } + } + catch (Exception ex) + { + // Most commonly a missing binary (Win32Exception "file not found"). This is the ops + // prerequisite failing — log loudly so it is unmistakable in the deploy logs. + _logger.LogError(ex, + "Opus transcode: could not launch ffmpeg ({Ffmpeg}). Is the ffmpeg binary installed on the host?", + ffmpeg); + return false; + } + + using var timeout = CancellationTokenSource.CreateLinkedTokenSource(ct); + timeout.CancelAfter(TimeSpan.FromSeconds(_options.TimeoutSeconds)); + + // Drain stderr concurrently — ffmpeg can block writing diagnostics if the pipe is not read. + var stderrTask = process.StandardError.ReadToEndAsync(timeout.Token); + + try + { + await process.WaitForExitAsync(timeout.Token); + } + catch (OperationCanceledException) when (ct.IsCancellationRequested) + { + TryKill(process); + await SafeStderr(stderrTask); // observe to avoid unobserved-task warnings + throw; // genuine shutdown cancellation — let it propagate + } + catch (OperationCanceledException) + { + TryKill(process); + await SafeStderr(stderrTask); // observe to avoid unobserved-task warnings + _logger.LogError("Opus transcode: ffmpeg exceeded the {Timeout}s timeout for {Source}.", + _options.TimeoutSeconds, sourcePath); + return false; + } + + var stderr = await SafeStderr(stderrTask); + if (process.ExitCode != 0) + { + _logger.LogError("Opus transcode: ffmpeg exited {Code} for {Source}. stderr: {Stderr}", + process.ExitCode, sourcePath, stderr); + return false; + } + + if (!File.Exists(destinationPath) || new FileInfo(destinationPath).Length == 0) + { + _logger.LogError("Opus transcode: ffmpeg exited 0 but produced no output for {Source}.", sourcePath); + return false; + } + + return true; + } + + private void TryKill(Process process) + { + try + { + if (!process.HasExited) + process.Kill(entireProcessTree: true); + } + catch (Exception ex) + { + _logger.LogWarning(ex, "Opus transcode: failed to kill timed-out ffmpeg process."); + } + } + + private static async Task SafeStderr(Task stderrTask) + { + try { return await stderrTask; } + catch { return ""; } + } +} diff --git a/DeepDrftContent/Processors/Opus/OggOpusConstants.cs b/DeepDrftContent/Processors/Opus/OggOpusConstants.cs new file mode 100644 index 0000000..fbba891 --- /dev/null +++ b/DeepDrftContent/Processors/Opus/OggOpusConstants.cs @@ -0,0 +1,65 @@ +namespace DeepDrftContent.Processors.Opus; + +/// +/// Wire-format constants for the Ogg-Opus derived artifacts. Centralised so the seek-index codec, +/// the page walker, and the tests agree on one set of magic numbers. +/// +public static class OggOpusConstants +{ + /// Opus granule positions are always sample counts at 48 kHz, regardless of input rate. + public const double OpusSampleRate = 48000.0; + + /// One seek-index entry per this many seconds of audio (OQ7 — 0.5 s buckets). + public const double SeekBucketSeconds = 0.5; + + /// The Ogg page capture pattern "OggS" — every page starts with these four bytes. + public static ReadOnlySpan CapturePattern => "OggS"u8; + + /// Magic signature opening an OpusHead identification header packet. + public static ReadOnlySpan OpusHeadSignature => "OpusHead"u8; + + /// Magic signature opening an OpusTags comment header packet. + public static ReadOnlySpan OpusTagsSignature => "OpusTags"u8; + + /// + /// Fixed size of an Ogg page header before the segment table: capture(4) + version(1) + + /// header-type(1) + granulepos(8) + serial(4) + sequence(4) + checksum(4) + page-segments(1). + /// + public const int OggPageHeaderSize = 27; + + /// Byte offset of the 64-bit granule position within an Ogg page header. + public const int GranulePositionOffset = 6; + + /// Byte offset of the page-segment count (the segment-table length) within the header. + public const int PageSegmentCountOffset = 26; + + /// Sentinel granule position for a page that ends mid-packet (no usable timestamp). + public const ulong NoGranulePosition = 0xFFFFFFFFFFFFFFFFUL; + + /// + /// Minimum byte length of an OpusHead packet payload to safely read pre_skip. + /// RFC 7845 §5.1: "OpusHead"(8) + version(1) + channels(1) + pre_skip(2) = 12 bytes minimum. + /// + public const int OpusHeadMinSize = 12; + + /// + /// Byte offset of pre_skip within the full OpusHead packet payload (including the + /// magic). RFC 7845 §5.1: "OpusHead"(8) + version(1) + channels(1) = 10 bytes before pre_skip. + /// + public const int OpusHeadPreSkipOffset = 10; + + /// + /// Header size of the serialized seek-index blob: + /// totalBytes(8) + duration(8) + count(4) + preSkip(2) + reserved(2) = 24 bytes. + /// + public const int SeekIndexHeaderSize = 24; + + /// Size of one serialized seek point: granulepos(8) + byteOffset(8). + public const int SeekPointSize = 16; + + /// Vault-resource extension for the Opus audio bytes. + public const string OpusExtension = ".opus"; + + /// Vault-resource extension for the combined setup-header + seek-index sidecar. + public const string SidecarExtension = ".opusidx"; +} diff --git a/DeepDrftContent/Processors/Opus/OggOpusParser.cs b/DeepDrftContent/Processors/Opus/OggOpusParser.cs new file mode 100644 index 0000000..0e4f850 --- /dev/null +++ b/DeepDrftContent/Processors/Opus/OggOpusParser.cs @@ -0,0 +1,146 @@ +using System.Buffers.Binary; + +namespace DeepDrftContent.Processors.Opus; + +/// +/// The result of walking an encoded Ogg Opus stream once: the captured setup header (the leading +/// OpusHead + OpusTags pages, verbatim) and the bucketed granule→byte seek index. This +/// is everything the sidecar artifact carries (§3.4a) — built at transcode time so delivery never +/// re-walks the stream. +/// +/// The leading setup pages (OpusHead + OpusTags), exactly as they +/// appear at the start of the stream, ready to prepend to any mid-stream page run before decode. +/// The accurate, 0.5 s-bucketed granule→byte transfer function. +public sealed record OggOpusWalk(byte[] SetupHeaderBytes, OggOpusSeekIndex SeekIndex); + +/// +/// Pure Ogg-Opus stream walker. Reads the page structure directly (the OggS capture pattern and +/// the 27-byte page header) to (1) capture the setup-header pages and (2) record, for every audio page, +/// its end granule position and exact byte offset — bucketed to 0.5 s with each bucket boundary snapped +/// to the nearest enclosing page start. No external dependency: the encoder (FFmpeg) produces the bytes; +/// this turns them into the seek artifact deterministically, so it is unit-testable without a codec. +/// +public static class OggOpusParser +{ + /// + /// Walks and produces the setup header + seek index, or null if the + /// bytes are not a recognisable Ogg Opus stream (no setup header, no audio pages, or truncated + /// structure). A null is the caller's signal to treat the transcode as failed and leave the track + /// lossless-only (C6) — it does not throw for malformed input. + /// + public static OggOpusWalk? Walk(ReadOnlySpan oggBytes) + { + var setupHeaderEnd = -1; + var sawOpusHead = false; + var sawOpusTags = false; + ushort preSkip = 0; + + var points = new List(); + ulong lastGranule = 0; + var nextBucketTime = 0.0; + var firstAudioPointTaken = false; + + var offset = 0; + while (offset + OggOpusConstants.OggPageHeaderSize <= oggBytes.Length) + { + var page = oggBytes.Slice(offset); + if (!page[..4].SequenceEqual(OggOpusConstants.CapturePattern)) + { + // Not on a page boundary — the encoder writes contiguous pages, so this means the + // stream is malformed or we mis-stepped. Either way it is unrecoverable here. + return null; + } + + var segmentCount = page[OggOpusConstants.PageSegmentCountOffset]; + var segmentTableEnd = OggOpusConstants.OggPageHeaderSize + segmentCount; + if (segmentTableEnd > page.Length) + return null; // truncated header + + var payloadSize = 0; + for (var i = 0; i < segmentCount; i++) + payloadSize += page[OggOpusConstants.OggPageHeaderSize + i]; + + var pageTotalSize = segmentTableEnd + payloadSize; + if (pageTotalSize > page.Length) + return null; // truncated payload + + var payload = page.Slice(segmentTableEnd, payloadSize); + var granule = BinaryPrimitives.ReadUInt64LittleEndian( + page.Slice(OggOpusConstants.GranulePositionOffset, 8)); + + // The setup pages carry no audio granule (OpusHead has granulepos 0; OpusTags too). They + // are the leading pages whose payload opens with the Opus magic signatures. + if (!sawOpusHead && StartsWith(payload, OggOpusConstants.OpusHeadSignature)) + { + sawOpusHead = true; + setupHeaderEnd = offset + pageTotalSize; + + // RFC 7845 §5.1 — OpusHead layout after the 8-byte "OpusHead" magic: + // [0] version (1 byte), [1] channel count (1 byte), + // [2-3] pre_skip (little-endian uint16) ← at packet bytes 10-11 + // pre_skip is the number of decoder samples to discard before presenting audio; + // all granule→time conversions must subtract it (RFC 7845 §4.3). + if (payload.Length >= OggOpusConstants.OpusHeadMinSize) + preSkip = BinaryPrimitives.ReadUInt16LittleEndian( + payload.Slice(OggOpusConstants.OpusHeadPreSkipOffset, 2)); + } + else if (sawOpusHead && !sawOpusTags && StartsWith(payload, OggOpusConstants.OpusTagsSignature)) + { + sawOpusTags = true; + setupHeaderEnd = offset + pageTotalSize; + } + else if (sawOpusHead && sawOpusTags) + { + // Audio page. Record the first audio page unconditionally (the seek anchor at t=0), + // then one entry per 0.5 s bucket. A page with no end-granule (mid-packet continuation, + // granulepos == -1) is skipped for indexing — its time is unknown — but still advances + // the byte cursor. + if (granule != OggOpusConstants.NoGranulePosition) + { + // RFC 7845 §4.3: presentation time = max(0, granule − preSkip) / 48000. + // Use this corrected time for bucketing so that a stream with pre-skip 3840 (~80 ms) + // does not systematically offset every indexed time by that amount. + var correctedTime = Math.Max(0.0, + (granule - (double)preSkip) / OggOpusConstants.OpusSampleRate); + + if (!firstAudioPointTaken) + { + // Anchor the first seek point at corrected time = 0 by storing the granule as + // preSkip. This guarantees that a binary search for t=0 ("largest entry with + // corrected time ≤ 0") always resolves to the first audio page's byte offset — + // even when the real granule is slightly above preSkip due to encoder lead-in. + points.Add(new OpusSeekPoint(preSkip, (ulong)offset)); + firstAudioPointTaken = true; + nextBucketTime = OggOpusConstants.SeekBucketSeconds; + } + else if (correctedTime >= nextBucketTime) + { + points.Add(new OpusSeekPoint(granule, (ulong)offset)); + // Advance past every bucket this page crossed so a long page does not emit a + // backlog of entries; the next bucket is the first boundary strictly after it. + while (nextBucketTime <= correctedTime) + nextBucketTime += OggOpusConstants.SeekBucketSeconds; + } + + lastGranule = granule; + } + } + + offset += pageTotalSize; + } + + if (!sawOpusHead || setupHeaderEnd < 0 || points.Count == 0) + return null; + + var setupHeader = oggBytes[..setupHeaderEnd].ToArray(); + // RFC 7845 §4.3: total duration is also pre-skip-corrected, matching the time a listener + // experiences (the last audio page's corrected time, clamped to ≥ 0). + var totalDuration = Math.Max(0.0, + (lastGranule - (double)preSkip) / OggOpusConstants.OpusSampleRate); + var index = new OggOpusSeekIndex(points, totalDuration, (ulong)oggBytes.Length, preSkip); + return new OggOpusWalk(setupHeader, index); + } + + private static bool StartsWith(ReadOnlySpan payload, ReadOnlySpan signature) => + payload.Length >= signature.Length && payload[..signature.Length].SequenceEqual(signature); +} diff --git a/DeepDrftContent/Processors/Opus/OggOpusSeekIndex.cs b/DeepDrftContent/Processors/Opus/OggOpusSeekIndex.cs new file mode 100644 index 0000000..465a1b5 --- /dev/null +++ b/DeepDrftContent/Processors/Opus/OggOpusSeekIndex.cs @@ -0,0 +1,124 @@ +using System.Buffers.Binary; + +namespace DeepDrftContent.Processors.Opus; + +/// +/// A single seek-index entry: an authoritative 48 kHz (Opus granule +/// positions are always sample counts at 48 kHz) paired with the exact byte offset of the Ogg page that +/// carries it. Every is a real page-start boundary, so a +/// Range: bytes={ByteOffset}- fetch lands the decoder Ogg-sync-aligned. +/// +/// +/// Per RFC 7845 §4.3, the PCM presentation time is (granulepos − preSkip) / 48000. The raw +/// is stored here as-is; callers should subtract the containing +/// before converting to a presentation time. Use +/// for the corrected value. +/// +/// The page's end granule position (48 kHz sample count). +/// The byte offset of the page start in the Opus file. +public readonly record struct OpusSeekPoint(ulong GranulePosition, ulong ByteOffset) +{ + /// + /// Raw granule-position-to-time conversion (granulepos / 48 kHz). Does NOT subtract pre-skip — use + /// for the RFC 7845-correct presentation time. + /// + public double RawTimeSeconds => GranulePosition / OggOpusConstants.OpusSampleRate; +} + +/// +/// The accurate, precomputed transfer function from seek-time to true file byte offset for one Ogg +/// Opus stream (§3.4a A). Built once at transcode time by walking the encoded stream; the client reads +/// it back and binary-searches instead of doing inaccurate VBR byte-rate math. +/// One entry per 0.5 s of audio (), each snapped to the +/// nearest enclosing page start, plus the totals needed to clamp a seek to range. +/// +/// Ordered (granulepos, byteOffset) entries, ascending. The first entry always +/// has == (corrected time = 0) +/// and points at the first audio page start, ensuring a seek to t=0 always resolves. +/// +/// Pre-skip-corrected total stream duration: max(0, lastGranule − preSkip) / 48000. +/// +/// Total Opus file byte length, for clamping a seek past the end. +/// +/// The pre_skip value from the OpusHead identification header (RFC 7845 §5.1). Opus +/// decoders must discard this many samples from the decoded start before presenting audio. The client +/// (wave 18.4) needs this to trim the first decoded buffer; storing it here avoids a re-parse of the +/// Ogg stream at delivery time. +/// +public sealed record OggOpusSeekIndex( + IReadOnlyList Points, + double TotalDurationSeconds, + ulong TotalByteLength, + ushort PreSkip) +{ + /// + /// Returns the RFC 7845-correct presentation time for a seek point: max(0, granule − preSkip) / 48000. + /// Use this for all time comparisons; raw omits the pre-skip. + /// + public double PresentationTimeSeconds(OpusSeekPoint point) => + Math.Max(0.0, (point.GranulePosition - (double)PreSkip) / OggOpusConstants.OpusSampleRate); + + /// + /// Serializes the index to the compact little-endian binary blob the sidecar stores. Layout: + /// [uint64 totalByteLength][double totalDurationSeconds][uint32 pointCount][uint16 preSkip][uint16 reserved] + /// then pointCount × (uint64 granulepos, uint64 byteOffset). The four-byte preSkip+reserved + /// region pads the header to 24 bytes, keeping the point table 8-byte-aligned. + /// Fixed-width records keep the client parse to a single typed-array read. + /// + public byte[] ToBytes() + { + var size = OggOpusConstants.SeekIndexHeaderSize + Points.Count * OggOpusConstants.SeekPointSize; + var bytes = new byte[size]; + var span = bytes.AsSpan(); + + BinaryPrimitives.WriteUInt64LittleEndian(span[..8], TotalByteLength); + BinaryPrimitives.WriteDoubleLittleEndian(span.Slice(8, 8), TotalDurationSeconds); + BinaryPrimitives.WriteUInt32LittleEndian(span.Slice(16, 4), (uint)Points.Count); + BinaryPrimitives.WriteUInt16LittleEndian(span.Slice(20, 2), PreSkip); + // bytes 22-23: reserved (zero-initialized by array allocation) + + var cursor = OggOpusConstants.SeekIndexHeaderSize; + foreach (var point in Points) + { + BinaryPrimitives.WriteUInt64LittleEndian(span.Slice(cursor, 8), point.GranulePosition); + BinaryPrimitives.WriteUInt64LittleEndian(span.Slice(cursor + 8, 8), point.ByteOffset); + cursor += OggOpusConstants.SeekPointSize; + } + + return bytes; + } + + /// + /// Parses a blob produced by . Returns null if the blob is too short or its + /// declared point count does not fit — the storage contract is exact, so a malformed blob is a + /// corruption signal, not a recoverable shape. (Provided so tests and any future server-side reader + /// share one codec with the writer.) + /// + public static OggOpusSeekIndex? FromBytes(ReadOnlySpan bytes) + { + if (bytes.Length < OggOpusConstants.SeekIndexHeaderSize) + return null; + + var totalByteLength = BinaryPrimitives.ReadUInt64LittleEndian(bytes[..8]); + var totalDuration = BinaryPrimitives.ReadDoubleLittleEndian(bytes.Slice(8, 8)); + var count = BinaryPrimitives.ReadUInt32LittleEndian(bytes.Slice(16, 4)); + var preSkip = BinaryPrimitives.ReadUInt16LittleEndian(bytes.Slice(20, 2)); + // bytes 22-23: reserved — ignored on read for forward-compatibility + + var expected = OggOpusConstants.SeekIndexHeaderSize + (long)count * OggOpusConstants.SeekPointSize; + if (bytes.Length < expected) + return null; + + var points = new OpusSeekPoint[count]; + var cursor = OggOpusConstants.SeekIndexHeaderSize; + for (var i = 0; i < count; i++) + { + var granule = BinaryPrimitives.ReadUInt64LittleEndian(bytes.Slice(cursor, 8)); + var offset = BinaryPrimitives.ReadUInt64LittleEndian(bytes.Slice(cursor + 8, 8)); + points[i] = new OpusSeekPoint(granule, offset); + cursor += OggOpusConstants.SeekPointSize; + } + + return new OggOpusSeekIndex(points, totalDuration, totalByteLength, preSkip); + } +} diff --git a/DeepDrftContent/Processors/Opus/OpusSidecar.cs b/DeepDrftContent/Processors/Opus/OpusSidecar.cs new file mode 100644 index 0000000..07cd594 --- /dev/null +++ b/DeepDrftContent/Processors/Opus/OpusSidecar.cs @@ -0,0 +1,57 @@ +using System.Buffers.Binary; + +namespace DeepDrftContent.Processors.Opus; + +/// +/// The single derived sidecar artifact per track (§3.4a B, recommended design): the Opus setup header +/// (OpusHead + OpusTags) followed by the granule→byte seek index. The client fetches this +/// once on track load and parses it into its OpusSeekData, so it always has both the setup bytes +/// (to prepend to any mid-stream slice) and the accurate seek transfer function before it ever issues a +/// Range fetch — including a window that opens away from byte 0 (UC9). +/// +/// The verbatim OpusHead + OpusTags pages. +/// The bucketed granule→byte seek index. +public sealed record OpusSidecar(byte[] SetupHeaderBytes, OggOpusSeekIndex SeekIndex) +{ + /// + /// Serializes to [uint32 setupHeaderLength][setup-header bytes][seek-index blob]. The + /// length prefix lets the client split the two regions with one read; the seek-index blob carries + /// its own self-describing header (), so it needs no trailing + /// length. + /// + public byte[] ToBytes() + { + var indexBytes = SeekIndex.ToBytes(); + var bytes = new byte[4 + SetupHeaderBytes.Length + indexBytes.Length]; + var span = bytes.AsSpan(); + + BinaryPrimitives.WriteUInt32LittleEndian(span[..4], (uint)SetupHeaderBytes.Length); + SetupHeaderBytes.CopyTo(span.Slice(4)); + indexBytes.CopyTo(span.Slice(4 + SetupHeaderBytes.Length)); + + return bytes; + } + + /// + /// Parses a blob produced by . Returns null on any structural inconsistency + /// (short blob, length prefix that overruns, or an unparseable index) — the format is exact, so a + /// malformed blob is corruption. + /// + public static OpusSidecar? FromBytes(ReadOnlySpan bytes) + { + if (bytes.Length < 4) + return null; + + var setupLength = BinaryPrimitives.ReadUInt32LittleEndian(bytes[..4]); + var indexStart = 4 + (long)setupLength; + if (bytes.Length < indexStart) + return null; + + var setupHeader = bytes.Slice(4, (int)setupLength).ToArray(); + var index = OggOpusSeekIndex.FromBytes(bytes.Slice((int)indexStart)); + if (index is null) + return null; + + return new OpusSidecar(setupHeader, index); + } +} diff --git a/DeepDrftContent/Processors/Opus/OpusTranscodeOptions.cs b/DeepDrftContent/Processors/Opus/OpusTranscodeOptions.cs new file mode 100644 index 0000000..ac81a7a --- /dev/null +++ b/DeepDrftContent/Processors/Opus/OpusTranscodeOptions.cs @@ -0,0 +1,33 @@ +namespace DeepDrftContent.Processors.Opus; + +/// +/// Host-supplied configuration for the Opus transcode. The only operationally significant knob is +/// — the transcode shells out to FFmpeg (libopus), which must be present on the +/// DeepDrftAPI host (see the wave handoff notes). Defaults target Ogg Opus fullband (48 kHz) at 320 kbps, +/// the artifact the spec fixes (§1). +/// +public sealed class OpusTranscodeOptions +{ + /// + /// Path to the ffmpeg executable. Empty/null resolves to "ffmpeg" (found on PATH). Override + /// with an absolute path when the binary is not on the host PATH. + /// + public string FfmpegPath { get; set; } = "ffmpeg"; + + /// Target Opus bitrate in kbps. 320 kbps fullband is the fixed artifact quality (§1). + public int BitrateKbps { get; set; } = 320; + + /// + /// Directory for the transient source/output files the transcode stages. Defaults to the system + /// temp path; the host overrides it to the data-disk upload-staging directory so large files never + /// land on the small RAM-backed /tmp tmpfs (same constraint the upload path already honours). + /// + public string StagingPath { get; set; } = Path.GetTempPath(); + + /// + /// Hard ceiling on a single transcode, in seconds. A run that exceeds it is killed and the track + /// stays lossless-only (C6). Generous by default — a 1 GB mix is CPU-expensive (§3.1) — but bounded + /// so a hung ffmpeg never wedges the background worker. + /// + public int TimeoutSeconds { get; set; } = 3600; +} diff --git a/DeepDrftContent/Processors/Opus/OpusTranscodeService.cs b/DeepDrftContent/Processors/Opus/OpusTranscodeService.cs new file mode 100644 index 0000000..dba73e9 --- /dev/null +++ b/DeepDrftContent/Processors/Opus/OpusTranscodeService.cs @@ -0,0 +1,154 @@ +using DeepDrftContent.Constants; +using DeepDrftContent.FileDatabase.Models; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; +using FileDb = DeepDrftContent.FileDatabase.Services.FileDatabase; + +namespace DeepDrftContent.Processors.Opus; + +/// +/// Derives and persists a track's low-data Ogg Opus artifacts (Phase 18.1). Mirrors +/// 's derived-artifact lifecycle: compute from the stored source, +/// store in a dedicated vault keyed by EntryKey, regenerable, failure-tolerant. For one track it +/// produces two entries in the vault — the Opus audio bytes and a +/// combined setup-header + seek-index sidecar (§3.4a). Strictly additive: the source tracks vault +/// is never touched, and a failure here leaves the track lossless-only and eligible for backfill (C2/C6). +/// +public sealed class OpusTranscodeService +{ + private readonly FileDb _fileDatabase; + private readonly FfmpegOpusEncoder _encoder; + private readonly OpusTranscodeOptions _options; + private readonly ILogger _logger; + + public OpusTranscodeService( + FileDb fileDatabase, + FfmpegOpusEncoder encoder, + IOptions options, + ILogger logger) + { + _fileDatabase = fileDatabase; + _encoder = encoder; + _options = options.Value; + _logger = logger; + } + + /// + /// Reads the source audio for from the tracks vault, transcodes it + /// to Ogg Opus 320, walks the encoded stream to build the seek index + capture the setup header, and + /// stores the Opus bytes and the sidecar in the vault under the + /// same key. Re-runnable — a second call overwrites the prior artifacts (backfill / replace-audio). + /// Returns false (logged) on any failure; never throws for expected failure modes (C6). The only + /// propagated exception is on genuine shutdown. + /// + public async Task TranscodeAndStoreAsync(string entryKey, CancellationToken ct) + { + var source = await _fileDatabase.LoadResourceAsync(VaultConstants.Tracks, entryKey); + if (source is null) + { + _logger.LogWarning("Opus transcode: no source audio in vault for {EntryKey}; skipping.", entryKey); + return false; + } + + Directory.CreateDirectory(_options.StagingPath); + var sourcePath = Path.Combine(_options.StagingPath, $"opus-src-{Guid.NewGuid():N}{source.Extension}"); + var opusPath = Path.Combine(_options.StagingPath, $"opus-out-{Guid.NewGuid():N}{OggOpusConstants.OpusExtension}"); + + try + { + await File.WriteAllBytesAsync(sourcePath, source.Buffer, ct); + + if (!await _encoder.EncodeAsync(sourcePath, opusPath, ct)) + return false; // encoder already logged the cause + + var opusBytes = await File.ReadAllBytesAsync(opusPath, ct); + + var walk = OggOpusParser.Walk(opusBytes); + if (walk is null) + { + _logger.LogError( + "Opus transcode: ffmpeg produced output but the Ogg stream could not be walked for {EntryKey}; " + + "no artifacts stored.", entryKey); + return false; + } + + await EnsureVaultAsync(); + + var opusBitrate = source.Duration > 0 + ? (int)(opusBytes.Length * 8 / source.Duration / 1000) + : _options.BitrateKbps; + var audioBinary = new AudioBinary(new AudioBinaryParams( + opusBytes, opusBytes.Length, OggOpusConstants.OpusExtension, source.Duration, opusBitrate)); + + var sidecar = new OpusSidecar(walk.SetupHeaderBytes, walk.SeekIndex).ToBytes(); + var sidecarBinary = new MediaBinary(new MediaBinaryParams( + sidecar, sidecar.Length, OggOpusConstants.SidecarExtension)); + + // Store the audio first, then the sidecar. If the sidecar write fails the Opus bytes are + // present but unseekable — treat that as a failed derive (return false) so a backfill re-runs + // it; do not leave a half-derived track that the delivery layer would treat as complete. + var audioStored = await _fileDatabase.RegisterResourceAsync( + VaultConstants.TrackOpus, OpusAudioKey(entryKey), audioBinary); + if (!audioStored) + { + _logger.LogError("Opus transcode: vault write of Opus audio failed for {EntryKey}.", entryKey); + return false; + } + + var sidecarStored = await _fileDatabase.RegisterResourceAsync( + VaultConstants.TrackOpus, OpusSidecarKey(entryKey), sidecarBinary); + if (!sidecarStored) + { + _logger.LogError("Opus transcode: vault write of sidecar failed for {EntryKey}.", entryKey); + return false; + } + + _logger.LogInformation( + "Opus transcode complete for {EntryKey}: {OpusBytes} bytes, {Points} seek points, {Duration:F1}s.", + entryKey, opusBytes.Length, walk.SeekIndex.Points.Count, walk.SeekIndex.TotalDurationSeconds); + return true; + } + catch (OperationCanceledException) when (ct.IsCancellationRequested) + { + throw; + } + catch (Exception ex) + { + _logger.LogError(ex, "Opus transcode failed for {EntryKey}; track stays lossless-only.", entryKey); + return false; + } + finally + { + TryDelete(sourcePath); + TryDelete(opusPath); + } + } + + /// The vault entry key under which a track's Opus audio bytes are stored. + public static string OpusAudioKey(string entryKey) => entryKey; + + /// The vault entry key under which a track's setup-header + seek-index sidecar is stored. + public static string OpusSidecarKey(string entryKey) => $"{entryKey}-sidecar"; + + private async Task EnsureVaultAsync() + { + // The TrackOpus vault is created at host startup (Startup.cs), so this guard is normally a + // no-op for the upload path. It is retained for the backfill path, which may run via a + // standalone CLI or a host that skips vault pre-creation, where the vault might not exist. + if (!_fileDatabase.HasVault(VaultConstants.TrackOpus)) + await _fileDatabase.CreateVaultAsync(VaultConstants.TrackOpus, MediaVaultType.Audio); + } + + private void TryDelete(string path) + { + try + { + if (File.Exists(path)) + File.Delete(path); + } + catch (Exception ex) + { + _logger.LogWarning(ex, "Opus transcode: failed to delete staging file {Path}.", path); + } + } +} diff --git a/DeepDrftContent/Processors/Opus/ResolvedAudio.cs b/DeepDrftContent/Processors/Opus/ResolvedAudio.cs new file mode 100644 index 0000000..85b3813 --- /dev/null +++ b/DeepDrftContent/Processors/Opus/ResolvedAudio.cs @@ -0,0 +1,23 @@ +using DeepDrftContent.FileDatabase.Models; +using DeepDrftModels.Enums; + +namespace DeepDrftContent.Processors.Opus; + +/// +/// The outcome of resolving a track + requested to a concrete artifact +/// (Phase 18.2). Carries the bytes, the content-type that matches what was actually returned, +/// and the format actually served — which may differ from the requested one when the C2 fallback fires +/// (Opus requested, no Opus artifact → the lossless artifact + its content-type). The delivery layer +/// (18.3) sets the response Content-Type from so the eventual decoder +/// picks the right decoder for the bytes it receives, not the bytes the listener asked for. +/// +/// The resolved audio artifact (never null when a resolution succeeds). +/// The MIME type of (e.g. audio/ogg for Opus, +/// or the source's real MIME for lossless). +/// The format actually returned. Equal to the requested format on a direct +/// hit; when an Opus request fell back. +public sealed record ResolvedAudio(AudioBinary Audio, string ContentType, AudioFormat ResolvedFormat) +{ + /// True when an Opus request was served the lossless artifact because no Opus existed (C2). + public bool DidFallBack(AudioFormat requested) => requested != ResolvedFormat; +} diff --git a/DeepDrftContent/Processors/Opus/TrackFormatResolver.cs b/DeepDrftContent/Processors/Opus/TrackFormatResolver.cs new file mode 100644 index 0000000..bf75d70 --- /dev/null +++ b/DeepDrftContent/Processors/Opus/TrackFormatResolver.cs @@ -0,0 +1,110 @@ +using DeepDrftContent.Constants; +using DeepDrftContent.FileDatabase.Models; +using DeepDrftModels.Enums; +using Microsoft.Extensions.Logging; +using FileDb = DeepDrftContent.FileDatabase.Services.FileDatabase; + +namespace DeepDrftContent.Processors.Opus; + +/// +/// The server-side format resolution + sidecar lookup seam (Phase 18.2). Given a track's +/// EntryKey and a requested , returns the correct audio artifact and the +/// content-type that matches it; given an EntryKey, returns the Opus seek/setup sidecar bytes. +/// Downstream waves call this — 18.3 wires it behind the ?format= stream param and serves the +/// sidecar over HTTP; this wave delivers only the seam, not the HTTP surface. +/// +/// Additive and non-breaking (C2): the lossless branch reads the source exactly as the existing stream +/// path does (via ), and an Opus request for a track +/// with no Opus artifact falls back to lossless rather than failing. Mirrors the +/// derived-artifact lookup precedent: read from the dedicated vault, +/// swallow misses to null (FileDatabase convention), let the caller decide. +/// +/// +public sealed class TrackFormatResolver +{ + private readonly FileDb _fileDatabase; + private readonly TrackContentService _trackContentService; + private readonly ILogger _logger; + + public TrackFormatResolver( + FileDb fileDatabase, + TrackContentService trackContentService, + ILogger logger) + { + _fileDatabase = fileDatabase; + _trackContentService = trackContentService; + _logger = logger; + } + + /// + /// Resolves + to the audio artifact to + /// serve plus its content-type. resolves the source artifact in the + /// tracks vault with its real MIME (WAV/MP3/FLAC). resolves the + /// derived Opus artifact (audio/ogg) when present, and falls back to lossless + /// when it is not (C2). Returns null only when even the lossless source is missing — i.e. the track has + /// no audio at all (an unknown key or a genuinely empty vault), the one case the caller treats as 404. + /// + public async Task ResolveAsync(string entryKey, AudioFormat requestedFormat) + { + if (requestedFormat == AudioFormat.Opus) + { + var opus = await _fileDatabase.LoadResourceAsync( + VaultConstants.TrackOpus, OpusTranscodeService.OpusAudioKey(entryKey)); + if (opus is not null) + return new ResolvedAudio(opus, MimeTypeExtensions.GetMimeType(opus.Extension), AudioFormat.Opus); + + // C2 fallback: no Opus artifact yet (legacy row, not backfilled, or transcode failed). Degrade + // to lossless rather than 404 — Opus is strictly additive; its absence never means "no audio". + _logger.LogInformation( + "Opus requested for {EntryKey} but no Opus artifact exists; falling back to lossless.", entryKey); + } + + return await ResolveLosslessAsync(entryKey); + } + + /// + /// Resolves the lossless source artifact and its real MIME — the existing read path, unchanged. Shared + /// by the explicit-lossless branch and the Opus fallback so both produce identical bytes + content-type. + /// + private async Task ResolveLosslessAsync(string entryKey) + { + var source = await _trackContentService.GetAudioBinaryAsync(entryKey); + if (source is null) + return null; + + return new ResolvedAudio(source, MimeTypeExtensions.GetMimeType(source.Extension), AudioFormat.Lossless); + } + + /// + /// Returns the Opus setup-header + seek-index sidecar bytes for , or null + /// when no sidecar is stored (no Opus artifact yet, or an older derive predating the sidecar). 18.3 + /// serves these on their own path; 18.4 fetches them once on track load and parses them into the + /// client's OpusSeekData. The bytes are the raw blob + /// ([uint32 setupHeaderLength][setup-header][seek-index]) exactly as 18.1 stored them. + /// + public async Task GetOpusSidecarAsync(string entryKey) + { + var sidecar = await _fileDatabase.LoadResourceAsync( + VaultConstants.TrackOpus, OpusTranscodeService.OpusSidecarKey(entryKey)); + return sidecar?.Buffer; + } + + /// + /// Reports whether already has a complete Opus derive — both the audio bytes + /// AND the seek/setup sidecar present in the track-opus vault. The Backfill-Opus pass (18.5) uses + /// this to enqueue only tracks that are missing or half-derived (audio without sidecar = unseekable, so + /// treated as incomplete and re-derived). Both halves are required because the transcode stores them in + /// sequence and a sidecar-write failure leaves a track the delivery layer must not treat as Opus-ready. + /// + public async Task HasOpusAsync(string entryKey) + { + var audio = await _fileDatabase.LoadResourceAsync( + VaultConstants.TrackOpus, OpusTranscodeService.OpusAudioKey(entryKey)); + if (audio is null) + return false; + + var sidecar = await _fileDatabase.LoadResourceAsync( + VaultConstants.TrackOpus, OpusTranscodeService.OpusSidecarKey(entryKey)); + return sidecar is not null; + } +} diff --git a/DeepDrftManager/Components/Pages/Tracks/BatchEdit.razor b/DeepDrftManager/Components/Pages/Tracks/BatchEdit.razor index c0422dc..eebb7cc 100644 --- a/DeepDrftManager/Components/Pages/Tracks/BatchEdit.razor +++ b/DeepDrftManager/Components/Pages/Tracks/BatchEdit.razor @@ -518,7 +518,10 @@ { var row = _tracks[i]; - if (row.Status == BatchRowStatus.Done) + // Skip rows already processed in a prior submit attempt. PostProcessing counts as processed: + // the track persisted successfully (only its background Opus derive is still settling), so a + // re-submit after a partial failure must NOT re-upload it and mint a duplicate. + if (row.Status is BatchRowStatus.Done or BatchRowStatus.PostProcessing) { _processedCount++; continue; @@ -638,7 +641,12 @@ } } - row.Status = BatchRowStatus.Done; + // §3.1a: a new-track upload persists the track (live + lossless) and the server + // derives Opus in the background, so the row enters the visible Post-Processing + // phase — same as BatchUpload. (The metadata-only update path above stays Done: it + // changes no audio, so it triggers no transcode.) Non-blocking; the Releases view + // polls the durable Opus status to settle it. + row.Status = BatchRowStatus.PostProcessing; succeeded++; } } diff --git a/DeepDrftManager/Components/Pages/Tracks/BatchRowModel.cs b/DeepDrftManager/Components/Pages/Tracks/BatchRowModel.cs index 1d4ab85..93730ae 100644 --- a/DeepDrftManager/Components/Pages/Tracks/BatchRowModel.cs +++ b/DeepDrftManager/Components/Pages/Tracks/BatchRowModel.cs @@ -40,4 +40,9 @@ public class BatchRowModel : 0; } -public enum BatchRowStatus { Queued, Uploading, Done, Failed } +// Done is the terminal success state (track persisted + playable losslessly). PostProcessing is the +// visible third upload phase (§3.1a): the byte transfer and server persist are finished and the track is +// live, but the server-side background Opus transcode is still running. It is NOT a failure and never +// blocks completion — the form may navigate away while a row sits in PostProcessing; the Releases browse +// view polls the durable Opus status from there. +public enum BatchRowStatus { Queued, Uploading, PostProcessing, Done, Failed } diff --git a/DeepDrftManager/Components/Pages/Tracks/BatchTrackList.razor b/DeepDrftManager/Components/Pages/Tracks/BatchTrackList.razor index 48c1121..3750ceb 100644 --- a/DeepDrftManager/Components/Pages/Tracks/BatchTrackList.razor +++ b/DeepDrftManager/Components/Pages/Tracks/BatchTrackList.razor @@ -126,6 +126,8 @@ { BatchRowStatus.Uploading => @ Uploading, + BatchRowStatus.PostProcessing => @ + Post-Processing, BatchRowStatus.Done => @Done, BatchRowStatus.Failed => @Failed, _ => @Queued diff --git a/DeepDrftManager/Components/Pages/Tracks/BatchUpload.razor b/DeepDrftManager/Components/Pages/Tracks/BatchUpload.razor index 00b0bda..9ce1244 100644 --- a/DeepDrftManager/Components/Pages/Tracks/BatchUpload.razor +++ b/DeepDrftManager/Components/Pages/Tracks/BatchUpload.razor @@ -69,6 +69,15 @@ Value="@_tracks[0].UploadPercent" aria-label="Uploading track" /> } + else if (_tracks[0].Status == BatchRowStatus.PostProcessing) + { + @* §3.1a: track is live + plays lossless; the Opus transcode runs in the background. + Indeterminate (no client-side progress for a server-side job) and non-blocking. *@ + + + Post-Processing (deriving Opus)… + + } } @@ -519,7 +528,13 @@ } } - row.Status = BatchRowStatus.Done; + // §3.1a: the byte transfer + server persist are done and the track is live and plays + // losslessly — the upload is successful HERE. The server then derives Opus on a + // background worker, so the row enters the visible Post-Processing phase rather than + // jumping straight to Done. This never blocks: the loop continues and the form may + // navigate away while rows sit in Post-Processing; the Releases view polls the durable + // Opus status to settle each one. + row.Status = BatchRowStatus.PostProcessing; succeeded++; } } diff --git a/DeepDrftManager/Components/Pages/Tracks/Releases.razor b/DeepDrftManager/Components/Pages/Tracks/Releases.razor index 1aaf845..66a125a 100644 --- a/DeepDrftManager/Components/Pages/Tracks/Releases.razor +++ b/DeepDrftManager/Components/Pages/Tracks/Releases.razor @@ -9,6 +9,7 @@ @inject ISnackbar Snackbar @inject ILogger Logger @inject NavigationManager NavigationManager +@implements IDisposable @attribute [Authorize] Releases — Deep DRFT Management @@ -51,6 +52,28 @@ Backfill High-res (@MissingHighResCount) } + @* Backfill-Opus (Phase 18.5 + 18.6 badge). The Opus derive runs on a server-side background + worker: pressing the button enqueues every track lacking a complete Opus artifact and reports + the (enqueued / skipped) outcome. The "missing N" badge (18.6) reads the same opus-status map + the page polls, giving visual parity with the two waveform backfill buttons — but unlike them, + the count is informational, not a per-track client loop (the work is scheduled, not driven from + here). The button stays pressable at N=0 (a no-op re-run is harmless); it only disables while a + press is in flight or another bulk run holds the page. *@ + + @if (_opusBackfillRunning) + { + + Scheduling… + } + else + { + Backfill Opus (@MissingOpusCount) + } + @@ -140,6 +163,22 @@ private int MissingProfileCount => _waveformStatus.Count(s => !s.HasProfile); private int MissingHighResCount => _waveformStatus.Count(s => !s.HasHighRes); + // EntryKey → HasOpus (a complete audio+sidecar derive). Loaded alongside the waveform status on init and + // re-read after a Backfill-Opus run so the "missing N" badge settles. Also the source the Post-Processing + // poll watches: a freshly uploaded track lands here with HasOpus=false and flips to true once the + // server-side background transcode finishes — the durable surface for the upload meter's Post-Processing + // phase after the form returns the admin to this view (§3.1a). + private IReadOnlyList _opusStatus = Array.Empty(); + + private int MissingOpusCount => _opusStatus.Count(s => !s.HasOpus); + + // Post-Processing poll: while any track is still missing Opus (a transcode in flight or not yet + // backfilled), re-read the opus-status map on an interval so the "missing N" badge and any per-track + // Post-Processing indicator settle without a manual refresh. Stops itself once nothing is missing, and is + // torn down on dispose. Non-blocking — it never gates an upload or a button; it only refreshes a count. + private const int OpusPollIntervalMs = 4000; + private CancellationTokenSource? _opusPollCts; + // Local state for the parent-owned "Generate All Profiles" bulk run. private bool _bulkRunning; private int _bulkTotal; @@ -150,6 +189,11 @@ private int _highResBulkTotal; private int _highResBulkDone; + // Local state for the "Backfill Opus" action. The Opus derive is server-side and background-queued, so + // there is no client-side per-track loop or progress total — this flag only guards the button while the + // single scheduling call is in flight. + private bool _opusBackfillRunning; + protected override async Task OnInitializedAsync() { // Seed the active tab from ?medium= so a catalogue card deep-links straight to its medium. Panel 0 @@ -170,9 +214,81 @@ _waveformStatus = result.Success && result.Value is not null ? result.Value : Array.Empty(); + + await RefreshOpusStatusAsync(); StateHasChanged(); } + /// + /// Re-read the per-track Opus derive status and (re)arm the Post-Processing poll when work is still + /// pending. Called on init, after a Backfill-Opus run, and from the poll itself. Best-effort: a failed + /// fetch leaves the previous map in place rather than zeroing the badge on a transient API blip. Does not + /// call StateHasChanged itself — callers batch it with their own render (the poll path renders explicitly). + /// + private async Task RefreshOpusStatusAsync() + { + var opusResult = await CmsTrackService.GetOpusStatusAsync(); + if (opusResult.Success && opusResult.Value is not null) + { + _opusStatus = opusResult.Value; + } + + if (MissingOpusCount > 0) + { + EnsureOpusPollRunning(); + } + } + + // Start the Post-Processing poll if it is not already running. The loop re-reads opus-status every + // OpusPollIntervalMs and renders; it exits as soon as nothing is missing (or on cancel/dispose). Guarded + // by a non-null CTS so overlapping callers (init + backfill) cannot start two loops. + private void EnsureOpusPollRunning() + { + if (_opusPollCts is not null) + { + return; + } + + _opusPollCts = new CancellationTokenSource(); + _ = PollOpusStatusAsync(_opusPollCts.Token); + } + + private async Task PollOpusStatusAsync(CancellationToken ct) + { + try + { + while (!ct.IsCancellationRequested && MissingOpusCount > 0) + { + await Task.Delay(OpusPollIntervalMs, ct); + + var result = await CmsTrackService.GetOpusStatusAsync(); + if (result.Success && result.Value is not null) + { + _opusStatus = result.Value; + await InvokeAsync(StateHasChanged); + } + } + } + catch (OperationCanceledException) + { + // Expected on dispose / navigation away — nothing to do. + } + catch (Exception ex) + { + Logger.LogWarning(ex, "Opus Post-Processing poll stopped on an unexpected error."); + } + finally + { + _opusPollCts?.Dispose(); + _opusPollCts = null; + } + } + + public void Dispose() + { + _opusPollCts?.Cancel(); + } + // Invalidates the cached per-track waveform status on all embedded grids so the next row expand // re-fetches fresh data. Called after each catalogue-wide bulk run so already-expanded rows // reflect the new waveform state on the next expand interaction. @@ -291,4 +407,54 @@ Snackbar.Add($"Backfilled {succeeded} high-res datum(s); {failures} failed.", Severity.Warning); } } + + /// + /// Kick off the catalogue-wide Backfill-Opus pass. The API enumerates the tracks lacking a complete Opus + /// artifact, enqueues a background derive for each, and returns the (enqueued, skipped) counts. This is a + /// single scheduling call — the transcodes run server-side afterward — so there is no per-track progress + /// to render here, just a busy flag and a result snackbar. Re-runnable: a second press only schedules + /// tracks still missing Opus. + /// + private async Task BackfillOpusAsync() + { + _opusBackfillRunning = true; + StateHasChanged(); + try + { + var result = await CmsTrackService.BackfillOpusAsync(); + if (!result.Success) + { + var error = result.Messages.FirstOrDefault()?.Message ?? "Failed to start the Opus backfill."; + Snackbar.Add(error, Severity.Error); + return; + } + + // Re-read the status map so the "missing N" badge reflects the just-enqueued work and the + // Post-Processing poll arms to watch the transcodes settle from N→0 as each finishes. + await RefreshOpusStatusAsync(); + + var (enqueued, skipped) = (result.Value.Enqueued, result.Value.Skipped); + if (enqueued == 0) + { + Snackbar.Add($"All {skipped} track(s) already have Opus — nothing to backfill.", Severity.Info); + } + else + { + Snackbar.Add( + $"Scheduled {enqueued} Opus transcode(s) in the background ({skipped} already had Opus). " + + "They will appear as each finishes.", + Severity.Success); + } + } + catch (Exception ex) + { + Logger.LogError(ex, "Opus backfill failed to start"); + Snackbar.Add("Failed to start the Opus backfill.", Severity.Error); + } + finally + { + _opusBackfillRunning = false; + StateHasChanged(); + } + } } diff --git a/DeepDrftManager/Services/CmsTrackService.cs b/DeepDrftManager/Services/CmsTrackService.cs index 663312e..622ba31 100644 --- a/DeepDrftManager/Services/CmsTrackService.cs +++ b/DeepDrftManager/Services/CmsTrackService.cs @@ -765,6 +765,89 @@ public class CmsTrackService : ICmsTrackService } } + public async Task> BackfillOpusAsync(CancellationToken ct = default) + { + var client = _httpClientFactory.CreateClient(ContentCmsClientName); + + HttpResponseMessage response; + try + { + response = await client.PostAsync("api/track/opus/backfill", null, ct); + } + catch (Exception ex) + { + _logger.LogError(ex, "Content API call failed for Opus backfill"); + return ResultContainer.CreateFailResult("Content API is unreachable."); + } + + using (response) + { + if (!response.IsSuccessStatusCode) + { + var body = await response.Content.ReadAsStringAsync(ct); + _logger.LogError("Content API Opus backfill failed: {Status} {Body}", (int)response.StatusCode, body); + return ResultContainer.CreateFailResult("Failed to start the Opus backfill."); + } + + OpusBackfillResult payload; + try + { + payload = await response.Content.ReadFromJsonAsync(ct); + } + catch (Exception ex) + { + _logger.LogError(ex, "Failed to deserialize Opus backfill response from Content API"); + return ResultContainer.CreateFailResult("Content API returned an unexpected response."); + } + + return ResultContainer.CreatePassResult(payload); + } + } + + public async Task> GetOpusStatusAsync(CancellationToken ct = default) + { + var client = _httpClientFactory.CreateClient(ContentCmsClientName); + + HttpResponseMessage response; + try + { + response = await client.GetAsync("api/track/opus-status", ct); + } + catch (Exception ex) + { + _logger.LogError(ex, "Content API call failed for Opus status"); + return ResultContainer.CreateFailResult("Content API is unreachable."); + } + + using (response) + { + if (!response.IsSuccessStatusCode) + { + _logger.LogError("Content API Opus status failed: {Status}", (int)response.StatusCode); + return ResultContainer.CreateFailResult("Failed to load Opus status."); + } + + OpusStatusDto[]? status; + try + { + status = await response.Content.ReadFromJsonAsync(ct); + } + catch (Exception ex) + { + _logger.LogError(ex, "Failed to deserialize Opus status from Content API response"); + return ResultContainer.CreateFailResult("Content API returned an unexpected response."); + } + + if (status is null) + { + _logger.LogError("Content API returned a null Opus status list"); + return ResultContainer.CreateFailResult("Content API returned an empty response."); + } + + return ResultContainer.CreatePassResult(status); + } + } + public async Task>> GetReleasesAsync(CancellationToken ct = default) { var client = _httpClientFactory.CreateClient(ContentCmsClientName); diff --git a/DeepDrftManager/Services/ICmsTrackService.cs b/DeepDrftManager/Services/ICmsTrackService.cs index 3796c51..57e1c36 100644 --- a/DeepDrftManager/Services/ICmsTrackService.cs +++ b/DeepDrftManager/Services/ICmsTrackService.cs @@ -152,6 +152,23 @@ public interface ICmsTrackService /// Task GenerateHighResWaveformAsync(string entryKey, CancellationToken ct = default); + /// + /// Trigger the catalogue-wide Backfill-Opus pass via POST api/track/opus/backfill (Phase 18.5). + /// The API enqueues a background Opus derive for every track lacking a complete Opus artifact and returns + /// the (enqueued, skipped) counts. Enqueue-only — the transcodes run server-side on a serial background + /// worker, so this call returns as soon as the work is scheduled, not when transcoding finishes. The + /// Enqueued count is how many derives were scheduled; Skipped is how many already had Opus. + /// + Task> BackfillOpusAsync(CancellationToken ct = default); + + /// + /// Fetch per-track Opus derive status from GET api/track/opus-status (Phase 18.6) for the CMS + /// Post-Processing surfaces. Unpaged — the admin catalogue is small. Each row's HasOpus is true only + /// when the track carries a complete Opus artifact (audio + sidecar). Drives the Backfill-Opus "missing N" + /// badge and the post-upload Post-Processing poll. Idempotent read — safe to poll on an interval. + /// + Task> GetOpusStatusAsync(CancellationToken ct = default); + /// Returns all releases with track counts from GET api/track/albums. Task>> GetReleasesAsync(CancellationToken ct = default); @@ -160,3 +177,11 @@ public interface ICmsTrackService /// Task> GetTrackCountAsync(CancellationToken ct = default); } + +/// +/// Outcome of a Backfill-Opus pass (Phase 18.5): how many tracks had a background derive scheduled +/// () and how many were skipped because they already carry a complete Opus +/// artifact (). Both are counts of tracks, not finished transcodes — the work +/// runs asynchronously on the API's background worker after this returns. +/// +public readonly record struct OpusBackfillResult(int Enqueued, int Skipped); diff --git a/DeepDrftManager/appsettings.json b/DeepDrftManager/appsettings.json index f1dc861..30a88ba 100644 --- a/DeepDrftManager/appsettings.json +++ b/DeepDrftManager/appsettings.json @@ -1,8 +1,7 @@ { "Logging": { "LogLevel": { - "Default": "Information", - "Microsoft.AspNetCore": "Warning" + "Default": "Warning" } }, "AllowedHosts": "*", diff --git a/DeepDrftModels/DTOs/OpusStatusDto.cs b/DeepDrftModels/DTOs/OpusStatusDto.cs new file mode 100644 index 0000000..a6edb40 --- /dev/null +++ b/DeepDrftModels/DTOs/OpusStatusDto.cs @@ -0,0 +1,19 @@ +namespace DeepDrftModels.DTOs; + +/// +/// Per-track Opus derive status for the CMS Post-Processing surfaces (Phase 18.6). Mirrors +/// : one row per track, flagging whether the track already carries a +/// complete Opus artifact. "Complete" means BOTH the Opus audio bytes AND the seek/setup +/// sidecar are present in the track-opus vault — a half-derived track (audio without sidecar) is +/// unseekable and counts as missing, so the Backfill-Opus pass re-derives it. is the +/// vault key the per-track enqueue trigger and the polling Post-Processing affordance key on. +/// +public class OpusStatusDto +{ + public long TrackId { get; set; } + public string EntryKey { get; set; } = string.Empty; + public string TrackName { get; set; } = string.Empty; + + /// True only when both the Opus audio and the seek/setup sidecar are stored (a complete derive). + public bool HasOpus { get; set; } +} diff --git a/DeepDrftModels/Enums/AudioFormat.cs b/DeepDrftModels/Enums/AudioFormat.cs new file mode 100644 index 0000000..9b95067 --- /dev/null +++ b/DeepDrftModels/Enums/AudioFormat.cs @@ -0,0 +1,24 @@ +namespace DeepDrftModels.Enums; + +/// +/// The delivery format a listener requests for a track's audio (Phase 18). One TrackEntity / +/// EntryKey addresses both renderings — "one source, multiple views" applied to delivery (C5). +/// Lives here, not in the content library, because it is a cross-boundary contract: the API stream +/// endpoint (18.3) parses it off the ?format= query param, the WASM client (18.4 / 18.6) selects +/// it, and the content-side resolver (18.2) resolves it to bytes — all three reference one enum. +/// +public enum AudioFormat +{ + /// + /// The existing source artifact in the tracks vault, served byte-for-byte with its real MIME + /// (WAV/MP3/FLAC — do not assume WAV). The universal, always-present rendering. + /// + Lossless, + + /// + /// The derived low-data Ogg Opus 320 artifact in the track-opus vault (audio/ogg). A + /// best-effort derived artifact: when absent (not yet transcoded, or transcode failed) a request for + /// it falls back to rather than 404ing (C2). + /// + Opus +} diff --git a/DeepDrftPublic.Client/CLAUDE.md b/DeepDrftPublic.Client/CLAUDE.md index 97468d7..4cadde0 100644 --- a/DeepDrftPublic.Client/CLAUDE.md +++ b/DeepDrftPublic.Client/CLAUDE.md @@ -51,7 +51,7 @@ All interactive UI for the site. Blazor WebAssembly. Pages, controls, the stream - `StreamingAudioPlayerService`: Production implementation. Chunked stream from `TrackMediaClient`, adaptive 16–64 KB buffer, early-playback, **seek-beyond-buffer** via offset request to the content API. - `AudioInteropService`: JS interop wrapper over `window.DeepDrftAudio`. Manages `DotNetObjectReference` lifetimes for progress, end-of-playback, spectrum callbacks. - Dark-mode services: `DarkModeServiceBase` (cookie name constant), `DarkModeCookieService` (JS cookie read/write). - - `WaveformVisualizerControlState`: Scoped session-persistent holder for the visualizer's **eight** continuous control positions, **two subsystem on/off toggles** (Phase 15), and one **Theater-Mode flag** (Phase 20): `ScrollSpeed`, `GradientRotationSpeed`, `LavaGravity`, `LavaHeat`, `FluidAmount` (wax count/volume), `FluidViscosity` (cohesion — the second half of the Phase 10 "bubbles" split; `BlobDensity` is gone), `CollisionStrength`, `WaveformWidth`, `LavaEnabled` (bool, default `true`), `WaveformEnabled` (bool, default `true`), `TheaterMode` (bool, default `false` — `DefaultTheaterMode`). Each has a matching `Default*` const. `Changed` event is the decoupling seam — controls mutate state + raise `Changed`; the bridge (`WaveformVisualizer`) subscribes and pushes the affected uniform or subsystem-enable; the Theater observers (the three detail pages and `AudioPlayerBar`) subscribe to react to `TheaterMode`. **`CoerceTheaterMode()`**: enforces the invariant that Theater Mode cannot remain on when both subsystems are off — called from `WaveformVisualizerControls.ToggleLava`/`ToggleWaveform` **before** `NotifyChanged()` so all observers see a consistent, coerced state in the same `Changed` cycle. `TheaterMode` is a page-chrome presentation flag; the visualizer bridge ignores it. Scoped DI so state survives SPA nav within a session and resets on fresh page load. **Phase 20 Wave 2 — playing-release predicates** live in `ReleaseDetailBase` / `CutDetailBase` (not in this state holder): `IsThisReleasePlaying` (`PlayerService?.CurrentTrack?.Release?.EntryKey == EntryKey`), `IsContentHidden` (`TheaterMode && IsThisReleasePlaying`), `ShowTheaterToggle` (`(LavaEnabled || WaveformEnabled) && IsThisReleasePlaying`). Both base classes also subscribe to `IStreamingPlayerService.StateChanged` (idempotent, reference-guarded, disposed) so the predicates re-evaluate live when playback moves between releases. + - `WaveformVisualizerControlState`: Scoped session-persistent holder for the visualizer's **eight** continuous control positions, **two subsystem on/off toggles** (Phase 15), and one **Theater-Mode flag** (Phase 20): `ScrollSpeed`, `GradientRotationSpeed`, `LavaGravity`, `LavaHeat`, `FluidAmount` (wax count/volume), `FluidViscosity` (cohesion — the second half of the Phase 10 "bubbles" split; `BlobDensity` is gone), `CollisionStrength`, `WaveformWidth`, `LavaEnabled` (bool, default `true`), `WaveformEnabled` (bool, default `true`), `TheaterMode` (bool, default `false` — `DefaultTheaterMode`). Each has a matching `Default*` const. `Changed` event is the decoupling seam — controls mutate state + raise `Changed`; the bridge (`WaveformVisualizer`) subscribes and pushes the affected uniform or subsystem-enable; the Theater observers (the three detail pages and `AudioPlayerBar`) subscribe to react to `TheaterMode`. **`CoerceTheaterMode()`**: enforces the invariant that Theater Mode cannot remain on when both subsystems are off — called from `WaveformVisualizerControls.ToggleLava`/`ToggleWaveform` **before** `NotifyChanged()` so all observers see a consistent, coerced state in the same `Changed` cycle. **`ApplyCapabilityDefault(bool hardwareAccelerated)`**: one-time scoped capability default (guarded by `_capabilityDefaultApplied`; never re-applies on SPA navigation, never overrides an explicit in-session toggle). When `hardwareAccelerated` is false (positive software-renderer match from `hwAccel.ts`'s `UNMASKED_RENDERER_WEBGL` probe, or total WebGL failure) sets `LavaEnabled = false` while leaving `WaveformEnabled` at its default on, then calls `CoerceTheaterMode()` + `NotifyChanged()` once so all observers see the default in a single cycle. Called by the visualizer bridge on first interactive render once JS interop (the HW-accel probe via `detectHardwareAcceleration()` exported from `WaveformVisualizer.ts`) is available; a no-op when HW accel is present. `TheaterMode` is a page-chrome presentation flag; the visualizer bridge ignores it. Scoped DI so state survives SPA nav within a session and resets on fresh page load. **Phase 20 Wave 2 — playing-release predicates** live in `ReleaseDetailBase` / `CutDetailBase` (not in this state holder): `IsThisReleasePlaying` (`PlayerService?.CurrentTrack?.Release?.EntryKey == EntryKey`), `IsContentHidden` (`TheaterMode && IsThisReleasePlaying`), `ShowTheaterToggle` (`(LavaEnabled || WaveformEnabled) && IsThisReleasePlaying`). Both base classes also subscribe to `IStreamingPlayerService.StateChanged` (idempotent, reference-guarded, disposed) so the predicates re-evaluate live when playback moves between releases. - `PlayTracker`: Per-session play-session tracker (Phase 16 wave 16.1). Opens on playback start, advances a high-water position on each progress tick (from `StreamingAudioPlayerService` — not the HTTP layer, so seek-beyond-buffer re-fetches are the same play), closes on track-switch / stop / organic-end / page-unload. Engagement floor: ≥3 s OR ≥5% of duration. Three-bucket classification (`partial`/`sampled`/`complete`). Emits at most one event per session via `IPlayEventSink`. No player or JS dependency — testable against a fake sink. - `ShareTracker`: Per-session share tracker (Phase 16 wave 16.1). Called by `SharePopover` after a successful clipboard write; applies a 60-second per-(target, channel) debounce. Sends via `BeaconInterop`. Scoped so debounce memory resets on fresh page load. **Wave 16.3:** injects `IAnonIdProvider`; attaches `_anonId.Current` to `ShareEventDto.AnonId` (omitted when null). - `BeaconInterop`: `navigator.sendBeacon` JS interop wrapper (Phase 16 wave 16.1). Fires JSON payloads to `api/event/{play,share}` fire-and-forget. Also wires a page-unload handler that flushes any pending play event when the page is torn down. diff --git a/DeepDrftPublic.Client/Clients/TrackMediaClient.cs b/DeepDrftPublic.Client/Clients/TrackMediaClient.cs index f38d1d7..191cd2e 100644 --- a/DeepDrftPublic.Client/Clients/TrackMediaClient.cs +++ b/DeepDrftPublic.Client/Clients/TrackMediaClient.cs @@ -2,6 +2,8 @@ using System.Net; using System.Net.Http.Headers; using System.Net.Http.Json; using DeepDrftModels.DTOs; +using DeepDrftModels.Enums; +using Microsoft.AspNetCore.Components.WebAssembly.Http; using Microsoft.Extensions.DependencyInjection; using NetBlocks.Models; @@ -18,13 +20,24 @@ public class TrackMediaResponse : IDisposable /// public string ContentType { get; } + /// + /// The total file length in bytes, parsed from the 206 response's Content-Range: + /// bytes start-end/TOTAL header (Phase 21 Direction B). Null when the server returned + /// 200 (no Content-Range) — callers fall back to as the total. + /// This is the EOF boundary the segment loop advances its cursor toward, and the full + /// logical length the JS decoder must see (so a bounded segment's small Content-Length + /// never trips the decoder's byte-count completion early). + /// + public long? TotalLength { get; } + private readonly HttpResponseMessage _response; - public TrackMediaResponse(Stream stream, long contentLength, string contentType, HttpResponseMessage response) + public TrackMediaResponse(Stream stream, long contentLength, string contentType, long? totalLength, HttpResponseMessage response) { Stream = stream; ContentLength = contentLength; ContentType = contentType; + TotalLength = totalLength; _response = response; } @@ -45,24 +58,61 @@ public class TrackMediaClient } /// - /// Fetches the WAV stream for a track via an HTTP Range request starting at a + /// Fetches the audio stream for a track via an HTTP Range request starting at a /// file-absolute byte offset. is the position from - /// the start of the file on disk (including the WAV header) — callers seeking into - /// audio data must add the header size themselves. The cancellation token aborts - /// the in-flight server connection rather than leaving the server draining bytes - /// into a dead socket. + /// the start of the file on disk (including any container/header bytes) — callers + /// seeking into audio data must add the header size themselves. The cancellation + /// token aborts the in-flight server connection rather than leaving the server + /// draining bytes into a dead socket. + /// + /// (Phase 21 Direction B) bounds the request to a single + /// segment: when set, the Range header is bytes={byteOffset}-{byteEnd} (inclusive), + /// so the browser holds at most ~one segment of raw bytes regardless of file size — the + /// network-memory bound this phase exists for. When null the request is open-ended + /// (bytes={byteOffset}-), the pre-Direction-B behaviour. Either way the response's + /// Content-Range total is surfaced via + /// so the caller knows the EOF boundary and the full logical length the decoder must see. + /// + /// + /// selects the delivery rendering (Phase 18): the default + /// sends no format query param, so existing + /// callers hit the byte-identical pre-Phase-18 endpoint; + /// requests the low-data Ogg Opus artifact, which the server resolves and falls back to + /// lossless when absent (C2). The response + /// reports the format actually served, so the JS decoder dispatches on the real bytes. + /// /// public async Task> GetTrackMedia( string trackId, long byteOffset = 0, + long? byteEnd = null, + AudioFormat format = AudioFormat.Lossless, CancellationToken cancellationToken = default) { try { - // Same URL for every seek — only the Range header differs. byteOffset 0 is + // Same URL for every fetch — only the Range header differs. byteOffset 0 is // not special-cased: "bytes=0-" requests the whole file from the start. - using var request = new HttpRequestMessage(HttpMethod.Get, $"api/track/{trackId}"); - request.Headers.Range = new RangeHeaderValue(byteOffset, null); + // Lossless omits the format param entirely so the request is byte-identical to + // the pre-Phase-18 endpoint; only Opus appends ?format=opus. + var uri = format == AudioFormat.Lossless + ? $"api/track/{trackId}" + : $"api/track/{trackId}?format={format.ToString().ToLowerInvariant()}"; + using var request = new HttpRequestMessage(HttpMethod.Get, uri); + // Bounded (byteEnd set) → "bytes=start-end" so the server returns a finite 206 + // slice and the browser buffers only that segment; open-ended (byteEnd null) → + // "bytes=start-". The server honours both via File(..., enableRangeProcessing: true), + // which parses the full RFC 7233 range grammar and slices accordingly. + request.Headers.Range = new RangeHeaderValue(byteOffset, byteEnd); + + // Stream the response body incrementally instead of buffering it whole (Phase 21.4 fix). + // In Blazor WebAssembly the HttpClient is backed by the browser fetch API; without this the + // browser buffers the ENTIRE body before the response stream yields a byte. With Direction B + // each request is already bounded to one segment, so the body is small regardless — but + // streaming still lets us read it incrementally and is harmless on the SSR server-to-server + // path (SocketsHttpHandler ignores the unknown option). Kept for both the initial and the + // seek/refill paths since both share this method. + request.SetBrowserResponseStreamingEnabled(true); // Use HttpCompletionOption.ResponseHeadersRead to get stream immediately var response = await _http.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, cancellationToken); @@ -72,11 +122,15 @@ public class TrackMediaClient // Default to WAV when the server omits the header — the only format shipping // today — so the JS factory always receives a usable media type. var contentType = response.Content.Headers.ContentType?.MediaType ?? "audio/wav"; + // Content-Range "bytes start-end/TOTAL" carries the full file length on a 206; on a 200 + // there is no Content-Range, so TotalLength is null and callers use ContentLength. + var totalLength = response.Content.Headers.ContentRange?.Length; var stream = await response.Content.ReadAsStreamAsync(cancellationToken); // TrackMediaResponse takes ownership of both stream and response; // do NOT dispose response here — the caller disposes via TrackMediaResponse.Dispose(). - return ApiResult.CreatePassResult(new TrackMediaResponse(stream, contentLength, contentType, response)); + return ApiResult.CreatePassResult( + new TrackMediaResponse(stream, contentLength, contentType, totalLength, response)); } catch (Exception e) { @@ -115,4 +169,33 @@ public class TrackMediaClient return ApiResult.CreateFailResult(e.Message); } } + + /// + /// Fetches a track's Opus seek/setup sidecar — the combined OpusHead/OpusTags setup header plus the + /// granule→byte seek index (Phase 18). The caller (18.5 player wiring) fetches this once on track load + /// and parses it into the JS-side OpusSeekData before issuing any Opus seek. A 404 means no Opus + /// artifact / sidecar exists for the track (legacy row, not backfilled, or transcode failed); callers + /// treat that as "this track has no Opus seek data — stay on lossless" rather than an error, so it + /// surfaces as a fail result with a stable message rather than throwing (mirrors GetWaveformProfileAsync). + /// + public async Task> GetOpusSidecarAsync(string trackId, CancellationToken cancellationToken = default) + { + try + { + var response = await _http.GetAsync($"api/track/{trackId}/opus/seekdata", cancellationToken); + if (response.StatusCode == HttpStatusCode.NotFound) + { + return ApiResult.CreateFailResult("No Opus sidecar available"); + } + + response.EnsureSuccessStatusCode(); + + var bytes = await response.Content.ReadAsByteArrayAsync(cancellationToken); + return ApiResult.CreatePassResult(bytes); + } + catch (Exception e) + { + return ApiResult.CreateFailResult(e.Message); + } + } } diff --git a/DeepDrftPublic.Client/Common/PublicSiteSettings.cs b/DeepDrftPublic.Client/Common/PublicSiteSettings.cs new file mode 100644 index 0000000..1b83b16 --- /dev/null +++ b/DeepDrftPublic.Client/Common/PublicSiteSettings.cs @@ -0,0 +1,29 @@ +using Microsoft.AspNetCore.Components; + +namespace DeepDrftPublic.Client.Common; + +/// +/// The single public-site listener-settings object (Phase 18 wave 18.6, §4a). The generalized analogue of +/// : one scoped holder for every remembered listener preference, seeded at +/// server prerender, carried into WASM via , and persisted to a cookie on +/// change. Today it carries one preference — streaming quality; tomorrow dark mode (and whatever follows) +/// folds in here as another property without disturbing the menu that reads it. +/// +/// Built design-for-adaptability per §4a: a new preference is a new [PersistentState] property here +/// plus a new in the menu — not a rewire. Dark mode is intentionally +/// not migrated in now (it keeps its own seam); this object is shaped +/// so that consolidation is later a merge of two identical seams, not a reconciliation of two different ones. +/// +/// +public class PublicSiteSettings +{ + /// + /// The listener's streaming-quality preference. Defaults to (Opus, + /// capability-gated — OQ2). Seeded from the streamQuality cookie at prerender; persisted on change + /// by the client cookie service. The player reads this to decide which ?format= to request, but + /// the capability gate and C2 fallback still apply on top, so a + /// preference never forces an unplayable stream. + /// + [PersistentState] + public StreamQuality StreamQuality { get; set; } = StreamQuality.LowData; +} diff --git a/DeepDrftPublic.Client/Common/SettingsItem.cs b/DeepDrftPublic.Client/Common/SettingsItem.cs new file mode 100644 index 0000000..9cdfa6b --- /dev/null +++ b/DeepDrftPublic.Client/Common/SettingsItem.cs @@ -0,0 +1,12 @@ +using Microsoft.AspNetCore.Components; + +namespace DeepDrftPublic.Client.Common; + +/// +/// One entry in the public-site Settings menu (Phase 18 wave 18.6, §4a). The settings-item abstraction the +/// menu renders instead of a hard-coded control list: a plus a +/// fragment bound to a persisted preference. Adding a future tenant (e.g. dark mode) is appending one of +/// these — not rewiring the menu. The control fragment owns its own binding to +/// and its own persistence call, so each item is self-contained and the menu stays preference-agnostic. +/// +public sealed record SettingsItem(string Label, RenderFragment Control); diff --git a/DeepDrftPublic.Client/Common/StreamQuality.cs b/DeepDrftPublic.Client/Common/StreamQuality.cs new file mode 100644 index 0000000..5dd1508 --- /dev/null +++ b/DeepDrftPublic.Client/Common/StreamQuality.cs @@ -0,0 +1,18 @@ +namespace DeepDrftPublic.Client.Common; + +/// +/// The listener's streaming-quality preference (Phase 18 wave 18.6, §4). This is the user's intent, +/// not the wire format that ultimately gets served: means "give me Opus if you can," +/// but the player still capability-gates and C2-falls-back to lossless when Opus can't play (a browser that +/// can't decode Ogg Opus, or a track with no Opus artifact). It is therefore deliberately distinct from +/// DeepDrftModels.Enums.AudioFormat (the delivery rendering resolved per request): one is the +/// remembered preference, the other is what a given stream request actually asks for. +/// +public enum StreamQuality +{ + /// Bandwidth-friendly Opus (capability-gated; the default before any choice — OQ2). + LowData, + + /// The lossless WAV path, always playable everywhere. + Lossless +} diff --git a/DeepDrftPublic.Client/Controls/AudioPlayerProvider.razor.cs b/DeepDrftPublic.Client/Controls/AudioPlayerProvider.razor.cs index 5202023..22e4a8a 100644 --- a/DeepDrftPublic.Client/Controls/AudioPlayerProvider.razor.cs +++ b/DeepDrftPublic.Client/Controls/AudioPlayerProvider.razor.cs @@ -1,3 +1,4 @@ +using DeepDrftPublic.Client.Common; using DeepDrftPublic.Client.Services; using DeepDrftPublic.Client.Clients; using Microsoft.AspNetCore.Components; @@ -13,6 +14,7 @@ public partial class AudioPlayerProvider : ComponentBase, IAsyncDisposable [Inject] public required BeaconInterop Beacon { get; set; } [Inject] public required IPlayEventSink PlayEventSink { get; set; } [Inject] public required IAnonIdProvider AnonId { get; set; } + [Inject] public required PublicSiteSettings Settings { get; set; } private IStreamingPlayerService? _audioPlayerService; private QueueService? _queueService; @@ -26,7 +28,12 @@ public partial class AudioPlayerProvider : ComponentBase, IAsyncDisposable // EnsureInitializedAsync — that path is correct because audio contexts // require a user gesture anyway. Initializing eagerly here causes 4+ // SignalR round-trips before any content is stable. - var player = new StreamingAudioPlayerService(AudioInterop, TrackMediaClient, Logger); + // Construct the preference-aware player (Phase 18 wave 18.6): it honours the listener's streaming- + // quality choice via the ResolveStreamFormatAsync seam while inheriting the 18.5 capability gate and + // C2 fallback. PublicSiteSettings is scoped data (already prerender-seeded + WASM-bridged), so passing + // it through the constructor is cheap and carries no lifecycle — the telemetry tracker still binds + // post-construction below, exactly as before. + var player = new PreferenceAwareStreamingPlayerService(AudioInterop, TrackMediaClient, Logger, Settings); // Phase 16: bind the play-session tracker to the player after construction, the same way the // queue binds — the player is built with `new`, not DI, so threading telemetry through its diff --git a/DeepDrftPublic.Client/Controls/Settings/SettingsMenu.razor b/DeepDrftPublic.Client/Controls/Settings/SettingsMenu.razor new file mode 100644 index 0000000..0e6d884 --- /dev/null +++ b/DeepDrftPublic.Client/Controls/Settings/SettingsMenu.razor @@ -0,0 +1,55 @@ +@using DeepDrftPublic.Client.Common +@using DeepDrftPublic.Client.Controls.Settings +@using DeepDrftPublic.Client.Services + +@* + The public-site Settings menu (Phase 18 wave 18.6, §4a). An app-bar trigger opening a MudMenu that renders + a settings-item list — NOT a hard-coded control stack. Each entry is a SettingsItem (label + a control + fragment bound to a persisted preference), so a future tenant (dark mode) plugs in as a new list entry, not + a menu rewire. Today the list holds one item: the streaming-quality toggle. + + The MudMenu items carry OnClick="@(() => {})" + OnTouch so a click inside a control row does not dismiss the + menu (MudMenu auto-closes on item activation otherwise), keeping the radio group usable. +*@ +
+ +
+
Settings
+ @foreach (var item in _items) + { +
+
@item.Label
+ @item.Control +
+ } +
+ +
+ +@code { + // The active player, cascaded by AudioPlayerProvider. SettingsMenu sits in the app bar inside the + // provider, so it receives the cascade here — but the MudMenu PANEL content below is portaled to + // (outside the provider), so a cascade cannot reach it. We thread the player into + // the setting as an explicit parameter instead: an explicit value captured in the item fragment flows + // into portaled content fine, where a [CascadingParameter] would land null. + [CascadingParameter] public IStreamingPlayerService? Player { get; set; } + + // The settings-item list. Built once; adding a preference is appending one SettingsItem with its control + // fragment — the menu body above renders whatever is here without knowing what each item is. The item + // fragment reads Player at render time (when the menu opens), so it picks up the cascaded instance even + // though the list itself is initialized before the cascade is set. + private readonly List _items; + + public SettingsMenu() + { + _items = + [ + new SettingsItem("Streaming quality", @) + ]; + } +} diff --git a/DeepDrftPublic.Client/Controls/Settings/StreamQualitySetting.razor b/DeepDrftPublic.Client/Controls/Settings/StreamQualitySetting.razor new file mode 100644 index 0000000..1dbdf19 --- /dev/null +++ b/DeepDrftPublic.Client/Controls/Settings/StreamQualitySetting.razor @@ -0,0 +1,97 @@ +@using DeepDrftPublic.Client.Common +@using DeepDrftPublic.Client.Services + +@* + The streaming-quality control (Phase 18 wave 18.6, §4) — the first occupant of the Settings menu. Binds + the listener's choice to PublicSiteSettings.StreamQuality and persists it via the cookie seam. Honest + capability gate (OQ2 / AC7): on a browser that cannot decode Ogg Opus the Low-data option still selects, + but a note tells the listener the effective stream is lossless — we never let the choice silently imply a + format that can't play. +*@ +
+ + + Low-data (Opus) + + + Lossless (WAV) + + + + @if (_opusUnavailable && _quality == StreamQuality.LowData) + { +
+ This browser can't decode Opus — you'll stream lossless. +
+ } +
+ + APPLY + +
+
+ +@code { + [Inject] public required PublicSiteSettings Settings { get; set; } + [Inject] public required SettingsCookieService CookieService { get; set; } + [Inject] public required AudioInteropService AudioInterop { get; set; } + + // The active player, threaded in from SettingsMenu (which reads it off the AudioPlayerProvider cascade). + // It is an explicit [Parameter], NOT a [CascadingParameter], because this control renders inside the + // MudMenu panel, which MudBlazor portals to — outside AudioPlayerProvider's cascade + // scope, so a cascade would land null here. Null during prerender or when no provider is present — Apply + // then just persists the preference, with no live track to restart. + [Parameter] public IStreamingPlayerService? Player { get; set; } + + private StreamQuality _quality; + + public bool IsApplyEnabled => _quality != Settings.StreamQuality; + + // Null until the capability probe runs (post-render JS interop). false → can decode Opus; true → cannot. + private bool _opusUnavailable; + + protected override void OnInitialized() + { + // Read the current preference (already seeded at prerender + bridged into WASM). + _quality = Settings.StreamQuality; + } + + protected override async Task OnAfterRenderAsync(bool firstRender) + { + if (!firstRender) return; + + // Capability probe is JS interop — only valid once interactive. Surfaces the honest note when the + // browser can't decode Ogg Opus, so a Low-data pick reads as "effectively lossless" rather than + // silently failing. The player applies the same gate independently; this is purely the UI honesty. + var canDecodeOpus = await AudioInterop.CanDecodeOggOpus(); + if (canDecodeOpus == _opusUnavailable) + { + _opusUnavailable = !canDecodeOpus; + StateHasChanged(); + } + } + + private async Task OnQualityChanged(StreamQuality quality) + { + _quality = quality; + } + + private async Task ApplyStreamQualitySetting(MouseEventArgs arg) + { + // Persist the choice first so the cookie + in-memory PublicSiteSettings.StreamQuality both reflect + // the new value BEFORE the restart: the reload re-resolves the delivery format via + // PreferenceAwareStreamingPlayerService, which reads PublicSiteSettings.StreamQuality fresh. + await CookieService.SetStreamQualityAsync(_quality); + + // Switch the currently-playing track to the new format immediately, preserving the listener's + // position. No-op inside the player when nothing is loaded — the new preference then simply + // applies to the next track played. Fire-and-forget: the reload runs the streaming loop for the + // life of the track, so awaiting it would pin this handler open; UI updates flow through the + // player's state notifications, not this await. + _ = Player?.ReloadPreservingPositionAsync(); + } + +} diff --git a/DeepDrftPublic.Client/Controls/WaveformVisualizer.razor.cs b/DeepDrftPublic.Client/Controls/WaveformVisualizer.razor.cs index 5fb5abd..e47c4fb 100644 --- a/DeepDrftPublic.Client/Controls/WaveformVisualizer.razor.cs +++ b/DeepDrftPublic.Client/Controls/WaveformVisualizer.razor.cs @@ -265,6 +265,24 @@ public partial class WaveformVisualizer : ComponentBase, IAsyncDisposable return; } + // Apply the hardware-capability default ONCE per session before seeding the controls: if the + // browser has no WebGL hardware acceleration, the lava subsystem (which software-renders on + // the main thread and starves audio decode) defaults off while the waveform stays on. The + // probe lives in JS (it needs a real WebGL context); the scoped state guards the one-time + // application, so a remounted visualizer never re-applies and a later explicit toggle is + // never clobbered. Sequenced before PushControlsAsync so the seed already carries the + // corrected enables; ApplyCapabilityDefault also raises Changed for the controls UI. + try + { + var hardwareAccelerated = await _module.InvokeAsync("detectHardwareAcceleration"); + ControlState.ApplyCapabilityDefault(hardwareAccelerated); + } + catch (JSException ex) + { + // A probe failure must not regress the HW-accel majority: leave the defaults (lava on). + Logger.LogWarning(ex, "WaveformVisualizer: hardware-acceleration probe failed; leaving lava at its default."); + } + // Seed the module with the current state now that it exists. All control values (the eight // dials + the two Phase 15 subsystem enables) come from the shared (session-persisted) state, // so a mix opened mid-session seeds the module with the knob/toggle positions the listener diff --git a/DeepDrftPublic.Client/Layout/DeepDrftMenu.razor b/DeepDrftPublic.Client/Layout/DeepDrftMenu.razor index f8808bf..27b7028 100644 --- a/DeepDrftPublic.Client/Layout/DeepDrftMenu.razor +++ b/DeepDrftPublic.Client/Layout/DeepDrftMenu.razor @@ -1,5 +1,6 @@ @using DeepDrftPublic.Client.Common @using DeepDrftPublic.Client.Controls +@using DeepDrftPublic.Client.Controls.Settings @using DeepDrftPublic.Client.Services @* Desktop Menu *@ @@ -42,6 +43,7 @@
+
@@ -74,7 +76,8 @@ @onclick="ToggleMobileMenu"> - + + diff --git a/DeepDrftPublic.Client/Layout/MainLayout.razor b/DeepDrftPublic.Client/Layout/MainLayout.razor index bdfc432..66ec0d5 100644 --- a/DeepDrftPublic.Client/Layout/MainLayout.razor +++ b/DeepDrftPublic.Client/Layout/MainLayout.razor @@ -42,6 +42,7 @@ @code { private string _audioPlayerClass = "minimized"; private const string DarkModeKey = "darkMode"; + private const string StreamQualityKey = "streamQuality"; private bool _isDarkMode = false; private bool? _lastAppliedDarkMode = null; private PersistingComponentStateSubscription _persistingSubscription; @@ -49,6 +50,7 @@ [Inject] public required PersistentComponentState PersistentState { get; set; } [Inject] public required DarkModeSettings DarkModeSettings { get; set; } + [Inject] public required PublicSiteSettings PublicSiteSettings { get; set; } [Inject] public required IJSRuntime JS { get; set; } protected override void OnInitialized() @@ -66,8 +68,17 @@ _isDarkMode = DarkModeSettings.IsDarkMode; } + // Restore the prerender-seeded streaming-quality preference (Phase 18 wave 18.6). Same bridge dark + // mode uses: the server SettingsService seeded PublicSiteSettings from the streamQuality cookie, and + // this carries it into WASM so the client boots already knowing the preference (no re-read flash, no + // wrong default before the first stream). + if (PersistentState.TryTakeFromJson(StreamQualityKey, out var restoredQuality)) + { + PublicSiteSettings.StreamQuality = restoredQuality; + } + // Register to persist state when prerendering completes - _persistingSubscription = PersistentState.RegisterOnPersisting(PersistDarkMode); + _persistingSubscription = PersistentState.RegisterOnPersisting(PersistState); } // Sync dark mode class on so portaled MudBlazor elements (popovers, menus, selects) @@ -91,9 +102,10 @@ // Theme wrapper class for CSS targeting private string ThemeWrapperClass => _isDarkMode ? "deepdrft-theme-dark" : "deepdrft-theme-light"; - private Task PersistDarkMode() + private Task PersistState() { PersistentState.PersistAsJson(DarkModeKey, _isDarkMode); + PersistentState.PersistAsJson(StreamQualityKey, PublicSiteSettings.StreamQuality); return Task.CompletedTask; } diff --git a/DeepDrftPublic.Client/Services/AudioInteropService.cs b/DeepDrftPublic.Client/Services/AudioInteropService.cs index 48ea050..e90c353 100644 --- a/DeepDrftPublic.Client/Services/AudioInteropService.cs +++ b/DeepDrftPublic.Client/Services/AudioInteropService.cs @@ -70,6 +70,37 @@ public class AudioInteropService : IAsyncDisposable return await InvokeJsAsync("DeepDrftAudio.initializeStreaming", playerId, totalStreamLength, contentType); } + /// + /// Probes whether this browser can stream-decode Ogg Opus via WebCodecs (AudioDecoder + + /// codec:'opus'; Safari < 16.4 / older Firefox cannot). Phase 18 capability gate (OQ2): the + /// player only requests Opus when this returns true, otherwise it stays on the universal lossless path + /// (AC7 — no listener ever gets silence over a codec gap). Probe failures degrade to false + /// (assume incapable) so an interop error can never silence playback. + /// + public async Task CanDecodeOggOpus() + { + try + { + return await _jsRuntime.InvokeAsync("DeepDrftAudio.canDecodeOggOpus"); + } + catch + { + return false; + } + } + + /// + /// Hands the raw Opus seek/setup sidecar bytes (setup header + granule→byte seek index) to the JS player + /// so the next Opus stream's decoder has them BEFORE init (the 18.4 set-before-init contract). The player + /// parses and stashes them; applies them when it builds the Opus decoder. + /// Must be called before on an Opus stream. Returns the parse result — + /// a failure means the bytes were not a valid sidecar, and the caller falls back to lossless. + /// + public async Task SetOpusSidecar(string playerId, byte[] sidecarBytes) + { + return await InvokeJsAsync("DeepDrftAudio.setOpusSidecar", playerId, sidecarBytes); + } + public async Task ProcessStreamingChunk(string playerId, byte[] audioChunk) { return await InvokeJsAsync("DeepDrftAudio.processStreamingChunk", playerId, audioChunk); @@ -115,6 +146,18 @@ public class AudioInteropService : IAsyncDisposable return await InvokeJsAsync("DeepDrftAudio.seek", playerId, position); } + /// + /// Resolve the file-absolute byte offset to begin a stream at with no + /// active playback or buffered audio — the "load at timestamp" seam (Phase 18 wave 18.6 format switch). + /// Returns on success; is + /// false when the decoder cannot yet resolve an offset (e.g. a WAV stream whose header has not been + /// parsed), so the caller can feed header bytes and retry. + /// + public async Task ResolveStreamOffsetAsync(string playerId, double position) + { + return await InvokeJsAsync("DeepDrftAudio.resolveStreamOffset", playerId, position); + } + // New methods for seek-beyond-buffer support public async Task GetBufferedDuration(string playerId) { @@ -128,11 +171,42 @@ public class AudioInteropService : IAsyncDisposable } } + /// + /// Phase 21.2a back-pressure poll: ask whether the scheduler is still over its forward + /// high/low-water band. The read loop calls this only WHILE already throttled, to learn when it + /// may resume reading — the steady-state loop reads the piggybacked ProductionPaused flag + /// off each chunk result instead. Defaults to false on any interop failure so a torn-down player + /// never wedges a loop that is exiting anyway. + /// + public async Task IsProductionPaused(string playerId) + { + try + { + return await _jsRuntime.InvokeAsync("DeepDrftAudio.isProductionPaused", playerId); + } + catch + { + return false; + } + } + public async Task ReinitializeFromOffset(string playerId, long totalStreamLength, double seekPosition) { return await InvokeJsAsync("DeepDrftAudio.reinitializeFromOffset", playerId, totalStreamLength, seekPosition); } + /// + /// Phase 21.3 / AC6 clean-failure recovery: after a window-miss refill (seek-back past the + /// retained tail) fails its Range fetch or reinit, halt the starved scheduler and leave the + /// player paused-but-loaded at so no silent false end fires and a + /// retry is possible. Routes through so an interop failure during + /// recovery still yields a failure result rather than throwing into the seek path. + /// + public async Task RecoverFromFailedRefill(string playerId, double seekPosition) + { + return await InvokeJsAsync("DeepDrftAudio.recoverFromFailedRefill", playerId, seekPosition); + } + public async Task SetVolumeAsync(string playerId, double volume) { return await InvokeJsAsync("DeepDrftAudio.setVolume", playerId, volume); @@ -388,6 +462,11 @@ public class StreamingResult : AudioOperationResult public bool HeaderParsed { get; set; } public int BufferCount { get; set; } public double? Duration { get; set; } // Duration in seconds calculated from WAV header + + // Phase 21.2a back-pressure: true when the scheduler's forward decoded fill is over the + // high-water mark and the C# read loop should stop calling ReadAsync until it drains. Read off + // the chunk result the loop already awaits — no extra interop hop in the unthrottled steady state. + public bool ProductionPaused { get; set; } } public class AudioPlayerState diff --git a/DeepDrftPublic.Client/Services/AudioPlayerService.cs b/DeepDrftPublic.Client/Services/AudioPlayerService.cs index 778cc1d..c33fe8f 100644 --- a/DeepDrftPublic.Client/Services/AudioPlayerService.cs +++ b/DeepDrftPublic.Client/Services/AudioPlayerService.cs @@ -118,6 +118,16 @@ public abstract class AudioPlayerService : IPlayerService, IAsyncDisposable IsPlaying = true; IsPaused = false; } + else if (IsPaused) + { + // Play failed while the player is paused — the scheduler may be empty after a + // failed refill (AC6 recovery). Re-issue a seek at the current position: the + // seek path routes to seekBeyondBuffer when the scheduler is empty (Phase 21.3 + // fix), triggering a real refetch rather than returning "Streaming not ready". + // We return early here; Seek owns its own state mutations and NotifyStateChanged. + await Seek(CurrentTime); + return; + } } if (!result.Success) @@ -128,7 +138,7 @@ public abstract class AudioPlayerService : IPlayerService, IAsyncDisposable { ErrorMessage = null; } - + await NotifyStateChanged(); } catch (Exception ex) @@ -298,7 +308,6 @@ public abstract class AudioPlayerService : IPlayerService, IAsyncDisposable /// protected virtual void OnPlaybackEnded() { } - protected async Task EnsureInitializedAsync() { if (!IsInitialized) diff --git a/DeepDrftPublic.Client/Services/DarkModeCookieService.cs b/DeepDrftPublic.Client/Services/DarkModeCookieService.cs index 177894e..30e6b2f 100644 --- a/DeepDrftPublic.Client/Services/DarkModeCookieService.cs +++ b/DeepDrftPublic.Client/Services/DarkModeCookieService.cs @@ -14,9 +14,7 @@ public class DarkModeCookieService(DarkModeSettings darkModeSetting, IJSRuntime public async ValueTask SetDarkModeAsync(bool isDarkMode) { - var expires = DateTime.UtcNow.AddDays(EXPIRY_DAYS).ToString("R"); - await js.InvokeVoidAsync("eval", - $"document.cookie = '{COOKIE_NAME}={isDarkMode.ToString().ToLower()}; expires={expires}; path=/; SameSite=Lax'"); + await js.InvokeVoidAsync("DeepDrftSettings.setCookie", COOKIE_NAME, isDarkMode.ToString().ToLower(), EXPIRY_DAYS); darkModeSetting.IsDarkMode = isDarkMode; } } \ No newline at end of file diff --git a/DeepDrftPublic.Client/Services/IPlayerService.cs b/DeepDrftPublic.Client/Services/IPlayerService.cs index c0ff051..33ffc56 100644 --- a/DeepDrftPublic.Client/Services/IPlayerService.cs +++ b/DeepDrftPublic.Client/Services/IPlayerService.cs @@ -91,4 +91,16 @@ public interface IStreamingPlayerService : IPlayerService /// and notifies; performs no JS interop. /// Task StageTrack(TrackDto track); + + /// + /// Re-streams the current track in the freshly-resolved delivery format while preserving the + /// listener's playback position (Phase 18 wave 18.6 — the Settings "Apply" restart). The format is + /// re-resolved on the new load via the ResolveStreamFormatAsync seam, so this picks up a just- + /// changed streaming-quality preference. A cross-format byte offset can only be resolved once the NEW + /// format's decoder has parsed its header, so the reload runs a fresh load from byte 0 to initialize + /// that decoder, then seeks back to the saved position through the existing seek-beyond-buffer path. + /// No-op when no track is loaded (nothing playing to switch). Safe to call while a track is playing; + /// a track switch during the brief restore window abandons the position restore. + /// + Task ReloadPreservingPositionAsync(); } \ No newline at end of file diff --git a/DeepDrftPublic.Client/Services/PreferenceAwareStreamingPlayerService.cs b/DeepDrftPublic.Client/Services/PreferenceAwareStreamingPlayerService.cs new file mode 100644 index 0000000..1375bed --- /dev/null +++ b/DeepDrftPublic.Client/Services/PreferenceAwareStreamingPlayerService.cs @@ -0,0 +1,49 @@ +using DeepDrftModels.Enums; +using DeepDrftPublic.Client.Clients; +using DeepDrftPublic.Client.Common; +using Microsoft.Extensions.Logging; + +namespace DeepDrftPublic.Client.Services; + +/// +/// The production player that honours the listener's streaming-quality preference (Phase 18 wave 18.6). +/// Extends through the single deliberately-overridable seam, +/// , so the rest of the streaming stack +/// (seek, telemetry, the seek-beyond-buffer format reuse) is inherited verbatim. +/// +/// The override is one branch: a preference returns +/// immediately; anything else falls through to base, which keeps +/// the 18.5 invariants intact — the capability gate (AC7: a browser that can't decode Ogg Opus gets lossless) +/// and the sidecar-absent → lossless fallback (C2: a legacy / un-backfilled / failed-transcode track gets +/// lossless). So a Lossless pick always yields lossless; a Low-data pick yields Opus only when it can +/// actually play, and lossless otherwise. No path produces an unplayable stream. +/// +/// +public class PreferenceAwareStreamingPlayerService : StreamingAudioPlayerService +{ + private readonly PublicSiteSettings _settings; + + public PreferenceAwareStreamingPlayerService( + AudioInteropService audioInterop, + TrackMediaClient trackMediaClient, + ILogger logger, + PublicSiteSettings settings) + : base(audioInterop, trackMediaClient, logger) + { + _settings = settings; + } + + protected override async Task ResolveStreamFormatAsync(string entryKey, CancellationToken cancellationToken) + { + // Listener explicitly chose lossless — request it directly, no Opus probe / sidecar fetch needed. + if (_settings.StreamQuality == StreamQuality.Lossless) + { + return AudioFormat.Lossless; + } + + // Low-data preference: defer to the base capability-gated resolution, which probes Opus support and + // the sidecar's presence and degrades to lossless when either is missing. Both 18.5 invariants are + // inherited here, not re-implemented. + return await base.ResolveStreamFormatAsync(entryKey, cancellationToken); + } +} diff --git a/DeepDrftPublic.Client/Services/SettingsCookieService.cs b/DeepDrftPublic.Client/Services/SettingsCookieService.cs new file mode 100644 index 0000000..d0bbce7 --- /dev/null +++ b/DeepDrftPublic.Client/Services/SettingsCookieService.cs @@ -0,0 +1,32 @@ +using DeepDrftPublic.Client.Common; +using Microsoft.JSInterop; + +namespace DeepDrftPublic.Client.Services; + +/// +/// Client-side runtime writer for public-site settings (Phase 18 wave 18.6), the analogue of +/// . Reads the current preference off the in-memory +/// (already seeded at prerender and bridged into WASM), and writes a +/// 365-day cookie via document.cookie interop when the listener changes it in the Settings menu — +/// the same durable-truth seam dark mode uses, so the choice survives the session and seeds the next visit's +/// prerender (no flash). +/// +public class SettingsCookieService(PublicSiteSettings settings, IJSRuntime js) : SettingsServiceBase +{ + private const int ExpiryDays = 365; + + public StreamQuality GetStreamQuality() => settings.StreamQuality; + + public async ValueTask SetStreamQualityAsync(StreamQuality quality) + { + if (settings.StreamQuality == quality) return; + + await WriteCookieAsync(StreamQualityCookieName, FormatStreamQuality(quality)); + settings.StreamQuality = quality; + } + + private async ValueTask WriteCookieAsync(string name, string value) + { + await js.InvokeVoidAsync("DeepDrftSettings.setCookie", name, value, ExpiryDays); + } +} diff --git a/DeepDrftPublic.Client/Services/SettingsServiceBase.cs b/DeepDrftPublic.Client/Services/SettingsServiceBase.cs new file mode 100644 index 0000000..73845c5 --- /dev/null +++ b/DeepDrftPublic.Client/Services/SettingsServiceBase.cs @@ -0,0 +1,28 @@ +using DeepDrftPublic.Client.Common; + +namespace DeepDrftPublic.Client.Services; + +/// +/// Shared cookie contract for the public-site settings seam (Phase 18 wave 18.6), the analogue of +/// . Holds the cookie names and the (de)serialization for each preference +/// so the server prerender-read service and the client cookie-write service agree on one wire format — +/// the load-bearing reason this is shared rather than duplicated. Each new preference adds its cookie name +/// and a parse/format pair here, keeping the round-trip in one place. +/// +public abstract class SettingsServiceBase +{ + protected const string StreamQualityCookieName = "streamQuality"; + + /// + /// Parses the streamQuality cookie value into , defaulting to + /// (the OQ2 default) for an absent, empty, or unrecognized value so + /// a missing/garbled cookie never produces a surprising preference. + /// + protected static StreamQuality ParseStreamQuality(string? cookieValue) => + Enum.TryParse(cookieValue, ignoreCase: true, out var parsed) + ? parsed + : StreamQuality.LowData; + + /// Formats a for cookie storage (round-trips with ). + protected static string FormatStreamQuality(StreamQuality quality) => quality.ToString(); +} diff --git a/DeepDrftPublic.Client/Services/StreamingAudioPlayerService.cs b/DeepDrftPublic.Client/Services/StreamingAudioPlayerService.cs index f795535..0d8c79a 100644 --- a/DeepDrftPublic.Client/Services/StreamingAudioPlayerService.cs +++ b/DeepDrftPublic.Client/Services/StreamingAudioPlayerService.cs @@ -1,4 +1,5 @@ using DeepDrftModels.DTOs; +using DeepDrftModels.Enums; using DeepDrftPublic.Client.Clients; using System.Buffers; using Microsoft.Extensions.Logging; @@ -15,6 +16,36 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS // Adaptive chunk sizing private const int MinBufferSize = 16 * 1024; // 16KB minimum private const int MaxBufferSize = 64 * 1024; // 64KB maximum + + // Phase 21.2a back-pressure poll interval. While the scheduler is over its forward high-water + // mark, the segment loop stops fetching the next segment and polls IsProductionPaused at this + // cadence until the fill drains below low-water. 100 ms is well under the low-water lookahead + // (seconds), so resume is prompt relative to the playhead — no starvation (AC3) — while keeping + // the poll cheap. The poll honors the loop's cancellation token, so a track switch/seek during a + // pause exits through the same drain discipline as a pause during ReadAsync (C6). + private const int BackpressurePollMs = 100; + + // Phase 21 Direction B — forward Range-segment size. The forward stream is fetched as a + // sequence of bounded "bytes=cursor-(cursor+SegmentSizeBytes-1)" 206 requests, the next issued + // only when the scheduler drains below low-water. Because each request is bounded and fully + // consumed before the next is issued, the browser fetch holds AT MOST ~one segment of raw bytes + // regardless of file size — this is the network-memory bound the phase exists for (the open-ended + // single GET buffered the whole ~970 MB body in the browser even when reads were paused, the + // 21.4 finding). 4 MB balances request overhead (a 1 GB mix is ~250 segments) against memory: + // at the 30 s high-water mark a fast connection holds well under a segment of unplayed raw bytes, + // so the bound is the segment size, not the decoded window. Tunable; not magic. + private const long SegmentSizeBytes = 4 * 1024 * 1024; + + // Phase 18 wave 18.6 — position-preserving format switch ("load at timestamp"). When the listener + // changes streaming quality mid-track, the new-format stream is started DIRECTLY at the saved + // position rather than from byte 0: the load resolves the byte offset for the target time in the + // freshly-initialized decoder and streams from there (never audibly playing the start). For WAV the + // byte-offset math needs the header, so the byte-0 segment is probed (fed to the decoder WITHOUT + // starting playback) until the header parses; this caps that probe so a header that never appears + // (corrupt stream) can't read unbounded. The decoder's own header-search ceiling is 256 KB, so this + // matches it. Opus needs no probe (its sidecar resolves offsets immediately after init). + private const int MaxHeaderProbeBytes = 256 * 1024; + private int _currentBufferSize = DefaultBufferSize; private int _consecutiveSlowReads = 0; @@ -33,6 +64,12 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS private readonly ILogger _logger; private string? _currentTrackId; + // The delivery format the active load resolved to (Phase 18). Captured once per LoadTrackStreaming and + // reused by the seek-beyond-buffer re-fetch so the Range continuation requests the SAME artifact the + // initial stream did — a seek must never switch formats mid-track (the JS decoder, the cached setup + // header, and the byte offsets all belong to one artifact). Defaults to Lossless until a load resolves. + private AudioFormat _currentFormat = AudioFormat.Lossless; + // Phase 16 play-session telemetry (§2.1). The tracker observes the playback lifecycle and emits at // most one bucketed play event per session, behind the engagement floor. Attached after construction // by AudioPlayerProvider (the player is not DI-registered), mirroring how QueueService binds — no @@ -129,7 +166,30 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS await NotifyStateChanged(); } - private async Task LoadTrackStreaming(TrackDto track) + /// + public async Task ReloadPreservingPositionAsync() + { + // Nothing playing → nothing to switch. The new preference simply takes effect on the next play. + if (CurrentTrack is not { } track || !IsStreamingMode) return; + + // Capture the position to restore before the reload resets streaming state. Near the very start + // there is nothing worth preserving — a plain restart in the new format is simpler and avoids a + // needless seek-offset resolution. + var resumeAt = CurrentTime; + + await EnsureInitializedAsync(); + await _audioInterop.EnsureAudioContextReady(PlayerId); + await NotifyTrackSelected(); + + // Reload the same track in the newly-resolved delivery format. A near-start position restarts + // from byte 0; otherwise the load begins DIRECTLY at the saved position (no audible playback + // from the start). LoadTrackStreaming runs the whole forward segment loop, so this is the last + // meaningful await — the caller already fires this fire-and-forget. + await LoadTrackStreaming(track, startPosition: resumeAt > 1.0 ? resumeAt : null); + await NotifyStateChanged(); + } + + private async Task LoadTrackStreaming(TrackDto track, double? startPosition = null) { // Always reset to clean state before loading new track. ResetToIdle // both cancels and awaits any in-flight streaming loop, so by the time @@ -174,22 +234,35 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS await NotifyStateChanged(); - // Pass the streaming token to the HTTP layer so a navigation/track switch - // aborts the server connection instead of leaving it draining bytes. - var mediaResult = await _trackMediaClient.GetTrackMedia( + // Resolve the delivery format for this load BEFORE requesting bytes (Phase 18, default policy + // OQ2). When Opus is chosen the sidecar is fetched and injected into the JS player here, ahead of + // InitializeStreaming, honouring the 18.4 set-before-init contract. The result is captured so the + // seek-beyond-buffer re-fetch reuses the same artifact. + _currentFormat = await ResolveStreamFormatAsync(track.EntryKey, loadCts.Token); + + // Direction B: fetch the FIRST bounded segment to learn the total file length and the + // content type. The 206 Content-Range carries the total; the segment loop advances its + // cursor toward it. The decoder is initialized with the TOTAL length (not the segment + // length) so a bounded segment's small Content-Length never trips its byte-count + // completion early — segment boundaries are invisible to the decoder, which sees one + // continuous in-order byte stream. Passing the streaming token aborts the server + // connection on a navigation/track switch instead of leaving it draining bytes. + var firstSegment = await _trackMediaClient.GetTrackMedia( track.EntryKey, byteOffset: 0, + byteEnd: SegmentSizeBytes - 1, + format: _currentFormat, cancellationToken: loadCts.Token); - if (!mediaResult.Success) + if (!firstSegment.Success) { - var technicalError = mediaResult.GetMessage(); + var technicalError = firstSegment.GetMessage(); _logger.LogError("Failed to get track media for {TrackId}: {Error}", track.EntryKey, technicalError); ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(technicalError); return; } - if (mediaResult.Value == null) + if (firstSegment.Value == null) { const string technicalError = "No audio returned from server"; _logger.LogError("No audio data returned for track {TrackId}", track.EntryKey); @@ -197,13 +270,22 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS return; } - using var audio = mediaResult.Value; + // Ownership of the first segment transfers to the segment loop, which disposes it (and + // every subsequent segment). No `using` here — a double dispose is avoided and the socket + // is released the moment the loop finishes consuming the segment. + var audio = firstSegment.Value; - // Initialize streaming mode with content length and media type (drives - // JS format-decoder selection). - var streamingResult = await _audioInterop.InitializeStreaming(PlayerId, audio.ContentLength, audio.ContentType); + // The total logical length the decoder must see. On a 206 the Content-Range carries it; + // a 200 (server ignored Range / file ≤ one segment) has no Content-Range, so fall back to + // the body's own Content-Length — that body IS the whole file in that case. + var totalLength = audio.TotalLength ?? audio.ContentLength; + + // Initialize streaming mode with the TOTAL length and media type (drives JS + // format-decoder selection). See above: total, not segment, length. + var streamingResult = await _audioInterop.InitializeStreaming(PlayerId, totalLength, audio.ContentType); if (!streamingResult.Success) { + audio.Dispose(); var technicalError = $"Failed to initialize streaming: {streamingResult.Error}"; _logger.LogError("Streaming initialization failed for track {TrackId}: {Error}", track.EntryKey, technicalError); @@ -211,8 +293,23 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS return; } - _activeStreamingTask = StreamAudioWithEarlyPlayback(audio, loadCts.Token); - await _activeStreamingTask; + if (startPosition is { } startAt) + { + // "Load at timestamp" (Phase 18 wave 18.6 format switch): begin the stream DIRECTLY at + // startAt rather than byte 0, so the listener never hears the track restart from the + // beginning. The byte-0 segment in hand is used only to parse the header for byte-offset + // math (WAV) — Opus resolves the offset from its sidecar with no probe — and then a fresh + // segment is fetched from the resolved offset and pumped via the shared seek/refill loop. + await StartFromPositionAsync(track.EntryKey, audio, totalLength, startAt, loadCts.Token); + } + else + { + // Forward segmentation from byte 0. The first segment is already in hand; the loop pumps + // it, then fetches subsequent bounded segments gated on the scheduler fill signal. + _activeStreamingTask = RunSegmentedStreamAsync( + track.EntryKey, audio, cursor: 0, totalLength, seekPosition: null, loadCts.Token); + await _activeStreamingTask; + } } catch (OperationCanceledException) when (loadCts.IsCancellationRequested) { @@ -233,10 +330,33 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS catch (Exception ex) { StreamingErrorHandler.LogError(_logger, ex, "LoadTrackStreaming", track.EntryKey); - ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(ex.Message); - LoadProgress = 0; - IsLoaded = false; - IsStreamingMode = false; + var userError = StreamingErrorHandler.GetUserFriendlyMessage(ex.Message); + + // Mid-stream failure (playback was already underway): halt the JS scheduler into a clean + // paused-but-loaded state exactly as the seek path does via RecoverFromFailedRefill, rather + // than resetting to unloaded and letting the scheduler's buffered tail drain into a silent + // false end (AC6). Apply only when this load is still the active operation — a superseding + // seek owns state and has already replaced _streamingCancellation with its own CTS. + if (_streamingPlaybackStarted && ReferenceEquals(_streamingCancellation, loadCts)) + { + await RecoverFromFailedRefill(CurrentTime, userError); + } + else if (ReferenceEquals(_streamingCancellation, loadCts)) + { + // First-segment failure (nothing buffered / playing yet), still the active operation: + // the normal unload-to-error path is correct — nothing is in the scheduler to halt. + ErrorMessage = userError; + LoadProgress = 0; + IsLoaded = false; + IsStreamingMode = false; + } + else + { + // Superseded load: a newer seek (or track switch) has already claimed _streamingCancellation + // and owns all shared state. Writing IsLoaded/IsStreamingMode here would corrupt the live + // operation — mirror the OCE catch's identity guard and do nothing to shared state. + _logger.LogDebug("Generic throw on superseded load for track {TrackId} — newer operation owns state, skipping unload", track.EntryKey); + } } finally { @@ -250,6 +370,178 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS } } + /// + /// Begin streaming the freshly-initialized track DIRECTLY at instead + /// of byte 0 (Phase 18 wave 18.6 — the position-preserving format switch). The decoder has already been + /// built by InitializeStreaming; this resolves the file-absolute byte offset for the target time + /// and then converges onto the shared seek/refill loop ( with a + /// non-null seekPosition), which reinitializes the decoder for a header-less Range continuation and + /// starts playback at the target — so nothing is ever audibly played from the start. + /// + /// Opus resolves the offset from its sidecar immediately; WAV needs its header, so the byte-0 segment + /// already in hand is fed to the decoder (WITHOUT starting playback) until the header parses, then the + /// offset resolves. The probe is bounded by . The byte-0 segment is + /// disposed once the header is in hand; the continuation is a fresh fetch from the resolved offset. + /// + /// + private async Task StartFromPositionAsync( + string trackId, + TrackMediaResponse headerSegment, + long totalLength, + double startPosition, + CancellationToken cancellationToken) + { + // Resolve the byte offset for the target time. Opus answers immediately from its sidecar; WAV + // returns failure until its header is parsed, so we probe the byte-0 segment and retry. The + // byte-0 segment is disposed once it has served its purpose (header probe / nothing for Opus), + // even if the probe throws, so its socket never leaks before the continuation fetch. + SeekResult resolved; + try + { + resolved = await _audioInterop.ResolveStreamOffsetAsync(PlayerId, startPosition); + if (!resolved.Success || !resolved.SeekBeyondBuffer) + { + await ProbeHeaderAsync(headerSegment, cancellationToken); + resolved = await _audioInterop.ResolveStreamOffsetAsync(PlayerId, startPosition); + } + } + finally + { + headerSegment.Dispose(); + } + + if (!resolved.Success || !resolved.SeekBeyondBuffer) + { + // Could not resolve an offset even after probing — the stream is unusable for a positioned + // start. Surface as an error rather than silently restarting from 0 (which would contradict + // the "preserve position" contract the listener invoked). The catch in LoadTrackStreaming + // settles the error state. + throw new Exception(resolved.Error ?? "Could not resolve a stream offset for the requested position"); + } + + // Fetch the FIRST bounded segment from the resolved offset and pump it through the shared loop + // exactly as a seek-beyond-buffer does (reinit for the header-less continuation happens inside, + // and playback starts at startPosition). Reuse the format the load already resolved to. + var byteOffset = resolved.ByteOffset; + var firstSegment = await _trackMediaClient.GetTrackMedia( + trackId, + byteOffset, + byteEnd: byteOffset + SegmentSizeBytes - 1, + format: _currentFormat, + cancellationToken: cancellationToken); + if (!firstSegment.Success || firstSegment.Value == null) + { + var technicalError = firstSegment.GetMessage() ?? "Failed to load audio from position"; + _logger.LogError("Failed to get track media from offset {Offset} for {TrackId}: {Error}", + byteOffset, trackId, technicalError); + throw new Exception(technicalError); + } + + var audio = firstSegment.Value; + // The absolute EOF boundary the segment loop targets. On a 206 the Content-Range carries the file + // total; on a 200 (single-segment tail) fall back to the offset plus this body's length. + var continuationTotal = audio.TotalLength ?? (byteOffset + audio.ContentLength); + + // Fresh playback-start transition for the positioned stream (it has not started yet). + _streamingPlaybackStarted = false; + CanStartStreaming = false; + BufferedChunks = 0; + // Reflect the landing position immediately so the UI seek bar shows the right spot while the + // first post-offset buffers decode. + CurrentTime = startPosition; + + _activeStreamingTask = RunSegmentedStreamAsync( + trackId, audio, cursor: byteOffset, continuationTotal, seekPosition: startPosition, cancellationToken); + await _activeStreamingTask; + } + + /// + /// Feed bytes from the byte-0 into the decoder until its header parses + /// (), WITHOUT starting playback — the WAV byte-offset math needs the header + /// before can answer. Bounded by + /// so a stream that never yields a header cannot read unbounded. The + /// decoded buffers this queues are dropped by the subsequent Range-continuation reinit (clearForSeek), + /// so nothing is audible and nothing leaks. Opus never reaches here (its offset resolves pre-probe). + /// + private async Task ProbeHeaderAsync(TrackMediaResponse segment, CancellationToken cancellationToken) + { + var buffer = ArrayPool.Shared.Rent(MaxBufferSize); + try + { + var probed = 0; + while (!HeaderParsed && probed < MaxHeaderProbeBytes) + { + var read = await segment.Stream.ReadAsync(buffer, 0, _currentBufferSize, cancellationToken); + if (read <= 0) break; // segment exhausted before a header — let the caller surface the failure. + probed += read; + + // Slice to the exact bytes read (the pooled buffer may carry stale tail bytes). + var chunk = buffer.AsSpan(0, read).ToArray(); + var result = await _audioInterop.ProcessStreamingChunk(PlayerId, chunk); + if (!result.Success) + { + throw new Exception($"Failed to process header probe chunk: {result.Error}"); + } + + HeaderParsed = result.HeaderParsed; + // Capture the once-only duration the header yields so the UI and play session have it. + if (result.Duration.HasValue && Duration == null) + { + Duration = result.Duration.Value; + _playTracker?.SetDuration(result.Duration.Value); + } + } + } + finally + { + ArrayPool.Shared.Return(buffer); + } + } + + /// + /// Resolves which delivery format this load should request (Phase 18 default policy, OQ2): Opus when the + /// browser can decode Ogg Opus AND a sidecar exists for the track, otherwise lossless. When Opus is + /// chosen the sidecar is injected into the JS player here (set-before-init, the 18.4 contract) so the + /// decoder has its setup header + seek index before InitializeStreaming builds it. + /// + /// This is the single, deliberately-overridable seam for the listener quality preference (wave 18.6). + /// 18.6 overrides this to honour the user's "streaming quality" toggle — returning lossless when the + /// listener picked it, and otherwise falling through to this capability-gated default. The capability + /// gate (AC7) and the sidecar-absent → lossless fallback (C2) stay here so any override inherits both: + /// a browser that cannot decode Opus, or a track with no sidecar, always lands on lossless and plays. + /// + /// + protected virtual async Task ResolveStreamFormatAsync(string entryKey, CancellationToken cancellationToken) + { + // Capability gate first (AC7): never hand Ogg Opus to a browser that cannot decode it. + if (!await _audioInterop.CanDecodeOggOpus()) + { + return AudioFormat.Lossless; + } + + // The sidecar must be present (and parseable by the JS decoder) to seek an Opus stream. Its absence + // means the track has no Opus artifact yet (legacy / not backfilled / transcode failed) — request + // lossless rather than Opus-without-a-sidecar (the server would C2-fall-back anyway, but asking for + // lossless keeps the request honest and avoids a wasted Opus-then-fallback round-trip). + var sidecar = await _trackMediaClient.GetOpusSidecarAsync(entryKey, cancellationToken); + if (!sidecar.Success || sidecar.Value is not { Length: > 0 } sidecarBytes) + { + return AudioFormat.Lossless; + } + + // Inject BEFORE InitializeStreaming (the set-before-init contract). A parse failure here means the + // bytes are not a usable sidecar — fall back to lossless so a malformed sidecar never breaks playback. + var injected = await _audioInterop.SetOpusSidecar(PlayerId, sidecarBytes); + if (!injected.Success) + { + _logger.LogWarning("Opus sidecar for {EntryKey} failed to parse ({Error}); falling back to lossless.", + entryKey, injected.Error); + return AudioFormat.Lossless; + } + + return AudioFormat.Opus; + } + /// /// Fetches and decodes the track's waveform loudness profile, then notifies state so the /// seek zone re-renders with real bars. Best-effort: a 404 (no stored profile) or any other @@ -311,112 +603,223 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS return profile; } - private async Task StreamAudioWithEarlyPlayback(TrackMediaResponse audio, CancellationToken cancellationToken) + /// + /// Phase 21 Direction B — the single segmented forward read loop, shared by the initial load and + /// the seek/refill path (the convergence C1/C5 require: one cursor, one fetch mechanism, no forked + /// path). It pumps the FIRST segment (already fetched by the caller), then fetches subsequent + /// bounded bytes=cursor-(cursor+SegmentSizeBytes-1) 206 segments — each only AFTER the + /// scheduler drains below low-water — until the cursor reaches . + /// Because each segment is bounded and fully consumed before the next is requested, the browser + /// holds at most ~one segment of raw bytes (the network-memory bound), while the decoder sees one + /// continuous in-order byte stream across segment boundaries (the demuxer/decoder buffer partial + /// frames/pages across the boundary exactly as for arbitrary chunks today — no per-segment reinit). + /// + /// The already-fetched first segment (byte ). + /// Owned by this method, which disposes it; subsequent segments are fetched and disposed inline. + /// File-absolute byte offset the first segment starts at (0 for a fresh load, + /// the resolved seek offset for a refill). + /// Total file length in bytes — the EOF boundary the cursor advances + /// toward. The decoder is initialized/reinitialized against this, not the per-segment length. + /// Non-null for a seek/refill: the decoder is reinitialized for the + /// header-less Range continuation at this time before the first segment's bytes are fed (WAV + /// retains its header, Opus re-applies the cached setup + lead-trim). Null for a forward load from + /// byte 0, where the first segment carries the header and no reinit is needed. + private async Task RunSegmentedStreamAsync( + string trackId, + TrackMediaResponse firstSegment, + long cursor, + long totalLength, + double? seekPosition, + CancellationToken cancellationToken) { byte[]? buffer = null; + var segment = firstSegment; try { - long totalBytesRead = 0; - buffer = ArrayPool.Shared.Rent(MaxBufferSize); // Rent larger buffer to accommodate adaptive sizing - int currentBytes; - var readTimer = System.Diagnostics.Stopwatch.StartNew(); - - do + // Seek/refill: reinitialize the active decoder for the header-less continuation ONCE, + // before any continuation bytes are fed. Forward-from-zero (seekPosition null) skips this + // — its first segment carries the real header the decoder parses. Done here, inside the + // single loop, so seek and forward share the same fetch+pump mechanism (no forked path). + if (seekPosition is { } resumeAt) { - readTimer.Restart(); - currentBytes = await audio.Stream.ReadAsync(buffer, 0, _currentBufferSize, cancellationToken); - readTimer.Stop(); - - // Adapt buffer size based on read performance - AdaptBufferSize(currentBytes, readTimer.ElapsedMilliseconds); - - if (currentBytes > 0) + // The decoder byte-counts the header-less continuation against the bytes REMAINING + // from the range start to EOF (total − cursor), not the absolute total — that is what + // reinitializeForRangeContinuation expects (StreamDecoder.remainingByteLength). The + // loop's own cursor still targets the absolute totalLength for EOF. + var remainingBytes = Math.Max(0, totalLength - cursor); + var reinitResult = await _audioInterop.ReinitializeFromOffset(PlayerId, remainingBytes, resumeAt); + if (!reinitResult.Success) { - totalBytesRead += currentBytes; - - // Always slice to the exact number of bytes read. The pooled buffer - // is rented at MaxBufferSize and may carry stale bytes past - // currentBytes from a prior rental — handing the full array to JS - // interop would serialise that garbage into the audio stream. - var actualBuffer = buffer.AsSpan(0, currentBytes).ToArray(); - - // Process chunk for streaming - var chunkResult = await _audioInterop.ProcessStreamingChunk(PlayerId, actualBuffer); - if (!chunkResult.Success) - { - var error = $"Failed to process streaming chunk: {chunkResult.Error}"; - _logger.LogWarning("Chunk processing failed: {Error}", error); - throw new Exception(error); - } - - // Update streaming state - CanStartStreaming = chunkResult.CanStartStreaming; - HeaderParsed = chunkResult.HeaderParsed; - BufferedChunks = chunkResult.BufferCount; - - // Set duration from WAV header when available (only set once) - if (chunkResult.Duration.HasValue && Duration == null) - { - Duration = chunkResult.Duration.Value; - _logger.LogInformation("Duration set from WAV header: {Duration:F2} seconds", Duration); - // Feed the same once-only duration to the play session so it can compute the - // completion fraction at close. Safe before/after session open — SetDuration - // is a no-op when no session is open and idempotent otherwise. - _playTracker?.SetDuration(chunkResult.Duration.Value); - } - - // Start playback as soon as we can - if (!_streamingPlaybackStarted && CanStartStreaming) - { - var playbackResult = await _audioInterop.StartStreamingPlayback(PlayerId); - if (playbackResult.Success) - { - _streamingPlaybackStarted = true; - IsPlaying = true; - IsPaused = false; - IsLoaded = true; // Track is loaded and ready to play (even if still downloading) - ErrorMessage = null; - - // Open the play session exactly once per load, at the moment playback truly - // begins (§2.1). The _sessionOpened guard keeps the SeekBeyondBuffer re-stream - // — which re-enters this transition with _streamingPlaybackStarted reset — - // from opening a second session for the same play. Duration may already be - // known from a prior chunk, so re-feed it after opening. - if (!_sessionOpened && _currentTrackId is { } trackKey) - { - _sessionOpened = true; - _playTracker?.OnPlaybackStarted(trackKey); - if (Duration is { } d) - _playTracker?.SetDuration(d); - } - - await NotifyStateChanged(); // Immediate notification for critical state change - } - else - { - var technicalError = $"Failed to start streaming playback: {playbackResult.Error}"; - _logger.LogError("Failed to start playback: {Error}", technicalError); - ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(technicalError); - } - } - - // Update progress - if (audio.ContentLength > 0) - { - LoadProgress = Math.Min(1.0, (double)totalBytesRead / audio.ContentLength); - } - - await ThrottledNotifyStateChanged(); + throw new Exception($"Failed to reinitialize for offset streaming: {reinitResult.Error}"); } - } while (currentBytes > 0); + } - // Notify the JS decoder that the stream is finished. When the server omits - // Content-Length the StreamDecoder cannot determine completion via byte counting - // alone; this explicit signal ensures the tail-decoding path (streamComplete=true) - // fires regardless of whether Content-Length was present. + buffer = ArrayPool.Shared.Rent(MaxBufferSize); // larger rental to fit adaptive sizing + var readTimer = System.Diagnostics.Stopwatch.StartNew(); + + // Segment loop. Each iteration fully consumes one bounded 206 body, advancing the cursor by + // the bytes received. The next segment is fetched only when the scheduler is below + // high-water (the inter-segment gate). EOF is the cursor reaching totalLength, or a short + // segment (server returned fewer bytes than requested — the final slice). + while (true) + { + long segmentBytesRead = 0; + int currentBytes; + do + { + readTimer.Restart(); + currentBytes = await segment.Stream.ReadAsync(buffer, 0, _currentBufferSize, cancellationToken); + readTimer.Stop(); + + AdaptBufferSize(currentBytes, readTimer.ElapsedMilliseconds); + + if (currentBytes > 0) + { + segmentBytesRead += currentBytes; + + // Slice to the exact bytes read: the pooled buffer is rented at MaxBufferSize + // and may carry stale bytes past currentBytes from a prior rental — handing the + // full array to JS would serialise that garbage into the audio stream. + var actualBuffer = buffer.AsSpan(0, currentBytes).ToArray(); + + var chunkResult = await _audioInterop.ProcessStreamingChunk(PlayerId, actualBuffer); + if (!chunkResult.Success) + { + var error = $"Failed to process streaming chunk: {chunkResult.Error}"; + _logger.LogWarning("Chunk processing failed: {Error}", error); + throw new Exception(error); + } + + CanStartStreaming = chunkResult.CanStartStreaming; + HeaderParsed = chunkResult.HeaderParsed; + BufferedChunks = chunkResult.BufferCount; + + // Set duration from header when available (only set once) + if (chunkResult.Duration.HasValue && Duration == null) + { + Duration = chunkResult.Duration.Value; + _logger.LogInformation("Duration set from header: {Duration:F2} seconds", Duration); + // Feed the once-only duration to the play session for the completion + // fraction. No-op when no session is open; idempotent otherwise. + _playTracker?.SetDuration(chunkResult.Duration.Value); + } + + // Start playback as soon as we can — at the min-buffer threshold, exactly as + // before (C2: first audio is not gated on the segment boundary; the first + // segment alone clears the threshold). + if (!_streamingPlaybackStarted && CanStartStreaming) + { + await TryStartPlaybackAsync(); + } + + // Progress against the total file length (cursor + bytes consumed so far). + if (totalLength > 0) + { + LoadProgress = Math.Min(1.0, (double)(cursor + segmentBytesRead) / totalLength); + } + + await ThrottledNotifyStateChanged(); + + // Per-chunk back-pressure — the bound that actually holds for high-density codecs. + // The inter-segment gate alone is matched to WAV's byte density (~24 s of audio per + // 4 MB segment) but NOT to Opus: at 320 kbps a 4 MB segment is ~100 s of decodable + // audio. The inner loop has the whole segment's bytes already in hand, so with no + // network wait to pace it, it would decode the ENTIRE segment eagerly — piling tens + // of MB of decoded f32 PCM AHEAD of a playhead that has barely moved, before the + // inter-segment gate ever runs. With HW accel off that lookahead lives in main- + // process RAM, and the byte ceiling cannot save us because nothing on this path + // polls it. So drain to low-water per chunk once the scheduler is over high-water. + // + // Gated on _streamingPlaybackStarted so this can NEVER block first audio (C2): until + // playback starts the playhead does not advance, so the forward fill would never + // drain and the loop would deadlock. The 30 s high-water sits far above the + // 6-buffer playback-start minimum, so in practice the gate is not even reached + // before playback begins — the guard is the correctness backstop, not the common + // case. Reads the piggybacked flag (no extra interop hop) to DECIDE to drain; the + // drain helper then polls IsProductionPaused — the same steady-state-reads-flag / + // throttled-state-polls split the inter-segment gate uses. + if (_streamingPlaybackStarted && chunkResult.ProductionPaused) + { + await DrainBackpressureAsync(cancellationToken); + } + } + } while (currentBytes > 0); + + // Segment fully consumed; advance the cursor and release this segment's stream/socket + // before deciding whether to fetch the next. Disposing here keeps exactly one segment's + // raw bytes resident at a time. + cursor += segmentBytesRead; + segment.Dispose(); + segment = null!; + + // EOF: cursor reached the total file length. This is the sole forward-EOF condition. + // A short segment body (segmentBytesRead < SegmentSizeBytes) is NOT an EOF signal — + // the inner read loop fully drains the HTTP body, so a short body means the server + // sent fewer bytes than the bounded range we requested. While cursor < totalLength that + // can only be a connection drop / truncated stream, NOT the file tail — route it to + // the same clean-failure recovery as a fetch error rather than silently completing. + var reachedTotal = totalLength > 0 && cursor >= totalLength; + if (reachedTotal) + { + break; + } + + // Guard: if the body was short but we haven't reached totalLength, the stream was + // truncated mid-segment (connection drop / premature close). Surface as an error so + // the scheduler is halted rather than left to drain its buffered tail into a false end. + if (segmentBytesRead < SegmentSizeBytes) + { + throw new Exception( + $"Stream truncated at byte {cursor} of {totalLength}: received {segmentBytesRead} bytes " + + $"but expected up to {SegmentSizeBytes} and have not reached EOF"); + } + + // Inter-segment back-pressure gate (Phase 21.2 fill signal, gating SEGMENT FETCH). Do not + // fetch the next segment while the scheduler is over high-water; wait until it drains + // below low-water. Because the browser only buffers bounded segments and we hold off + // requesting the next one, raw network memory stays at ~one segment. Shares the same + // drain helper as the per-chunk gate above. No _streamingPlaybackStarted guard is needed + // here (unlike the per-chunk gate): reaching this point means a full segment was consumed, + // which is ~24 s (WAV) / ~100 s (Opus) of audio — far past the 6-buffer playback-start + // minimum — so playback is always running by now and the fill can drain. A file that fits + // in one segment hits EOF and breaks above, never reaching this gate. + await DrainBackpressureAsync(cancellationToken); + + // Fetch the next bounded segment. The end offset is clamped implicitly by the server + // (a request past EOF yields the available tail as a short slice, caught above). + var nextEnd = cursor + SegmentSizeBytes - 1; + var nextResult = await _trackMediaClient.GetTrackMedia( + trackId, + byteOffset: cursor, + byteEnd: nextEnd, + format: _currentFormat, + cancellationToken: cancellationToken); + if (!nextResult.Success || nextResult.Value == null) + { + var technicalError = nextResult.GetMessage() ?? "Failed to fetch next stream segment"; + _logger.LogError("Failed to fetch segment at offset {Offset} for {TrackId}: {Error}", + cursor, trackId, technicalError); + throw new Exception(technicalError); + } + segment = nextResult.Value; + } + + // Notify the JS decoder that the stream is finished. The decoder marks completion by byte + // count against the total it was initialized with; this explicit signal flushes the + // residual tail and covers the (rare) case where the total was unknown. await _audioInterop.MarkStreamCompleteAsync(PlayerId); - // Mark as fully loaded + // Complete-without-start fallback: if the track's total decodable audio never crossed the + // start threshold (e.g. total Opus audio < 1s lead, or WAV < 6 buffers), the in-loop + // CanStartStreaming check never fired and _streamingPlaybackStarted is still false. Now that + // streamComplete is set on the JS scheduler, calling StartStreamingPlayback lets it drain + // the accumulated buffers and fires onPlaybackEnded exactly once — same transition the + // normal path uses, so session/_sessionOpened/Duration handling is identical. + if (!_streamingPlaybackStarted) + { + await TryStartPlaybackAsync(); + } + LoadProgress = 1.0; await NotifyStateChanged(); } @@ -427,7 +830,7 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS } catch (Exception ex) { - StreamingErrorHandler.LogError(_logger, ex, "StreamAudioWithEarlyPlayback"); + StreamingErrorHandler.LogError(_logger, ex, "RunSegmentedStreamAsync"); ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(ex.Message); LoadProgress = 0; IsLoaded = false; @@ -437,6 +840,8 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS } finally { + // Release the last segment (if a fetch failed mid-loop it may still be held) and the buffer. + segment?.Dispose(); if (buffer != null) { ArrayPool.Shared.Return(buffer); @@ -444,6 +849,46 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS } } + /// + /// Call StartStreamingPlayback on the JS player and apply the resulting state transitions. + /// This is the single playback-start transition shared by the in-loop threshold path and the + /// completion-path fallback — both callers set the guard and apply session/Duration handling + /// identically so neither path diverges. + /// + private async Task TryStartPlaybackAsync() + { + var playbackResult = await _audioInterop.StartStreamingPlayback(PlayerId); + if (playbackResult.Success) + { + _streamingPlaybackStarted = true; + IsPlaying = true; + IsPaused = false; + IsLoaded = true; // loaded and ready, even while still downloading + ErrorMessage = null; + + // Open the play session exactly once per load, at the moment playback + // truly begins (§2.1). The _sessionOpened guard keeps a seek/refill + // re-stream — which re-enters this transition with + // _streamingPlaybackStarted reset — from opening a second session for + // the same play. Duration may already be known, so re-feed it. + if (!_sessionOpened && _currentTrackId is { } trackKey) + { + _sessionOpened = true; + _playTracker?.OnPlaybackStarted(trackKey); + if (Duration is { } d) + _playTracker?.SetDuration(d); + } + + await NotifyStateChanged(); // immediate — critical state change + } + else + { + var technicalError = $"Failed to start streaming playback: {playbackResult.Error}"; + _logger.LogError("Failed to start playback: {Error}", technicalError); + ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(technicalError); + } + } + /// /// In streaming mode, Stop fully resets to Idle state since audio data is consumed. /// This is equivalent to Unload for streaming playback. @@ -515,6 +960,10 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS return; } + // Capture into a non-null local: _currentTrackId is the field a track-switch could clear, but + // this seek operates against the track loaded NOW; the segment loop needs a stable id. + var trackId = _currentTrackId; + IsSeekingBeyondBuffer = true; // Cancel the current streaming loop AND wait for it to fully exit before @@ -538,46 +987,68 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS await DrainActiveStreamingTaskAsync(); oldCts?.Dispose(); + // Single-writer discipline (C6/AC8): all three failure exits must share the same guard. + // TrackMediaClient.GetTrackMedia swallows OperationCanceledException and returns + // Success==false, so a superseded seek lands in the media-fetch-fail branch below + // rather than in the OCE catch. Without the guard those branches would call + // RecoverFromFailedRefill — running clearForSeek + setPlaybackOffset against the player + // state the NEWER seek now owns. A local predicate keeps all three exits symmetric so a + // future exit cannot forget the check. + bool IsStillActiveSeek() => ReferenceEquals(_streamingCancellation, seekCts); + try { // Update UI immediately CurrentTime = seekPosition; await NotifyStateChanged(); - // Request new stream from offset - var mediaResult = await _trackMediaClient.GetTrackMedia( - _currentTrackId, + // Request the FIRST bounded segment from the resolved offset (Direction B — converged with + // the forward path). Reuse the format the initial load resolved to (_currentFormat): an + // Opus seek must come back as Opus bytes so the cached setup header + page-aligned + // byteOffset (resolved JS-side from the Opus seek index) match the continuation; WAV resolves + // its offset from the header — one seam, format-appropriate math (AC9 / §3.4a C). The + // segment loop then continues forward segmentation from this offset exactly as a fresh load + // does from 0 — no forked fetch path (C1/C5). + var firstSegment = await _trackMediaClient.GetTrackMedia( + trackId, byteOffset, + byteEnd: byteOffset + SegmentSizeBytes - 1, + format: _currentFormat, cancellationToken: seekCts.Token); - if (!mediaResult.Success || mediaResult.Value == null) + if (!firstSegment.Success || firstSegment.Value == null) { - var technicalError = mediaResult.GetMessage() ?? "Failed to load audio from position"; + var technicalError = firstSegment.GetMessage() ?? "Failed to load audio from position"; _logger.LogError("Failed to get track media from offset {Offset}: {Error}", byteOffset, technicalError); - ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(technicalError); - IsSeekingBeyondBuffer = false; + // Guard: a superseded seek must NOT touch shared state. The newer seek owns teardown. + if (IsStillActiveSeek()) + { + await RecoverFromFailedRefill(seekPosition, StreamingErrorHandler.GetUserFriendlyMessage(technicalError)); + } + else + { + _logger.LogDebug("Media-fetch failed on superseded seek to {Position} — newer seek owns state, skipping recovery", seekPosition); + } return; } - using var audio = mediaResult.Value; + var audio = firstSegment.Value; + // The absolute EOF boundary the segment loop's cursor targets. On a 206 the Content-Range + // carries the file total; on a 200 (single-segment file) fall back to cursor + body length. + var totalLength = audio.TotalLength ?? (byteOffset + audio.ContentLength); - // Reinitialize JS player for offset streaming - var reinitResult = await _audioInterop.ReinitializeFromOffset(PlayerId, audio.ContentLength, seekPosition); - if (!reinitResult.Success) - { - _logger.LogError("Failed to reinitialize for offset streaming: {Error}", reinitResult.Error); - ErrorMessage = "Failed to seek to position"; - IsSeekingBeyondBuffer = false; - return; - } - - // Reset streaming state for new stream + // Reset streaming state for the new stream. The decoder reinit for the header-less + // continuation happens INSIDE RunSegmentedStreamAsync (seekPosition non-null), so seek and + // forward share one fetch+pump+reinit mechanism. A reinit failure there throws and lands in + // the catch below, which recovers when still the active seek — the same clean-failure path + // (AC6) the old explicit reinit branch had, now unified with the fetch-failure path. _streamingPlaybackStarted = false; CanStartStreaming = false; HeaderParsed = false; BufferedChunks = 0; - // Stream audio from offset - _activeStreamingTask = StreamAudioWithEarlyPlayback(audio, seekCts.Token); + // Stream from offset via the shared segment loop. Ownership of `audio` transfers to it. + _activeStreamingTask = RunSegmentedStreamAsync( + trackId, audio, cursor: byteOffset, totalLength, seekPosition, seekCts.Token); await _activeStreamingTask; IsSeekingBeyondBuffer = false; @@ -588,22 +1059,65 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS // still the active seek — if _streamingCancellation has been replaced, a // newer seek is in progress and owns the flag. _logger.LogDebug("Seek beyond buffer cancelled"); - if (ReferenceEquals(_streamingCancellation, seekCts)) + if (IsStillActiveSeek()) { IsSeekingBeyondBuffer = false; } } catch (Exception ex) { + // A refill fetch can fail deep into a long mix (the listener didn't initiate it). Recover + // into a clean paused-but-loaded state (AC6) rather than leaving the starved scheduler to + // fire a silent false end. Only when we are still the active seek — a superseding seek owns + // the state and the OCE catch above handles its own teardown. _logger.LogError(ex, "Error during seek beyond buffer to position {Position}", seekPosition); - ErrorMessage = StreamingErrorHandler.GetUserFriendlyMessage(ex.Message); - IsSeekingBeyondBuffer = false; - await NotifyStateChanged(); + if (IsStillActiveSeek()) + { + await RecoverFromFailedRefill(seekPosition, StreamingErrorHandler.GetUserFriendlyMessage(ex.Message)); + } } } /// - /// Single method to reset all state - called by both Stop and Unload. + /// Clean-failure recovery for a window-miss refill (Phase 21.3 / AC6). A backward seek past the + /// retained tail re-fetches via the existing Range path; that mid-stream fetch the listener did not + /// initiate can fail deep into a long mix. When it does, the pre-seek loop has already been + /// cancelled and drained, but the JS scheduler is still holding stale pre-seek buffers and still + /// "playing" — left alone it drains them and fires a silent false end (the wedged/starved state AC6 + /// forbids). This halts the scheduler into a paused-but-loaded state at , + /// surfaces a clear error, and leaves the track loaded so the listener can retry the seek or pick + /// another track. Mirrors PlaybackScheduler.playFromPosition's end-of-buffer recovery: stop + /// pretending to play. + /// + private async Task RecoverFromFailedRefill(double seekPosition, string userFacingError) + { + // Halt the starved scheduler JS-side (stop sources, drop stale buffers, anchor at the target). + // Best-effort: if even this interop fails the player is no worse off, and we still surface the + // error and settle C# state below. + var recovered = await _audioInterop.RecoverFromFailedRefill(PlayerId, seekPosition); + if (!recovered.Success) + { + _logger.LogWarning("Refill-failure recovery interop did not succeed: {Error}", recovered.Error); + } + + // Settle C# into the matching recoverable state: not playing, paused at the target, still loaded + // and still in streaming mode. IsLoaded = true and IsStreamingMode = true are both load-bearing — + // the "paused-but-loaded" contract lets the listener retry the seek (Seek early-returns when + // !IsLoaded || !IsStreamingMode), resume via TogglePlayPause, or pick another track. Resetting + // either to false would wedge at least one of the three retry routes (AC6 / Phase 21.3). + ErrorMessage = userFacingError; + IsPlaying = false; + IsPaused = true; + IsLoaded = true; + IsStreamingMode = true; + CurrentTime = seekPosition; + IsSeekingBeyondBuffer = false; + await NotifyStateChanged(); + } + + /// + /// Single method to reset all state - called by both Stop and Unload, and as the prologue of a new + /// load. /// private async Task ResetToIdle() { @@ -653,6 +1167,7 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS _streamingPlaybackStarted = false; IsSeekingBeyondBuffer = false; _currentTrackId = null; + _currentFormat = AudioFormat.Lossless; await NotifyStateChanged(); } @@ -691,6 +1206,27 @@ public class StreamingAudioPlayerService : AudioPlayerService, IStreamingPlayerS } } + /// + /// Block the segment loop while the scheduler's decoded forward fill is over high-water, resuming + /// once it drains below low-water (Phase 21.2 hysteresis). Shared by the per-chunk gate (inside a + /// segment) and the inter-segment gate so both honor identical drain discipline — a guard present on + /// one path and absent on the other would let one path overshoot the memory bound. + /// + /// The poll awaits on , so a track switch/seek mid-wait throws + /// OCE and unwinds through the existing drain discipline (C6). UC5: a user pause freezes the playhead + /// so the fill never drains on its own — hold here until playback resumes (IsPaused clears) OR the + /// fill drains. Returns immediately when nothing is throttled (the steady-state common case). + /// + /// + private async Task DrainBackpressureAsync(CancellationToken cancellationToken) + { + while (IsPaused || await _audioInterop.IsProductionPaused(PlayerId)) + { + cancellationToken.ThrowIfCancellationRequested(); + await Task.Delay(BackpressurePollMs, cancellationToken); + } + } + private async Task ThrottledNotifyStateChanged() { var now = DateTime.UtcNow; diff --git a/DeepDrftPublic.Client/Services/WaveformVisualizerControlState.cs b/DeepDrftPublic.Client/Services/WaveformVisualizerControlState.cs index c20faa1..9695548 100644 --- a/DeepDrftPublic.Client/Services/WaveformVisualizerControlState.cs +++ b/DeepDrftPublic.Client/Services/WaveformVisualizerControlState.cs @@ -160,6 +160,47 @@ public sealed class WaveformVisualizerControlState /// public event Action? Changed; + // Whether the one-time, capability-driven default has been applied this session. The default-set + // (lava off when the browser has no WebGL hardware acceleration) must run exactly once — on the + // first interactive render, before the listener has touched a toggle — so it sets the *initial + // default* and never clobbers a later explicit in-session toggle. Scoped with the rest of this + // state, so it survives SPA navigation (a remounted visualizer does not re-apply) and resets on a + // fresh page load (F5 re-probes). + private bool _capabilityDefaultApplied; + + /// + /// Applies the hardware-capability default exactly once per session: when the browser reports no + /// WebGL hardware acceleration, the lava subsystem (the expensive, main-thread software-rendered + /// part that starves audio decode) defaults off while the waveform stays on. With + /// acceleration present this is a no-op — lava keeps its on-state. + /// + /// + /// Idempotent and guarded: only the FIRST call this session has any effect, so it sets the initial + /// default and never overrides a listener's explicit toggle (the control remains fully functional — + /// a user on a software renderer may re-enable lava at their own risk). Mutates, coerces + /// Theater Mode, then raises once so the controls UI, the visualizer bridge, + /// and the Theater observers all reflect the default in a single cycle. Called by the visualizer + /// bridge on first interactive render, once JS interop (the probe) is available. + /// + /// + /// + /// The probe result — true when WebGL hardware acceleration is present (or the renderer is + /// unknown/masked, favoring the common case), false only on a positive software-renderer + /// match or total WebGL failure. + /// + public void ApplyCapabilityDefault(bool hardwareAccelerated) + { + if (_capabilityDefaultApplied) return; + _capabilityDefaultApplied = true; + + // Accelerated (or unknown): keep the as-shipped defaults — no observer churn. + if (hardwareAccelerated) return; + + LavaEnabled = false; // the expensive subsystem off; WaveformEnabled stays at its default (true). + CoerceTheaterMode(); + NotifyChanged(); + } + /// /// Enforces the Theater-Mode invariant: Theater Mode cannot remain on when both visualizer /// subsystems are off (there is nothing to go to theater FOR). Call this after mutating diff --git a/DeepDrftPublic.Client/Startup.cs b/DeepDrftPublic.Client/Startup.cs index 59ebd23..70bc7bf 100644 --- a/DeepDrftPublic.Client/Startup.cs +++ b/DeepDrftPublic.Client/Startup.cs @@ -14,6 +14,13 @@ public static class Startup services.AddScoped(); services.AddScoped(); + // Public-site listener settings (Phase 18 wave 18.6). PublicSiteSettings is the generalized, + // prerender-seeded preference object (today: streaming quality); SettingsCookieService writes the + // 365-day cookie at runtime. Same scoped lifetime + cookie seam as the dark-mode pair above, so the + // preference survives SPA nav within a session and seeds the next visit's prerender. + services.AddScoped(); + services.AddScoped(); + // Track Client. The HTTP-backed ITrackDataService is used by both WASM and SSR // prerender — both call DeepDrftAPI over the "DeepDrft.API" client. services.AddScoped(); diff --git a/DeepDrftPublic/Components/App.razor b/DeepDrftPublic/Components/App.razor index bbec5d5..0d1ac87 100644 --- a/DeepDrftPublic/Components/App.razor +++ b/DeepDrftPublic/Components/App.razor @@ -25,6 +25,7 @@