288 Commits

Author SHA1 Message Date
daniel-c-harvey 8d1272e36f docs: fix stale anonid.ts path after telemetry module rename
Deploy DeepDrftPublic / Build & Publish (push) Successful in 4m1s
Deploy DeepDrftPublic / Deploy (push) Successful in 1m23s
2026-06-26 22:22:05 -04:00
daniel-c-harvey 6a043b622e Merge telemetry-transport-resilience into dev (first-party fetch for play/share; beacon only on unload) 2026-06-26 22:17:27 -04:00
daniel-c-harvey 2af0d8650b fix(telemetry): first-party fetch for play/share, beacon only on unload
Route normal play closes (end/switch/stop) and all shares through a same-origin
HttpClient POST so privacy-hardened browsers stop blocking them; keep sendBeacon
for the tab-unload edge. Rename the JS module off telemetry/beacon to session/
lifecycle so the retained fallback isn't name-matched. No new data or identifiers.
2026-06-26 21:11:43 -04:00
daniel-c-harvey ca44979b08 docs: record Opus/derived read-path streaming and index-only opus-status
Deploy DeepDrftAPI / Build, Publish & Bundle (push) Successful in 2m25s
Deploy DeepDrftManager / Build & Publish (push) Successful in 1m28s
Deploy DeepDrftPublic / Build & Publish (push) Successful in 4m2s
Package install tarball / package (push) Successful in 6s
Deploy DeepDrftAPI / Deploy (push) Successful in 1m33s
Deploy DeepDrftManager / Deploy (push) Successful in 1m59s
Deploy DeepDrftPublic / Deploy (push) Successful in 1m22s
2026-06-26 15:32:18 -04:00
daniel-c-harvey bf5b314aed Merge opus-read-streaming into dev (stream Opus/derived read path; last read-path OOM fix + logger-leak hardening) 2026-06-26 15:16:38 -04:00
daniel-c-harvey afa862a67b fix: move LogInformation inside disposal guard in both GetTrack streaming arms
Prevents an OS handle leak if the logger throws after the FileStream is opened but before File() takes ownership. Also corrects a stale "finally block" comment in the lossless arm — it has always been a catch.
2026-06-26 15:16:29 -04:00
daniel-c-harvey d72263aea1 Stream Opus/derived read path: serve from seekable disk FileStream, never a whole-file byte[]; HasOpusAsync is index-only 2026-06-26 14:58:11 -04:00
daniel-c-harvey 1e17ffc380 docs: record Opus transcode store-path streaming 2026-06-26 14:30:24 -04:00
daniel-c-harvey a98cef1ba7 Merge opus-transcode-streaming into dev (stream Opus transcode store path; last store-path OOM fix) 2026-06-26 14:15:36 -04:00
daniel-c-harvey 4351ae04be Stream Opus transcode source and encoded output; removes last store-path OOM
Source read via streamed vault open + bounded staging copy (index-only duration/extension); encoded output walked from a bounded stream (new OggOpusParser.WalkAsync, byte-identical to the buffer oracle) and stored via streaming vault write. Adds parity tests.
2026-06-26 14:06:33 -04:00
daniel-c-harvey c1f2cafd8b Merge streaming-overhaul into dev (Opus low-data streaming, windowed streaming, HW-accel-off stabilization) 2026-06-26 11:14:59 -04:00
daniel-c-harvey 634eb611eb docs: record streaming-stabilization arc (Opus + HW-accel-off fixes)
Document the back-pressure water marks, genuine end-of-playback gating, rebuffer hysteresis, 48kHz Opus pre-align, decodePressure auto-throttle, and HW-accel detection / lava default-off, plus the off-main-thread-decode fallback note.
2026-06-26 10:59:48 -04:00
daniel-c-harvey d7a373cdb0 Merge HW-acceleration detection (default lava off on software renderer) into streaming-overhaul 2026-06-26 10:42:25 -04:00
daniel-c-harvey 020a945843 Detect HW acceleration; default lava off on software renderer; release probe WebGL context
Probes UNMASKED_RENDERER_WEBGL once per page via a throwaway WebGL context; defaults the lava subsystem off on a positive software-renderer match or total WebGL failure; releases the throwaway context via WEBGL_lose_context after reading the renderer string to avoid exhausting the browser's per-page context limit.
2026-06-26 10:41:07 -04:00
daniel-c-harvey 1aef30f67d Merge visualizer-viewport-framing: anchor ambient visualizer to viewport bottom 2026-06-26 10:37:09 -04:00
daniel-c-harvey 0e8b85bbcb Merge visualizer auto-throttle under decode pressure + instrumentation strip into streaming-overhaul 2026-06-26 06:00:53 -04:00
daniel-c-harvey 374f09150f Auto-throttle visualizer under sustained Opus decode pressure; strip streaming investigation instrumentation 2026-06-26 06:00:05 -04:00
daniel-c-harvey 9bfa921703 docs: reflect streaming upload store path and waveform compute (OOM fix) 2026-06-26 05:55:54 -04:00
daniel-c-harvey 4fe2d564d9 fix: logging levels in Prod 2026-06-26 05:44:19 -04:00
daniel-c-harvey 5d9f410cd8 Merge Wave 2: stream waveform compute to bound large-upload memory into dev 2026-06-26 05:40:06 -04:00
daniel-c-harvey 76f7f389a3 Merge forward-cushion widening + back-pressure diagnostics into streaming-overhaul 2026-06-26 05:17:00 -04:00
daniel-c-harvey 61e185a2f7 audio: widen forward decode cushion 30/15->60/30s + add [BP-DIAG] back-pressure instrumentation
Byte cap (96MB) unchanged as the hard OOM bound; the wider time window only lets sparse Opus use existing memory headroom to ride out decode jitter. Diag logs pin whether the block is back-pressure or decode throughput.
2026-06-25 21:52:20 -04:00
daniel-c-harvey 9347f11ff0 Stream the waveform compute so large uploads no longer buffer the whole file (Wave 2 OOM) 2026-06-25 21:49:11 -04:00
daniel-c-harvey f0d1463619 Merge Opus rebuffer hysteresis + ultra-short-track start fix into streaming-overhaul 2026-06-25 16:02:12 -04:00
daniel-c-harvey aa0b64329f Merge Wave 1: stream audio store path to fix large-upload OOM into dev 2026-06-25 15:59:43 -04:00
daniel-c-harvey 4ab430d232 Fix complete-without-start hang for ultra-short tracks; add Opus rebuffer hysteresis
Tracks whose total audio falls below the playback-start threshold (Opus <1s lead, WAV <6 buffers) silently hung loaded-but-not-playing. After MarkStreamCompleteAsync, call TryStartPlaybackAsync when _streamingPlaybackStarted is still false so the scheduler drains its buffers and fires onPlaybackEnded exactly once.
2026-06-25 15:54:53 -04:00
daniel-c-harvey beec36a382 fix(vault): atomic streamed write via temp→rename, suppress OCE log noise
AddEntryStreamingAsync now writes to a temp file in the same vault directory,
renames it into place (POSIX rename(2) — atomic on Linux), and updates the
index only after the rename succeeds. A client disconnect or I/O fault during
the write leaves the original backing file intact and the index unchanged; the
temp file is cleaned up best-effort on failure. Fixes the data-corruption
regression on the replace path where a cancelled write could truncate the live
backing file after the index update and FileMode.Create already ran.

Also filters OperationCanceledException from error-level logging in
RegisterResourceStreamingAsync — a normal client disconnect is not an error.

Two tests added to AudioStoreStreamingTests covering cancel and fault on the
replace path.
2026-06-25 15:46:58 -04:00
daniel-c-harvey 79bbbd4956 fix(api): stream audio store path to eliminate whole-file buffering (OOM)
Processors now emit a ProcessedAudio plan with a streamed writer instead of a whole-file AudioBinary; vault writes stream via RegisterResourceStreamingAsync. Header parsing is bounded. Wave 2 (waveform/Opus) still re-reads the full file by design.
2026-06-25 15:27:28 -04:00
daniel-c-harvey 48e58c266d Merge false end-of-playback fix (Opus startup underrun gating) into streaming-overhaul 2026-06-25 15:17:17 -04:00
daniel-c-harvey 67422e922d fix(audio): guard underrun/stream-complete against false end-of-playback
pause() clears underrun_ so setStreamComplete can't fire TrackEnded while paused; resetToStart() resets streamComplete. Prior fix: underrun_ park + streamComplete discriminator prevent the Opus-startup false-end. Tests: 18 PlaybackScheduler cases including pause-during-underrun and underrun->resume->genuine-end-once.
2026-06-25 15:16:22 -04:00
daniel-c-harvey 3aed5c129f Merge guard revert + reload-path instrumentation into streaming-overhaul 2026-06-24 23:59:19 -04:00
daniel-c-harvey e98e616997 Remove harmful single-load guard; instrument reload/reset path for double-header-parse hunt 2026-06-24 23:59:09 -04:00
daniel-c-harvey 8fa37f995b Merge Opus 48kHz up-front AudioContext init (root-cause fix) into streaming-overhaul 2026-06-24 23:27:26 -04:00
daniel-c-harvey 0800167511 fix(audio): align AudioContext to 48kHz up front for Opus streams
Opus resolved its 48kHz context lazily on the first chunk, close()ing and rebuilding the live graph mid-decode. Move the recreate into initializeStreaming so it runs before any bytes flow; the lazy call early-returns. WAV path unchanged.
2026-06-24 23:26:42 -04:00
daniel-c-harvey 8a6acd5f5f Merge Opus single-load guard (instrument double-load) into streaming-overhaul 2026-06-24 23:09:08 -04:00
daniel-c-harvey be9de8d77c Collapse duplicate same-track streaming loads to enforce one load per play
A second LoadTrackStreaming for the same in-flight track (UI double-fire, queue re-entry, or JS false-end auto-advance) is now dropped; a different-track load still supersedes. Targets the Opus double-load; keeps load-gen diagnostics.
2026-06-24 23:08:58 -04:00
daniel-c-harvey d686fe48ce Apply stream-quality change live by reloading at current position
Finish the Settings "Apply" behavior so changing streaming quality mid-track
switches format immediately instead of only persisting the cookie for the next
play.

- SettingsMenu reads the AudioPlayerProvider cascade and threads the player into
  StreamQualitySetting as an explicit parameter (the MudMenu panel portals to
  MudPopoverProvider, outside the cascade scope, so a [CascadingParameter] there
  lands null). StreamQualitySetting's Apply persists the cookie, then asks the
  player to reload preserving position.

- Add a "load at timestamp" path to the player rather than restart-from-0-then-
  seek (which audibly played the start and raced the just-started scheduler into
  a crash). ReloadPreservingPositionAsync loads the track in the newly-resolved
  format beginning DIRECTLY at the saved position:
    * new JS resolveStreamOffset(position) resolves the file-absolute byte offset
      with no playback/buffer state (Opus from its sidecar immediately; WAV after
      a header probe),
    * StartFromPositionAsync converges onto the existing seek/refill loop
      (RunSegmentedStreamAsync with a non-null seekPosition) so the decoder
      reinitializes for a header-less Range continuation and starts playback at
      the target,
    * ProbeHeaderAsync feeds the byte-0 segment to the decoder WITHOUT starting
      playback until the WAV header parses (bounded by 256 KB); the probe buffers
      are dropped by the continuation's clearForSeek, so nothing is audible.

- IStreamingPlayerService gains ReloadPreservingPositionAsync; the QueueService
  test fake implements it.
2026-06-24 22:55:03 -04:00
daniel-c-harvey 7adc35dd5d docs: record streaming raw-queue and per-chunk memory bounds
Document the three-layer memory bound (raw queue, decoded queue, network) in the streaming seam after the HW-accel-off OOM fix landed on streaming-overhaul.
2026-06-24 22:09:01 -04:00
daniel-c-harvey 8206c0bdaf Merge streaming OOM cross-format memory bound into streaming-overhaul 2026-06-24 19:54:12 -04:00
daniel-c-harvey aeec582957 Bound decoded forward fill per chunk in streaming read loop
The inter-segment back-pressure gate matched WAV byte density but let a 4MB Opus segment (~100s at 320kbps) decode eagerly into main-process RAM, OOMing the tab with HW accel off. Drain per chunk past high-water, gated on playback start. Adds load-generation diagnostics for the double-load hypothesis.
2026-06-24 19:50:33 -04:00
daniel-c-harvey 036ee1f78e docs: record Phase 21 (windowed streaming) as landed; note Direction A to B pivot
Move Phase 21 from PLAN to COMPLETED with the as-built record, and annotate
the spec that Direction B shipped after WASM fetch buffering defeated A.
2026-06-24 16:05:30 -04:00
daniel-c-harvey c1e6930c70 Merge Phase 21 Direction B (Range-segmented forward fetch) into streaming-overhaul 2026-06-24 15:54:45 -04:00
daniel-c-harvey cc9d20184d Restore IsStreamingMode on recovery; guard superseded-load else-branch
RecoverFromFailedRefill now sets IsStreamingMode=true so the in-place
seek-retry route isn't wedged. The generic-catch unload path is gated on
the loadCts identity, so a superseded load no longer clobbers a newer
operation's state.
2026-06-24 15:37:38 -04:00
daniel-c-harvey e7762e35e8 Fix truncated-segment and mid-stream failure paths in segmented loop
cursor>=totalLength is the sole forward-EOF test; a short non-final body is
a truncation error, not EOF. Mid-stream forward-load failures now invoke
RecoverFromFailedRefill so the scheduler halts instead of a silent false end.
Two regression tests pin both paths.
2026-06-24 15:16:46 -04:00
daniel-c-harvey 11faf8888f Phase 21 Direction B: bound network memory via Range-segmented forward fetch
Replace the open-ended forward GET with sequential bounded bytes=start-end
segments, the next fetched only when the scheduler drains below low-water,
so the browser holds ~one segment regardless of file size. Seek converges
on the same loop. Strip BP-DIAG.
2026-06-24 13:20:37 -04:00
daniel-c-harvey adbd376d42 Fix stale spacer-observe comment: drop visualizer/clipping ref, name spacer as sole consumer 2026-06-24 10:40:52 -04:00
daniel-c-harvey cb899a2913 Anchor ambient visualizer to viewport bottom; occlude via z-index not clip
Drop the --player-height bottom inset so the fixed visualizer fills the
viewport; the inset player bar no longer leaves a page-background gap. The
spacer now occludes via opaque page-surface + z-index. Visualizer no longer
reads --player-height, so spacer.ts coalescing is removed.
2026-06-24 09:06:45 -04:00
daniel-c-harvey def297e7d9 Merge Phase 21.4 streaming fix + back-pressure diagnostics into streaming-overhaul 2026-06-24 09:02:34 -04:00
daniel-c-harvey 369cb86437 Add [BP-DIAG] back-pressure instrumentation for Phase 21.4 browser run
Temporary, grep-tagged diagnostics at the read-loop pause, the scheduler
latch, and the chunk-result path to show whether ProductionPaused latches,
reaches C#, and parks the loop. Strip once the cause is confirmed.
2026-06-24 09:00:38 -04:00
daniel-c-harvey c7629c15a4 Enable WASM response streaming on audio media fetch
Without SetBrowserResponseStreamingEnabled the browser buffers the whole
body before yielding, so the Phase 21.2 read-loop pause backpressured an
already-downloaded payload. Set it on both the initial and seek/refill
requests; safe no-op on the SSR path.
2026-06-24 08:45:33 -04:00
daniel-c-harvey 9c95a5f23e Merge Phase 21.3 (seek-back-past-window refill + AC6 recovery) into streaming-overhaul 2026-06-23 23:58:51 -04:00
daniel-c-harvey b93881cd66 21.3 review fixes: guard superseded-seek failures; restore post-recovery retry
C6/AC8: IsStillActiveSeek() predicate guards all three SeekBeyondBuffer
failure exits, so a superseded seek never recovers over a newer seek's
state. AC6: empty scheduler routes to seekBeyondBuffer so a same-target
retry (seek or play) refetches instead of no-oping.
2026-06-23 23:55:28 -04:00
daniel-c-harvey af4cb186f3 Phase 21.3: seek-back-past-window refill + clean refill-failure recovery
Seek-back past the retained tail reuses the existing seek-beyond-buffer
Range path (per-path resolver). A failed refill now halts the scheduler
into a paused-but-loaded state (AC6) instead of a silent false end.
2026-06-23 23:43:17 -04:00
daniel-c-harvey 121983b19d Merge Phase 21.2 (streaming back-pressure) into streaming-overhaul 2026-06-23 23:29:35 -04:00
daniel-c-harvey 29e8747c69 21.2 review remediation: pause-spin, OQ7 comment, rename, C2 cross-check
Skip the back-pressure interop poll while paused (UC5). Document complete()
draining the stash in full by design. Rename scheduler isProductionPaused to
evaluateProductionPause (latch-advancing); window exposure name unchanged.
2026-06-23 23:28:42 -04:00
daniel-c-harvey 518479e7ae Phase 21.2: back-pressure to bound the unplayed decoded region
Shared scheduler fill signal (forward water-marks + hard byte cap) pauses
the C# read loop above high-water and, for Opus, stops the demux/decode
feed so WebCodecs queues stay near-empty. Routes through the existing
cancellation discipline; releases the latch on clear/seek.
2026-06-23 23:16:08 -04:00
daniel-c-harvey a2becf45d6 Merge Phase 21.1 (PlaybackScheduler partial eviction) into streaming-overhaul 2026-06-23 22:56:48 -04:00
daniel-c-harvey 07f29a8216 Reconcile eviction comment wording; add handleSourceEnded cascade test (Phase 21.1)
The inclusive <= bound is correct; comments now say 'at or behind'. New
test drives eviction through the real onended trigger with a live mid-array
source pinning the frontier.
2026-06-23 22:49:12 -04:00
daniel-c-harvey ed606d94c7 Add partial eviction to PlaybackScheduler (Phase 21.1)
Drop already-played buffers from the front while advancing the time
anchor so position/index bookkeeping stays exact. Shared by both decode
paths, no format branch. Back-retain is a config seam for 21.2.
2026-06-23 22:39:05 -04:00
daniel-c-harvey ccf7d3dbe3 docs: reconcile Phase 21 spec with as-built Phase 18 (two decode paths)
Window both the WAV StreamDecoder and Opus WebCodecs paths feeding one PlaybackScheduler — shared eviction, per-path back-pressure; reuse the now-live index-driven Opus seek for refill. Drops stale approximate-seek language; adds OQ6/OQ7.
2026-06-23 22:01:49 -04:00
daniel-c-harvey bbcf8be677 docs: record Phase 18 (Opus low-data streaming) in COMPLETED; stage PLAN for Phase 21 2026-06-23 21:48:39 -04:00
daniel-c-harvey 8902ce4d63 Merge Opus duration/seek/visualizer fix into streaming-overhaul 2026-06-23 21:25:26 -04:00
daniel-c-harvey eb58ae4a72 Fix Opus duration reporting so seekbar and visualizer work
Surface the sidecar duration on the first Opus chunk instead of gating it on the first decoded buffers; C# locks UI Duration on chunk 1, and async WebCodecs decode left it at 0 — killing seek and the duration-gated visualizer.
2026-06-23 21:23:43 -04:00
daniel-c-harvey d80b777e9f Merge WebCodecs streaming Opus decoder (+ AC9 seek re-sync) into streaming-overhaul 2026-06-23 21:03:02 -04:00
daniel-c-harvey 5a75da1769 fix: AC9 seek fine re-sync + deterministic decoder drain (WebCodecs Opus)
Seek now trims the lead-in so playback lands at the requested time, not the page start; decoder drain polls decodeQueueSize (bounded) instead of a single timeout. Minor cleanups.
2026-06-23 20:57:05 -04:00
daniel-c-harvey 7f3fb74126 Replace broken per-segment Opus decode with WebCodecs AudioDecoder streaming pipeline 2026-06-23 17:42:06 -04:00
daniel-c-harvey d0118997b6 Merge Opus capability-probe fix into streaming-overhaul 2026-06-23 17:03:07 -04:00
daniel-c-harvey 5b78efaad4 fix: replace hand-assembled Opus probe blob with real ffmpeg/libopus output
Previous probe sample had invalid Ogg page CRC32s, so Chrome/Firefox rejected it and the capability check always returned false. New 176-byte libopus sample has verified-correct CRCs. Adds structural-validity tests.
2026-06-23 16:57:49 -04:00
daniel-c-harvey 81d4b42b72 Merge Phase 18.6 Track A (public Settings menu + streaming-quality toggle) into streaming-overhaul 2026-06-23 14:52:28 -04:00
daniel-c-harvey 77c6c42c94 remediate: replace eval cookie writes with safe JS helper + add tests (18.6 Track A)
Both SettingsCookieService and DarkModeCookieService now call window.DeepDrftSettings.setCookie (new Interop/settings/settings.ts) instead of eval. New tests cover SettingsServiceBase parse/format round-trip and the PreferenceAwareStreamingPlayerService invariant (Lossless skips probe; LowData inherits base).
2026-06-23 14:17:34 -04:00
daniel-c-harvey ab75bbf6c1 Merge Phase 18.6 Track B (CMS Opus status: backfill badge + Post-Processing) into streaming-overhaul 2026-06-23 14:09:48 -04:00
daniel-c-harvey 59f48bb8cb feature: CMS Opus status surfaces — backfill missing-N badge + upload Post-Processing phase (18.6) 2026-06-23 14:06:21 -04:00
daniel-c-harvey c63c7ca033 feature: Phase 18.6 Track A — public Settings menu + streaming-quality toggle 2026-06-23 14:06:19 -04:00
daniel-c-harvey e5366bc4ec Merge Phase 18.5 (Opus end-to-end integration + Backfill-Opus) into streaming-overhaul 2026-06-23 12:52:07 -04:00
daniel-c-harvey 2bde4908d7 Wire Opus end-to-end playback + Backfill-Opus action (Phase 18.5)
Player picks Opus when the browser can decode it and a sidecar exists (else lossless), injecting the sidecar before stream init; seek reuses the same format. Adds the Backfill-Opus bulk API endpoint + CMS action.
2026-06-23 12:39:13 -04:00
daniel-c-harvey 1e063d95f4 chore: Trigger CI
Deploy DeepDrftAPI / Build, Publish & Bundle (push) Successful in 2m21s
Deploy DeepDrftManager / Build & Publish (push) Successful in 1m33s
Deploy DeepDrftPublic / Build & Publish (push) Successful in 4m17s
Package install tarball / package (push) Successful in 7s
Deploy DeepDrftAPI / Deploy (push) Successful in 1m33s
Deploy DeepDrftManager / Deploy (push) Successful in 1m28s
Deploy DeepDrftPublic / Deploy (push) Successful in 1m30s
2026-06-23 08:50:22 -04:00
daniel-c-harvey dce5530890 Merge Phase 18.4 (OpusFormatDecoder + index-based seek) into streaming-overhaul 2026-06-23 08:48:06 -04:00
daniel-c-harvey 8afcd3784f Merge Phase 18.3 (Opus delivery transport) into streaming-overhaul 2026-06-23 08:44:53 -04:00
daniel-c-harvey 261289c1b8 feature: OpusFormatDecoder — Ogg-page-aligned segmenting, sidecar parser, accurate index-based seek (Phase 18.4) 2026-06-23 08:34:39 -04:00
daniel-c-harvey 740d01a67f feature: Phase 18.3 — Opus delivery transport (?format= stream + seek sidecar endpoint) 2026-06-23 08:34:37 -04:00
daniel-c-harvey 1fdbec2533 Merge cors-manager-origin into dev
Deploy DeepDrftAPI / Build, Publish & Bundle (push) Successful in 2m15s
Package install tarball / package (push) Successful in 6s
Deploy DeepDrftAPI / Deploy (push) Successful in 1m35s
2026-06-23 08:21:33 -04:00
daniel-c-harvey 70842cb576 docs: add production install checklist 2026-06-23 08:15:56 -04:00
daniel-c-harvey f2a0d39521 config: add app.deepdrft.com to API CORS allowlist 2026-06-23 08:15:55 -04:00
daniel-c-harvey e807ddb91b Merge Phase 18.2 (Opus format resolution + sidecar lookup contract) into streaming-overhaul 2026-06-23 07:49:28 -04:00
daniel-c-harvey 19793ba1c3 feature: Opus format resolution + sidecar lookup contract (Phase 18.2) 2026-06-23 07:45:06 -04:00
daniel-c-harvey 1bda2b7bea docs: reflect Phase 23 SEO crawl directives as landed
Deploy DeepDrftManager / Build & Publish (push) Successful in 1m29s
Deploy DeepDrftPublic / Build & Publish (push) Successful in 4m7s
Deploy DeepDrftManager / Deploy (push) Successful in 1m23s
Deploy DeepDrftPublic / Deploy (push) Successful in 1m28s
2026-06-23 07:40:57 -04:00
daniel-c-harvey 8773803712 feature: og default image 2026-06-23 07:40:42 -04:00
daniel-c-harvey 3cc11bcbb5 Merge p23-w1-t2-cms-noindex into dev
Phase 23 Track B: make DeepDrftManager uncrawlable — static robots.txt (Disallow: /) + blanket noindex meta in the CMS head. No env gate; the CMS is always uncrawlable.
2026-06-23 07:36:01 -04:00
daniel-c-harvey 0ba4fc6597 Merge p23-w1-t1-public-crawl-endpoints into dev
Phase 23 Track A: env-gated /robots.txt + /sitemap.xml on DeepDrftPublic. Thin controller + pure builders, reuses api/release + ReleaseRoutes + SeoOptions.BaseUrl. Non-prod uncrawlable; sitemap loc equals page canonical by construction.
2026-06-23 07:35:52 -04:00
daniel-c-harvey 7a0ccdd784 fix: correct WalkPageSize to 100 (actual server PageSize cap) and update comment 2026-06-23 07:33:24 -04:00
daniel-c-harvey e845dc3496 Merge ffmpeg host provisioning into streaming-overhaul 2026-06-23 07:30:21 -04:00
daniel-c-harvey ba064cc136 provision ffmpeg on DeepDrftAPI host for Opus transcode
Phase 18.1 needs ffmpeg (libopus). Add it to bootstrap.sh apt prereqs and a
preflight guard in install.sh; resolves via the systemd user unit's default
PATH (/usr/bin), no config change.
2026-06-23 07:30:13 -04:00
daniel-c-harvey ca057dc630 chore: make DeepDrftManager uncrawlable and noindex (Phase 23.3)
Static robots.txt (Disallow: /) in wwwroot + blanket noindex meta in App.razor head. No env gate — the CMS is always uncrawlable. Defense in depth per spec OQ-C1.
2026-06-23 07:23:49 -04:00
daniel-c-harvey 5f4807cc4a feature: Phase 23 Track A — env-gated /robots.txt + /sitemap.xml public crawl endpoints 2026-06-23 07:23:42 -04:00
daniel-c-harvey b3dadbb572 Merge Phase 18.1 (Opus ingest transcode + seek-index sidecar) into streaming-overhaul 2026-06-23 07:13:46 -04:00
daniel-c-harvey 9a4b79d377 docs: spec Phase 23 — SEO crawl directives (sitemap.xml, robots.txt, CMS noindex) 2026-06-23 07:10:20 -04:00
daniel-c-harvey 33383cd675 Merge p22-w2-jsonld-type-fix into dev
Deploy DeepDrftPublic / Build & Publish (push) Successful in 4m20s
Package install tarball / package (push) Successful in 6s
Deploy DeepDrftPublic / Deploy (push) Successful in 1m25s
Fix JSON-LD @type serialization: concrete nodes were emitting a bare Type alongside @type because the attribute sat only on the abstract base override. Validator now clean.
2026-06-23 06:57:44 -04:00
daniel-c-harvey 56f7013314 fix: put [JsonPropertyName("@type")] on each concrete JsonLdNode override
System.Text.Json emitted both "@type" and a bare "Type" because the attribute was only on the abstract base member. Adds regression assertions for all node types.
2026-06-23 06:57:05 -04:00
daniel-c-harvey 6add30a4ff fix: Wave 18.1 review — pre-skip subtraction, t=0 anchor, PreSkip in sidecar, stderr on cancel 2026-06-23 06:55:31 -04:00
daniel-c-harvey 33d6f34d8a feature: Phase 18.1 — derive Opus 320 + seek-index sidecar at ingest
Background-job transcode (ffmpeg/libopus) after source store; pure C# Ogg
walker builds the 0.5s-bucketed granule→byte seek index + captures the
OpusHead/OpusTags setup header into a per-track sidecar in a new track-opus
vault. Best-effort, additive, regenerated on replace-audio.
2026-06-23 06:30:10 -04:00
daniel-c-harvey 2653e62eeb docs: reflect Phase 22 SEO metadata component as landed 2026-06-23 06:21:52 -04:00
daniel-c-harvey 45bd599bdd Merge p22-w1-seo-metadata-component into dev
Phase 22: parameterized SEO metadata component for the public site — SeoHead + typed JSON-LD builders, per-medium release schema, env-gated noindex (beta uncrawled), inline-safe JSON-LD escaping.
2026-06-23 06:16:31 -04:00
daniel-c-harvey f976af0f7c fix(seo): escape inline JSON-LD, per-release byArtist, soft-404 + env-gated noindex
Escape </>& in JSON-LD body to kill script-breakout; byArtist now uses the release artist; detail-page not-found branches emit noindex; default robots gated to Production via a PersistentState SeoEnvironment bridge.
2026-06-23 06:10:03 -04:00
daniel-c-harvey f3b89ca9d7 feature: Phase 22 SEO metadata component for public site
One presentational SeoHead renders the full OG/Twitter/JSON-LD head surface at prerender via typed schema.org builders. Per-medium release schema, config-sourced canonicals, 404 noindex. Zero CMS change.
2026-06-23 05:41:55 -04:00
daniel-c-harvey 8752fc0c98 docs: resolve Phase 18 OQ7 seek-index granularity to 0.5s buckets 2026-06-23 05:36:25 -04:00
daniel-c-harvey 274d0ace62 Merge install-prep-analysis: installer prompts for AuthBlocks:Email:From 2026-06-23 05:28:17 -04:00
daniel-c-harvey e3a4364b8c docs(plan): Phase 18 OQ resolutions + VBR-safe accurate Opus seek model 2026-06-23 05:26:58 -04:00
daniel-c-harvey 564b704803 fix(installer): prompt for and write AuthBlocks:Email:From
Without this field, DeepDrftAPI throws InvalidOperationException on
startup. Adds the EMAIL_FROM prompt after EMAIL_TOKEN, writes "From"
into the Email JSON object, and unsets the variable on cleanup.
2026-06-23 05:26:48 -04:00
daniel-c-harvey 6af6677a12 docs: spec Phase 22 — parameterized SEO metadata component (public site) 2026-06-23 05:12:31 -04:00
daniel-c-harvey 1bdaeaa164 docs(plan): add Phase 18 Opus low-data streaming; resolve Phase 21 OQ5 (no MSE) 2026-06-23 04:58:21 -04:00
daniel-c-harvey a84a99c309 docs: spec Phase 21 — windowed streaming buffer for bounded client memory 2026-06-23 00:14:44 -04:00
daniel-c-harvey 2c1571057a feature: Manager Menu Styles and Page Titles
Deploy DeepDrftAPI / Build, Publish & Bundle (push) Successful in 2m13s
Deploy DeepDrftManager / Build & Publish (push) Successful in 1m23s
Deploy DeepDrftPublic / Build & Publish (push) Successful in 4m4s
Deploy DeepDrftAPI / Deploy (push) Successful in 1m33s
Deploy DeepDrftManager / Deploy (push) Successful in 1m28s
Deploy DeepDrftPublic / Deploy (push) Successful in 1m28s
2026-06-22 23:04:49 -04:00
daniel-c-harvey 0b7d8e41e7 Merge account-nav-menu into dev 2026-06-22 22:42:48 -04:00
daniel-c-harvey 4833935925 feature: About Bio text 2026-06-22 22:41:39 -04:00
daniel-c-harvey 7917d56af3 feature: Manager Logos 2026-06-22 22:41:30 -04:00
daniel-c-harvey 1fd63fe368 Add AccountNavMenu to CmsLayout nav drawer 2026-06-22 22:39:21 -04:00
daniel-c-harvey 4e1f540945 Merge bump-cerebellum-final into dev 2026-06-22 22:28:10 -04:00
daniel-c-harvey 1ed518b018 chore: bump Cerebellum stack to NetBlocks 10.3.32 / BlazorBlocks 10.3.35 / AuthBlocks 10.3.39
Delivers the ResultDtoBase.From() null-crash fix to DeepDrft's
Users/Registrations pages.
2026-06-22 22:27:57 -04:00
daniel-c-harvey 7c41aa678d Revert "Merge bisect-match-skipper into dev"
This reverts commit 475e5e671c, reversing
changes made to 0d1da9e63c.
2026-06-22 12:47:02 -04:00
daniel-c-harvey 475e5e671c Merge bisect-match-skipper into dev 2026-06-22 12:24:00 -04:00
daniel-c-harvey 9971474403 bisect: pin DeepDrftHome to Skipper's known-good package versions
AuthBlocks* → 10.3.35, BlazorBlocks* → 10.3.32. Diagnostic downgrade to
isolate null-ref crash on Users/Registrations pages.
2026-06-22 12:23:19 -04:00
daniel-c-harvey 0d1da9e63c docs: note Phase 20 visualizer-flash fix (coalesced --player-height publish) 2026-06-22 08:38:55 -04:00
daniel-c-harvey d47c186045 Merge p20-theater-visualizer-flash into dev 2026-06-22 08:36:05 -04:00
daniel-c-harvey 670eaab34d fix(visualizer): coalesce --player-height publish so Theater ease doesn't thrash the WebGL backing store 2026-06-22 08:19:53 -04:00
daniel-c-harvey c58b1c9386 Merge bump-cerebellum-deps into dev 2026-06-21 11:55:40 -04:00
daniel-c-harvey 450204cdbf Bump Cerebellum packages to fix null-Items crash on Users/Registrations pages
AuthBlocks → 10.3.38, BlazorBlocks → 10.3.34, NetBlocks → 10.3.31.
Pulls server-side null-Items guard (AuthBlocks) and BlazorBlocks render
guard. Direct refs for BlazorBlocks/NetBlocks raised to avoid NU1605
downgrade conflicts with AuthBlocks 10.3.38's transitive requirements.
2026-06-21 11:50:05 -04:00
daniel-c-harvey 5c22c1626a docs: reflect Phase 20 Wave 2 theater refinements (full-screen body, eased collapse, playing-release scoping) 2026-06-21 10:18:19 -04:00
daniel-c-harvey 8628fbf215 Merge Theater Mode refinements (Phase 20 Wave 2) into dev 2026-06-21 09:23:56 -04:00
daniel-c-harvey a23a22a2a3 fix(css): visibility transition 0s->0.45s so allow-discrete defers collapse flip to end of ease-out 2026-06-21 09:20:18 -04:00
daniel-c-harvey 6e12d0161a fix(theater): replace max-height collapse with grid-rows + visibility; fix keyboard-focus leak when collapsed 2026-06-21 09:12:24 -04:00
daniel-c-harvey 9716092805 feat(theater): full-screen detail body, eased content collapse, playing-release scoping
Detail bodies fill 100vh below the nav so the visualizer reads full-screen; Theater toggle eases page content and the player-bar now-showing panel in/out instead of popping (reduced-motion honored); Theater only applies to the currently-playing release.
2026-06-21 08:59:09 -04:00
daniel-c-harvey a577df88dd docs: reflect Phase 20 Theater Mode landing in PLAN, COMPLETED, CLAUDE.md, and spec status 2026-06-20 22:17:58 -04:00
daniel-c-harvey 011dbe8d81 Merge Theater Mode (Phase 20) into dev 2026-06-20 22:12:23 -04:00
daniel-c-harvey 2fc2d4eb6d test: fix PascalCase nit in CoerceTheaterMode_BothOff_TheaterBecomesFalse 2026-06-20 22:09:34 -04:00
daniel-c-harvey 14f3af41e4 fix(theater): auto-exit Theater Mode when both visualizer subsystems are disabled
Adds CoerceTheaterMode() to WaveformVisualizerControlState; ToggleLava/ToggleWaveform
call it before NotifyChanged so all observers see consistent state in one Changed cycle.
Covers the dead-end escape route bug (Phase 20 review finding).
2026-06-20 22:03:39 -04:00
daniel-c-harvey fa01b9c8e0 feat(public): add Theater Mode to release detail pages
Toggle left of the lava popover hides release content so the visualizer fills
the surface; player bar grows to carry the playing release's cover, title, and
share. State on WaveformVisualizerControlState; pages and bar observe it.
2026-06-20 21:51:30 -04:00
daniel-c-harvey 835fb71337 docs(plan): mark Phase 20 Theater Mode scoped after sign-off 2026-06-20 21:40:56 -04:00
daniel-c-harvey 021801999c docs(plan): add Phase 20 Theater Mode spec and roadmap entry 2026-06-20 19:08:44 -04:00
daniel-c-harvey 54cba7eea0 docs(queue): sync client CLAUDE.md to deque cleanup — cached QueueItems, scaffold/StreamNow PLAY routing 2026-06-20 19:05:18 -04:00
daniel-c-harvey fbaf545c90 Merge queue-deque-redesign into dev
Two-level deque queue model + five bug fixes, plus review cleanup.
2026-06-20 19:01:07 -04:00
daniel-c-harvey d3f89c494a fix: Waveform Visualizer Controls layout 2026-06-20 18:56:53 -04:00
daniel-c-harvey c3ec3acafa fix(queue): route scaffold masthead PLAY through queue; cache QueueItems snapshot 2026-06-20 18:51:30 -04:00
daniel-c-harvey 214f708e65 feat(queue): two-level deque model — PLAY prepends, add appends, last-track-end empties
Fixes five queue bugs: Playlist relabel, last-track-empties, dormant-seed-from-player on first add, immediate panel reactivity, and front/back deque semantics. Adds JumpTo for row jumps.
2026-06-20 15:26:37 -04:00
daniel-c-harvey 5058c72375 fix(rcl): commit theme.js so RCL interop JS ships via MapStaticAssets
Deploy DeepDrftManager / Build & Publish (push) Successful in 2m0s
Deploy DeepDrftPublic / Build & Publish (push) Successful in 4m32s
Deploy DeepDrftManager / Deploy (push) Successful in 1m26s
Deploy DeepDrftPublic / Deploy (push) Successful in 1m29s
theme/ was missing from the per-module .gitignore allowlist (only
parallax/ and knob/ were re-included), so theme.js never got committed,
was absent from publish output, and 404'd at runtime. Broaden the
allowlist to the whole DeepDrftShared.Client/wwwroot/js/ tree so every
compiled RCL interop module ships automatically.
2026-06-20 12:31:49 -04:00
daniel-c-harvey f5edcba7b2 feature: Waveform Controls Restructuring
Deploy DeepDrftPublic / Build & Publish (push) Successful in 4m2s
Deploy DeepDrftPublic / Deploy (push) Successful in 1m27s
2026-06-20 03:12:41 -04:00
daniel-c-harvey 64e1f71e18 docs: reflect gas-lamp self-coloring in theming section
Deploy DeepDrftAPI / Build, Publish & Bundle (push) Successful in 2m13s
Deploy DeepDrftManager / Build & Publish (push) Successful in 1m26s
Deploy DeepDrftPublic / Build & Publish (push) Successful in 3m56s
Deploy DeepDrftAPI / Deploy (push) Successful in 1m33s
Deploy DeepDrftManager / Deploy (push) Successful in 1m28s
Deploy DeepDrftPublic / Deploy (push) Successful in 1m28s
GasLampLit now uses an explicit #2A5C4F frame fill; the removed dark-only nav rule is no longer described as live.
2026-06-20 03:11:33 -04:00
daniel-c-harvey 7807d4ebe1 Merge theme-icon-followups into dev
Fix PlayStateIcon green-on-green chip and gas-lamp frame in dark theme.
2026-06-20 03:07:40 -04:00
daniel-c-harvey 4410132409 docs: correct PlayStateIcon compiled-selector specificity tuple (0,4,0) to (0,5,0)
The [b-xxx] Blazor scope attribute is a fifth class/attribute simple selector; the prior count dropped it.
2026-06-20 03:06:59 -04:00
daniel-c-harvey 00ff9e2702 fix(dark-theme): PlayStateIcon glyph beats .dd-accent-icon; GasLampLit self-colored frame
PlayStateIcon.razor.css adds a .mud-icon-root rule !important so the play chip always shows
navy on moss-green in dark. GasLampLit frame path changed from currentColor to #2A5C4F;
dead nav dark rule removed.
2026-06-20 03:03:18 -04:00
daniel-c-harvey bb086e5869 docs: update Provision User nav target to /useradmin/users/new (AuthBlocks 10.3.37) 2026-06-20 02:53:53 -04:00
daniel-c-harvey 674d772986 Merge p19-w6-authblocks-1037-adopt into dev (adopt AuthBlocks 10.3.37: account-creation normalization + paged-route null-guard fix; repoint Provision User nav) 2026-06-20 02:39:58 -04:00
daniel-c-harvey ee296db7f6 Merge theme-accent-icon-consolidation into dev
Consolidate per-site dark-icon overrides into reusable .dd-accent-icon treatment; fix hero glyphs in dark.
2026-06-20 02:35:08 -04:00
daniel-c-harvey 8a4da2f0b9 chore: bump Cerebellum.AuthBlocks to 10.3.37 in DeepDrftAPI
Picks up the server-side null-guard fix in RouteHelpers.GetPage/GetAll and UserService.GetPage, resolving the ArgumentNullException on the CMS User Accounts and Registrations pages.
2026-06-20 02:33:38 -04:00
daniel-c-harvey c28a2b1cf5 docs: correct specificity arithmetic and spinner-clause accuracy in .dd-accent-icon comments
Glyph rule is (0,3,0) > (0,1,0) — beats .mud-secondary-text on specificity, not source order.
ReleaseHeroOverlay spinner comment now distinguishes dead glyph clauses from the live spinner clause that produced the intentional light delta.
2026-06-20 02:32:12 -04:00
daniel-c-harvey 1427c92092 feat(manager): adopt AuthBlocks.Web 10.3.37; repoint Provision User nav to /useradmin/users/new
10.3.37 retires /account/superregister in favour of the new canonical /useradmin/users/new route. Bump the package and update the CmsLayout nav link accordingly.
2026-06-20 02:31:38 -04:00
daniel-c-harvey 2fbb1c9b95 fix(theme): green hero Share/Play/Queue glyphs in dark via shared .dd-accent-icon
Fold Session/Mix hero glyphs into the reusable accent-icon treatment so they reach
the glyph (beating .mud-secondary-text) green-accent in both themes; drop the dead
wrapper white rule and the redundant dark-only hero override. Light pixel-identical.
2026-06-20 02:21:11 -04:00
daniel-c-harvey 4c56eededc Merge dark-theme-hero-buttons into dev
Green hero Share/Play/Queue, lava-lamp, and gas-lamp affordances in dark theme.
2026-06-20 01:51:01 -04:00
daniel-c-harvey f9d99b2c98 fix: dark-theme hero buttons green in dark mode; correct source-order comment
Both ::deep and global selectors are (0,3,0); override wins on source order
(deepdrft-styles.css linked after scoped bundle in App.razor).
2026-06-20 01:49:55 -04:00
daniel-c-harvey 59608a23c5 Merge dark-theme-green-buttons into dev
Green Play/Share/Queue buttons on Cut detail in dark theme.
2026-06-20 01:30:20 -04:00
daniel-c-harvey 2ddc57edb1 fix(dark-theme): green Play/Share/Queue buttons in Cut detail
Color.Secondary renders off-white in dark mode, making the filled Play
button and the Share/Queue icon buttons in .cut-detail-actions and track
rows unreadable. Override to green (--deepdrft-primary) in dark only;
hero-overlay icons untouched.
2026-06-20 01:29:59 -04:00
daniel-c-harvey 0bb656a512 docs: log Phase 18 Wave 5 light-glass panel theming in COMPLETED
Record the new theme-aware --deepdrft-panel-* token family making the queue,
visualizer, and privacy overlays light-glass in light theme (dark-glass unchanged in
dark), and the lifted dark-glass exemption.
2026-06-20 01:12:04 -04:00
daniel-c-harvey bb5a1fcad4 fix: Privacy Message 2026-06-20 01:10:19 -04:00
daniel-c-harvey 896b37792e Merge light-glass-panels into dev
Queue, waveform-visualizer control deck, and privacy overlays now render as light
translucent glass with legible dark text in light theme via a new theme-aware
--deepdrft-panel-* token family; dark-glass charcoal unchanged in dark theme.
Lifts the prior dark-glass exemption for these three panels.
2026-06-20 01:08:30 -04:00
daniel-c-harvey 2619fc67c8 fix: wire --deepdrft-panel-text-muted into queue rows; refresh stale light/dark comments
Replace opacity-reduced color on .deepdrft-queue-position and .deepdrft-queue-artist with
var(--deepdrft-panel-text-muted) so the token earns its place in the family.
Update .wvc-section-label and .waveform-visualizer-control-icon comments to reflect
theme-aware (not static-light) behavior.
2026-06-20 01:06:58 -04:00
daniel-c-harvey 4c14c67c33 feat(theme): light-glass panels in light theme
Queue, visualizer control deck, and privacy overlays now bind a theme-aware
--deepdrft-panel-* family (surface/text/text-muted/border/row-hover): light
translucent glass with dark text in light theme, unchanged dark-glass charcoal
in dark. Tokens re-declared in body.deepdrft-theme-dark for the body-portaled overlays.
2026-06-20 00:59:22 -04:00
daniel-c-harvey 494668bf24 Merge p19-w5-mailtrap-testinbox into dev (wire optional Mailtrap TestInbox sandbox routing in DeepDrftAPI) 2026-06-20 00:36:11 -04:00
daniel-c-harvey c4e22c706c docs: record sponsor approval of NewUser normalization decisions
Mark brief §5 decisions resolved (all recommendations accepted 2026-06-20): NewUser canonical for direct provision, SuperRegister deleted + redirected, Registration label tidied.
2026-06-20 00:34:44 -04:00
daniel-c-harvey c747f3200f docs: clarify TestInbox placeholder in authblocks.example.json
Empty string gave no hint what value is expected; <sandbox-id> signals the Mailtrap sandbox inbox ID that must be supplied.
2026-06-20 00:34:41 -04:00
daniel-c-harvey 1dd1646cce docs: record popover-surface retune and portaled-popover body-class bridge
Note the 4%/bluer-navy --deepdrft-popover-surface values, the new
--deepdrft-popover-surface-dark source token, the theme TS interop module, and the
<body>-class bridge in CLAUDE.md; log Phase 18 Wave 4 in COMPLETED.md.
2026-06-20 00:32:13 -04:00
daniel-c-harvey 6bbec2fc8e Merge popover-surface-retune into dev
Retune public-site popover surfaces: light reads as a near-page-background light
surface (8%->4% navy), dark skews bluer (navy-mid + green-accent). Root cause: popovers
portal to <body>, outside the theme wrapper; MainLayout now stamps the theme class on
<body> via a TS interop helper so portaled popovers receive the dark token.
2026-06-20 00:28:20 -04:00
daniel-c-harvey 0c22ce8f09 docs: add AuthBlocks NewUser/SuperRegister normalization team brief
Brief the AuthBlocks team to make NewUser the canonical direct-provision page (absorbing SuperRegister) and keep Registration as the invite flow.
2026-06-20 00:28:06 -04:00
daniel-c-harvey 67645cfd05 wire Mailtrap TestInbox config in DeepDrftAPI
Read AuthBlocks:Email:TestInbox from config (no throw — optional sandbox key). Add TestInbox placeholder to authblocks.example.json.
2026-06-20 00:27:01 -04:00
daniel-c-harvey 2591710f09 refactor: replace eval dark-mode body-class with TS theme interop helper
Extracts setBodyThemeClass into DeepDrftShared.Client/Interop/theme/theme.ts;
MainLayout lazy-imports the compiled module and calls it, matching the
established knob/parallax IJSObjectReference pattern. DisposeAsync added.
2026-06-20 00:26:52 -04:00
daniel-c-harvey 30999b038c fix: gate OnAfterRenderAsync body-class JS call; hoist dark popover token
Only stamps body class on firstRender or _isDarkMode change; adds base call.
Hoists duplicate dark popover mix value to --deepdrft-popover-surface-dark in :root;
both .deepdrft-theme-dark and body.deepdrft-theme-dark reference it via var().
2026-06-20 00:21:53 -04:00
daniel-c-harvey b5106d090f fix: popover surface — body-class bridge for portal scope, retune light/dark
MudBlazor popovers portal to <body>, outside the theme wrapper, so the dark token
was unreachable. MainLayout now stamps deepdrft-theme-dark on <body>. Light: 8%->4%
navy (near page background); dark: navy-mid + 20% green-accent (bluer).
2026-06-20 00:15:42 -04:00
daniel-c-harvey a2ed334d0d docs: mark ModelView DI briefs resolved (shipped in BlazorBlocks 10.3.33 / AuthBlocks 10.3.36) 2026-06-19 23:58:14 -04:00
daniel-c-harvey 9300c794b4 Merge p19-w4-authblocks-1036-bump into dev (AuthBlocks 10.3.36: JWT refresh fix + ModelView DI fix; drop stopgap) 2026-06-19 23:57:08 -04:00
daniel-c-harvey 95dd48018a chore: bump AuthBlocks to 10.3.36, drop EditModalSaveContextHolder stopgap
10.3.36 fixes JWT refresh for idle sessions and registers EditModalSaveContextHolder via AddBlazorBlocksWeb() — making the manual stopgap in DeepDrftManager/Program.cs redundant. BlazorBlocks direct refs (10.3.30) resolved without conflict; left unchanged.
2026-06-19 23:54:10 -04:00
daniel-c-harvey c21b85afdf docs: note BatchUpload captures user id at init to survive mid-session token expiry 2026-06-19 23:39:10 -04:00
daniel-c-harvey 234a57d6b7 Merge cms-upload-userid-capture into dev (capture upload-form user id at init so mid-session token expiry can't discard a composed release) 2026-06-19 23:28:45 -04:00
daniel-c-harvey 4bec507aab docs: split ModelView DI brief into per-team BlazorBlocks and AuthBlocks briefs
Two self-contained team briefs with explicit ship-ordering; original trimmed to an index pointing to both.
2026-06-19 23:26:56 -04:00
daniel-c-harvey a30d15f79d fix: correct BatchUpload comments — no prerender pass on this host, single init pass on live interactive circuit 2026-06-19 23:23:16 -04:00
daniel-c-harvey b90604d311 docs: add brief for upstream BlazorBlocks ModelView DI-registration fix
EditModalSaveContextHolder is required by ModelView but registered by no BlazorBlocks/AuthBlocks setup extension. Recommends AddBlazorBlocksWeb() called from ConfigureAuthServices.
2026-06-19 23:16:29 -04:00
daniel-c-harvey 77d0562b08 feature: Dark Theme Home & About Styles 2026-06-19 23:15:26 -04:00
daniel-c-harvey aeda7e67a8 Merge p19-w3-editmodal-holder into dev (register EditModalSaveContextHolder so AuthBlocks Users/Registrations pages render) 2026-06-19 23:12:36 -04:00
daniel-c-harvey bd9c67fc65 fix: capture upload-form user id at init, not submit, so token expiry mid-session can't discard a composed release 2026-06-19 23:12:26 -04:00
daniel-c-harvey 62fe27224c fix: register EditModalSaveContextHolder in DeepDrftManager DI
ModelView has a required [Inject] of this type; without it navigating to /useradmin/users or /useradmin/registrations terminated the circuit. Matches the registration pattern from SkipperHaven.
2026-06-19 23:10:08 -04:00
daniel-c-harvey 0708bb7352 docs: correct pending-registration route references to api/pendingregistration 2026-06-19 22:53:59 -04:00
daniel-c-harvey e6d5b9b77a Merge p19-w2-mailtrap-fromaddress into dev (wire AuthBlocks:Email:From so invite-email sends succeed) 2026-06-19 22:48:06 -04:00
daniel-c-harvey 04847391ad fix: wire AuthBlocks:Email:From into EmailConnection.FromAddress
Mailtrap rejected invite sends because FromAddress was never populated. Adds the missing config assignment alongside Host/Token, and documents the From key in authblocks.example.json.
2026-06-19 22:45:49 -04:00
daniel-c-harvey 3d71b6836e docs: correct Wave 2 hero detail, add Wave 3 note to Phase 18 COMPLETED entry 2026-06-19 22:08:04 -04:00
daniel-c-harvey 833b5a921e Merge p18-w3-hero-dark-legibility into dev (Phase 18 Wave 3 — hero text + button dark-mode legibility) 2026-06-19 22:05:58 -04:00
daniel-c-harvey 3bf95538bd fix: dark btn-primary hover uses green-interactive (#429d6a) not green-light (#2A5C4F) so contrast improves on hover 2026-06-19 22:05:32 -04:00
daniel-c-harvey eb7e977f3c feature: AppBar clearance & Theming 2026-06-19 22:04:57 -04:00
daniel-c-harvey 0b8593950b docs: reflect Phase 19.1/19.2 landing (CMS nav drawer + auth-state DefaultLayout) 2026-06-19 22:04:02 -04:00
daniel-c-harvey 51ac1a76de fix(dark): hero text + button legibility on navy ground (Phase 18 W3)
Bind page-text/page-text-muted tokens directly in hero base rules (drop
:global overrides); dark-mode overrides for btn-primary (green-accent fill)
and btn-ghost (white text, light border).
2026-06-19 22:00:26 -04:00
daniel-c-harvey 949bccfb8e Merge p19-w1-t2-public-route-layout into dev (auth-state-driven DefaultLayout for public CMS routes) 2026-06-19 21:57:44 -04:00
daniel-c-harvey cfaf63468d Merge p19-w1-t1-cms-nav-drawer into dev (CMS nav drawer surfacing AuthBlocks user-admin + SuperRegister) 2026-06-19 21:57:37 -04:00
daniel-c-harvey d6dcd82a53 fix: gate SuperRegister nav link to UserAdmin role
Provision User nav link was visible to all authenticated CMS users but its target page is UserAdmin-gated. Wraps the MudNavLink in HierarchicalRoleAuthorizeView matching the UserAdminMenu pattern.
2026-06-19 21:57:00 -04:00
daniel-c-harvey 3485acf3a8 feat: auth-state-driven DefaultLayout for CMS public routes
Resolve Routes.razor DefaultLayout from cascaded AuthenticationState so unauthenticated AuthBlocks pages (/account/login, /account/register) render in lean CmsHomeLayout instead of the authenticated CmsLayout shell.
2026-06-19 21:16:42 -04:00
daniel-c-harvey c04c2a9e98 docs: reflect Phase 18 landing; fix palette-file claim in CLAUDE.md 2026-06-19 21:16:40 -04:00
daniel-c-harvey f1276faabc feat(cms): add nav drawer to CmsLayout
Add a MudDrawer with app-bar toggle linking Catalogue, Releases, Upload, SuperRegister, and the self-gating UserAdminMenu fragment so user-admin pages are reachable.
2026-06-19 21:06:47 -04:00
daniel-c-harvey 6029e226d5 Merge p18-w2-theme-followups into dev (Phase 18 Wave 2 — appbar navy, dark hero legibility, true page ground, green-on-green play chip) 2026-06-19 21:01:24 -04:00
daniel-c-harvey 135cc48301 fix: correct AppbarBackground dark-mode comment — appbar is lighter than #0D1B2A page ground, not the ground itself 2026-06-19 21:00:44 -04:00
daniel-c-harvey 54766fd5fc docs: correct Phase 19 to CMS-only host model (drop DeepDrftPublic track)
All three AuthBlocks account paths live on DeepDrftManager; public registration is an unauthenticated CMS route like the CMS login. Path 2 reduces to a single auth-state-driven DefaultLayout fix (SkipperHaven pattern).
2026-06-19 20:46:14 -04:00
daniel-c-harvey fcc95b9195 style: Phase 18 Wave 2 — appbar navy, dark hero legibility, true page ground, green-on-green play chip 2026-06-19 20:32:21 -04:00
daniel-c-harvey 042641d841 docs: expand Phase 19 to all three AuthBlocks registration paths + reset brief
Cover admin provision-now, public self-service redeem, and admin invite-by-email across CMS + public-site tracks. Add standalone AuthBlocks password-reset team brief.
2026-06-19 19:18:53 -04:00
daniel-c-harvey 0358df82ac feat: Player & Menu Styles 2026-06-19 19:18:40 -04:00
daniel-c-harvey 0f7088fe86 Merge p18-w1-theme-dark-remediation into dev (Phase 18 dark-mode token pass) 2026-06-19 19:12:26 -04:00
daniel-c-harvey 5408d0779c fix: scope play-glyph override to dark mode, fix connect-option hover, tokenize bio placeholder, correct popover comment 2026-06-19 19:04:05 -04:00
daniel-c-harvey abe94953b9 docs: add Phase 19 user-management CMS wiring plan + product note 2026-06-19 19:02:40 -04:00
daniel-c-harvey 03fdcda054 style: theme-aware token pass for dark-mode surfaces (Phase 18)
Re-point neutral page surfaces, play-chip, and default popover from constant brand tokens to theme-aware aliases defined twice in deepdrft-tokens.css. Decorative navy/green sections and bespoke dark-glass panels untouched. Appbar-navy symptom deferred (palette C#, out of CSS scope).
2026-06-19 18:12:35 -04:00
daniel-c-harvey 5298cab9b1 feature: Re-enable Dark Mode Toggle & App Bar Styles & Mobile App Bar Fixes
Deploy DeepDrftAPI / Build, Publish & Bundle (push) Successful in 2m9s
Deploy DeepDrftPublic / Build & Publish (push) Successful in 4m7s
Deploy DeepDrftAPI / Deploy (push) Successful in 1m34s
Deploy DeepDrftPublic / Deploy (push) Successful in 1m31s
2026-06-19 17:48:26 -04:00
daniel-c-harvey e05d93a67b docs: document upload staging directory and Upload:StagingPath config 2026-06-19 17:45:52 -04:00
daniel-c-harvey fd4fdd2624 docs: add Phase 18 theme/dark-mode remediation plan + product note 2026-06-19 17:41:11 -04:00
daniel-c-harvey 639f4741e6 Merge upload-temp-disk-fix into dev (stage large audio uploads on data disk instead of /tmp) 2026-06-19 17:37:26 -04:00
daniel-c-harvey d7071fdbc2 fix: always delete staging file on mid-copy abort
Build the staging path before the copy in both UploadTrack and ReplaceAudio so the finally block deletes it on cancellation or IO error, not only on success.
2026-06-19 17:36:06 -04:00
daniel-c-harvey 37cf19c405 fix: stage audio uploads on data disk instead of /tmp
Relocate both the framework multipart buffer (via ASPNETCORE_TEMP) and the controller staging file to a configurable data-disk directory, so large WAV/FLAC/MP3 uploads no longer fail on the host's small tmpfs.
2026-06-19 17:25:51 -04:00
daniel-c-harvey 37bbfb947f docs: note footer PRIVACY button + centered MudOverlay privacy modal 2026-06-19 17:09:37 -04:00
daniel-c-harvey 261b11436e Merge privacy-footer-overlay into dev (PRIVACY footer button + centered overlay note)
Deploy DeepDrftAPI / Build, Publish & Bundle (push) Successful in 2m11s
Deploy DeepDrftManager / Build & Publish (push) Successful in 1m29s
Deploy DeepDrftPublic / Build & Publish (push) Successful in 3m56s
Deploy DeepDrftAPI / Deploy (push) Successful in 1m33s
Deploy DeepDrftManager / Deploy (push) Successful in 1m29s
Deploy DeepDrftPublic / Deploy (push) Successful in 1m29s
2026-06-19 17:02:17 -04:00
daniel-c-harvey 280dbbcbc9 style: DRY footer btn CSS, add trailing newline, drop wrong section ordinal 2026-06-19 16:59:01 -04:00
daniel-c-harvey ce17a685e0 docs: reflect Phase 17 Wave 17.3 landing; Phase 17 complete 2026-06-19 16:48:48 -04:00
daniel-c-harvey 64379c8901 feat: move footer privacy note behind PRIVACY overlay button 2026-06-19 16:48:46 -04:00
daniel-c-harvey 1f8802363c Merge p17-w3-embed-panel into dev (Phase 17 Wave 17.3: Fixed embed queue panel + collapse/resize handshake) 2026-06-19 16:38:38 -04:00
daniel-c-harvey 58cdb4d9dc fix: isolate multi-embed resize handshake with per-snippet token
ForRelease mints a per-call token used as the iframe id and threaded into the src as EmbedId; the host script matches on it so multiple embeds resize independently. ForTrack unchanged.
2026-06-19 16:32:59 -04:00
daniel-c-harvey 97cce691db docs: document upload duplicate-detection rule, release/exists endpoint, and FindOrCreateRelease WasCreated contract 2026-06-19 16:25:50 -04:00
daniel-c-harvey d0be26bb3e Merge upload-duplicate-detection into dev (block duplicate-release uploads by title+artist) 2026-06-19 16:22:28 -04:00
daniel-c-harvey 466084b5a3 feat: Phase 17.3 — Fixed embed queue panel with collapse/expand iframe resize (OQ1 Option A)
Read-only inline queue panel below the release embed's player bar; row-jump reuses PlayRelease. ForRelease mints a taller iframe plus a postMessage resize listener for the collapse toggle; ForTrack unchanged.
2026-06-19 16:21:45 -04:00
daniel-c-harvey 558ff4b4c6 fix: close TOCTOU in CREATE path; add anti-forgery, loose-track, and case-sensitivity tests
FindOrCreateRelease now returns (ReleaseDto, bool WasCreated); the CREATE path in UploadAsync
rejects WasCreated=false as a duplicate rather than silently attaching on a lost race.
2026-06-19 15:55:08 -04:00
daniel-c-harvey bd85507308 Block duplicate-release uploads by (title, artist): pre-flight check + server 409 backstop, with within-batch Cut attach via releaseId 2026-06-19 15:44:41 -04:00
daniel-c-harvey fbd298b9c3 docs: reflect Phase 17 Wave 2 (docked overlay + Add-to-Queue) and Phase 16.5 capstone landing
Deploy DeepDrftAPI / Build, Publish & Bundle (push) Successful in 3m2s
Deploy DeepDrftManager / Build & Publish (push) Successful in 1m24s
Deploy DeepDrftPublic / Build & Publish (push) Successful in 4m2s
Package install tarball / package (push) Successful in 6s
Deploy DeepDrftAPI / Deploy (push) Successful in 1m34s
Deploy DeepDrftManager / Deploy (push) Successful in 1m31s
Deploy DeepDrftPublic / Deploy (push) Successful in 2m0s
2026-06-19 15:42:17 -04:00
daniel-c-harvey 3da6591194 docs(phase-16): reflect live Plays card in stats CLAUDE.md
HomeStatsDto gains TotalPlays + UniqueListeners; StatsController now composes ITrackService + IEventService (best-effort play/listener reads).
2026-06-19 15:41:17 -04:00
daniel-c-harvey da60296cf8 Merge p17-w2-t2-add-to-queue into dev (Phase 17 Wave 17.4: Add-to-Queue affordance) 2026-06-19 15:35:19 -04:00
daniel-c-harvey 4320ea8029 Merge p17-w2-t1-docked-overlay into dev (Phase 17 Wave 17.2: docked queue overlay + ClearUpcoming) 2026-06-19 15:34:59 -04:00
daniel-c-harvey 678d3f66ad Merge p16-w5-t2-privacy-footer into dev (anonId privacy disclosure footer line) 2026-06-19 15:33:28 -04:00
daniel-c-harvey be04e53a97 Merge p16-w5-t1-plays-card into dev (Phase 16 Wave 16.5: home Plays-card live) 2026-06-19 15:31:37 -04:00
daniel-c-harvey 58b30d3c13 feat(footer): add anonId privacy disclosure line
Wraps existing footer row in .deepdrft-footer-main and adds a
.deepdrft-footer-privacy paragraph below it with the approved
Variant 1 copy. Mono fine-print at 0.55 rem / 70% opacity.
2026-06-19 15:26:07 -04:00
daniel-c-harvey be1a55fd37 feat(stats): flip home Plays card live (Phase 16.5)
Add TotalPlays + UniqueListeners to HomeStatsDto, composed at
StatsController from IEventService (no migration). Card reads via
existing persistent-state-bridged round-trip.
2026-06-19 15:26:07 -04:00
daniel-c-harvey 9d0ce99a5d fix: PlayRelease always materialises a defensive copy so Items alias can't wipe the queue on jump; add aliasing regression test 2026-06-19 15:23:20 -04:00
daniel-c-harvey 1d387c2a34 feat(player): add append-only "Add to Queue" buttons beside detail-page play affordances
Cut header (release → EnqueueRange), Cut track rows + Session/Mix hero (track → Enqueue). Reuses existing engine path; add is not play.
2026-06-19 15:18:38 -04:00
daniel-c-harvey fe3819f378 feat(player): docked queue overlay with reorder, remove, jump, and clear-upcoming
Add a Queue toggle to the docked player bar opening a centered editable queue
overlay. New additive QueueService.ClearUpcoming keeps the playing track while
dropping the rest. Current track is non-removable.
2026-06-19 15:18:25 -04:00
daniel-c-harvey cfcc2693f2 docs: reflect raised upload cap (~1.86 GB) and 1200s response timeout 2026-06-19 15:14:07 -04:00
daniel-c-harvey 621c4f9cb3 docs(phase-16): draft anonId privacy-note copy; note deferred Postgres integration harness 2026-06-19 15:10:15 -04:00
daniel-c-harvey 67eeb38529 Merge fix-large-upload-cap into dev (raise CMS upload cap to ~1.86 GB + nginx timeouts) 2026-06-19 15:08:48 -04:00
daniel-c-harvey 9aa66e8a62 docs: resolve remaining seven Phase 17 open questions (all 11 now closed) 2026-06-19 15:08:39 -04:00
daniel-c-harvey 3b9ca700c9 raise upload size cap to ~1.86 GB and nginx timeouts to 1200s
Raise RequestSizeLimit/MultipartBodyLengthLimit on upload+replace-audio,
MaxUploadBytes in BatchUpload/BatchEdit, and DefaultResponseTimeoutSeconds to
1200s. Add client_max_body_size 2000m and proxy_read/send_timeout 1200s to the
nginx manager/public confs.
2026-06-19 15:02:49 -04:00
daniel-c-harvey 4317a2f9e7 docs(phase-16): record 16.2 absorption + 16.3 anonId landing
PLAN/COMPLETED mark 16.2 absorbed into 16.1 and 16.3 landed (no migration). Folder CLAUDE.md files reflect anonId now accepted/persisted + the distinct-listener queries.
2026-06-19 14:57:23 -04:00
daniel-c-harvey 297805b5a8 Merge p16-w3-anonid into dev (Phase 16 Wave 16.3: unique-listener anonId layer) 2026-06-19 14:43:46 -04:00
daniel-c-harvey 944f23a88c docs: reflect Phase 17 Wave 17.1 landing (queue Move/RemoveAt + QueueList) 2026-06-19 14:43:36 -04:00
daniel-c-harvey 75e5d99aea Merge p17-w1-queue-engine-list into dev (Phase 17 Wave 17.1: queue Move/RemoveAt + shared QueueList) 2026-06-19 14:38:25 -04:00
daniel-c-harvey c084efa78e feat(phase-16.3): light up anonId unique-listener layer
Mint a first-party localStorage anonId, thread it onto play/share beacons,
persist it via EventController, and add all-time distinct-listener counts
(site/track/release). Storage columns + indexes already existed from 16.1.
2026-06-19 14:37:55 -04:00
daniel-c-harvey f296bbdf00 Add queue Move/RemoveAt + dormant-Enqueue coherence and shared QueueList (Phase 17.1) 2026-06-19 14:32:08 -04:00
daniel-c-harvey ebbaa3f84f docs: resolve four Phase 17 open questions (OQ1/OQ4/OQ8/OQ10), defer ReleaseGallery card affordance 2026-06-19 13:42:19 -04:00
daniel-c-harvey a715f4b28d Merge p16-w1-foundation into dev (Phase 16 Wave 16.1: anonymous play & share telemetry substrate) 2026-06-19 13:34:01 -04:00
daniel-c-harvey 90555dc4e0 docs: spec Phase 17 player-bar queue view (queue button, overlay/embed modes, add-to-queue) 2026-06-19 13:29:57 -04:00
daniel-c-harvey 0fbf81b23e Merge branch 'dev' into p16-w1-foundation
# Conflicts:
#	DeepDrftPublic.Client/Controls/SharePopover.razor.cs
2026-06-19 13:28:50 -04:00
daniel-c-harvey 4114aa0be4 docs: reflect embed new-tab title link and embed queue skip buttons 2026-06-19 13:22:29 -04:00
daniel-c-harvey 884ccab826 Merge p16-embed-newwindow into dev (embed: new-tab title link + queue skip buttons) 2026-06-19 13:17:36 -04:00
daniel-c-harvey 3c1998de4f feat(embed): show skip-prev/next buttons in embed when queue exists 2026-06-19 13:10:50 -04:00
daniel-c-harvey 622ee940f4 fix(phase-16): forward X-Forwarded-For from EventProxyController so the API rate limiter partitions per client IP
Proxy chains any inbound XFF with the connection IP before relaying upstream; UseForwardedHeaders resolves it to the limiter's partition key. Documents the EventRepository first-play counter race (unique index is the backstop).
2026-06-19 13:09:21 -04:00
daniel-c-harvey 18e171213c feat: open player title link in new tab when embedded (Fixed mode) 2026-06-19 13:08:04 -04:00
daniel-c-harvey e9c61bac1a docs: reflect whole-release embeds, queue armed-idle state, and per-track share 2026-06-19 13:00:13 -04:00
daniel-c-harvey dbd90ee52a feat(phase-16): anonymous play & share telemetry substrate (wave 16.1)
Player-service play-session tracker (floor + 3-bucket classify), SharePopover share tracker with debounce, sendBeacon interop, proxied rate-limited POST api/event/{play,share}, append-only event logs + incremental play_counter with server-side release resolution. Migration authored, not applied. No anonId, no read surface.
2026-06-19 12:59:00 -04:00
daniel-c-harvey 1b7861e168 Merge p16-release-embed into dev (whole-release embeds + per-track share) 2026-06-19 12:55:11 -04:00
daniel-c-harvey 098020db32 feat: add per-track SharePopover to Cut detail track rows 2026-06-19 12:08:27 -04:00
daniel-c-harvey 912256d99a Add whole-release embeds to FramePlayer and un-gate the release embed share affordance
The queue gains an armed-but-idle state (Arm/Start) so a release embed stages track 0 prerender-safe, then queues the full release on first play and auto-advances.
2026-06-19 12:05:35 -04:00
daniel-c-harvey 1931574ad4 Merge gitattributes-knob-eol into dev (pin knob.js to LF, stop CRLF churn on Windows checkout) 2026-06-19 11:39:40 -04:00
daniel-c-harvey 25aba1cbb7 docs(phase-16): resolve decisions D1-D7; re-sequence waves bottom-up, card last 2026-06-19 11:32:24 -04:00
daniel-c-harvey 81d0028f2b fix: pin knob.js to LF in .gitattributes to stop CRLF churn on Windows checkout 2026-06-19 11:32:18 -04:00
daniel-c-harvey 62007a6517 fix: Icons
Deploy DeepDrftManager / Build & Publish (push) Successful in 2m9s
Deploy DeepDrftPublic / Build & Publish (push) Successful in 4m18s
Deploy DeepDrftManager / Deploy (push) Successful in 1m31s
Deploy DeepDrftPublic / Deploy (push) Successful in 1m28s
2026-06-19 11:15:19 -04:00
daniel-c-harvey 13b07beb0b fix: Styles & Links & Content 2026-06-19 11:15:09 -04:00
daniel-c-harvey 7711c5067c docs: reflect DurationSeconds write on replace-audio
Deploy DeepDrftAPI / Build, Publish & Bundle (push) Successful in 4m3s
Deploy DeepDrftAPI / Deploy (push) Successful in 1m35s
Replace path now updates SQL DurationSeconds via unconditional SetDuration; document SetDuration vs null-guarded UpdateDuration and correct the stale 'SQL is not written' note.
2026-06-19 10:15:59 -04:00
daniel-c-harvey eaa71ebea3 Merge replace-audio-duration-sync into dev (sync DurationSeconds on audio replace via unconditional SetDuration) 2026-06-19 10:13:19 -04:00
daniel-c-harvey e8359d5473 fix: replace-audio duration write now unconditional via SetDuration
UpdateDuration's null guard matched zero rows for tracks that already had a duration (all normally-uploaded tracks). Add SetDurationAsync/SetDuration/ITrackService.SetDuration with no null guard; fail on zero rows. ReplaceAudioAsync now calls SetDuration.
2026-06-19 04:19:39 -04:00
daniel-c-harvey 7265754c27 fix: write DurationSeconds to SQL after replace-audio vault swap 2026-06-18 15:03:38 -04:00
daniel-c-harvey abc832467d docs(plan): add Phase 16 spec — anonymous play & share tracking
Design spec for the telemetry layer behind the home-hero Plays card:
completion-bucketed plays, shares, optional anonymous unique listeners
under a no-PII constraint. Seven open decisions flagged for Daniel.
2026-06-18 14:28:02 -04:00
daniel-c-harvey 47919a226e feature: Home page graphics
Deploy DeepDrftAPI / Build, Publish & Bundle (push) Successful in 2m22s
Deploy DeepDrftManager / Build & Publish (push) Successful in 1m23s
Deploy DeepDrftPublic / Build & Publish (push) Successful in 4m16s
Deploy DeepDrftAPI / Deploy (push) Successful in 1m35s
Deploy DeepDrftManager / Deploy (push) Successful in 1m29s
Deploy DeepDrftPublic / Deploy (push) Successful in 1m28s
2026-06-18 14:25:08 -04:00
daniel-c-harvey 933b7145e5 Merge knob-js-deploy-fix into dev (commit compiled RCL knob JS so it ships in publish output) 2026-06-18 13:18:43 -04:00
daniel-c-harvey f21647423a docs: document track replace-audio endpoint and edit-form gating 2026-06-18 13:17:30 -04:00
daniel-c-harvey df7acd9e80 docs: reflect live home-hero stats (duration column, stats endpoint, backfill, NowPlayingStats wiring) 2026-06-18 13:14:52 -04:00
daniel-c-harvey 3a4db834ac fix: track compiled RCL knob JS for MapStaticAssets deployment 2026-06-18 13:14:09 -04:00
daniel-c-harvey d12151278a Merge cms-track-replace-gating into dev
Replace track audio in CMS edit form + gate last-track delete.
2026-06-18 13:14:08 -04:00
daniel-c-harvey ca90302f21 fix: register-new-then-remove-old in ReplaceTrackAudioAsync; replace wording in timeout messages; doc comment on ExistingTrackCount
On partial failure the old path deleted the original audio before
confirming the new write succeeded. Now: load old extension, register
new audio first (original untouched on failure), then clean up stale
backing file only on success and only when extension changed.
2026-06-18 13:11:59 -04:00
daniel-c-harvey 16784b37f2 feat(cms): replace track audio in edit form, gate last-track delete
Swap a track's audio by EntryKey (metadata/release/position preserved, waveform regenerated); hide per-track remove on a release's sole persisted track so it can only be replaced or release-deleted.
2026-06-18 12:59:56 -04:00
daniel-c-harvey e9e6b6054f Merge nowplaying-stats into dev (live home-hero aggregate stats + track duration column) 2026-06-18 12:58:54 -04:00
daniel-c-harvey 8fa330fbd3 fix: exclude live tracks under soft-deleted releases from home stats cut/mix figures 2026-06-18 12:42:23 -04:00
daniel-c-harvey 5f0422a263 Wire NowPlayingStats to live aggregates: add SQL track duration column, stats endpoint, and duration backfill 2026-06-18 11:53:49 -04:00
daniel-c-harvey 8ddecb4acc about styles
Deploy DeepDrftPublic / Build & Publish (push) Successful in 3m57s
Deploy DeepDrftPublic / Deploy (push) Successful in 1m28s
2026-06-17 22:35:41 -04:00
307 changed files with 30101 additions and 1216 deletions
+1
View File
@@ -9,3 +9,4 @@
*.conf text eol=lf
# Vendor JS pinned LF — avoids CRLF churn on Windows checkout
DeepDrftShared.Client/wwwroot/js/parallax/parallax.js text eol=lf
DeepDrftShared.Client/wwwroot/js/knob/knob.js text eol=lf
+4 -1
View File
@@ -317,4 +317,7 @@ Database/Vaults/*
!DeepDrftPublic.Client/wwwroot/js/*.js
# RCL compiled JS must be committed — MapStaticAssets serves from build-time manifest;
# gitignored TS output is absent when manifest is generated, so absent from publish output.
!DeepDrftShared.Client/wwwroot/js/parallax/
# Re-include the whole RCL js/ tree so every compiled module (parallax, knob, theme, and
# any added later) ships, rather than maintaining a per-module allowlist.
!DeepDrftShared.Client/wwwroot/js/
!DeepDrftShared.Client/wwwroot/js/**
+39 -17
View File
@@ -8,12 +8,12 @@ DeepDrftHome is a **net10.0** solution consisting of ten projects implementing a
### Core Projects
- **DeepDrftPublic**: ASP.NET Core host. Blazor Web App with Server + WASM render modes. Owns browser-facing proxy controller for `api/track/*` (metadata listing and audio streaming), MudBlazor theme prerender, and TypeScript→JS audio interop. Public-facing site for listeners.
- **DeepDrftPublic.Client**: Blazor WebAssembly assembly. All interactive UI (pages, player stack, dark-mode plumbing, HTTP clients for both backends). Pages include the public `/about` editorial page (`Pages/About.razor` — three-movement **"Liner Notes"** editorial treatment: numbered left-rail (oversized Bodoni numerals + vertical hairline spine + mono marginalia captions), asymmetric content column, pull-quotes breaking into the margin, hand-authored SVG waveform movement dividers (self-contained motif, not the live `WaveformVisualizer`), and stacked editorial definition list for CUTS/SESSIONS/MIXES; active-movement highlight via `about-rail.ts` IntersectionObserver interop; registered in `Layout/Pages.cs`). Consumed by the public site.
- **DeepDrftManager**: ASP.NET Core host. Blazor Web App with server-rendered `InteractiveServer` render mode. Hosts all CMS Razor components and pages under `Components/Pages/Cms/`, `Components/Pages/Tracks/`, `Components/Layout/CmsLayout.razor`, and `Components/Shared/` (all inlined from the former `DeepDrftCms` RCL). Public entry point: `Components/Pages/Home.razor` (`@page "/"`, no `[Authorize]`, uses lean `CmsHomeLayout`) — unauthenticated visitors see a DeepDrft-branded splash with a Login CTA; authenticated admins are redirected to `/catalogue` via `RedirectToCatalogue`. The catalogue dashboard (`Components/Pages/Index.razor`) lives at `@page "/catalogue"` and remains `[Authorize]`-gated with `CmsLayout`; its cards are **CUTS / SESSIONS / MIXES**, each deep-linking to `/releases?medium=<medium>` with the matching tab pre-selected. The consolidated browse surface is `Components/Pages/Tracks/Releases.razor` (`@page "/releases"`): bulk-action buttons (Generate All Profiles / Backfill High-res) → medium tab strip (ALL / CUTS / SESSIONS / MIXES) → the active tab's grid; waveform columns (Profile / High-res) — each showing a status icon when a datum is present and an always-visible generate/regenerate button — and per-track info tooltip live in `CmsAlbumBrowser`'s expanded child-row track table. Old list routes `/tracks`, `/tracks/albums`, `/tracks/archive` are kept as aliases on `Releases.razor` so bookmarks don't 404; operational sub-routes (`/tracks/upload`, edit routes, etc.) remain at `/tracks/*`. Gated by AuthBlocks login and hierarchical `Admin` role authorization. All track operations (upload, metadata read/write, delete) are HTTP proxies via `ICmsTrackService` / `CmsTrackService` injected directly into Blazor components; no in-process data layer. Two named HttpClients: `DeepDrft.Content.Cms` (bounded 100 s default, for all non-upload calls) and `DeepDrft.Content.Cms.Upload` (`InfiniteTimeSpan`, for large WAV uploads). Upload progress and idle/heartbeat timeout are driven by a single `ProgressStreamContent` wrapper (`Services/ProgressStreamContent.cs`); `CmsTrackService.UploadTrackAsync` adds a two-phase cancellation (idle window resets per progress tick; separate response-wait budget arms when the body completes).
- **DeepDrftPublic**: ASP.NET Core host. Blazor Web App with Server + WASM render modes. Owns browser-facing proxy controller for `api/track/*` (metadata listing and audio streaming), crawl-directive endpoints (`GET /robots.txt` and `GET /sitemap.xml`, environment-gated via `IWebHostEnvironment.IsProduction()` directly — server-side only, no PersistentState bridge — served by `CrawlDirectiveController` with pure builders in `Seo/RobotsTxt.cs` and `Seo/SitemapXml.cs`), MudBlazor theme prerender, and TypeScript→JS audio interop. Public-facing site for listeners.
- **DeepDrftPublic.Client**: Blazor WebAssembly assembly. All interactive UI (pages, player stack, dark-mode plumbing, HTTP clients for both backends). Pages include the public `/about` editorial page (`Pages/About.razor` — three-movement **"Liner Notes"** editorial treatment: numbered left-rail (oversized Bodoni numerals + vertical hairline spine + mono marginalia captions), asymmetric content column, pull-quotes breaking into the margin, hand-authored SVG waveform movement dividers (self-contained motif, not the live `WaveformVisualizer`), and stacked editorial definition list for CUTS/SESSIONS/MIXES; active-movement highlight via `about-rail.ts` IntersectionObserver interop; registered in `Layout/Pages.cs`). Home hero stat row (`NowPlayingStats.razor`) is live-data-backed via `IStatsDataService` / `StatsClient` (named `"DeepDrft.API"` client) with a `PersistentComponentState` prerender bridge; `RuntimeFormat` helper converts mix runtime seconds to `hh:mm`. **SEO component** (`Controls/SeoHead.razor` + `Common/SeoModel`, `SeoJsonLd`, `SeoOptions`, `SeoUrls`, `SeoEnvironment`): `SeoHead` is a presentational `<HeadContent>` emitter (one line per page, no fetch); `SeoModel` named factories (`ForRelease`/`ForHome`/`ForAbout`/`ForBrowse`/`ForNotFound`) encode the medium→schema.org mapping in one place; `SeoJsonLd` builds typed JSON-LD (MusicGroup / MusicAlbum+LiveAlbum / MusicRecording / CollectionPage) with inline-safe escaping; `SeoOptions` holds site-wide config (`BaseUrl https://deepdrft.com`, title suffix, default OG image seam, IG `sameAs`) registered via the static `Startup` seam; `SeoEnvironment` is a scoped `[PersistentState]` bridge (mirrors `DarkModeSettings`) seeded in `DeepDrftPublic/Components/App.razor` from `IWebHostEnvironment.IsProduction()` — robots defaults to `index,follow` only in Production, `noindex,nofollow` everywhere else (fail-safe is noindex); per-page `SeoModel.Robots` overrides the default. Tags are present in prerendered HTML (rides the existing `PersistentComponentState` bridge; no new fetch). Canonical/OG origins come from `SeoOptions.BaseUrl` (config), not `window.location` — no `window` at server prerender and the origin cannot be derived behind the nginx proxy. Consumed by the public site.
- **DeepDrftManager**: ASP.NET Core host. Blazor Web App with server-rendered `InteractiveServer` render mode. **Always uncrawlable**: a static `wwwroot/robots.txt` (`Disallow: /`, no env gate) plus a blanket `<meta name="robots" content="noindex,nofollow">` in `Components/App.razor` — defense in depth so the CMS is never indexed regardless of how it is discovered. Hosts all CMS Razor components and pages under `Components/Pages/Cms/`, `Components/Pages/Tracks/`, `Components/Layout/CmsLayout.razor`, and `Components/Shared/` (all inlined from the former `DeepDrftCms` RCL). Public entry point: `Components/Pages/Home.razor` (`@page "/"`, no `[Authorize]`, uses lean `CmsHomeLayout`) — unauthenticated visitors see a DeepDrft-branded splash with a Login CTA; authenticated admins are redirected to `/catalogue` via `RedirectToCatalogue`. `Routes.razor` resolves `DefaultLayout` from the cascaded `Task<AuthenticationState>`: unauthenticated → `CmsHomeLayout`, authenticated → `CmsLayout`; this means the AuthBlocks `Login`/`Register` pages (which declare no `@layout`) render in the lean layout for unauthenticated visitors. `CmsLayout` carries a left `MudDrawer` (app-bar hamburger toggle) holding the CMS destinations (Catalogue `/catalogue`, Releases `/releases`, Upload `/tracks/upload`), the AuthBlocks `UserAdminMenu` fragment (self-gates to `UserAdmin`+, links Users/Registrations/Permissions), and a "Provision User" link to `/useradmin/users/new` wrapped in a `HierarchicalRoleAuthorizeView` (`UserAdmin`-gated) — making the AuthBlocks user-administration surface reachable from the CMS UI. The catalogue dashboard (`Components/Pages/Index.razor`) lives at `@page "/catalogue"` and remains `[Authorize]`-gated with `CmsLayout`; its cards are **CUTS / SESSIONS / MIXES**, each deep-linking to `/releases?medium=<medium>` with the matching tab pre-selected. The consolidated browse surface is `Components/Pages/Tracks/Releases.razor` (`@page "/releases"`): bulk-action buttons (Generate All Profiles / Backfill High-res) → medium tab strip (ALL / CUTS / SESSIONS / MIXES) → the active tab's grid; waveform columns (Profile / High-res) — each showing a status icon when a datum is present and an always-visible generate/regenerate button — and per-track info tooltip live in `CmsAlbumBrowser`'s expanded child-row track table. Old list routes `/tracks`, `/tracks/albums`, `/tracks/archive` are kept as aliases on `Releases.razor` so bookmarks don't 404; operational sub-routes (`/tracks/upload`, edit routes, etc.) remain at `/tracks/*`. Gated by AuthBlocks login and hierarchical `Admin` role authorization. All track operations (upload, metadata read/write, delete, replace audio) are HTTP proxies via `ICmsTrackService` / `CmsTrackService` injected directly into Blazor components; no in-process data layer. The per-track "Replace audio" affordance in `BatchEdit` / `BatchTrackList` / `BatchTrackDetail` swaps the vault bytes, regenerates both waveform datums server-side, and re-derives `DurationSeconds` from the new audio; the track id, `EntryKey`, release membership, position, and all other metadata are preserved. The remove control on a persisted track is hidden when it is the release's sole remaining persisted track — a release can reach zero live tracks only via replace or release-level delete, not per-track removal. Two named HttpClients: `DeepDrft.Content.Cms` (bounded 100 s default, for all non-upload calls) and `DeepDrft.Content.Cms.Upload` (`InfiniteTimeSpan`, for large WAV uploads). Upload progress and idle/heartbeat timeout are driven by a single `ProgressStreamContent` wrapper (`Services/ProgressStreamContent.cs`); `CmsTrackService.UploadTrackAsync` adds a two-phase cancellation (idle window resets per progress tick; separate response-wait budget arms when the body completes). The upload form is create-only: `BatchUpload.razor` calls `GET api/track/release/exists` as a pre-flight before transferring bytes and blocks the submit with a visible message if a (title, artist) match already exists; the server also rejects duplicates with 409. The authenticated user's id (`NameIdentifier` claim) is captured once into `_createdByUserId` at component initialization (`OnInitializedAsync`) — not re-read at submit — so a mid-session token expiry cannot discard a long-composed release; the page is `[Authorize]`-gated and runs `prerender: false`, so the auth state is fully available at init and only one init pass occurs. Within-batch multi-track Cuts still work by passing the release id from row 1 as `releaseId` on rows 2..N (the ATTACH path), while `BatchEdit.razor` uses the same ATTACH path for its legitimate adds-to-existing-release.
- **DeepDrftShared.Client**: Razor Class Library. Shared Blazor components consumed by both `DeepDrftPublic` and `DeepDrftManager` for consistency across public and admin surfaces.
- **DeepDrftData**: Class library. EF Core domain logic: `DeepDrftContext`, `TrackConfiguration`, `Migrations`, `TrackRepository`, `TrackService`, `TrackManager`. Consumed by `DeepDrftAPI` and tests.
- **DeepDrftAPI**: ASP.NET Core host. Dual-database authority (SQL metadata + FileDatabase binary). AuthBlocks API host (owns registration, migration/seed, JWT endpoints). Track endpoints: streaming, vault write, upload+persist, delete+cleanup, paged list with filters, single metadata (ApiKey-gated operations), metadata update, waveform profiles (512-bucket seeker + per-track high-res visualizer datum in the `track-waveforms` vault), release-track join operations. Release endpoints: paged list with medium filter, single read, session hero-image upload (all unauthenticated reads; authenticated writes via ApiKey). Image endpoints: authenticated upload, unauthenticated streaming.
- **DeepDrftAPI**: ASP.NET Core host. Dual-database authority (SQL metadata + FileDatabase binary). AuthBlocks API host (owns registration, migration/seed, JWT endpoints). Track endpoints: streaming, vault write, upload+persist, delete+cleanup, paged list with filters, single metadata (ApiKey-gated operations), metadata update, waveform profiles (512-bucket seeker + per-track high-res visualizer datum in the `track-waveforms` vault), release-track join operations, `POST api/track/duration/backfill` (ApiKey-gated one-time backfill of `DurationSeconds` for existing rows from vault audio). Stats endpoints: `GET api/stats/home` (unauthenticated; returns `HomeStatsDto` with cut track count, per-`ReleaseType` cut release counts, mix release count, and total mix runtime seconds). Release endpoints: paged list with medium filter, single read, session hero-image upload (all unauthenticated reads; authenticated writes via ApiKey). Image endpoints: authenticated upload, unauthenticated streaming.
- **DeepDrftContent**: Class library. The FileDatabase implementation in full (Models, Services, Utils, Abstractions, Constants), `AudioProcessor`, content-side `TrackService`. Consumed by hosts and tests.
- **DeepDrftModels**: Shared contracts. `TrackEntity`, `TrackDto`, `PagingParameters<T>`, `PagedResult<T>`. Every project references this.
- **DeepDrftTests**: NUnit test suite. Comprehensive FileDatabase tests (vault creation, media storage, indexing, factory patterns, utilities). Integration-focused with temp-directory test isolation.
@@ -34,7 +34,7 @@ Server-side (SSR): Both clients point directly at DeepDrftAPI (server-to-server,
1. **SQL Database (PostgreSQL)**: Metadata and track info via Entity Framework
- Connection string: Read from `environment/connections.json` via `CredentialTools.ResolvePathOrThrow("connections")` with key `ConnectionStrings:DefaultConnection`.
- Entity: `TrackEntity` with `Id`, `EntryKey`, `TrackName`, `Artist`, `Album?`, `Genre?`, `ReleaseDate?`, `ImagePath?`
- Entity: `TrackEntity` with `Id`, `EntryKey`, `TrackName`, `Artist`, `Album?`, `Genre?`, `ReleaseDate?`, `ImagePath?`, `DurationSeconds?`
- Context: `DeepDrftContext` in `DeepDrftData`
2. **FileDatabase**: Custom file-based binary storage system
@@ -42,7 +42,7 @@ Server-side (SSR): Both clients point directly at DeepDrftAPI (server-to-server,
- Root contains typed **MediaVaults** (Media, Image, Audio)
- Each vault has a JSON `index` file listing entries + per-entry metadata
- Entries are user-supplied strings sanitized to `[a-zA-Z0-9-]` + file extension
- Binary hierarchy: `FileBinary``MediaBinary` (+ Extension/MIME) → `AudioBinary` (+ Duration/Bitrate) | `ImageBinary` (+ AspectRatio)
- Binary hierarchy (model shapes): `FileBinary``MediaBinary` (+ Extension/MIME) → `AudioBinary` (+ Duration/Bitrate) | `ImageBinary` (+ AspectRatio). **Non-delivery read path** (`LoadResourceAsync<AudioBinary>`) returns a full-buffer `AudioBinary` — still used for non-delivery operations (e.g., duration backfill). **Audio delivery path** streams via `GetEntryStreamAsync` (Opus artifact, `track-opus` vault) or `OpenAudioMediaStreamAsync` (lossless source, `tracks` vault) — a seekable, disk-backed `Stream` per request, never a whole-file `byte[]` (read-side OOM fix, parallel to the store-side). The **write/store path** is streaming: audio processors return a `ProcessedAudio` plan (metadata + streamed `WriteToAsync` callback); `RegisterResourceStreamingAsync` / `MediaVault.AddEntryStreamingAsync` write bytes to a temp file then `File.Move` atomic-rename into place — the full `AudioBinary` buffer is never materialized on this path.
- **Error-handling philosophy**: public operations swallow exceptions and return `null`/`false` — callers must check return values, not catch.
## Key Architectural Decisions
@@ -57,36 +57,58 @@ The split between host projects (`DeepDrftPublic`, `DeepDrftManager`, `DeepDrftC
`TrackEntity` holds *only* metadata. The link to binary content is `EntryKey` (string) — the entry id inside the `tracks` vault in FileDatabase. Dual-database add flow:
1. `DeepDrftContent.TrackService.AddTrackFromWavAsync` processes WAV, generates entry GUID, stores audio in vault, returns unpersisted `TrackEntity`.
1. `TrackContentService.AddTrackAsync` routes the audio file by extension (`AudioProcessorRouter`), produces a `ProcessedAudio` plan (bounded-header metadata + streamed `WriteToAsync` callback — no whole-file `AudioBinary` buffer), and stores it in the vault via `FileDatabase.RegisterResourceStreamingAsync` / `MediaVault.AddEntryStreamingAsync` (atomic temp→rename on the Linux host). Returns an unpersisted `TrackEntity` with `DurationSeconds` populated from the header parse. Wave 1 OOM fix.
2. `DeepDrftAPI.Services.UnifiedTrackService.UploadAsync` persists the entity to SQL via `DeepDrftData.TrackManager` and returns the persisted entity with `Id`.
If step 1 succeeds and step 2 fails, audio is orphaned in the vault (no rollback today).
The Opus transcode derived-artifact path (`OpusTranscodeService.TranscodeAndStoreAsync`) also streams its entire pipeline: extension and duration are read from the vault index (no body load); the source bytes are opened via `TrackContentService.OpenAudioMediaStreamAsync` and bounded-copied to a staging file; the encoded Ogg output is walked from a `FileStream` via `OggOpusParser.WalkAsync(Stream)` (bounded one-page-at-a-time buffer; byte-identical to the retained whole-buffer oracle `Walk(ReadOnlySpan<byte>)`); and stored via `RegisterResourceStreamingAsync`. The sidecar (a few KB, inherently bounded) retains the whole-buffer write. Completes the store-path OOM-fix arc.
### Streaming-first audio playback
The player is not fetch-then-play:
1. Client calls `GET api/track/{id}` on DeepDrftContent and receives WAV bytes as a stream (`HttpCompletionOption.ResponseHeadersRead`).
2. `StreamingAudioPlayerService` reads in adaptive 1664 KB chunks, pushes each via `AudioInteropService.processStreamingChunk`.
3. TypeScript `StreamDecoder` parses WAV header, decodes chunks to `AudioBuffer`s. `PlaybackScheduler` schedules them on a Web Audio graph.
1. Client calls `GET api/track/{id}` on DeepDrftContent and receives the first bounded segment (`Range: bytes=0-{SegmentSizeBytes-1}`, 4 MB) via `HttpCompletionOption.ResponseHeadersRead`.
2. `StreamingAudioPlayerService` reads in adaptive 1664 KB chunks within each segment, pushes each via `AudioInteropService.processStreamingChunk`.
3. TypeScript `StreamDecoder` parses the format header, decodes chunks to `AudioBuffer`s. `PlaybackScheduler` schedules them on a Web Audio graph.
4. Playback starts as soon as a min buffer is queued; UI duration from parsed header (not waiting for full file).
5. **Seek beyond buffer**: if seek target is past what's decoded, client issues `GET api/track/{id}` with `Range: bytes={byteOffset}-`. Server streams raw bytes from that file-absolute offset with a `206 Partial Content` response. Player retains the parsed WAV header and feeds the raw PCM continuation into the existing decode pipeline.
5. **Seek beyond buffer**: if seek target is past what's decoded, `StreamingAudioPlayerService` issues a new bounded segment starting at the seek byte offset. Server responds 206; player retains the parsed header and feeds the raw continuation into the existing decode pipeline.
**Memory bounding (three complementary layers, all required):**
- **Raw-queue bound (`StreamDecoder`):** `releaseConsumedChunks()` front-compacts `rawChunks` after each aligned segment is decoded, using a `discardedBytes` absolute cursor so all offsets remain absolute even as the array's front moves. Without this, a long WAV (e.g. a 92-min mix ≈ 970 MB raw) accumulates its entire decoded-from body in `rawChunks` regardless of the decode-side bounds below.
- **Decoded-queue bound (`PlaybackScheduler`):** `evictPlayedBuffers()` discards already-played `AudioBuffer`s, capping the scheduler's forward fill to a 96 MB ceiling (Phase 21.2). Decoded PCM is larger than source (Web Audio uses 32-bit float — a 16-bit stereo WAV roughly doubles; Opus decodes to the same float footprint).
- **Network bound (segmented fetch, Phase 21 Direction B):** The forward stream is fetched as sequential bounded `bytes=cursor-{cursor+4MB-1}` Range requests via `RunSegmentedStreamAsync`; the next segment is fetched only after `DrainBackpressureAsync` confirms the scheduler is below low-water. Because each segment is fully consumed before the next is issued, the browser holds at most ~one segment of raw bytes. A per-chunk drain inside the segment loop (gated on `_streamingPlaybackStarted` so it cannot deadlock first-audio) additionally prevents high-density codecs (e.g. Opus, where a 4 MB segment is ~100 s of audio) from decoding the whole segment eagerly ahead of the playhead before the inter-segment gate runs.
**Playback stability invariants (streaming-stabilization arc):**
- **Back-pressure water marks:** forward fill 60s high / 30s low (`PlaybackScheduler.DEFAULT_FORWARD_HIGH_WATER_SECONDS` / `...LOW_WATER_SECONDS`). Production pauses on `lookahead ≥ high OR decoded bytes > 96 MB ceiling`, whichever first. The time window is a jitter cushion for Opus's async decode ramp; the byte cap is the hard OOM guarantee and is unchanged.
- **Genuine end-of-playback:** `PlaybackScheduler` uses a `streamComplete` flag (set by `setStreamComplete`) combined with an `underrun_` park/resume state to distinguish a drained-but-still-streaming queue (startup/underrun gap → park, resume on refill) from a truly finished track (stream complete AND queue drained → `finishPlayback()`, the single genuine-end path). Prevents the false `onPlaybackEnded` that previously fired when the Opus WebCodecs queue momentarily drained during the decode ramp.
- **Rebuffer hysteresis:** Opus playback start and underrun-resume are gated on `hasMinimumPlaybackLead()` (1s decoded lead, `DEFAULT_MIN_PLAYBACK_LEAD_SECONDS`); WAV keeps `hasMinimumBuffers(6)`. `streamComplete` overrides the gate so a short tail still plays out. `StreamingAudioPlayerService.TryStartPlaybackAsync()` force-starts after `MarkStreamCompleteAsync` when the threshold was never crossed (ultra-short track protection).
- **Opus `AudioContext` pre-aligned to 48 kHz** in `AudioPlayer.initializeStreaming` before any bytes flow, eliminating the mid-decode context teardown that previously OOM'd the tab under software rendering.
**Visualizer / decode contention (decodePressure + hwAccel):**
- `DeepDrftPublic/Interop/audio/decodePressure.ts` exports a shared `DecodePressureSignal` singleton. The audio pipeline (`OpusStreamDecoder` yield-cap events, `PlaybackScheduler` underrun-parking events) calls `report()` on sustained lag; the visualizer calls `isUnderPressure()` each rAF frame. The signal engages only after ≥ 5 reports within 2500 ms with a 1s minimum hold (hysteresis); a lone startup-ramp blip never engages.
- Under pressure, `WaveformVisualizer.ts` throttles its rAF loop to ~15 fps (`PRESSURE_THROTTLE_FRAME_MS = 1000/15`), cutting main-thread WebGL software-render + physics cost so WebCodecs decode recovers. A no-op under HW accel.
- `DeepDrftPublic/Interop/visualizer/hwAccel.ts` probes `WEBGL_debug_renderer_info.UNMASKED_RENDERER_WEBGL` for software-renderer signatures (SwiftShader, llvmpipe, softpipe, Microsoft Basic Render, etc.). Absence of the debug extension is treated as accelerated — lava is disabled only on positive evidence. On first interactive render, `WaveformVisualizerControlState.ApplyCapabilityDefault(bool)` applies a one-time scoped default: `LavaEnabled = false` (the expensive subsystem) when no HW accel is detected, `WaveformEnabled` stays on. Guarded by `_capabilityDefaultApplied`; never overrides an explicit user toggle.
- **Known limitation / deferred escalation:** HW-accel-off Opus playback is made usable by defaulting lava off and throttling under pressure. If starvation recurs (e.g. waveform-only path also proves too costly, or a software renderer slips the probe), the documented next step is moving Opus WebCodecs decode off the main thread (Web Worker / AudioWorklet) so it stops competing with main-thread rendering. Not a current implementation — a recorded fallback.
Keep this seam clean — it is the most architecturally load-bearing part of the playback path.
### Theming and dark mode
- MudBlazor is the UI framework. Light and dark palettes (bespoke "Charleston in the Day" / "Lowcountry Summer Nights") defined inline in `MainLayout.razor`.
- MudBlazor is the UI framework. Light and dark palettes (bespoke "Charleston in the Day" / "Lowcountry Summer Nights") defined in `DeepDrftShared.Client/Common/DeepDrftPalettes.cs`. `MainLayout.razor` mounts `<MudThemeProvider Theme="@DeepDrftPalettes.Default" IsDarkMode="_isDarkMode" />` — the palettes are not inline in the layout.
- Dark mode toggles via cookie (`darkMode`, 365 days). Client-side via JS interop.
- During server prerender, `DarkModeService` (in `DeepDrftPublic`) reads the cookie and seeds `DarkModeSettings.IsDarkMode`, which carries into WASM render via `PersistentComponentState`. Avoids "wrong theme flash" on initial paint.
- `DarkModeSettings` lives in `DeepDrftPublic.Client.Common` (consumed by both server prerender and client components).
- **Theme-aware token layer:** `DeepDrftShared.Client/wwwroot/styles/deepdrft-tokens.css` defines two kinds of CSS custom properties. *Source tokens* (`--deepdrft-navy`, `--deepdrft-white`, `--deepdrft-green-accent`, etc.) are brand constants — identical in `:root` and `.deepdrft-theme-dark`. *Theme-aware aliases* are defined in both blocks and flip when the theme wrapper class changes. Component and page CSS must bind the **alias**, not the source token, so neutral surfaces invert for free. Current alias families: `--deepdrft-page-surface`/`-text`/`-text-muted` (neutral page backgrounds and text), `--deepdrft-play-chip`/`-glyph`/`-chip-soft` (play-state icon chip and glyph), `--deepdrft-popover-surface` (default MudBlazor popover background — light: `color-mix(navy 4%, white)`, a near-page-background surface; dark: references source token `--deepdrft-popover-surface-dark`, a `color-mix(navy-mid 80%, green-accent 20%)` bluer navy defined once in `:root` and referenced by both the `.deepdrft-theme-dark` wrapper block and `body.deepdrft-theme-dark` so portaled popovers are reached). The bespoke glass panels (visualizer/queue/privacy) now bind their own theme-aware `--deepdrft-panel-surface`/`-text`/`-text-muted`/`-border`/`-row-hover` family: dark-glass charcoal (sourced from the `--deepdrft-panel-ground` constant) with light text in dark theme, and a light translucent glass with dark text in light theme. These tokens are re-declared in `body.deepdrft-theme-dark` because the panels are MudOverlay panels that portal to `<body>` (same portal scope as popovers); the `--deepdrft-panel-ground` source token is now consumed only via the dark `--deepdrft-panel-surface` value.
- **Portaled-popover body-class bridge:** MudBlazor popovers portal to `<body>`, outside the `.deepdrft-theme-dark` wrapper `<div>`, so the dark popover token never reached them. Fix: `MainLayout.razor` stamps `deepdrft-theme-dark` on `<body>` via the `setBodyThemeClass(isDark)` helper in `DeepDrftShared.Client/Interop/theme/theme.ts` (lazy-imported as `_content/DeepDrftShared.Client/js/theme/theme.js`). The call fires only on first render or when `_isDarkMode` actually changes (gated by `_lastAppliedDarkMode` comparison) to avoid redundant JS calls on unrelated re-renders. The `body.deepdrft-theme-dark` selector in `deepdrft-tokens.css` resolves `--deepdrft-popover-surface` from `--deepdrft-popover-surface-dark` for these portaled elements.
- **Interactive-accent icon treatment (`.dd-accent-icon` / `.dd-accent-fill`):** one reusable rule in `DeepDrftPublic/wwwroot/styles/deepdrft-styles.css` for green-accent interactive icon affordances (Play / Share / Add-to-Queue / lava-lamp trigger), replacing the former pile of per-site dark overrides. Wrap the affordance container in `.dd-accent-icon` to colour its glyphs green-accent in both themes; add `.dd-accent-fill` when the container also holds a `Color.Secondary` filled button that must go green-accent in dark. It is a CSS class (not a palette `Color`) because no MudBlazor `Color` enum is green in both themes, and it targets `.dd-accent-icon .mud-icon-button .mud-icon-root` (0,3,0) `!important` to beat MudBlazor's standalone `.mud-secondary-text` (0,1,0) `!important` on the glyph svg — specificity wins; source order is not load-bearing for the glyph clause. The Session/Mix release-detail hero Share/Play glyphs use this class too (already green-accent in light via `Color.Secondary`, so folding them in keeps light pixel-identical and fixes dark). The gas-lamp toggle (`GasLampLit`) is self-colored in its SVG (`fill="#2A5C4F"` on the frame) — no dark-only CSS rule is needed; `GasLamp` (unlit, light mode) continues to use `currentColor` and inherits nav text colour. New green-accent icons use this class, not a new override. (Convention detail in `DeepDrftPublic.Client/CLAUDE.md`.)
- Typography: Google Fonts (Bodoni Moda, Cormorant, DM Sans). Hand-rolled gas-lamp icon (lit/unlit) lives in `DeepDrftShared.Client/Common/DDIcons.cs`.
### TypeScript interop, not raw JS
Audio interop authored in TypeScript under `DeepDrftPublic/Interop/audio/`, compiled to `wwwroot/js/audio/` via `Microsoft.TypeScript.MSBuild`. One module per responsibility (AudioContextManager, StreamDecoder, PlaybackScheduler, SpectrumAnalyzer, AudioPlayer), plus `index.ts` exposing `window.DeepDrftAudio`. `tsconfig.json` is **not** copied to output. In dev, raw `.ts` served from `/Interop/` for source-map debugging. A second interop module lives at `DeepDrftPublic/Interop/about/about-rail.ts` (IntersectionObserver for the About page active-movement rail highlight; compiled output gitignored).
**`DeepDrftShared.Client` also hosts TypeScript interop.** Its `tsconfig.json` maps `rootDir: "Interop"``outDir: "wwwroot/js"`, compiled by the same `Microsoft.TypeScript.MSBuild` package. Current modules: `Interop/parallax/parallax.ts` (parallax scroll for `ParallaxImage`) and `Interop/knob/knob.ts` (`capturePointer`/`releasePointer` for `RadialKnob`). Consumers lazy-import via the static-asset path `_content/DeepDrftShared.Client/js/<module>/<file>.js`.
**`DeepDrftShared.Client` also hosts TypeScript interop.** Its `tsconfig.json` maps `rootDir: "Interop"``outDir: "wwwroot/js"`, compiled by the same `Microsoft.TypeScript.MSBuild` package. Current modules: `Interop/parallax/parallax.ts` (parallax scroll for `ParallaxImage`), `Interop/knob/knob.ts` (`capturePointer`/`releasePointer` for `RadialKnob`), and `Interop/theme/theme.ts` (`setBodyThemeClass(isDark)` — stamps/removes `deepdrft-theme-dark` on `<body>` so portaled MudBlazor elements inherit the dark popover token; consumed by `MainLayout.razor`). Consumers lazy-import via the static-asset path `_content/DeepDrftShared.Client/js/<module>/<file>.js`.
## Development Commands
@@ -114,10 +136,10 @@ dotnet run --project DeepDrftAPI
### Entity Framework (SQL Database)
```bash
# Add migration (from solution root)
dotnet ef migrations add MigrationName --project DeepDrftData --startup-project DeepDrftPublic
dotnet ef migrations add MigrationName --project DeepDrftData --startup-project DeepDrftAPI
# Update database
dotnet ef database update --project DeepDrftData --startup-project DeepDrftPublic
dotnet ef database update --project DeepDrftData --startup-project DeepDrftAPI
```
## Key Configuration Files
@@ -125,8 +147,8 @@ dotnet ef database update --project DeepDrftData --startup-project DeepDrftPubli
All projects load secrets via `CredentialTools.ResolvePathOrThrow()` from gitignored `environment/` files:
- `DeepDrftPublic/appsettings.json`: Logging and URL config. Secrets loaded from `environment/api.json` (DeepDrftAPI base URL via `Api:ContentApiUrl`).
- `DeepDrftManager/appsettings.json`: Logging and URL config. Secrets loaded from `environment/api.json` (DeepDrftAPI base URL via `Api:ContentApiUrl` and API key via `Api:ContentApiKey`). Non-secret upload tunables (in `appsettings.json` itself, not `environment/`): `Upload:IdleTimeoutSeconds` (default 90 — aborts a stalled body-streaming phase) and `Upload:ResponseTimeoutSeconds` (default 600 — budget for server-side persist after the body is fully sent).
- `DeepDrftAPI/appsettings.json`: Logging and hosting config. Secrets loaded from `environment/filedatabase.json` (FileDatabase vault path), `environment/apikey.json` (API key), `environment/connections.json` (SQL and Auth connection strings), `environment/authblocks.json` (AuthBlocks JWT/email/admin creds).
- `DeepDrftManager/appsettings.json`: Logging and URL config. Secrets loaded from `environment/api.json` (DeepDrftAPI base URL via `Api:ContentApiUrl` and API key via `Api:ContentApiKey`). Non-secret upload tunables (in `appsettings.json` itself, not `environment/`): `Upload:IdleTimeoutSeconds` (default 90 — aborts a stalled body-streaming phase) and `Upload:ResponseTimeoutSeconds` (default 1200 — budget for server-side persist after the body is fully sent).
- `DeepDrftAPI/appsettings.json`: Logging and hosting config. Non-secret upload tunable: `Upload:StagingPath` (default empty → a `staging` subdirectory under the FileDatabase vault path) — the data-disk directory where large audio bodies are staged during upload/replace-audio, kept off the system temp mount (`/tmp` is a small tmpfs on the Linux host); `Startup` also points the framework's multipart buffer here via `ASPNETCORE_TEMP`. Secrets loaded from `environment/filedatabase.json` (FileDatabase vault path), `environment/apikey.json` (API key), `environment/connections.json` (SQL and Auth connection strings), `environment/authblocks.json` (AuthBlocks JWT/email/admin creds).
## Folder-Level Guidance
+2 -2
View File
@@ -40,7 +40,7 @@ The CMS is now inlined as the primary content of `DeepDrftManager`, a dedicated
- A `[HierarchicalRoleAuthorize("Admin")]` attribute (from `AuthBlocksWeb.HierarchicalAuthorize`) on every CMS page component, so `Admin` and any descendant role are admitted by the bundled hierarchical role handler.
- Controllers and minimal-API endpoints for CMS operations (`POST api/cms/track`, `DELETE api/cms/track/{id}`, `PUT api/cms/track/{id}`). Controllers are host-owned per the existing convention. Protected by `[Authorize(Roles = "Admin")]` — the JWT bearer middleware AuthBlocks installs validates the access token on each request.
- The `AddAuthBlocks(...)` call in `Program.cs` and the matching `await app.Services.UseAuthBlocksStartupAsync()` post-build hook. This installs JWT bearer middleware, the hierarchical role authorization handler, the `AuthDbContext`, the EF migrations, and seeds system roles plus the configured admin user on first boot.
- The `app.MapAuthBlocks()` call that registers `/api/auth/*`, `/api/users/*`, `/api/roles/*`, `/api/user-roles/*`, and `/api/pending-registrations/*` minimal-API endpoints. The CMS UI uses `/api/auth/login`, `/api/auth/logout`, `/api/auth/refresh`, and `/api/auth/me`; the rest are available if Wave 3 account-management ever lands.
- The `app.MapAuthBlocks()` call that registers `/api/auth/*`, `/api/users/*`, `/api/roles/*`, `/api/user-roles/*`, and `/api/pendingregistration/*` minimal-API endpoints. The CMS UI uses `/api/auth/login`, `/api/auth/logout`, `/api/auth/refresh`, and `/api/auth/me`; the rest are available if Wave 3 account-management ever lands.
**Render mode:** `InteractiveServer` for all CMS pages and routes. AuthBlocks's bundled UI (`AuthBlocksWeb` pages) is server-rendered MudBlazor with `JwtAuthenticationStateProvider` reading tokens from browser `localStorage` via JS interop. `InteractiveServer` is the right fit because: (a) it matches what the bundled login UI uses, (b) `InputFile` uploads are natively server-side, (c) CMS endpoints live in the `DeepDrftManager` process with direct access to services.
@@ -79,7 +79,7 @@ Concretely, from reading the library source:
- Real per-user accounts (`ApplicationUser` table). No shared password.
- One seeded admin on first boot via `AdminUserSettings`. Username, email, password come from `DeepDrftManager/environment/authblocks.json` (gitignored, same pattern as `apikey.json`).
- No public signup in Wave 1. The `/account/register` page that AuthBlocks bundles requires a registration code (generated by an admin via `/api/pending-registrations`). We do not surface `/account/register` in any nav until Wave 3 account management lands; the route exists but is uninteresting until then.
- No public signup in Wave 1. The `/account/register` page that AuthBlocks bundles requires a registration code (generated by an admin via `/api/pendingregistration`). We do not surface `/account/register` in any nav until Wave 3 account management lands; the route exists but is uninteresting until then.
- **Mutation attribution.** `TrackEntity` gains a nullable `CreatedByUserId : long?` column in the W1.2 migration. Populated on every CMS-originated mutation; null for historical CLI-added rows and for any pre-CMS data. Captures attribution from day one even though Wave 1 has exactly one user (`feedback_design_for_adaptability`).
- **Role gate.** Every CMS page and every `api/cms/*` endpoint requires the `Admin` system role. We use `Admin` rather than introducing a new `CmsAdmin` role because the collective is small and the existing hierarchy already covers the case; if Wave 3 ever needs finer grain (e.g. a `ContentEditor` role that can edit but not delete), that is a `SystemRole.cs` edit upstream, not a redesign here.
+363
View File
@@ -6,6 +6,369 @@ Newest entries at the top. Group by phase/wave header (mirroring `PLAN.md` / `CM
---
## Phase 23 — SEO Crawl Directives (landed 2026-06-23)
**Landed:** 2026-06-23 on dev.
- **What:** Server-side crawl-directive endpoints for `DeepDrftPublic` (`GET /robots.txt` and `GET /sitemap.xml`) plus a defense-in-depth noindex layer for `DeepDrftManager`. The endpoint/file-shaped follow-on to Phase 22's per-page `SeoHead` component. Phase 22 is the *content* of discoverability; Phase 23 is the *directives* layer above it — telling crawlers **which** pages exist and **whether** to crawl at all. No new `DeepDrftAPI` endpoint, no schema change.
- **Why:** Without robots.txt a crawler has no machine-readable signal about which routes to include or exclude (e.g. `/FramePlayer`, `/api/*`). Without sitemap.xml Google/Bing must discover release detail pages by link-following alone. Without noindex/robots protection the CMS could be inadvertently crawled if an admin link ever appeared on a public page.
- **Shape:**
- **`DeepDrftPublic/Controllers/CrawlDirectiveController.cs`** (new): thin controller serving both endpoints. Reads `IWebHostEnvironment.IsProduction()` **directly** — no `SeoEnvironment` PersistentState bridge needed because these are server-side only (nothing crosses the server→WASM seam). Env gate is fail-safe closed: non-production robots.txt emits `Disallow: /` and the sitemap returns 404.
- **`DeepDrftPublic/Seo/RobotsTxt.cs`** (new): pure builder for the robots.txt body. Production: `Allow: /` + `Disallow: /FramePlayer` + `Disallow: /api/` + `Sitemap:` pointer. Non-production: `Disallow: /`.
- **`DeepDrftPublic/Seo/SitemapXml.cs`** (new): pure builder for the sitemap XML body. Walks `GET api/release` (server-to-server via the existing `"DeepDrft.API"` named client, paged) and emits a sitemaps.org `urlset`. Six explicit static roots (`/`, `/about`, `/cuts`, `/sessions`, `/mixes`, `/archive`) plus one `<url>` per release — `<loc>` = `SeoOptions.BaseUrl` + `ReleaseRoutes.DetailHref`, equal to the page's `SeoHead` canonical by construction; `<lastmod>` from `ReleaseDate`. Resilient: a partial/failed release read yields a well-formed roots-only document, never a 500.
- **`DeepDrftManager/wwwroot/robots.txt`** (new static file): `Disallow: /` with no environment gate — the CMS is always uncrawlable, including in production.
- **`DeepDrftManager/Components/App.razor`** (updated): blanket `<meta name="robots" content="noindex,nofollow">` in the CMS host `<head>` — defense in depth against de-indexing URLs discovered via external links, complementing the robots.txt directive.
- **Design memo:** `product-notes/phase-23-seo-crawl-directives.md`.
---
## Phase 22 — SEO Metadata Component (landed 2026-06-23)
**Landed:** 2026-06-23 on dev.
- **What:** A parameterized, reusable SEO head component (`SeoHead.razor`) that emits the full modern-SEO head surface — standard meta, canonical, robots, Open Graph, Twitter Card, and schema.org JSON-LD — for every public page in one line of markup. **Public listener site only** (`DeepDrftPublic` host + `DeepDrftPublic.Client`); the CMS is explicitly out of scope. No data-model/schema change, no new API endpoint.
- **Why:** `App.razor` had a static `<head>` with no description, canonical, OG, Twitter Card, or JSON-LD anywhere; pages set only an ad-hoc `<PageTitle>` with an inconsistent suffix. A shared `/mixes/{key}` link unfurled as a bare title + URL. Crawlers and social unfurlers saw nothing useful.
- **Shape:**
- **`Controls/SeoHead.razor`** (new): purely presentational `<HeadContent>` + `<PageTitle>` emitter. Accepts a single `SeoModel` parameter; owns no data fetch. Each page wires it in one line.
- **`Common/SeoModel.cs`** (new): typed per-page input with named factories — `ForRelease(release, baseUrl, options)` (medium-dispatched), `ForHome`, `ForAbout`, `ForBrowse`, `ForNotFound`. Factories encode the medium→schema mapping in one place. Explicit `SeoModel.Robots` override available; default is environment-gated (see `SeoEnvironment`).
- **`Common/SeoJsonLd.cs`** (new): typed schema.org JSON-LD builders. Cut → `MusicAlbum` with ordered `MusicRecording` track list; Session → `MusicAlbum`/`LiveAlbum`; Mix → single `MusicRecording` with ISO-8601 duration; Home/About → `MusicGroup` (with `sameAs: ["https://instagram.com/deepdrft.music"]`); Browse → `CollectionPage`. `byArtist` wired per-release. JSON-LD body is inline-safe-escaped (`<`/`>`/`&``\uXXXX`) to prevent script-breakout from CMS-authored text.
- **`Common/SeoOptions.cs`** (new): site-wide config — `BaseUrl` (`https://deepdrft.com`), title suffix (`Deep DRFT`, middot separator), default OG image seam (uses `ImageProxyController` route), IG handle in `sameAs`, no Twitter handle. Registered via the static `Startup` seam (runs in both server and WASM `Program.cs`).
- **`Common/SeoUrls.cs`** (new): URL helpers for canonical and `og:image` construction from `SeoOptions.BaseUrl` (config, not `window.location` — no `window` at server prerender and the origin can't be derived behind the nginx proxy).
- **`Common/SeoEnvironment.cs`** (new): scoped `[PersistentState]` bridge seeded in `DeepDrftPublic/Components/App.razor` from `IWebHostEnvironment.IsProduction()` — mirrors the `DarkModeSettings` bridge. Default robots is `index,follow` only in Production; `noindex,nofollow` in every non-production environment so the beta/staging site stays uncrawled. Explicit per-page `SeoModel.Robots` overrides this default. Fail-safe default is `noindex`.
- **Wired into:** Home, About, Cut/Session/Mix detail pages (incl. their not-found branches → `noindex`), the browse views (Albums/Sessions/Mixes/Archive), and the 404 NotFound page.
- **Render-mode correctness:** `SeoHead` rides the existing `PersistentComponentState` bridge (the same `ReleaseDto` the detail pages already bridge) — no new fetch. The `InteractiveAuto` double-render produces identical head content across prerender and WASM passes (fed from bridged state, guarded on id/key equality).
- **Design memo:** `product-notes/phase-22-seo-metadata-component.md`.
---
## Phase 21 — Windowed Streaming Buffer (landed 2026-06-24)
**Landed:** 2026-06-24 on `streaming-overhaul`.
- **What:** Bounded client memory for long audio streams. Playing a 1 GB+ DJ Mix (the Phase 9 `Mix` medium — a single long track) no longer accumulates the full decoded PCM in the browser; instead the player holds only a sliding forward window and discards what has already played. Four waves, all landed. Public listener site only (`DeepDrftPublic.Client` player stack + `DeepDrftPublic` TypeScript audio interop); no CMS, no API endpoint, no schema change.
- **Why:** The `PlaybackScheduler` held an `AudioBuffer[]` it never evicted — both decode paths (`StreamDecoder`/`IFormatDecoder` for WAV/MP3/FLAC, and `OggDemuxer``OpusStreamDecoder` for Opus) pushed into it without limit. Decoded PCM is larger than the source (Web Audio is 32-bit float per sample/channel — a 16-bit stereo WAV roughly doubles once decoded; a low-data Opus mix decodes to the same float footprint regardless of how few compressed bytes arrived), so a 1 GB WAV or Opus mix could accumulate ~2 GB of retained float in the browser. The Opus path also had a second upstream accumulation locus: the WebCodecs `AudioDecoder` work queue and `decodedQueue: AudioData[]`.
- **Shape (by wave):**
- **21.1 — Partial eviction in `PlaybackScheduler` (cold-start; load-bearing; shared by both paths).** Drop already-played buffers while keeping position/index/time-anchor bookkeeping exact against a buffer array that no longer begins at absolute time 0. Written once, serves both decode paths (they `addBuffer` identically). This is the hardest correctness work in the phase. No refill yet.
- **21.2 — Back-pressure (one fill signal, two throttle sites).** Bound the unplayed region by stopping production above a high-water mark and resuming below low-water, driven by `PlaybackScheduler.evaluateProductionPause()` — a single shared scheduler-fill signal (OQ6/OQ7 resolved: shared controller, per-path hook). **21.2a**`StreamingAudioPlayerService` stops fetching the next segment above high-water, polling `evaluateProductionPause()` at 100 ms cadence until the fill drains below low-water (serves both paths). **21.2b** — the Opus demux/decode feed is additionally stopped when the same signal is set, so the WebCodecs decode queue and `decodedQueue` do not balloon behind a paused segment loop (Opus only; no WAV analogue). Together with 21.1 this bounds both the played and unplayed sides on both formats.
- **21.3 — Seek-back-past-window refill + clean failure recovery.** When a backward seek lands earlier than the retained tail, the existing seek-beyond-buffer path is reused pointed at the earlier offset (whichever resolver the active path ships: `IFormatDecoder`/`StreamDecoder` for WAV; the live `resolveOpusByteOffset` + `OpusStreamDecoder.reinitializeForRangeContinuation` for Opus). Minimal AC6 refill-failure handling added: `RecoverFromFailedRefill` clears the scheduler and surfaces a user-visible error rather than leaving the player wedged.
- **21.4 — Validation + Direction A→B pivot (the critical wave — see as-built divergence below).** Network-memory bounding confirmed in Daniel's browser run: the segmented approach delivers ~4 MB segments pacing with playback, with the browser holding ~one segment of raw bytes rather than the full artifact.
- **As-built divergence — Direction A→B pivot (important).** The spec recommended **Direction A** (sliding window on one open-ended forward stream, relying on pausing `ReadAsync` / the segment loop to backpressure the socket) with **Direction B** (discrete bounded `Range: bytes=start-end` segments, each fetched only when the scheduler drains below low-water) held as the documented fallback "if Direction A's back-pressure proves leaky in practice."
**21.4 browser validation proved it leaky.** On Blazor WASM, the browser `fetch` API buffers the entire HTTP response body regardless of how slowly the application reads it: pausing reads bounded the *decode* but not the *network download*, so the entire ~970 MB body accumulated in browser memory even though the application only decoded a window of it. Direction A's core assumption — that pausing `ReadAsync` closes the TCP flow-control window before the browser caches the whole body — does not hold in the WASM `fetch` runtime.
**We pivoted to Direction B.** The forward stream now issues sequential bounded `bytes=cursor-(cursor+SegmentSizeBytes-1)` Range requests (`SegmentSizeBytes = 4 MB`), each fetched only after the scheduler drains below low-water — via `RunSegmentedStreamAsync` in `StreamingAudioPlayerService`. Because each 4 MB request is fully consumed before the next is issued, the browser holds at most ~one segment of raw bytes regardless of file size. The decode-side windowing (21.1/21.2) pairs with it: the segment loop's segment-gate replaces the raw `ReadAsync` pause as the production throttle (21.2a), and the Opus decode-ahead throttle (21.2b) hooks the same `evaluateProductionPause()` signal. Seek and refill converge on the same segmented loop. Direction B is recorded as the shipped approach; Direction A is recorded as tried-in-validation and found insufficient for the WASM `fetch` runtime.
- **What Phase 21 does NOT include:** the full AC matrix re-run (Opus seek-storm, visualizer, rapid-seek concurrency under the new segmented loop) beyond the network-memory bounding confirmed in 21.4. Those acceptance criteria remain the validation baseline for any follow-on work touching this seam.
- **Design memo:** `product-notes/phase-21-windowed-streaming-buffer.md` (note: the spec recommended Direction A; the as-built pivot to Direction B is annotated at the top of that doc and in §3.2/§3.3).
### Phase 21 — Post-landing: raw-queue and per-chunk bounds (landed 2026-06-24)
**Landed:** 2026-06-24 on `streaming-overhaul`. Fixes a Chrome tab crash (main-process OOM with hardware acceleration OFF) on long-track playback. Two unbounded memory queues remained after Phase 21 landed; this patch adds complementary bounds on both.
1. **`StreamingAudioPlayerService` — per-chunk drain inside segments.** Phase 21 Direction B's inter-segment back-pressure gate is matched to WAV byte density (~24 s per 4 MB segment), but not to Opus: at 320 kbps a 4 MB segment is ~100 s of decodable audio. The inner chunk loop had no fill check, so the entire segment decoded eagerly ahead of the playhead — piling decoded f32 PCM into main-process RAM before the inter-segment gate ever ran. Fix: an extracted `DrainBackpressureAsync` helper (shared with the inter-segment gate) is called per chunk once the scheduler is over high-water, gated on `_streamingPlaybackStarted` so it cannot deadlock first-audio (the playhead doesn't advance until playback starts, so draining to low-water would block indefinitely before the first buffer plays). Also added load-generation diagnostic logging (`_loadGeneration`) to confirm single-load-per-play-action in-browser.
2. **`StreamDecoder.ts``releaseConsumedChunks()` front-compaction of `rawChunks`.** `rawChunks` retained the entire decoded-from stream body because consumed chunks were never released — a 92-min WAV mix (≈ 970 MB raw) accumulated all of it in the browser. Phase 21.2's 96 MB decoded-side ceiling does not reach this raw queue. Fix: `releaseConsumedChunks()` walks `rawChunks` after each aligned segment is decoded and splices off all fully-consumed chunks (any chunk whose end ≤ `processedBytes`), advancing `discardedBytes` (an absolute cursor into the logical stream) so `extractAlignedData` can continue walking from the correct position even though `rawChunks[0]` no longer begins at byte 0. This is the raw-side analogue of `PlaybackScheduler.evictPlayedBuffers` (the decoded side).
Together with the existing decoded-side 96 MB ceiling (Phase 21.2) and the segmented fetch (Phase 21 Direction B), these complete the three-layer memory bound: raw queue, decoded queue, and network.
### Phase 21 — Post-landing: streaming stabilization (landed 2026-06-26)
**Landed:** 2026-06-26 on `streaming-overhaul`. Resolves playback instability that surfaced specifically with Opus + hardware acceleration off: the Opus underrun/false-end cycle, visualizer starvation of WebCodecs decode, and an OOM path from mid-decode `AudioContext` teardown. All changes are TypeScript-only (`DeepDrftPublic/Interop/audio/` + `DeepDrftPublic/Interop/visualizer/`) plus the C# service that calls them; no API, no schema, no CMS change.
1. **Back-pressure water marks widened to 60s high / 30s low.** `PlaybackScheduler.ts` `DEFAULT_FORWARD_HIGH_WATER_SECONDS` / `DEFAULT_FORWARD_LOW_WATER_SECONDS` doubled from 30/15. The 96 MB decoded-byte ceiling is unchanged and remains the hard OOM bound; production pauses on `lookahead ≥ high OR bytes > cap`, whichever first. The wider time window absorbs per-packet WebCodecs decode jitter (Opus 48 kHz stereo ≈ 0.37 MB/s → 60 s ≈ 23 MB — well within the cap) without false back-pressure on the Opus decode ramp.
2. **End-of-playback gated on genuine completion (`streamComplete` + `underrun_`).** `PlaybackScheduler` now distinguishes a drained-but-still-streaming queue (startup/underrun gap) from a truly finished track (stream complete AND queue drained). `streamComplete` (set by `setStreamComplete`, called from `AudioPlayer.markStreamComplete`) is the discriminator. When the queue drains while `streamComplete` is false the scheduler parks in `underrun_` state (anchors frozen, `isActive_` false) and resumes from `scheduleNewBuffers` when buffers arrive. `finishPlayback()` is the single genuine-end path, reached only when both conditions hold simultaneously. Previously a drained Opus queue during the WebCodecs startup ramp fired `onPlaybackEnded`, nulling Duration and advancing the queue.
3. **Rebuffer hysteresis — 1s decoded-lead gate.** `hasMinimumPlaybackLead()` gates Opus playback START and underrun RESUME (`DEFAULT_MIN_PLAYBACK_LEAD_SECONDS = 1.0`), preventing the decode-ramp thrash (resume on a 20 ms Opus packet → drain immediately → re-park → repeat). `streamComplete` overrides the gate so a short tail plays out rather than parking forever. WAV keeps its `hasMinimumBuffers(6)` buffer-count start gate unchanged — large synchronous WAV segments rarely underrun and that gate's character must not change. **Complete-without-start force-start:** `StreamingAudioPlayerService.TryStartPlaybackAsync()` is called after `MarkStreamCompleteAsync` when `_streamingPlaybackStarted` is still false, preventing an ultra-short Opus track (total audio below the 1s lead) from sitting loaded-but-not-playing.
4. **Opus `AudioContext` pre-aligned to 48 kHz.** `AudioPlayer.initializeStreaming` calls `contextManager.recreateWithSampleRate(OPUS_SAMPLE_RATE)` when the content type resolves as Opus, before any bytes flow. The context is already at 48 kHz when the decoder's own lazy `recreateWithSampleRate` check runs, so it short-circuits (no-op). Eliminates the mid-decode context teardown-and-rebuild that previously double-decoded the stream and OOM'd the tab under HW accel off.
5. **`decodePressure` singleton + `WaveformVisualizer` auto-throttle (~15 fps under pressure).** `DeepDrftPublic/Interop/audio/decodePressure.ts` exports a process-wide `DecodePressureSignal` singleton shared between the audio pipeline (producer: `OpusStreamDecoder` yield-cap events and `PlaybackScheduler` underrun-parking events) and the visualizer (consumer). Engages only on sustained stress (≥ 5 `report()` calls within a 2500 ms window) and holds for a 1000 ms minimum dwell before releasing, suppressing lone startup-ramp blips. `WaveformVisualizer.ts` checks `decodePressure.isUnderPressure()` each rAF frame; when engaged it skips frames so at most one renders per `PRESSURE_THROTTLE_FRAME_MS` (`1000/15` ms, ~15 fps), cutting main-thread WebGL software-render + physics cost so WebCodecs decode recovers. A no-op under HW accel — healthy decode never calls `report()` and `isUnderPressure()` stays false indefinitely.
6. **Hardware-acceleration detection + `ApplyCapabilityDefault`.** `DeepDrftPublic/Interop/visualizer/hwAccel.ts` probes `WEBGL_debug_renderer_info.UNMASKED_RENDERER_WEBGL` for known software-renderer signatures (SwiftShader, llvmpipe, softpipe, Microsoft Basic Render Driver, Mesa Offscreen, "software"). Absence of the debug extension is treated as accelerated — the probe disables lava only on positive evidence of software rendering, not on uncertainty. `WaveformVisualizerControlState.ApplyCapabilityDefault(bool hardwareAccelerated)` is called once per session by the visualizer bridge on first interactive render: when `hardwareAccelerated` is false it sets `LavaEnabled = false` (the expensive main-thread subsystem) while leaving `WaveformEnabled` on, coerces Theater Mode, and raises `Changed` once so all observers see the default in a single cycle. Guarded by `_capabilityDefaultApplied` so it runs at most once per session and never overrides an explicit in-session user toggle. With HW accel present the call is a no-op.
---
## Phase 18 — Opus Low-Data Streaming (landed 2026-06-23)
_Note: Two distinct efforts share the "Phase 18" label — phase numbers are organisational, not sequential. See also **Phase 18 — Theme / Dark-Mode Remediation (landed 2026-06-19)** below._
**Landed:** 2026-06-23 on `streaming-overhaul`.
- **What:** Dual-format audio delivery — the existing lossless WAV path plus a new low-data Ogg Opus (fullband, 320 kbps) path — giving listeners a quality choice. Opus is the default (capability-gated). Six waves, all landed.
- **Why:** Lossless WAV works but imposes high bandwidth cost, particularly for long DJ Mixes. Opus at 320 kbps is indistinguishable for typical listening and an order of magnitude smaller, making the site usable on metered or constrained connections. The bespoke Web Audio decode→schedule graph is retained by deliberate choice; Opus feeds the same `IFormatDecoder` seam rather than an HTML `<media>` element or MSE.
- **Shape (by wave):**
- **18.1 — Ingest transcode + seek-index sidecar.** New `OpusTranscodeService`/processor in `DeepDrftContent`, invoked post-store from `UnifiedTrackService.UploadAsync` as a background job (failure-tolerant; the track plays lossless first). Produces the Ogg Opus 320 artifact and a combined seek/setup **sidecar** (0.5 s-bucketed granule→byte index + `OpusHead`/`OpusTags` setup header), both stored in a new `track-opus` vault. **ffmpeg/libopus** added to the deploy bootstrap as a runtime prereq on the API host.
- **18.2 — Storage + lookup contract.** Server-side "given `EntryKey` + format, return the right artifact + content-type, with lossless fallback when no Opus artifact exists."
- **18.3 — Delivery: `?format=opus|lossless` + sidecar serving + proxy threading.** `?format=` param on the stream endpoint (Range preserved); sidecar-serving endpoint; param forwarded through `TrackProxyController`; `TrackMediaClient` sends the format param.
- **18.4 — Client Opus decoder + index-based seek + capability detection.** New `OpusFormatDecoder` (`IFormatDecoder`): Ogg-page demux, setup-header carry from the cached sidecar, `calculateByteOffset` binary-searching the precomputed seek index (VBR-safe accurate seek — not interpolation). Capability gate falls back to the lossless path on browsers without Opus support. Testing-phase fixes also landed: a valid ffmpeg-generated capability-probe sample and correct duration reporting (fixed the seekbar + visualizer in Opus mode).
- **18.5 — End-to-end integration.** Player format selection (Opus-by-default, capability-gated); sidecar fetch/inject on track load; **Backfill-Opus** CMS bulk action (third sibling to Generate-Profiles / Backfill-High-res); replace-audio regenerates Opus + sidecar.
- **18.6 — Settings menu + quality toggle + CMS status.** New public-site **Settings menu** (app-bar trigger + settings-item abstraction + `ListenerSettings`, persisted via the dark-mode-pattern seam: `streamQuality` cookie → prerender-read → `PersistentComponentState` → client cookie service); streaming-quality toggle (Low-data/Lossless) as its first occupant. CMS **Opus-status badge** on the track grid. CMS upload meter **Post-Processing** phase visualizing background Opus transcode.
- **As-built divergence — the Opus decoder (important).** The originally-spec'd "segment-and-decode-each-page via `decodeAudioData`" model was implemented and then **replaced**: browser testing showed it glitches at every segment boundary and broke seeking (Opus has pre-skip + inter-frame state, so independent per-segment decode is architecturally wrong). The landed decoder is a **WebCodecs `AudioDecoder` streaming pipeline** (Ogg demux → packets → stateful continuous decode), with index-driven seek and a WebCodecs-based capability gate. Daniel's call at the decision point — offered a decode-whole stopgap (Option A) vs. the proper WebCodecs streaming decoder (Option B), he chose B: "Let's go for B, why do you like half-solutions so much" (the same no-stopgap posture as the Mix visualizer WebGL2 decision, where he said "no stopgap — WebGL as step 1, no pussyfooting"). The segmented approach is recorded as tried-and-replaced.
- **What Phase 18 does NOT include:** windowed / bounded-memory streaming. Phase 18 still fetches the whole (smaller) Opus artifact; bounding client memory to a sliding forward window is **Phase 21's job**.
- **Design memo:** `product-notes/phase-18-opus-low-data-streaming.md`.
---
## Phase 20 — Theater Mode (landed 2026-06-20)
**Landed:** 2026-06-20 on dev. Pending: final manual browser/GPU smoke-test on dev.
- **What:** A presentation-only Theater Mode toggle on the three public Release Detail views (`CutDetail.razor`, `MixDetail.razor`, `SessionDetail.razor`). Toggling ON hides the release page content via `@if` so the lava-lamp + waveform visualizer fills the surface unobstructed; the player bar grows to surface the playing release's cover art, release title (linked), and a release-mode `SharePopover`. Toggling OFF restores the page byte-for-byte. The top action row (back link, lava-lamp popover, Theater toggle) stays visible in both states. Behavior is identical across all three mediums. Persists across SPA navigation within a session; resets to OFF on fresh page load.
- **Why:** The visualizer is the site's most distinctive feature (Phases 10/12/15). Theater Mode makes it the *whole* thing on demand — a "lean back and watch the lamp" experience — and relocates the minimum release identity to the one piece of chrome that stays (the player bar), so nothing essential is lost.
- **Shape:**
- **`Controls/TheaterModeToggle.razor`** (new): shared toggle button placed immediately left of the lava-lamp `WaveformVisualizerControlPopover` on all three detail pages inside a `.dd-detail-top-actions` cluster. Material `Theaters` glyph; `.dd-accent-icon` for green-accent in both themes. Visible only when `LavaEnabled || WaveformEnabled`; disabled until interactive. Flips `WaveformVisualizerControlState.TheaterMode` and calls `NotifyChanged()`. Subscribes to `State.Changed` for its own active-state re-render; disposes cleanly.
- **`Controls/AudioPlayerBar/NowShowingPanel.razor`** (new): presentational "now showing" band rendered by `AudioPlayerBar` only when `TheaterMode && CurrentTrack?.Release is not null`. Shows cover art (`deepdrft-track-detail-cover-art` / `deepdrft-gradient-soft-secondary` placeholder), release title link (`ReleaseRoutes.DetailHref`), and release-mode `SharePopover` in `.dd-accent-icon`. Layout CSS in `AudioPlayerBar.razor.css` (`.now-showing-*`); surface/text bind `--deepdrft-page-*` aliases — no new dark overrides.
- **`Services/WaveformVisualizerControlState.cs`** (widened): gained `TheaterMode` bool + `DefaultTheaterMode = false` const, and `CoerceTheaterMode()` — enforces the invariant that Theater Mode cannot remain on when both subsystems are off. Called from `WaveformVisualizerControls.ToggleLava`/`ToggleWaveform` before `NotifyChanged()` so all observers see a consistent coerced state in the same `Changed` cycle.
- **`Controls/AudioPlayerBar/AudioPlayerBar.razor` + `.razor.cs` + `.razor.css`**: subscribes to `WaveformVisualizerControlState.Changed`; mounts `<NowShowingPanel>` above transport controls when Theater is on and a release is playing.
- **Three detail pages** (`CutDetail.razor`, `MixDetail.razor`, `SessionDetail.razor`): page-level `@if (!VisualizerControlState.TheaterMode)` gates content regions on each page individually (not in `ReleaseDetailScaffold`, so Session — which does not use the scaffold — is covered identically). Each page's top action cluster hosts `<TheaterModeToggle />` in a `.dd-detail-top-actions` flex wrapper.
- **`deepdrft-styles.css`**: new `.dd-detail-top-actions` layout-only class (`display:flex; align-items:center; gap:0.25rem`) — no colour; shared by all three pages.
- **`DeepDrftTests/WaveformVisualizerControlStateTests.cs`** (new): unit tests for the `CoerceTheaterMode()` auto-exit invariant.
- **Design memo:** `product-notes/phase-20-theater-mode.md`.
### Phase 20 — Wave 2 — Theater Mode refinements (landed 2026-06-21)
**Landed:** 2026-06-21 on dev.
- **What:** Three refinements to the base Phase 20 feature. (1) **Full-screen detail body:** each detail page's foreground container gained `.dd-detail-fill` (`min-height: calc(100vh - var(--deepdrft-nav-height, 88px))`), so the visualizer reads as full-screen and the footer is pushed below the fold regardless of Theater Mode. (2) **Eased collapse (no pop):** the hard `@if` content-hide on the three detail pages was replaced by a `.dd-theater-collapsible` / `.dd-theater-collapsible-inner` wrapper pair that receives `.dd-theater-collapsed` when `IsContentHidden` is true — animates `grid-template-rows: 1fr → 0fr`, `opacity`, and `visibility` (deferred via `transition-behavior: allow-discrete`) so Theater ON/OFF eases rather than pops; `prefers-reduced-motion` collapses instantly. The same wrapper pattern drives the player-bar `NowShowingPanel`, which is now kept mounted whenever a release is playing and collapsed (not `@if`-removed) when Theater is OFF — enabling the ease-in when Theater turns ON (resolves OQ2 design intent for a mounted-but-dormant panel). (3) **Playing-release scoping:** Theater Mode now only applies to the currently-playing release. `ReleaseDetailBase` and `CutDetailBase` each gained a cascaded `IStreamingPlayerService PlayerService`, a reference-guarded `StateChanged` subscription (disposed in `Dispose`), and three predicates: `IsThisReleasePlaying` (`CurrentTrack?.Release?.EntryKey == EntryKey`), `IsContentHidden` (`TheaterMode && IsThisReleasePlaying`), `ShowTheaterToggle` (`(LavaEnabled || WaveformEnabled) && IsThisReleasePlaying`). `TheaterModeToggle.razor` gained an `Available` parameter (default `true`) folded into its render gate; all three pages pass `Available="ShowTheaterToggle"`. A detail page whose release is not playing shows no toggle and ignores the global `TheaterMode` flag.
---
## Phase 18 — Theme / Dark-Mode Remediation (landed 2026-06-19)
_Note: Two distinct efforts share the "Phase 18" label — phase numbers are organisational, not sequential. See also **Phase 18 — Opus Low-Data Streaming (landed 2026-06-23)** above._
**Landed:** 2026-06-19 on dev (Wave 1 + Wave 2 + Wave 3).
- **What:** A DRY token pass resolving six theming symptoms (five in dark mode, one in light) that all traced to three root causes: neutral page surfaces bound to constant brand tokens, the play chip bound to a constant light-grey, and no theme-aware popover-surface token. Resolved as one coherent pass via a shared token layer rather than per-component patches.
- **Why:** Symptom consolidation and root-cause analysis showed all six symptoms shared the same underlying structure — component CSS bypassing the theme-aware alias layer and binding constant source tokens directly. A single additive token pass in `deepdrft-tokens.css` plus targeted re-pointing of consumers fixes all six without scattering dark-mode rules.
- **Shape:**
- **Token foundation (`deepdrft-tokens.css`):** Three new theme-aware token families added to `DeepDrftShared.Client/wwwroot/styles/deepdrft-tokens.css`, each defined in both `:root` (light) and `.deepdrft-theme-dark` (dark):
- `--deepdrft-page-surface` / `--deepdrft-page-text` / `--deepdrft-page-text-muted` — neutral page surface family. Light: `--deepdrft-white` / `--deepdrft-navy` / `--deepdrft-muted`. Dark: `var(--mud-palette-background)` (#0D1B2A, the true page ground) / `--deepdrft-white` / `color-mix(muted 70%, white)` — neutral sections dissolve into the site background as one continuous dark field rather than reading as raised panels.
- `--deepdrft-play-chip` / `--deepdrft-play-glyph` / `--deepdrft-play-chip-soft` — play-chip family. Light: soft-grey chip (matching prior `--deepdrft-soft`). Dark: `--deepdrft-green-accent` chip + `--deepdrft-navy` glyph (navy-on-green for solid chips); `--deepdrft-play-chip-soft` is `color-mix(green-accent 30%, transparent)` (the player-bar translucent override).
- `--deepdrft-popover-surface` — popover surface. Light: `color-mix(navy 8%, white)` soft desaturated-navy wash. Dark: `#162437` (pixel-identical to `DeepDrftPalettes.Dark.Surface` — dark popovers unchanged, only light is retoned).
- **Neutral-surface inversion (T2):** `Home.razor.css`, `About.razor.css`, `DeepDrftFooter.razor.css` re-pointed from constant `--deepdrft-white`/`--deepdrft-navy` to `--deepdrft-page-surface`/`--deepdrft-page-text`. Decorative navy/green sections (`.section-dark`, `.split-left`, `.cta-banner`, hero overlays) untouched — classification encoded in which token each section binds.
- **Play-chip theming (T3):** `PlayStateIcon.razor.css` `.icon-container` re-pointed to `--deepdrft-play-chip`; glyph to `--deepdrft-play-glyph`. Player-bar context overrides chip to `--deepdrft-play-chip-soft` (translucent green wash). Light-mode parity and connect-option hover also corrected.
- **Popover surface (T4):** `deepdrft-styles.css` binds `--deepdrft-popover-surface` to the MudBlazor default popover surface. Bespoke dark-glass panels (`--deepdrft-panel-ground`) untouched.
- **Wave 2 refinements (on top of T1T4):** App bar background moved to navy (`#112338`) from near-black (`#0D1B2A`). Neutral page surfaces re-pointed to `var(--mud-palette-background)` (`#0D1B2A`) as the true dark ground — sections dissolve into the body background rather than reading as navy-mid raised panels (resolves Wave 1's open question in favour of ground). Dark-mode hero legibility (superseded in Wave 3 — see below). Play-glyph settled on navy-on-green (solid chips) and green-on-green (player bar, via `--deepdrft-play-chip-soft`).
- **Wave 3 — hero dark-mode legibility fix:** `DeepDrftHero.razor.css` hero text re-worked to bind theme-aware tokens directly in the base rules rather than via `:global(.deepdrft-theme-dark)` overrides (matching the About page's proven pattern). `.hero-title` and `.hero-desc` now bind `--deepdrft-page-text` directly; `.hero-subtitle` (previously bound to the constant `--deepdrft-muted`) now binds `--deepdrft-page-text-muted`, making it theme-aware for the first time. Only `.hero-title em` retains an explicit dark override (`:global(.deepdrft-theme-dark) .hero-title em``--deepdrft-green-accent`, lifting the low-contrast `--deepdrft-green` on the dark ground). Global hero-button dark treatment added to `deepdrft-styles.css`: `.deepdrft-theme-dark .btn-primary``--deepdrft-green-accent` fill + `--deepdrft-navy` text (hover: `--deepdrft-green-interactive`); `.deepdrft-theme-dark .btn-ghost``--deepdrft-page-text` color + `--deepdrft-border-light` border.
- **Open questions resolved:** Dark neutral surface = ground (continuous field, `--mud-palette-background`) — not elevated navy-mid. Popover target: `color-mix(navy 8%, white)` in light; dark binds `#162437` (MudBlazor dark Surface) unchanged.
- **Design memo:** `product-notes/theme-dark-mode-remediation.md`.
### Phase 18 — Wave 4 — Popover-surface retune + portaled-popover body-class bridge (landed 2026-06-20)
**Landed:** 2026-06-20 on dev.
- **What:** Follow-on retune of `--deepdrft-popover-surface` values and a root-cause fix for portaled MudBlazor popovers that were never reaching the dark token.
- **Why:** Wave 13 shipped `--deepdrft-popover-surface` light at `color-mix(navy 8%, white)` (too saturated — read as a grey slab) and dark at flat `#162437`. More importantly, MudBlazor popovers portal to `<body>`, outside the `.deepdrft-theme-dark` wrapper `<div>`, so the dark token never applied to them at all. Both needed fixing as a pair.
- **Shape:**
- **Token retune (`deepdrft-tokens.css`):** Light value changed from 8% → 4% navy mix (near-page-background, clearly light). Dark value changed from `#162437` to `color-mix(in srgb, var(--deepdrft-navy-mid) 80%, var(--deepdrft-green-accent) 20%)` — a bluer navy with a slight green accent. Dark value hoisted into a new source token `--deepdrft-popover-surface-dark` (defined once in `:root`), referenced by both the `.deepdrft-theme-dark` wrapper block and a new `body.deepdrft-theme-dark` block so portaled content is reached from either selector.
- **Portaled-popover body-class bridge (`MainLayout.razor` + new TS module):** `MainLayout.razor` now stamps/removes `deepdrft-theme-dark` on `<body>` after each render via a new `DeepDrftShared.Client/Interop/theme/theme.ts` module exporting `setBodyThemeClass(isDark: boolean)`. Lazy-imported as `_content/DeepDrftShared.Client/js/theme/theme.js`. Call is gated to fire only on first render or when `_isDarkMode` changes (`_lastAppliedDarkMode` comparison) — no redundant JS calls on unrelated re-renders. `IJSObjectReference _themeModule` is disposed in `DisposeAsync` to clean up the module reference when the circuit tears down.
### Phase 18 — Wave 5 — Glass-panel theme-aware token family (landed 2026-06-20)
**Landed:** 2026-06-20 on dev.
- **What:** The three `MudOverlay`-based glass panels — the queue panel (`.deepdrft-queue-modal`), the waveform visualizer control deck, and the privacy modal — now render as a light translucent glass with legible dark text in light theme, while remaining the existing dark-glass charcoal in dark theme. Dark mode is visually unchanged; a latent white-on-light bug in the inline embed queue row was incidentally fixed by the token flip.
- **Why:** Prior to this wave, all three panels were bound to the constant `--deepdrft-panel-ground` token, exempting them from the theme-aware alias layer established in Waves 13. In light theme this produced white text on a near-white glass surface — unreadable. The panels needed their own theme-aware family (separate from `--deepdrft-popover-surface`, which targets MudBlazor default popovers) and the same `body.deepdrft-theme-dark` portal-scope treatment introduced for popovers in Wave 4.
- **Shape:**
- **New token family (`deepdrft-tokens.css`):** `--deepdrft-panel-surface` / `--deepdrft-panel-text` / `--deepdrft-panel-text-muted` / `--deepdrft-panel-border` / `--deepdrft-panel-row-hover` — each defined in `:root` (light values: translucent glass with dark text), `.deepdrft-theme-dark` (dark-glass charcoal with light text, sourced from the existing `--deepdrft-panel-ground` constant), and `body.deepdrft-theme-dark` (same dark values re-declared so the tokens resolve correctly when the panels portal to `<body>` via `MudOverlay`).
- **Consumer re-pointing:** The three panels and their descendants (queue rows, visualizer deck, privacy modal) previously bound `--deepdrft-panel-ground` directly; they are now re-pointed to the appropriate `--deepdrft-panel-surface`/`-text`/`-text-muted`/`-border`/`-row-hover` aliases.
- **Exemption lifted:** This deliberately removes the previously-documented exemption of these panels from the theme-aware layer. `--deepdrft-panel-ground` is now consumed only as the dark-theme value of `--deepdrft-panel-surface`, not directly by any component CSS.
---
## Phase 17 — Player-Bar Queue View: Wave 17.3 — Fixed embed panel + iframe resize (landed 2026-06-19)
**Landed:** 2026-06-19 on dev.
- **What:** The Fixed (embed) mode queue panel and the OQ1 Option-A iframe resize handshake. Release embeds now render an always-shown, read-only queue panel below the player-bar controls; the Queue button collapses/expands that panel and posts the iframe's new height to the host page so the outer `<iframe>` element resizes to match. Single-track embeds (TrackEntryKey mode) have no queue, no panel, and no Queue button — unchanged compact behaviour. Phase 17 is now complete (all four waves landed).
- **Why:** Phase 11 wave 11.F armed release embeds with a queue (skip navigation, auto-advance), but the viewer had no way to see or jump within the queue. Wave 17.3 surfaces it in Fixed mode — read-only because a shared embed is not an editable playlist — and resolves OQ1 (Option A confirmed feasible: `postMessage` resize degrades gracefully if the host strips the script).
- **Shape:**
- **Fixed embed queue panel** (`AudioPlayerBar.razor`): rendered conditionally on `ShowFixedPanel && _fixedPanelOpen` inside `.deepdrft-queue-embed-panel`; hosts `<QueueList Items="QueueItems" CurrentIndex="QueueCurrentIndex" Editable="false" OnJump="@OnQueueJump" />`. Read-only: no drag handles, no remove buttons. Row-jump (OQ2) calls `PlayRelease(Items, index)` — coherent from the armed-but-not-started state (`PlayRelease` already clears `IsArmed` and materializes a defensive copy).
- **Queue button in Fixed mode** (`PlayerTransportZone`): toggles `_fixedPanelOpen`; triggers a height post after the panel renders. Gated on `ShowFixedPanel` so single-track embeds see no button.
- **`EmbedSnippetBuilder.cs`** (`DeepDrftPublic.Client/Helpers/EmbedSnippetBuilder.cs`): `ForRelease` now mints a per-snippet random token (8 hex chars from `Guid.NewGuid().ToString("N")[..8]`). Token is used as the iframe id (`deepdrft-embed-{token}`) and threaded into the iframe src as `&EmbedId={token}`. Taller iframe height (release: 384 px vs. track: 196 px). Carries a host-side `<script>` listener that matches incoming `{type:"deepdrft-embed-resize", embedId}` messages against the snippet's own token and sets `iframe.style.height` — multiple release embeds on one host page resize independently (no cross-talk). Degrades to Option B if the host strips the script (panel still works inside the iframe at expanded height). `ForTrack` is unchanged (compact height 196 px, no script, no id token).
- **`embed-frame.ts`** (`DeepDrftPublic/Interop/embed/embed-frame.ts`; compiled output gitignored): new TypeScript interop module. Reads `EmbedId` from `window.location.search` once at module load; exports `postHeight(element: HTMLElement)` — measures the player element's rendered height (`Math.ceil(getBoundingClientRect().height) + 2`), builds `{type:"deepdrft-embed-resize", height, embedId?}` payload (omits `embedId` when absent for backward-compatible degradation), and calls `window.parent.postMessage(payload, "*")`. No-ops when not framed (`window.parent === window`) or the element is unmeasurable.
- **CSS** (`DeepDrftPublic/wwwroot/styles/deepdrft-styles.css`): new `deepdrft-queue-embed-panel` and related `deepdrft-` embed-panel classes for the fixed queue panel chrome.
- **Tests** (`EmbedSnippetBuilderTests`): height divergence (ForRelease taller than ForTrack), ForTrack-unchanged (height 196, no script), id uniqueness (two ForRelease calls yield distinct ids), id/script-token consistency (iframe id matches token in script), EmbedId-in-src (token appears as `EmbedId=` in the iframe src).
---
## Phase 17 — Player-Bar Queue View: Wave 17.4 — Add-to-Queue affordance (landed 2026-06-19)
**Landed:** 2026-06-19 on dev.
- **What:** The append-only Add-to-Queue affordance on detail pages — a new shared `AddToQueueButton.razor` control wired at every detail-page play site, enabling listeners to add a release or individual track to the queue without interrupting the current track. `ReleaseGallery` browse-grid cards are intentionally excluded (OQ10, deferred).
- **Why:** Phase 11 wave 11.F built the `Enqueue`/`EnqueueRange` append path in the queue engine but gave it no UI entry point. Wave 17.4 lights that dormant path, completing the Add-to-Queue capability Daniel stated as commitment 4 of Phase 17. It was split from 17.2 because it depends only on the existing engine append members (not on 17.1's new `Move`/`RemoveAt`), allowing it to land in parallel.
- **Shape:**
- **`AddToQueueButton.razor`** (`DeepDrftPublic.Client/Controls/AddToQueueButton.razor`): shared append-only button with two modes: track mode (`Enqueue` — called with a single `TrackDto`) and release mode (`EnqueueRange` — called with an ordered `IReadOnlyList<TrackDto>`). Material `PlaylistAdd` glyph; tooltip reads "Add to queue" (track mode) or "Add release to queue" (release mode). Reads the cascaded `IQueueService`; disabled until interactive / when the cascade is absent; append-only — does not play, does not navigate.
- **`CutDetail.razor`** (header): release-mode `AddToQueueButton` beside the header play affordance, passing the `TrackNumber`-ordered track list.
- **`CutDetail.razor`** (track rows): track-mode `AddToQueueButton` beside the per-row play affordance.
- **`SessionDetail.razor`** (hero play): track-mode `AddToQueueButton` beside the Session hero play button.
- **`MixDetail.razor`** (hero play): track-mode `AddToQueueButton` beside the Mix hero play button.
- **Excluded sites:** `StreamNowButton` (no fixed track to resolve — OQ9) and `ReleaseGallery` cards (no play button today — OQ10, deferred to `TODO.md`).
---
## Phase 17 — Player-Bar Queue View: Wave 17.2 — Docked queue overlay (landed 2026-06-19)
**Landed:** 2026-06-19 on dev.
- **What:** The editable docked-player queue overlay — a Queue toggle button in the non-Fixed (docked) player bar and a new `QueueOverlay.razor` modal that hosts the shared `QueueList` in editable mode. Listeners can now see, reorder, remove from, and jump within the queue while a release is playing. Also fixed a pre-existing `QueueChanged` unsubscribe leak in `AudioPlayerBar.DisposeAsync`, hardened `PlayRelease` with a defensive copy, and styled the global `deepdrft-queue-*` CSS classes for the first time (first styling for the `QueueList` classes that 17.1 shipped unstyled).
- **Why:** Phase 11 built the queue engine and Phase 17 wave 17.1 built the shared `QueueList` component, but neither surfaced the queue visually in the docked player. Wave 17.2 delivers Daniel's commitment 2 — a visible, editable queue panel in the non-Fixed player bar.
- **Shape:**
- **Queue toggle button** (`AudioPlayerBar` / `PlayerTransportZone`): shown only when `!Fixed && Items.Count > 0`, placed below the transport-button row and left of the timestamp. Material `QueueMusic` glyph; renders in an active/highlighted state when the overlay is open.
- **`QueueOverlay.razor`** (`DeepDrftPublic.Client/Controls/QueueOverlay.razor`): screen-centered tinted modal borrowing the `WaveformVisualizerControlPopover` `MudOverlay` idiom (`DarkBackground="true"`, `Modal="true"`). Panel stops click propagation; scrim-click closes the overlay (drag-safe: capture div sits above the scrim during a drag so releasing outside the panel never fires the close handler). Auto-closes if a removal empties the queue. Hosts `QueueList` in `Editable="true"` mode.
- **`AudioPlayerBar` wiring**: reorder → `Move(fromIndex, toIndex)`; remove → `RemoveAt(index)` (auto-closes overlay when queue empties); row-jump → `PlayRelease(Items, index)`; Clear header action → new `ClearUpcoming()`. Fixed pre-existing `QueueChanged` unsubscribe leak in `DisposeAsync`.
- **`QueueList.razor` — current-row remove suppression**: the remove (×) control is now hidden on the currently-playing row (`Editable && !isCurrent`), enforcing OQ3's "the current track cannot be removed" rule in the UI. Reorder of the current row is still permitted.
- **Engine — `ClearUpcoming()`** (`IQueueService` / `QueueService`): new additive member. Removes all queued items except the currently-playing one, leaving it as the sole item at `CurrentIndex == 0`; re-emits `QueueChanged`; touches no playback. Satisfies OQ5's requirement that Clear does not stop or remove the current track.
- **Engine — `PlayRelease` defensive copy**: `PlayRelease` now always materializes a defensive copy of its input list (`tracks.ToList()`) so it can never alias the caller's `Items` list — fixes a row-jump bug where jumping via `PlayRelease(Items, index)` could mutate the live `Items` reference mid-operation.
- **CSS — `deepdrft-queue-*` classes** (`DeepDrftPublic/wwwroot/styles/deepdrft-styles.css`): overlay/list chrome classes added to the global stylesheet (portaled overlay content cannot use scoped CSS). This is also the first styling pass for the `QueueList` classes 17.1 introduced without accompanying styles.
---
## Phase 17 — Player-Bar Queue View: Wave 17.1 — Engine additions + shared QueueList (landed 2026-06-19)
**Landed:** 2026-06-19 on dev.
- **What:** The queue-engine additions and the shared presentational list component that waves 17.2 and 17.3 will consume. No player-bar wiring, overlay, embed, or Add-to-Queue affordance landed — those remain in waves 17.2 and 17.3.
- **Why:** Wave 17.1 is the cold-start prerequisite for the full Phase 17 queue-view surface. The engine additions are interop-free state mutations that land without any UI decision being made; `QueueList` is the single presentational "view" both the docked overlay and the embedded panel will share (one source, multiple views), so it is cleanest to build and test it before the hosting contexts exist.
- **Shape:**
- **Engine — `Move(int fromIndex, int toIndex)`** (`IQueueService` / `QueueService`): reorders `Items` in-place, adjusts `CurrentIndex` so the same track stays current across the move, re-emits `QueueChanged`. Never re-streams or interrupts the playing track. No-op (no `QueueChanged`) when either index is out of range or the indices are equal. Interop-free; safe during prerender.
- **Engine — `RemoveAt(int index)`** (`IQueueService` / `QueueService`): removes the item at `index`, adjusts `CurrentIndex` (a track before current decrements the index; a track after current leaves it unchanged; removing the current track does not stop playback — the track runs to natural end while `CurrentIndex` resolves to the new slot occupant; removing the last remaining item leaves the queue empty and dormant with `CurrentIndex == -1`). Re-emits `QueueChanged`. No-op when `index` is out of range. Interop-free; safe during prerender.
- **Engine — dormant-`Enqueue` coherence (OQ8):** `Enqueue` and `EnqueueRange` into an empty/dormant queue (`CurrentIndex == -1`) now set `CurrentIndex` to 0 so a subsequent play/skip is correct. Does **not** auto-play — add is not play. `PlayCurrent` is never called from these paths; the methods remain interop-free.
- **`QueueList.razor`** (`DeepDrftPublic.Client/Controls/QueueList.razor`): purely presentational component. Renders `Items` as an ordered list with the current track marked (position number + `GraphicEq` now-playing icon on the current row). `Editable` flag gates drag-reorder handles and per-row remove controls: when `true`, wraps rows in a `MudDropContainer`/`MudDropZone` for reorder; when `false`, renders a plain `<div>` (read-only; the embed's fixed-order shared queue). Reorder, remove, and row-jump are surfaced to the parent as `EventCallback<(int FromIndex, int ToIndex)> OnReorder`, `EventCallback<int> OnRemove`, and `EventCallback<int> OnJump` respectively — the component calls no `IQueueService` method itself. Owns no data fetch or player wiring. Runs during prerender without JS interop (drag work is client-only and inert when no drag occurs).
- **`QueueServiceTests`**: T1T10 added, covering `Move` (in-range, out-of-range, same-index no-ops; current-track identity preserved across reorders) and `RemoveAt` (before/after/at current; last-item dormant; out-of-range no-op; playback not stopped).
---
## Phase 16 — Anonymous Play & Share Tracking: Wave 16.5 — Home Plays-card capstone (landed 2026-06-19)
**Landed:** 2026-06-19 on dev.
- **What:** The capstone wave — the live home hero Plays card and the `HomeStatsDto` extension that powers it. `NowPlayingStats.razor`'s third card, previously a static "XXX / Plays (Coming Soon)" odometer placeholder, now renders the live `TotalPlays` figure in the existing odometer treatment with a secondary "N listeners" line (`UniqueListeners`). No new fetch path, no new client service, no migration — the card consumes the same `HomeStatsDto` round-trip the other two cards already use. Privacy footer line (`DeepDrftFooter.razor`, `.deepdrft-footer-privacy`) also landed as part of the same merge: a quiet fine-print disclosure of the anonymous `anonId` token, using the Variant 1 approved copy from `product-notes/phase-16-privacy-note.md`.
- **Why:** The Plays card was deliberately held as the final wave (sequenced bottom-up per Daniel's directive) so the substrate (16.1 capture + rollup, 16.2 bucket/channel, 16.3 anonId + distinct-listener aggregation) would be solid before any read surface appeared. Wave 16.4 (per-target / CMS stats views) was speculative and skipped; the event log supports it later if wanted. With 16.5 landing, Phase 16 is complete.
- **Shape:**
- **`HomeStatsDto` extended** (`DeepDrftModels/DTOs/HomeStatsDto.cs`): two new fields — `TotalPlays` (`long`; site-wide sum of every `play_counter` row's `PartialCount + SampledCount + CompleteCount`, all-time; zero until the telemetry migration is applied — expected, not an error) and `UniqueListeners` (`int`; distinct non-null `anon_id` across all play events, all-time; over-counts by design, honestly labelled "listeners"). No other DTO changes.
- **`StatsController` composition** (`DeepDrftAPI/Controllers/StatsController.cs`): now injects `ITrackService` (existing) **and** `IEventService` (Phase 16 event domain). `GetHome` assembles `HomeStatsDto` in two sequential best-effort reads: track-domain aggregation via `ITrackService.GetHomeStats` (existing; failure returns 500 as before); play/listener figures via `IEventService.GetTotalPlayCount` and `IEventService.GetDistinctListenerCount` (Phase 16; a telemetry failure or not-yet-applied migration leaves them at 0 rather than 500-ing the whole endpoint). Neither domain reaches into the other's tables; the controller is the composition seam only.
- **`IEventService` additions** (`DeepDrftData/IEventService.cs`): `GetTotalPlayCount(ct)``ResultContainer<long>` and `GetDistinctListenerCount(ct)``ResultContainer<int>` (wave 16.3 added the distinct-listener overloads; `GetTotalPlayCount` is the one new member for 16.5).
- **`EventRepository.CountTotalPlaysAsync`** (`DeepDrftData/Repositories/EventRepository.cs`): sums `PartialCount + SampledCount + CompleteCount` directly over `PlayCounters` via LINQ — **not** `PlayCounter.TotalPlays` (which is an EF-ignored computed property and not translatable). An empty counter table sums to 0.
- **`NowPlayingStats.razor`** (`DeepDrftPublic.Client/Controls/NowPlayingStats.razor`): third card now renders `@_stats.TotalPlays` in `.hero-stat-odometer` and `@_stats.UniqueListeners listeners` as `.hero-stat-sub`. No change to the `PersistentComponentState` bridge or `IStatsDataService` fetch path — the DTO fields arrive in the same existing round-trip.
- **`DeepDrftFooter.razor`** (`DeepDrftPublic.Client/Layout/DeepDrftFooter.razor`): privacy disclosure paragraph (`.deepdrft-footer-privacy`) added: "We keep a random tag in your browser so we can count how many people a track reaches — not who they are. No account, no name, nothing personal, nothing shared with anyone else. Clear your browser data and the tag's gone."
- **No migration.** The `play_counter` rollup table was created by `20260619155610_AddPlayShareTelemetry` (wave 16.1; authored, not yet applied — Daniel-gated). The `CountTotalPlaysAsync` query returns 0 gracefully until that migration runs.
---
## Phase 16 — Anonymous Play & Share Tracking: Wave 16.1 — Foundation (landed 2026-06-19)
**Landed:** 2026-06-19 on dev.
- **What:** The anonymous telemetry **substrate** — foundation end-to-end with nothing reading it yet. No `anonId` written; no home-card/read surface changed (those are waves 16.3 and 16.5). The full capture-and-storage pipeline is in place: client-side play-session tracker and share tracker, `sendBeacon` transport with page-unload handler, proxied and rate-limited intake endpoints, append-only SQL event log with incremental rollup, and server-side release attribution.
- **Why:** The home hero's Plays stat card (`NowPlayingStats.razor`'s third card) has been a static "XXX / Plays (Coming Soon)" placeholder. Phase 16 builds the anonymous, privacy-light substrate that will eventually power it. Wave 16.1 is the cold-start foundation — nothing reads the log yet; correctness and storage are the deliverable, not the visible metric.
- **Shape:**
- **Client — `PlayTracker`** (`DeepDrftPublic.Client/Services/PlayTracker.cs`): opens a play session on playback start, advances a high-water position on each progress tick (instrumented at `StreamingAudioPlayerService`, not at the HTTP layer, so seek-beyond-buffer re-fetches count as the same play), closes on track-switch / stop / organic-end / page-unload. Engagement floor: ≥3s OR ≥5% of duration (whichever is smaller). Three-bucket classification: `partial` < 30%, `sampled` 3080%, `complete` > 80%. Emits at most one event per session via `IPlayEventSink`. Deliberately free of player, HTTP, and JS dependencies for testability.
- **Client — `ShareTracker`** (`DeepDrftPublic.Client/Services/ShareTracker.cs`): called by `SharePopover` after a successful clipboard write; applies a 60-second per-(target, channel) debounce so repeated copies of the same link in a session count as one share. Sends via `sendBeacon`. No `anonId` in wave 16.1.
- **Client — `BeaconInterop`** (`DeepDrftPublic.Client/Services/BeaconInterop.cs`): `navigator.sendBeacon` JS interop wrapper + page-unload handler that flushes any pending play event when the page is torn down.
- **Public proxy — `EventProxyController`** (`DeepDrftPublic/Controllers/EventProxyController.cs`): proxies `POST api/event/play` and `POST api/event/share` to DeepDrftAPI. Buffers and relays the small JSON body verbatim; forwards `X-Forwarded-For` for per-IP rate limiting on the API side. Opts out of antiforgery (`[IgnoreAntiforgeryToken]`) — `sendBeacon` cannot attach tokens.
- **API — `EventController`** (`DeepDrftAPI/Controllers/EventController.cs`): `POST api/event/play` and `POST api/event/share`, unauthenticated, rate-limited by the `"events"` fixed-window policy (30 requests / 60 s per IP, registered in `Program.cs`). Returns `202 Accepted` (fire-and-forget contract). Payload-validates the track key and enum values; delegates writes to `IEventService`.
- **API — rate limiter** (`DeepDrftAPI/Program.cs`): `AddRateLimiter` + `"events"` fixed-window policy keyed on `Connection.RemoteIpAddress`; `UseForwardedHeaders` in production resolves the XFF chain into the real client IP. `UseRateLimiter()` added to the middleware pipeline.
- **Data — `EventRepository`** (`DeepDrftData/Repositories/EventRepository.cs`): append-only writes to `play_event` and `share_event` tables; incremental-on-write bump of the `play_counter` rollup (D6); server-side track→release resolution at write time (D4) — the client sends only the track `EntryKey`, the repository stamps the release id.
- **Data — `EventManager` / `IEventService`** (`DeepDrftData/EventManager.cs`): `IEventService` boundary (`RecordPlay`, `RecordShare`); `EventManager` wraps `EventRepository` and returns NetBlocks `Result`. Registered scoped in `DeepDrftAPI/Program.cs` alongside the existing track and release domain services.
- **Migration `20260619155610_AddPlayShareTelemetry`**: adds `play_event`, `share_event`, and `play_counter` tables. **Authored but not yet applied** (Daniel-gated).
---
## Phase 16 — Anonymous Play & Share Tracking: Wave 16.3 — Unique-listener `anonId` layer (landed 2026-06-19)
**Landed:** 2026-06-19 on dev (merge `297805b`). No migration — `anon_id varchar(64)` columns and `IX_play_event_anon_id`/`IX_share_event_anon_id` indexes already shipped in the wave 16.1 migration.
- **What:** The unique-listener `anonId` seam end-to-end — the "last metric layer" of the Phase 16 substrate. Client mints a first-party `localStorage` GUID on first visit, threads it onto play and share beacon payloads (omitted when null), server accepts and length-clamps it (reject-not-truncate, ≤64 chars), persists it to the reserved nullable `anon_id` columns, and exposes all-time distinct-listener aggregation. The distinct-count capability is in place but not yet surfaced on any read surface (16.5 consumes it). Privacy-notice copy deliberately not authored (Daniel-gated).
- **Why:** The anonymous unique-listener metric (D5 / D3) is the final substrate wave before the home Plays card can be lit (16.5). It was sequenced last of the metric layers because it is the lowest-priority metric and carries no dependency — the event log captures `anon_id` on the same rows 16.1 already writes; 16.3 simply lights the seam that was reserved but unused.
- **Shape:**
- **Client — `IAnonIdProvider` / `AnonIdProvider`** (`DeepDrftPublic.Client/Services/IAnonIdProvider.cs`, `AnonIdProvider.cs`): `IAnonIdProvider` exposes `string? Current` (synchronous cached read, safe on the unload path) and `ValueTask EnsureLoadedAsync()` (warms the cache from `localStorage` via JS interop — idempotent, best-effort, never throws). `AnonIdProvider` is the production implementation over the `window.DeepDrftAnonId.get` interop call. Degrades to null when `localStorage` is unavailable (private mode / blocked / partitioned iframe) — missing id is the accepted graceful path; over-counting is the direction of error (§3). Scoped (per-session cache); the token itself outlives the session in `localStorage`.
- **Client — TypeScript interop** (`DeepDrftPublic/Interop/session/anonid.ts`): mints and reads the `localStorage` GUID. Exposes `window.DeepDrftAnonId.get`. Returns null without throwing when storage is unavailable.
- **Client — `BeaconPlayEventSink`** (`DeepDrftPublic.Client/Services/BeaconPlayEventSink.cs`): now injects `IAnonIdProvider`; reads `_anonId.Current` synchronously at emit time and sets `PlayEventDto.AnonId`. Null id produces an anonId-less payload (the field is omitted from the wire JSON entirely via `WhenWritingNull` — the API treats absent and null identically).
- **Client — `ShareTracker`** (`DeepDrftPublic.Client/Services/ShareTracker.cs`): now injects `IAnonIdProvider`; reads `_anonId.Current` at share time and sets `ShareEventDto.AnonId`. Same null-omit posture as the play sink.
- **API — `EventController`** (`DeepDrftAPI/Controllers/EventController.cs`): `TryNormalizeAnonId` helper on both `POST api/event/play` and `POST api/event/share` — whitespace-only / empty / null collapses to null (valid anonId-less event); a token longer than 64 chars is rejected with `400 Bad Request` rather than truncated (truncation would collide distinct listeners onto one prefix); valid tokens are trimmed and passed through.
- **Data — `EventRepository`** (`DeepDrftData/Repositories/EventRepository.cs`): three new distinct-count queries (already in the repository as of 16.3): `CountDistinctListenersAsync()` (site-wide, nulls excluded), `CountDistinctListenersForTrackAsync(trackEntryKey)` (per-track), `CountDistinctListenersForReleaseAsync(releaseId)` (per-release, uses the stamped `release_id` on the play event row — D4 attribution).
- **Data — `IEventService` / `EventManager`** (`DeepDrftData/EventManager.cs`): three new members exposing the distinct-count capability: `GetDistinctListenerCount()`, `GetDistinctListenerCountForTrack(trackEntryKey)`, `GetDistinctListenerCountForRelease(releaseId)` — each returns `ResultContainer<int>`. No read surface or card consumes them yet (16.5).
- **No migration** — the `anon_id varchar(64)` columns on `play_event` and `share_event` and their covering indexes (`IX_play_event_anon_id`, `IX_share_event_anon_id`) were already created by `20260619155610_AddPlayShareTelemetry` (wave 16.1). Wave 16.3 only wires the client seam and adds the server-side aggregation queries.
---
## Phase 16 — Anonymous Play & Share Tracking: Wave 16.2 — Completion-bucket classification + shares (landed by absorption into 16.1)
**Landed:** absorbed into wave 16.1 (2026-06-19). All §4.1 deliverables shipped inside the foundation wave.
- **What:** Three-bucket completion classification correct and exhaustive end to end, and share-channel split. Because these were structurally inseparable from the foundation (the tracker, payload, log table, and rollup all required the bucket column set from day one), they landed together with 16.1 rather than as a follow-on wave.
- **Why:** The §4.2 spec listed bucket classification and share-channel split as wave 16.2 items, but the implementation showed they could not be cleanly deferred — the `play_counter` rollup columns are per-bucket by design (D6), and the share `channel` discriminator is a single non-null column on the `share_event` table. Building the log without them would have required a migration to add them in 16.2 anyway.
- **Shape:**
- **`PlayBucket` enum** (`DeepDrftModels.Enums`): `Partial` (< 30%), `Sampled` (3080%), `Complete` (> 80%) — exhaustive, non-overlapping. D1 resolved.
- **`PlayCounter` rollup columns** (`DeepDrftModels.Entities.PlayCounter`): `PartialCount`, `SampledCount`, `CompleteCount` (each `long`), `TotalPlays` (computed `long` sum). `BumpCounterAsync` in `EventRepository` switches on the bucket to increment the correct column in the same transaction as the event append.
- **API-boundary bucket validation** (`EventController`): `Enum.IsDefined(payload.Bucket)` guard — an undefined bucket value returns `400 Bad Request` before the write reaches the repository.
- **`ShareChannel` enum** (`DeepDrftModels.Enums`): `Link` / `Embed` on `ShareEvent.Channel`. `ShareTracker` passes the channel through from the `SharePopover` clipboard action; `EventController` validates it is a defined `ShareChannel` value.
- **Deferred:** optional `share_count` rollup column on `play_counter` (per-track share count in the rollup table) — not built. Shares are not on the home-card hot path; per-target share reads are speculative wave 16.4 work.
---
## Home Hero Stats — Live data wiring (landed 2026-06-18)
**Landed:** 2026-06-18 on dev (commits `5f0422a` + `8fa330f`, merged `e9e6b60`).
- **What:** Replaced the hard-coded placeholder figures in the public home hero stat row (`NowPlayingStats`) with real SQL-backed aggregates. Resolves the "Real stat-row numbers" deferred item from Phase 0 §0.3.
- **Why:** The stat row ("47+ / 2 / ∞") was intentionally hard-coded at Phase 0 with a TODO; the data model now has enough shape (releases, medium discriminator, trackrelease join) to serve real numbers in a single efficient query.
- **Shape:**
- **New SQL column:** `DurationSeconds` (`double?`, column `duration_seconds`) on `TrackEntity` and `TrackDto`. Populated at upload via the existing dual-database add flow (`TrackContentService` extracts duration from vault audio; `UnifiedTrackService` persists it to SQL). Migration `20260618155002_AddTrackDuration`. Configured in `TrackConfiguration`.
- **New aggregate query:** `TrackRepository.GetHomeStatsAsync``HomeStatsDto` (new DTO in `DeepDrftModels/DTOs/`). Returns cut track count, per-`ReleaseType` cut release counts (zero-count types suppressed), mix release count, and total mix runtime seconds (null durations counted as 0; tracks under soft-deleted releases excluded). Surfaced via `ITrackService.GetHomeStats` on `TrackManager`.
- **New API endpoints:** `GET api/stats/home` (`StatsController`, unauthenticated; returns `HomeStatsDto` bare) and `POST api/track/duration/backfill` (ApiKey-gated; one-time backfill of `DurationSeconds` for pre-existing rows from vault audio, delegated to `UnifiedTrackService.BackfillDurationsAsync`).
- **New public proxy:** `StatsProxyController` in `DeepDrftPublic` mirrors `ReleaseProxyController`; forwards `GET api/stats/home` from the browser to DeepDrftAPI.
- **New client surface:** `StatsClient` (`Clients/`, named `"DeepDrft.API"` client) + `IStatsDataService` / `StatsClientDataService` (`Services/`) registered scoped in `Startup.ConfigureDomainServices`. `RuntimeFormat` static helper (`Helpers/`) converts seconds to `hh:mm`.
- **`NowPlayingStats.razor`:** now renders live data — Studio Cuts card (cut track count + zero-suppressed Single/EP/Album breakdown), Mixes card (`MixReleaseCount` "Sets" + `hh:mm` runtime), Plays card (static "XXX / Coming Soon" odometer placeholder). Uses `PersistentComponentState` to bridge the SSR prerender fetch across the WASM seam (only persists on a successful load).
---
## Phase 12 — About Page (public site editorial) (landed 2026-06-17)
**Landed:** 2026-06-17 on dev.
+96 -14
View File
@@ -11,12 +11,12 @@ Dual-database authority for tracks (SQL metadata + FileDatabase binary), release
## What lives here now (only)
- `Program.cs`, `Startup.cs`: HTTP host config, DI wiring, middleware setup, port binding. AuthBlocks startup: `AddAuthBlocks`, `UseAuthBlocksStartupAsync`, `MapAuthBlocks`, authentication/authorization middleware.
- `Services/UnifiedTrackService.cs`: Host-internal orchestrator. Coordinates vault write + SQL persist for upload (`UploadAsync`), and SQL delete + vault remove for delete (`DeleteAsync`).
- `Services/UnifiedTrackService.cs`: Host-internal orchestrator. Coordinates streaming vault write + SQL persist for upload (`UploadAsync`), and SQL delete + vault remove for delete (`DeleteAsync`). The upload/replace hot path streams audio into the vault via the `ProcessedAudio` plan (Wave 1 OOM fix) and then computes both waveform datums in a single bounded streaming pass via `TryStoreWaveformDatumsAsync` (Wave 2 OOM fix) — neither path buffers the whole audio file in a managed `byte[]`.
- `Services/UnifiedReleaseService.cs`: Host-internal orchestrator. Coordinates release mutations (mix waveform compute + store, session hero-image upload + link).
- `Controllers/TrackController.cs`: Track endpoints (see below).
- `Controllers/ReleaseController.cs`: Release endpoints (see below).
- `Middleware/ApiKeyAuthenticationMiddleware.cs`, `Middleware/ApiKeyAuthorizeAttribute.cs`: ApiKey validation logic (for track endpoints only).
- `Models/`: Settings POCOs only (`ApiKeySettings`, `CorsSettings`, `FileDatabaseSettings`). No domain code.
- `Models/`: Settings POCOs only (`ApiKeySettings`, `CorsSettings`, `FileDatabaseSettings`, `UploadSettings`, `UploadStagingDirectory`). No domain code.
- `environment/filedatabase.json`: FileDatabase vault path config (loaded via CredentialTools, not in repo).
- `environment/apikey.json`: API key for track endpoints (loaded via CredentialTools, not in repo, must be created locally or at deployment).
- `environment/connections.json`: SQL and Auth connection strings (loaded via CredentialTools, not in repo, format: `{ "ConnectionStrings": { "DefaultConnection": "...", "Auth": "..." } }`).
@@ -32,12 +32,12 @@ Dual-database authority for tracks (SQL metadata + FileDatabase binary), release
### GET api/track/{trackId} (unauthenticated)
Returns the WAV bytes from the `tracks` vault with HTTP Range support.
Streams the track's audio bytes from disk with HTTP Range support. An optional `?format=opus` query parameter selects between the derived Opus artifact and the lossless source.
- **Route parameter `trackId`** (string): the entry id inside the `tracks` vault (i.e. `TrackEntity.EntryKey`).
- **Range header** (optional): HTTP Range header for byte-range requests (e.g., `Range: bytes=1000-`). Server responds with `206 Partial Content` and streams from the requested offset.
- Streams the file directly from disk with `enableRangeProcessing: true`, supporting both full-file and partial-range requests without synthesizing WAV headers or buffering.
- Returns 200 for full-file requests, 206 for Range requests, 404 if track not found, 500 if vault operations fail (with error swallowing — the vault returns `null`).
- **Query parameter `format`** (optional): `opus` requests the derived Ogg Opus artifact from the `track-opus` vault when present, falling back to lossless when it is not (C2 — never 404 if any audio exists); omitted or any other value delivers the lossless source in its stored format (WAV/MP3/FLAC). Both arms stream from a seekable disk `FileStream` via `File(stream, contentType, enableRangeProcessing: true)` — no whole-file `byte[]`; `Content-Type` reflects what was actually served (e.g., `audio/ogg` on an Opus hit, the source's real MIME on a lossless request or C2 fallback).
- **Range header** (optional): HTTP Range header for byte-range requests (e.g., `Range: bytes=1000-`). Server responds with `206 Partial Content` and streams from the requested offset. Honoured on both arms (seekable `FileStream` in both cases).
- Returns 200 for full-file requests, 206 for Range requests, 404 if no audio artifact exists for the track, 500 if vault operations fail.
### GET api/track/albums (unauthenticated)
@@ -74,7 +74,7 @@ Admin backfill: computes and stores a waveform profile for an existing track fro
- **Header `ApiKey`**: required. Validated by `ApiKeyAuthenticationMiddleware`.
- **Route parameter `trackId`** (string): the entry id (TrackEntity.EntryKey).
- Fetches audio from vault, decodes it, computes a loudness profile, and stores the profile in the `waveform-profiles` vault.
- Streams the vault audio in bounded ≤80 KB chunks (no whole-file load) via `WaveformProfileService.ComputeAndStoreProfileStreamingAsync`. Tri-state result: `null` = no vault audio → 404; `false` = audio present but not WAV-decodable or vault write failed → 500; `true` = stored → 200.
- Returns 200 on success. Returns 404 if no audio is stored under that key. Returns 500 if WAV decoding or vault write fails.
### GET api/track/{trackId}/waveform/high-res (unauthenticated)
@@ -91,7 +91,7 @@ Server-side trigger: compute and store the per-track high-res datum for any trac
- **Header `ApiKey`**: required. Validated by `ApiKeyAuthenticationMiddleware`.
- **Route parameter `trackId`** (string): the entry id (TrackEntity.EntryKey).
- Calls `WaveformProfileService.ComputeAndStoreHighResAsync` via `UnifiedTrackService`.
- Reads the duration from vault index metadata (no audio body load), then streams the vault audio in bounded chunks via `WaveformProfileService.ComputeAndStoreHighResStreamingAsync`. Tri-state result: `null` = no vault audio → 404; `false` = audio present but not WAV-decodable or vault write failed → 500; `true` = stored → 200.
- Returns 200 on success. Returns 404 if no audio stored under that key. Returns 500 on compute/storage failure.
### GET api/track/meta/by-key/{entryKey} (unauthenticated)
@@ -110,6 +110,36 @@ Admin backfill view: returns every track with flags indicating whether each wave
- **Response**: `List<WaveformStatusDto>` with `TrackId`, `EntryKey`, `TrackName`, `HasProfile` (bool — 512-bucket player-bar seeker profile in `waveform-profiles` vault), and `HasHighRes` (bool — duration-derived high-res visualizer datum in `track-waveforms` vault).
- Returns 200 on success. Returns 500 on query error.
### GET api/track/opus-status ([ApiKeyAuthorize])
Admin Post-Processing view: returns every track with a flag indicating whether it has a complete Opus artifact (both audio AND seek/setup sidecar present in the `track-opus` vault). Used by the CMS to show the Backfill-Opus badge and to poll per-track Post-Processing status after an upload. Mirrors the shape and auth posture of `GET api/track/waveform-status`.
- **Header `ApiKey`**: required. Validated by `ApiKeyAuthenticationMiddleware`.
- **Response**: `List<OpusStatusDto>` with `TrackId`, `EntryKey`, `TrackName`, and `HasOpus` (bool — true only when both the Opus audio entry and the seek/setup sidecar entry are present in `track-opus`; a half-derived track counts as incomplete).
- `HasOpus` is resolved via `TrackFormatResolver.HasOpusAsync` — an **index-only** existence check (`MediaVault.HasIndexEntry` for both entries; no file-body load). The endpoint loops over the whole catalogue, so a body load per track would stream the full library sequentially; the index lookup costs zero disk reads per track.
- Returns 200 on success. Returns 500 on query error.
### POST api/track/duration/backfill ([ApiKeyAuthorize])
Admin backfill: for every track whose `DurationSeconds` SQL column is still null, reads the `AudioBinary.Duration` from the vault and writes it to SQL. Idempotent — a re-run only touches still-null rows; rows that already have a value are skipped.
- **Header `ApiKey`**: required. Validated by `ApiKeyAuthenticationMiddleware`.
- No request body.
- Calls `UnifiedTrackService.BackfillDurationsAsync`. Lives on `TrackController` in the literal-route block (before `{trackId}` routes, so the segment is never treated as a trackId).
- **Response**: `{ updated: int, skipped: int }` — counts of rows written vs. already-populated rows bypassed.
- Returns 200 on success. Returns 500 if the backfill operation fails.
### GET api/track/release/exists ([ApiKeyAuthorize])
Upload-form pre-flight: checks whether a release with the given (title, artist) already exists in the catalogue. Returns the matching `ReleaseDto` (so the caller can name it in a block message) or 404 when none exists. Uses the same `GetReleaseByTitleAndArtist` read the upload CREATE-path duplicate guard uses, so the pre-flight and the server backstop agree on the match by construction (exact ordinal comparison, soft-deleted rows excluded).
- **Header `ApiKey`**: required. Validated by `ApiKeyAuthenticationMiddleware`.
- **Query parameters**:
- `title` (string, required): the release title to check.
- `artist` (string, required): the artist name to check.
- Declared as a literal 2-segment route (`"release/exists"`) before the parameterized `{trackId}` route and distinct from `"release/{id:long}"` (different segment shape) — no routing ambiguity.
- Returns 200 with `ReleaseDto` JSON if a match exists. Returns 400 if either query parameter is missing or whitespace. Returns 404 if no match. Returns 500 on query error.
### DELETE api/track/release/{id:long} ([ApiKeyAuthorize])
Soft-delete a release row. Used by the albums browser to remove an orphaned release (one with no live tracks).
@@ -146,10 +176,11 @@ Soft-delete a release row. Used by the albums browser to remove an orphaned rele
- `releaseType` (string, optional): enum `ReleaseType` (e.g., `Single`, `Album`, `EP`). Defaults to `Single` if null or unrecognized.
- `medium` (string, optional): enum `ReleaseMedium` (e.g., `Cut`, `Mix`, `Session`). Defaults to `Cut` if null or unrecognized.
- `trackNumber` (int?, optional): track position within the release (1-based). Defaults to 1 if ≤ 0 or null.
- The upload stream is copied to a temp file under `Path.GetTempPath()` with the appropriate extension (`.wav`, `.mp3`, or `.flac`). The audio processor reads from disk and requires the correct extension for format detection. The temp file is always deleted in a `finally` block — success or failure.
- `[RequestSizeLimit(1 GB)]` + `[RequestFormLimits(MultipartBodyLengthLimit = 1 GB)]` lift the per-request ceiling above the framework default (~28 MB) so production-sized files are accepted. The body is streamed to the temp file, not buffered in memory.
- Calls `UnifiedTrackService.UploadAsync`, which orchestrates: `TrackContentService.AddTrackAsync` (format-agnostic vault write via router) → `TrackManager` (SQL persist with `createdByUserId`).
- Returns 200 with the **persisted** `TrackDto` JSON (Id populated) on success. Returns 400 for missing/invalid form fields or unsupported audio format. Returns 409 if the request violates domain cardinality rules (e.g., track number conflict). Returns 500 if processing fails.
- `releaseId` (long?, optional): the SQL release ID to attach this track to. Omit (null) on the first row of a submit — this is the **CREATE path**, which mints a new release and blocks a pre-existing (title, artist) with 409. Set to the release id returned by row 1 for rows 2..N of a within-batch multi-track Cut — this is the **ATTACH path**, which skips the (title, artist) pre-existing check and attaches directly to the already-created release after validating the id matches the natural key. The upload form is create-only; appending to a pre-existing release must go through the edit tools.
- The upload stream is copied to a staging file under the **upload staging directory** (resolved from `Upload:StagingPath`, defaulting to a `staging` subdirectory under the FileDatabase vault path — on the data disk, **never** `Path.GetTempPath()`) with the appropriate extension (`.wav`, `.mp3`, or `.flac`). The audio processor reads from disk and requires the correct extension for format detection. The staging file is always deleted in a `finally` block — success or failure. The framework's own multipart file-section buffer is relocated off the system temp mount too: `Startup.ConfigureDomainServices` sets the `ASPNETCORE_TEMP` env var to the same staging directory, so neither on-disk copy of a large body lands on `/tmp` (a small RAM-backed tmpfs on the Linux host).
- `[RequestSizeLimit(~1.86 GB / 2_000_000_000)]` + `[RequestFormLimits(MultipartBodyLengthLimit = 2_000_000_000)]` lift the per-request ceiling above the framework default (~28 MB) so production-sized files are accepted. The body is streamed to the staging file, not buffered in memory.
- `UnifiedTrackService.UploadAsync` orchestrates: release resolution (CREATE or ATTACH, see above) → `TrackContentService.AddTrackAsync` (format-agnostic streaming vault write via router — audio is streamed to the vault via the `ProcessedAudio` plan, no whole-file buffer — Wave 1 OOM fix) → `TrackManager` (SQL persist with `createdByUserId`) → `TryStoreWaveformDatumsAsync` (best-effort: reads duration from vault index metadata, then computes both waveform datums from a single bounded streaming pass over the stored audio — Wave 2 OOM fix). Release resolution runs the cardinality guard on both paths and, on the CREATE path, calls `ITrackService.FindOrCreateRelease` (returns `(ReleaseDto Release, bool WasCreated)`); if `WasCreated` is false, a concurrent upload won the race and the request is rejected as a duplicate rather than silently attaching.
- Returns 200 with the **persisted** `TrackDto` JSON (Id populated) on success. Returns 400 for missing/invalid form fields or unsupported audio format. Returns 409 for two distinct domain conditions: a pre-existing (title, artist) duplicate on the CREATE path (`DUPLICATE_RELEASE:` marker → 409 Conflict), or a track-number conflict within the release (`CARDINALITY_VIOLATION:` marker → 409 Conflict). Returns 500 if processing fails.
### DELETE api/track/{id:long} ([ApiKeyAuthorize])
@@ -160,6 +191,17 @@ Soft-delete a release row. Used by the albums browser to remove an orphaned rele
- Calls `UnifiedTrackService.DeleteAsync`, which: looks up SQL row → deletes SQL row → deletes vault entry via EntryKey.
- Returns 200 on success, 404 if track not found, 500 if deletion fails.
### POST api/track/{id:long}/replace-audio ([ApiKeyAuthorize])
**Authenticated endpoint.** Accepts a raw audio file upload (.wav, .mp3, .flac) as `multipart/form-data` and replaces the existing track's vault bytes in place, preserving the track id, `EntryKey`, SQL row (metadata/release/position), and release membership. Both waveform datums (512-bucket seeker profile + high-res visualizer datum) are regenerated after the swap; waveform regen failure is logged and swallowed — it does not fail the replace.
- **Header `ApiKey`**: required. Validated by `ApiKeyAuthenticationMiddleware`.
- **Route parameter `id`** (long): the SQL track ID.
- **Form field `audioFile`** (`IFormFile`, required): the replacement audio bytes. File name must end in `.wav`, `.mp3`, or `.flac`.
- `[RequestSizeLimit(~1.86 GB / 2_000_000_000)]` + `[RequestFormLimits(MultipartBodyLengthLimit = 2_000_000_000)]` mirror the upload ceiling. The body is streamed to a staging file under the upload staging directory (the same off-`/tmp` data-disk location as the upload path; correct extension preserved for the audio processor), always deleted in a `finally` block.
- Calls `UnifiedTrackService.ReplaceAudioAsync`, which: looks up SQL row by id → calls `TrackContentService.ReplaceTrackAudioAsync(entryKey, tempFilePath)` (streams new audio into the vault under the existing `EntryKey` via the `ProcessedAudio` plan, no whole-file buffer — Wave 1 OOM fix; removes the stale backing file only on a cross-format swap, after the new write succeeds) → regenerates both waveform datums from a single bounded streaming pass over the freshly stored audio via `TryStoreWaveformDatumsAsync` (best-effort; a datum failure is logged and swallowed — Wave 2 OOM fix) → writes the new audio's duration to `DurationSeconds` via `ITrackService.SetDuration` (unconditional overwrite; a failure is surfaced, not swallowed, to prevent derived aggregates like `MixRuntimeSeconds` from silently going stale).
- Returns 200 on success. Returns 400 if the file is missing or the format is unsupported. Returns 404 if the track id is not found. Returns 500 if vault processing fails.
### GET api/track/page (unauthenticated)
Paged metadata list from SQL with optional filtering. Public browser data, same auth posture as `GET api/track/{id}`.
@@ -255,7 +297,7 @@ Legacy endpoint: formerly served the high-res waveform datum for a Mix release f
### POST api/release/{id:long}/mix/waveform ([ApiKeyAuthorize])
Server-side trigger: fetch the Mix's track audio from the vault, compute the duration-derived high-res datum, store it in the `track-waveforms` vault under the track's `EntryKey`, and link it via `MixMetadata.WaveformEntryKey`. Delegates to `WaveformProfileService.ComputeAndStoreHighResAsync` — the same shared seam used by the upload path and the generalized CMS generate action. No request body.
Server-side trigger: fetch the Mix's track audio from the vault, compute the duration-derived high-res datum, store it in the `track-waveforms` vault under the track's `EntryKey`, and link it via `MixMetadata.WaveformEntryKey`. Delegates to `WaveformProfileService.ComputeAndStoreHighResStreamingAsync` (streams the vault audio in bounded chunks, no whole-file buffer) — the same shared seam used by the upload path and the generalized CMS generate action. No request body.
- **Header `ApiKey`**: required. Validated by `ApiKeyAuthenticationMiddleware`.
- **Route parameter `id`** (long): the SQL release ID.
@@ -272,6 +314,44 @@ Stores a hero image in the `images` vault and links it via `SessionMetadata.Hero
- Validates MIME type (rejects unsupported types with `.bin` sentinel). Calls `UnifiedReleaseService.SetHeroImageAsync`.
- Returns 200 on success. Returns 400 for missing file or unsupported MIME type. Returns 404 if release not found. Returns 500 on processing or vault failure.
## The stats endpoints
### GET api/stats/home (unauthenticated)
Aggregate figures behind the public home hero stat row (`NowPlayingStats`). A single read returns everything the three cards need so the client makes one round-trip. Public, same auth posture as `GET api/track/page`.
- **Response**: `HomeStatsDto` with:
- `CutTrackCount` (int): total non-deleted tracks on Cut-medium releases.
- `CutReleaseTypeCounts` (`List<CutReleaseTypeCount>`): per-`ReleaseType` Cut release counts; zero-count types are absent (zero-suppressed server-side).
- `MixReleaseCount` (int): total non-deleted Mix-medium releases.
- `MixRuntimeSeconds` (double): sum of `DurationSeconds` across all non-deleted tracks on Mix releases (null durations count as 0).
- `TotalPlays` (long): site-wide total plays — sum of every `play_counter` row's bucket columns (`PartialCount + SampledCount + CompleteCount`), all-time (Phase 16). Zero until the play-telemetry migration is applied.
- `UniqueListeners` (int): site-wide distinct anonymous listeners — distinct non-null `anon_id` across all play events, all-time (Phase 16). Zero until the migration is applied.
- `StatsController` injects **both** `ITrackService` (track-domain aggregation — Cuts/Mixes cards) and `IEventService` (event-domain aggregation — Plays card). Neither domain reaches into the other's tables; the controller is the thin composition seam. Track-domain aggregation comes from `TrackRepository.GetHomeStatsAsync` via `ITrackService.GetHomeStats`; play/listener figures come from `IEventService.GetTotalPlayCount` and `IEventService.GetDistinctListenerCount` (Phase 16 wave 16.5). Play/listener reads are **best-effort**: a telemetry failure or not-yet-applied migration leaves those fields at 0 rather than failing the whole endpoint with 500.
- Returns 200 on success. Returns 500 if the track-domain aggregation fails.
## The event endpoints (Phase 16 anonymous telemetry)
Both endpoints are unauthenticated and rate-limited by the `"events"` fixed-window policy (30 requests / 60 s per IP, keyed on `Connection.RemoteIpAddress` after `UseForwardedHeaders()` resolves XFF). Returns `202 Accepted` — fire-and-forget contract; the `sendBeacon` client ignores the response. Controller: `EventController`.
### POST api/event/play (unauthenticated, rate-limited)
Records an anonymous play event. Client sends the track `EntryKey`, a completion bucket, and an optional `anonId` (wave 16.3); server-side release resolution joins track→release at write time (D4). The `anonId` is length-clamped server-side: whitespace-only / empty / null collapses to null (valid anonId-less event); a token longer than 64 chars returns `400` rather than being truncated (truncation would collide distinct listeners).
- **Body** (`PlayEventDto`): `{ "trackEntryKey": "...", "bucket": "partial"|"sampled"|"complete", "anonId": "..." }` (`anonId` optional — omitted when null).
- Validates: non-empty `trackEntryKey`; `bucket` must be a defined `PlayBucket` enum value.
- Delegates to `IEventService.RecordPlay`, which appends to `play_event` and bumps `play_counter`.
- Returns 202 on success. Returns 400 for missing/invalid fields. Returns 429 when the rate limit is exceeded. Returns 500 on a write failure (logged; beacon ignores it).
### POST api/event/share (unauthenticated, rate-limited)
Records an anonymous share event (a clipboard write from `SharePopover`).
- **Body** (`ShareEventDto`): `{ "targetKey": "...", "targetType": "track"|"release", "channel": "link"|"embed", "anonId": "..." }` (`anonId` optional — omitted when null; same length-clamp as the play endpoint).
- Validates: non-empty `targetKey`; defined `ShareTargetType` and `ShareChannel` enum values; `anonId` ≤ 64 chars (reject-not-truncate).
- Delegates to `IEventService.RecordShare`, which appends to `share_event`.
- Returns 202 on success. Returns 400 for missing/invalid fields. Returns 429 on rate limit. Returns 500 on write failure.
## ApiKey middleware behaviour
`ApiKeyAuthenticationMiddleware` runs on every request but only enforces on endpoints with `[ApiKeyAuthorize]` metadata.
@@ -308,6 +388,7 @@ Configured in `Startup.ConfigureDomainServices()`, applied to all endpoints via
5. Ensure the `images` vault exists (type `MediaVaultType.Image`, created on first boot if missing) via `InitializeImageVault`.
5a. Ensure the `track-waveforms` vault exists (type `MediaVaultType.Media`, created on first boot if missing) — holds per-track high-res visualizer datum keyed by `TrackEntity.EntryKey`.
6. Register singletons: `AudioProcessor`, `ImageProcessor`, `TrackService` (the `DeepDrftContent` version for vault operations), `WaveformProfileService`.
6a. **Upload staging directory** — resolve and create the on-disk staging directory (read `Upload:StagingPath`; if empty, default to a `staging` subdirectory under the FileDatabase vault path via `Startup.ResolveStagingPath`). Set the `ASPNETCORE_TEMP` env var to this directory before any request is served, relocating the framework's multipart file-section buffer (Layer 1) off the system temp mount. Register `UploadStagingDirectory` as a singleton so both `UploadTrack` and `ReplaceAudio` in `TrackController` stage to the same data-disk location (Layer 2) and never write to `/tmp` (a small RAM-backed tmpfs on the Linux host).
**In `Program.cs`** (SQL + AuthBlocks + wiring):
@@ -330,8 +411,9 @@ Mapped in `Development` only. Swagger UI at `/swagger` for testing endpoints loc
## Configuration files
- `appsettings.json`: Logging, hosting, CORS, and AuthBlocks config. **Does not contain secrets.**
- `appsettings.json`: Logging, hosting, CORS, AuthBlocks, and non-secret upload config. **Does not contain secrets.**
- `Logging`: standard ASP.NET structure.
- `Upload:StagingPath`: non-secret string. Empty default → a `staging` subdirectory under the FileDatabase vault path (on the data disk). Override to an absolute path when the vault default is not suitable. Consumed by `Startup.ResolveStagingPath`.
- `CorsSettings.AllowedOrigins`: array of origin URLs allowed to call the API (required; throws on startup if missing).
- `AuthBlocks:Jwt:Issuer`, `AuthBlocks:Jwt:Audience`: JWT validation settings (loaded from `environment/authblocks.json`).
- `environment/filedatabase.json` (required, loaded via CredentialTools, not in repo):
+114
View File
@@ -0,0 +1,114 @@
using DeepDrftData;
using DeepDrftModels.DTOs;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.RateLimiting;
namespace DeepDrftAPI.Controllers;
/// <summary>
/// Anonymous play/share telemetry intake (Phase 16 §2.2 / §4.3). Unauthenticated — same posture as the
/// public reads — but IP rate-limited (the "events" limiter, registered in Program.cs) and payload-
/// validated to make casual inflation annoying (§2.5). Both endpoints return <c>202 Accepted</c>: these
/// are fire-and-forget telemetry, not transactions, and the client (a <c>sendBeacon</c>) never reads the
/// response. The release dimension on a play is resolved server-side from the track key (§2.3 / D4).
/// The controller is a thin HTTP boundary; all write logic lives in <see cref="IEventService"/>.
/// </summary>
[ApiController]
[Route("api/event")]
[EnableRateLimiting("events")]
public class EventController : ControllerBase
{
// Reject oversized bodies before deserialization — a coarse abuse guard (§2.5). The legitimate
// payloads are a track key + an enum, well under 1 KB.
private const int MaxBodyBytes = 1024;
// The anonId is a client-minted GUID string (~36 chars); the anon_id column is varchar(64). Reject
// anything longer as malformed rather than silently truncating — an over-long token is either a bug
// or an inflation attempt, and a truncated id would corrupt the distinct-listener count by colliding
// distinct listeners onto one prefix. Whitespace-only is treated as absent.
private const int MaxAnonIdLength = 64;
private readonly IEventService _eventService;
private readonly ILogger<EventController> _logger;
public EventController(IEventService eventService, ILogger<EventController> logger)
{
_eventService = eventService;
_logger = logger;
}
// POST api/event/play (unauthenticated, rate-limited)
[HttpPost("play")]
[RequestSizeLimit(MaxBodyBytes)]
public async Task<ActionResult> RecordPlay([FromBody] PlayEventDto payload, CancellationToken ct = default)
{
// Reject a missing track key and an out-of-range bucket (§2.5). [ApiController] model binding
// already 400s a malformed/oversized body and an undefined enum value, but the explicit guards
// keep the contract obvious and cover the empty-string key the model binder lets through.
if (string.IsNullOrWhiteSpace(payload.TrackEntryKey))
return BadRequest("trackEntryKey is required");
if (!Enum.IsDefined(payload.Bucket))
return BadRequest("bucket is invalid");
if (!TryNormalizeAnonId(payload.AnonId, out var anonId))
return BadRequest("anonId is invalid");
var result = await _eventService.RecordPlay(payload.TrackEntryKey, payload.Bucket, anonId, ct);
if (!result.Success)
{
// A telemetry failure must never surface to the listener as an error they can act on, but
// we still log it and answer 5xx so a monitor can see the substrate is unhealthy. The
// beacon ignores the status either way.
_logger.LogWarning("RecordPlay failed: {Error}", result.Messages.FirstOrDefault()?.Message);
return StatusCode(500);
}
return Accepted();
}
// POST api/event/share (unauthenticated, rate-limited)
[HttpPost("share")]
[RequestSizeLimit(MaxBodyBytes)]
public async Task<ActionResult> RecordShare([FromBody] ShareEventDto payload, CancellationToken ct = default)
{
if (string.IsNullOrWhiteSpace(payload.TargetKey))
return BadRequest("targetKey is required");
if (!Enum.IsDefined(payload.TargetType))
return BadRequest("targetType is invalid");
if (!Enum.IsDefined(payload.Channel))
return BadRequest("channel is invalid");
if (!TryNormalizeAnonId(payload.AnonId, out var anonId))
return BadRequest("anonId is invalid");
var result = await _eventService.RecordShare(payload.TargetType, payload.TargetKey, payload.Channel, anonId, ct);
if (!result.Success)
{
_logger.LogWarning("RecordShare failed: {Error}", result.Messages.FirstOrDefault()?.Message);
return StatusCode(500);
}
return Accepted();
}
// Normalize an incoming anonId (wave 16.3): whitespace-only / empty / null collapses to a null token
// (the listener didn't send one, or storage was unavailable — a valid, anonId-less event). A token
// over the column width is rejected (400) rather than truncated, since truncation would collide
// distinct listeners. Returns false only on the over-long case; null and a valid token both pass.
private static bool TryNormalizeAnonId(string? raw, out string? anonId)
{
if (string.IsNullOrWhiteSpace(raw))
{
anonId = null;
return true;
}
var trimmed = raw.Trim();
if (trimmed.Length > MaxAnonIdLength)
{
anonId = null;
return false;
}
anonId = trimmed;
return true;
}
}
+4 -4
View File
@@ -112,10 +112,10 @@ public class ReleaseController : ControllerBase
}
// POST api/release/{id}/mix/waveform ([ApiKeyAuthorize], no body)
// Server-side trigger: fetch the Mix's track audio from the vault, compute a duration-derived high-res
// waveform via ComputeAndStoreHighResAsync, store it in the track-waveforms vault, and set
// MixMetadata.WaveformEntryKey. 404 when the release is missing or has no stored audio; 500 on
// compute/storage failure. Declared before "{id:long}".
// Server-side trigger: stream the Mix's track audio from the vault, compute a duration-derived
// high-res waveform, store it in the track-waveforms vault, and set MixMetadata.WaveformEntryKey.
// 404 when the release is missing or has no stored audio; 500 on compute/storage failure. Declared
// before "{id:long}".
[ApiKeyAuthorize]
[HttpPost("{id:long}/mix/waveform")]
public async Task<ActionResult> GenerateMixWaveform(long id, CancellationToken ct = default)
@@ -0,0 +1,59 @@
using DeepDrftData;
using Microsoft.AspNetCore.Mvc;
namespace DeepDrftAPI.Controllers;
[ApiController]
[Route("api/[controller]")]
public class StatsController : ControllerBase
{
private readonly ITrackService _sqlTrackService;
private readonly IEventService _eventService;
private readonly ILogger<StatsController> _logger;
public StatsController(
ITrackService sqlTrackService, IEventService eventService, ILogger<StatsController> logger)
{
_sqlTrackService = sqlTrackService;
_eventService = eventService;
_logger = logger;
}
// GET api/stats/home (unauthenticated)
// Aggregate figures behind the public home hero stat row — one read for all three cards. Same auth
// posture as the other public browse reads (GET api/track/page). The figures span two domains:
// the track-domain aggregation (Cuts/Mixes cards) lives in the SQL track service; the play-domain
// figures (Phase 16 Plays card — total plays + unique listeners) live in the event service. This
// controller is the thin composition seam that assembles both into one HomeStatsDto — neither
// domain reaches into the other's tables. Play/listener figures are best-effort: a telemetry read
// failure (or the not-yet-applied migration) leaves them at zero rather than failing the whole card.
[HttpGet("home")]
public async Task<ActionResult> GetHome(CancellationToken ct = default)
{
var trackResult = await _sqlTrackService.GetHomeStats(ct);
if (!trackResult.Success || trackResult.Value is null)
{
var error = trackResult.Messages.FirstOrDefault()?.Message ?? "Unknown error";
_logger.LogError("GetHome stats failed: {Error}", error);
return StatusCode(500, "Failed to load stats");
}
var stats = trackResult.Value;
var playsResult = await _eventService.GetTotalPlayCount(ct);
if (playsResult is { Success: true })
stats.TotalPlays = playsResult.Value;
else
_logger.LogWarning("GetHome total-plays read failed; Plays card falls back to 0: {Error}",
playsResult.Messages.FirstOrDefault()?.Message);
var listenersResult = await _eventService.GetDistinctListenerCount(ct);
if (listenersResult is { Success: true })
stats.UniqueListeners = listenersResult.Value;
else
_logger.LogWarning("GetHome unique-listeners read failed; secondary line falls back to 0: {Error}",
listenersResult.Messages.FirstOrDefault()?.Message);
return Ok(stats);
}
}
+380 -49
View File
@@ -5,6 +5,7 @@ using DeepDrftContent.Constants;
using DeepDrftContent.FileDatabase.Services;
using DeepDrftContent.FileDatabase.Models;
using DeepDrftContent.Processors;
using DeepDrftContent.Processors.Opus;
using DeepDrftData;
using DeepDrftModels.DTOs;
using DeepDrftModels.Enums;
@@ -20,6 +21,8 @@ public class TrackController : ControllerBase
private readonly UnifiedTrackService _unifiedService;
private readonly ITrackService _sqlTrackService;
private readonly WaveformProfileService _waveformProfileService;
private readonly TrackFormatResolver _formatResolver;
private readonly UploadStagingDirectory _stagingDirectory;
private readonly ILogger<TrackController> _logger;
// FileDatabase is injected directly for PutTrack because that endpoint receives a pre-processed
@@ -34,6 +37,8 @@ public class TrackController : ControllerBase
UnifiedTrackService unifiedService,
ITrackService sqlTrackService,
WaveformProfileService waveformProfileService,
TrackFormatResolver formatResolver,
UploadStagingDirectory stagingDirectory,
ILogger<TrackController> logger)
{
_trackContentService = trackContentService;
@@ -41,9 +46,49 @@ public class TrackController : ControllerBase
_unifiedService = unifiedService;
_sqlTrackService = sqlTrackService;
_waveformProfileService = waveformProfileService;
_formatResolver = formatResolver;
_stagingDirectory = stagingDirectory;
_logger = logger;
}
// Builds a unique staging file path on the data disk with the validated extension. The caller MUST
// assign this to the local that its finally block guards BEFORE calling StageUploadAsync — that
// way a mid-copy abort (OperationCanceledException, IO error) still triggers deletion of the
// partially-written file. Staging lives under UploadStagingDirectory, never Path.GetTempPath() —
// on the Linux host /tmp is a small tmpfs that cannot hold a large WAV.
private string BuildStagingPath(string uploadExtension) =>
Path.Combine(_stagingDirectory.Path, Guid.NewGuid().ToString("N") + uploadExtension);
// Streams an uploaded audio body to the pre-allocated staging path. The caller owns the path and
// must delete it in a finally block; separating path generation from the copy ensures the finally
// guard fires even when CopyToAsync throws before returning.
private async Task StageUploadAsync(
IFormFile audioFile, string stagingPath, CancellationToken cancellationToken)
{
await using var stagingStream = new FileStream(
stagingPath, FileMode.CreateNew, FileAccess.Write, FileShare.None,
bufferSize: 81920, useAsync: true);
await using var uploadStream = audioFile.OpenReadStream();
await uploadStream.CopyToAsync(stagingStream, cancellationToken);
}
// Best-effort removal of a staging file. Logs and swallows — a stranded staging file is a
// disk-hygiene concern, not a request failure.
private void DeleteStagingFile(string stagingPath)
{
try
{
if (System.IO.File.Exists(stagingPath))
{
System.IO.File.Delete(stagingPath);
}
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to delete staging file {StagingPath}", stagingPath);
}
}
// --- Literal-segment routes first ---
// These are declared before the parameterized "{trackId}" / "{id:long}" actions so route
// resolution never treats "page", "upload", or "meta" as a trackId.
@@ -96,6 +141,37 @@ public class TrackController : ControllerBase
return Ok(result.Value);
}
// GET api/track/release/exists?title=...&artist=... ([ApiKeyAuthorize])
// Upload-form pre-flight: does a release with this exact (title, artist) already exist? Returns the
// matching ReleaseDto (so the caller can name it in the block message) or 404 when none exists. Uses
// the same GetReleaseByTitleAndArtist read the upload create-path duplicate guard uses, so the
// pre-flight and the server backstop agree on the match by construction (exact ordinal comparison,
// soft-deleted rows excluded). "release/exists" is a literal 2-segment route declared before the
// parameterized "{trackId}" route and distinct from "release/{id:long}" (different segment shape).
[ApiKeyAuthorize]
[HttpGet("release/exists")]
public async Task<ActionResult> ReleaseExists(
[FromQuery] string? title,
[FromQuery] string? artist,
CancellationToken ct = default)
{
if (string.IsNullOrWhiteSpace(title) || string.IsNullOrWhiteSpace(artist))
return BadRequest("title and artist are both required");
var result = await _sqlTrackService.GetReleaseByTitleAndArtist(title, artist, ct);
if (!result.Success)
{
var error = result.Messages.FirstOrDefault()?.Message ?? "Unknown error";
_logger.LogError("ReleaseExists failed for ({Title}, {Artist}): {Error}", title, artist, error);
return StatusCode(500, "Failed to check release");
}
if (result.Value is null)
return NotFound();
return Ok(result.Value);
}
// GET api/track/genres (unauthenticated)
// Distinct non-null genres with track counts. Public browse data, same posture as GET
// api/track/page. Literal segment, declared before the parameterized "{trackId}" route.
@@ -173,19 +249,95 @@ public class TrackController : ControllerBase
return Ok(status);
}
// GET api/track/opus-status ([ApiKeyAuthorize])
// Admin Post-Processing view (18.6): returns every track with a flag for whether it carries a COMPLETE
// Opus artifact — both the Opus audio AND the seek/setup sidecar present (TrackFormatResolver.HasOpusAsync,
// the same completeness rule the 18.5 Backfill-Opus pass enqueues against; a half-derived track counts as
// missing). Mirrors GET waveform-status exactly: same ApiKey auth, same unpaged whole-catalogue shape, same
// literal-route placement before "{trackId}". The CMS reads it to show the Backfill-Opus "missing N" badge
// and to poll per-track Post-Processing status after an upload.
[ApiKeyAuthorize]
[HttpGet("opus-status")]
public async Task<ActionResult> GetOpusStatus()
{
var tracks = await _sqlTrackService.GetAll();
if (!tracks.Success || tracks.Value is null)
{
var error = tracks.Messages.FirstOrDefault()?.Message ?? "Unknown error";
_logger.LogError("GetOpusStatus failed to load tracks: {Error}", error);
return StatusCode(500, "Failed to load tracks");
}
var status = new List<OpusStatusDto>(tracks.Value.Count);
foreach (var track in tracks.Value)
{
status.Add(new OpusStatusDto
{
TrackId = track.Id,
EntryKey = track.EntryKey,
TrackName = track.TrackName,
HasOpus = await _formatResolver.HasOpusAsync(track.EntryKey),
});
}
return Ok(status);
}
// POST api/track/duration/backfill ([ApiKeyAuthorize], no body)
// One-time admin backfill: for every track whose SQL duration is still null, read the duration from
// the vault audio and write it to SQL. Mirrors the waveform backfill posture. Idempotent — a re-run
// only touches still-missing rows. Returns { updated, skipped }. Declared in the literal-route block
// (before "{trackId}") so the segment is never treated as a trackId.
[ApiKeyAuthorize]
[HttpPost("duration/backfill")]
public async Task<ActionResult> BackfillDurations(CancellationToken cancellationToken)
{
var result = await _unifiedService.BackfillDurationsAsync(cancellationToken);
if (!result.Success)
{
var error = result.Messages.FirstOrDefault()?.Message ?? "Unknown error";
_logger.LogError("BackfillDurations failed: {Error}", error);
return StatusCode(500, error);
}
return Ok(new { updated = result.Value.Updated, skipped = result.Value.Skipped });
}
// POST api/track/opus/backfill ([ApiKeyAuthorize], no body)
// Backfill-Opus (18.5, OQ4): enqueue a background Opus derive for every track lacking a complete Opus
// artifact (audio + sidecar). Mirrors the duration-backfill posture — enqueue-only and non-blocking, the
// transcodes run on the shared serial worker. Idempotent: a re-run only schedules tracks still missing
// Opus. Returns { enqueued, skipped }. Declared in the literal-route block (before "{trackId}") so the
// "opus/backfill" segment is never treated as a trackId; distinct shape from "{trackId}/opus" (per-track).
[ApiKeyAuthorize]
[HttpPost("opus/backfill")]
public async Task<ActionResult> BackfillOpus(CancellationToken cancellationToken)
{
var result = await _unifiedService.BackfillOpusAsync(cancellationToken);
if (!result.Success)
{
var error = result.Messages.FirstOrDefault()?.Message ?? "Unknown error";
_logger.LogError("BackfillOpus failed: {Error}", error);
return StatusCode(500, error);
}
return Ok(new { enqueued = result.Value.Enqueued, skipped = result.Value.Skipped });
}
// POST api/track/upload: raw audio in (multipart/form-data) + metadata → persisted TrackDto out.
// Accepts .wav, .mp3, and .flac. Used by the CMS upload flow on DeepDrftManager; that host
// proxies the upload here so it never touches the vault disk path or SQL directly.
// UnifiedTrackService owns the two-database write.
//
// RequestSizeLimit/MultipartBodyLengthLimit set to 1 GB: audio uploads can be tens to hundreds
// of MB and the framework defaults (~28 MB) reject them outright. The IFormFile path streams
// the body to a temp file once Kestrel surfaces it, so the limit is the per-request ceiling,
// not a buffered allocation.
// RequestSizeLimit/MultipartBodyLengthLimit set to ~1.86 GB: audio uploads can be tens to
// hundreds of MB (or over a GB for high-res WAVs); the framework defaults (~28 MB) reject them
// outright. The IFormFile path streams the body to a temp file once Kestrel surfaces it, so the
// limit is the per-request ceiling, not a buffered allocation. 2_000_000_000 stays below
// int.MaxValue (2,147,483,647) so it is safe where limits are int-typed.
[ApiKeyAuthorize]
[HttpPost("upload")]
[RequestSizeLimit(1_073_741_824)]
[RequestFormLimits(MultipartBodyLengthLimit = 1_073_741_824)]
[RequestSizeLimit(2_000_000_000)]
[RequestFormLimits(MultipartBodyLengthLimit = 2_000_000_000)]
public async Task<ActionResult<DeepDrftModels.DTOs.TrackDto>> UploadTrack(
[FromForm] IFormFile? audioFile,
[FromForm] string? trackName,
@@ -199,6 +351,7 @@ public class TrackController : ControllerBase
[FromForm] string? releaseType,
[FromForm] string? medium,
[FromForm] int? trackNumber,
[FromForm] long? releaseId,
CancellationToken cancellationToken)
{
_logger.LogInformation("UploadTrack called: trackName={TrackName}, artist={Artist}, fileName={FileName}, size={Size}",
@@ -266,23 +419,15 @@ public class TrackController : ControllerBase
var resolvedTrackNumber = trackNumber is > 0 ? trackNumber.Value : 1;
// The processor router selects by extension and reads from disk, so the temp file must carry
// the upload's real extension. Path.GetTempFileName() yields .tmp, which the router rejects —
// generate our own path preserving the validated .wav/.mp3/.flac extension.
var tempPath = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N") + uploadExtension);
// Build the staging path before the copy so the finally block can delete the partial file
// even if CopyToAsync throws mid-stream (client cancellation, disk-full, IO error).
var stagingPath = BuildStagingPath(uploadExtension);
try
{
await using (var tempStream = new FileStream(
tempPath, FileMode.CreateNew, FileAccess.Write, FileShare.None,
bufferSize: 81920, useAsync: true))
await using (var uploadStream = audioFile.OpenReadStream())
{
await uploadStream.CopyToAsync(tempStream, cancellationToken);
}
await StageUploadAsync(audioFile, stagingPath, cancellationToken);
var result = await _unifiedService.UploadAsync(
tempPath,
stagingPath,
trackName,
artist,
string.IsNullOrWhiteSpace(album) ? null : album,
@@ -294,6 +439,7 @@ public class TrackController : ControllerBase
parsedReleaseType,
parsedMedium,
resolvedTrackNumber,
releaseId,
cancellationToken);
if (!result.Success || result.Value is null)
@@ -301,14 +447,19 @@ public class TrackController : ControllerBase
var error = result.Messages.FirstOrDefault()?.Message ?? "Failed to process and store audio";
_logger.LogWarning("UploadTrack: UnifiedTrackService failed for {TrackName}: {Error}", trackName, error);
// A cardinality rejection is a well-formed request that violates a domain rule, so it
// is 409 Conflict — distinct from the 500 used for processing failure. The marker is
// stripped so the client sees only the human-readable detail.
// A cardinality or duplicate-release rejection is a well-formed request that violates a
// domain rule, so it is 409 Conflict — distinct from the 500 used for processing failure.
// The marker is stripped so the client sees only the human-readable detail.
if (error.StartsWith(UnifiedTrackService.CardinalityViolationMarker, StringComparison.Ordinal))
{
return Conflict(error[UnifiedTrackService.CardinalityViolationMarker.Length..]);
}
if (error.StartsWith(UnifiedTrackService.DuplicateReleaseMarker, StringComparison.Ordinal))
{
return Conflict(error[UnifiedTrackService.DuplicateReleaseMarker.Length..]);
}
return StatusCode(500, error);
}
@@ -322,17 +473,7 @@ public class TrackController : ControllerBase
}
finally
{
try
{
if (System.IO.File.Exists(tempPath))
{
System.IO.File.Delete(tempPath);
}
}
catch (Exception ex)
{
_logger.LogWarning(ex, "UploadTrack: failed to delete temp file {TempPath}", tempPath);
}
DeleteStagingFile(stagingPath);
}
}
@@ -479,6 +620,69 @@ public class TrackController : ControllerBase
return StatusCode(500, error);
}
// POST api/track/{id}/replace-audio ([ApiKeyAuthorize])
// Swap an existing track's audio bytes from a raw upload, preserving the track's id, EntryKey,
// release membership, position, and metadata. UnifiedTrackService.ReplaceAudioAsync owns the
// vault swap + waveform regen; nothing in SQL is written. Mirrors the upload endpoint's temp-file
// streaming and ~1.86 GB ceiling (a WAV replace is a large-body upload like the original). The
// literal "{id:long}/replace-audio" segment is declared in the literal-route block so it never
// resolves to the parameterized "{trackId}" GET.
[ApiKeyAuthorize]
[HttpPost("{id:long}/replace-audio")]
[RequestSizeLimit(2_000_000_000)]
[RequestFormLimits(MultipartBodyLengthLimit = 2_000_000_000)]
public async Task<ActionResult> ReplaceAudio(
long id,
[FromForm] IFormFile? audioFile,
CancellationToken cancellationToken)
{
_logger.LogInformation("ReplaceAudio called: id={Id}, size={Size}", id, audioFile?.Length);
if (audioFile is null || audioFile.Length == 0)
{
return BadRequest("Audio file is required");
}
var uploadExtension = Path.GetExtension(audioFile.FileName).ToLowerInvariant();
if (uploadExtension is not (".wav" or ".mp3" or ".flac"))
{
return BadRequest("Uploaded file must have a .wav, .mp3, or .flac extension");
}
// Build the staging path before the copy so the finally block can delete the partial file
// even if CopyToAsync throws mid-stream (client cancellation, disk-full, IO error).
var stagingPath = BuildStagingPath(uploadExtension);
try
{
await StageUploadAsync(audioFile, stagingPath, cancellationToken);
var result = await _unifiedService.ReplaceAudioAsync(id, stagingPath, cancellationToken);
if (result.Success)
{
_logger.LogInformation("ReplaceAudio succeeded: id={Id}", id);
return Ok();
}
var error = result.Messages.FirstOrDefault()?.Message ?? "Failed to replace audio";
if (string.Equals(error, UnifiedTrackService.TrackNotFoundMessage, StringComparison.Ordinal))
{
return NotFound();
}
_logger.LogError("ReplaceAudio failed for id {Id}: {Error}", id, error);
return StatusCode(500, error);
}
catch (Exception ex)
{
_logger.LogError(ex, "ReplaceAudio failed for id {Id}", id);
return StatusCode(500, "Internal server error");
}
finally
{
DeleteStagingFile(stagingPath);
}
}
// DELETE api/track/release/{id} ([ApiKeyAuthorize])
// Soft-delete a release row directly. Used by the albums browser to remove an orphaned release
// (one with no live tracks). "release" is a literal segment, declared here in the literal-route
@@ -497,10 +701,28 @@ public class TrackController : ControllerBase
// --- Parameterized routes ---
// GET api/track/{trackId}?format=opus|lossless (unauthenticated)
// Streams the track's audio bytes with HTTP Range support. The optional `format` selector (Phase 18.3)
// picks the delivery rendering: absent or unrecognized ⇒ Lossless (byte-identical to pre-Phase-18 —
// the existing zero-copy disk-stream path, untouched); `opus` ⇒ the derived Ogg Opus 320 artifact
// when present, falling back to lossless when it is not (C2 — never 404/silence). The Opus path streams
// the resolved artifact from a seekable disk FileStream via File(..., enableRangeProcessing: true) —
// no whole-file byte[] — so Range: bytes=X- still yields 206 (load-bearing for streaming + seek),
// matching the lossless disk-stream's range behavior.
[HttpGet("{trackId}")]
public async Task<ActionResult> GetTrack(string trackId)
public async Task<ActionResult> GetTrack(string trackId, [FromQuery] string? format = null)
{
_logger.LogInformation("GetTrack called with trackId: {TrackId}", trackId);
_logger.LogInformation("GetTrack called with trackId: {TrackId}, format: {Format}", trackId, format);
// Only `opus` diverges from today's behavior; everything else (null, "lossless", garbage) takes the
// unchanged lossless disk-stream path below, preserving the large-file zero-copy streaming. Routing
// lossless through the resolver would force the whole source (up to ~1 GB) into memory per request —
// a regression the resolver's in-memory byte[] result is fine for Opus (small) but not for lossless.
if (Enum.TryParse<AudioFormat>(format, ignoreCase: true, out var requestedFormat)
&& requestedFormat == AudioFormat.Opus)
{
return await GetTrackOpusAsync(trackId);
}
try
{
@@ -519,7 +741,7 @@ public class TrackController : ControllerBase
}
// Resolve MIME and log before handing the stream to File().
// If anything here throws, the finally block disposes the wrapper
// If anything here throws, the catch block disposes the wrapper
// (and its inner FileStream) so neither leaks. On the success path
// File() takes ownership of the inner stream; ASP.NET Core disposes
// it after the response body is sent. The wrapper is a thin struct
@@ -533,16 +755,15 @@ public class TrackController : ControllerBase
streamMimeType = MimeTypeExtensions.GetMimeType(mediaStream.Extension);
streamLength = mediaStream.Stream.Length;
innerStream = mediaStream.Stream;
_logger.LogInformation(
"Streaming track from disk: {TrackId}, Size: {Size} bytes",
trackId, streamLength);
}
catch
{
await mediaStream.DisposeAsync();
throw;
}
_logger.LogInformation(
"Streaming track from disk: {TrackId}, Size: {Size} bytes",
trackId, streamLength);
// enableRangeProcessing: true — seek is served by HTTP Range requests.
// The FileStream is seekable, so ASP.NET Core honours an incoming
// Range header by slicing the file and responding 206 Partial Content.
@@ -555,6 +776,76 @@ public class TrackController : ControllerBase
}
}
// The ?format=opus arm of GetTrack. Resolves the Opus artifact (or the lossless fallback when none
// exists, C2) via TrackFormatResolver and streams the resolved bytes from a seekable disk FileStream —
// never a whole-file byte[] (a ~220 MB Opus / ~970 MB lossless managed allocation per request was the
// read-path OOM defect this closes). enableRangeProcessing:true is load-bearing: the seekable FileStream
// lets ASP.NET honour Range: bytes=X- with a 206 slice (seek + Phase 21 windowing). The resolver reports
// the *actually-served* format via ResolvedAudio, so the content-type matches the bytes (audio/ogg on a
// hit, the source MIME on a fallback) and the eventual client decoder dispatches correctly.
//
// Disposal mirrors the lossless GetTrack path exactly: File() takes ownership of the stream on success
// and disposes it after the response; the inner try disposes ResolvedAudio (and its FileStream) on any
// pre-handoff throw so a handle never leaks.
private async Task<ActionResult> GetTrackOpusAsync(string trackId)
{
try
{
var resolved = await _formatResolver.ResolveAsync(trackId, AudioFormat.Opus);
if (resolved is null)
{
_logger.LogWarning("Track not found for Opus request: {TrackId}", trackId);
return NotFound();
}
string contentType;
long streamLength;
Stream innerStream;
try
{
contentType = resolved.ContentType;
// Length from the seekable FileStream — a metadata read, not a body load.
streamLength = resolved.Stream.Length;
innerStream = resolved.Stream;
_logger.LogInformation(
"Streaming track {TrackId} as {Format} ({Size} bytes, {ContentType})",
trackId, resolved.ResolvedFormat, streamLength, contentType);
}
catch
{
await resolved.DisposeAsync();
throw;
}
return File(innerStream, contentType, enableRangeProcessing: true);
}
catch (Exception ex)
{
_logger.LogError(ex, "Error retrieving track as Opus: {TrackId}", trackId);
return StatusCode(500, "Internal server error");
}
}
// GET api/track/{trackId}/opus/seekdata (unauthenticated)
// Returns the Opus setup-header + granule→byte seek-index sidecar bytes (Phase 18.3). The client
// fetches this once on track load and parses it into OpusSeekData (18.4) before issuing any Opus seek.
// Raw octet-stream — the bytes are the OpusSidecar blob exactly as 18.1 stored them. 404 when no sidecar
// is stored (no Opus artifact yet, or an older derive predating the sidecar); the client then degrades
// to lossless, mirroring the C2 posture of the audio path. Same public auth posture as the audio stream.
// The "opus/seekdata" literal suffix keeps this distinct from the audio and waveform routes.
[HttpGet("{trackId}/opus/seekdata")]
public async Task<ActionResult> GetOpusSeekData(string trackId)
{
var sidecar = await _formatResolver.GetOpusSidecarAsync(trackId);
if (sidecar is null)
{
_logger.LogInformation("No Opus sidecar for track: {TrackId}", trackId);
return NotFound();
}
return File(sidecar, "application/octet-stream");
}
// GET api/track/{trackId}/waveform (unauthenticated)
// Returns the stored waveform loudness profile for a track, base64-encoded. Public listener
// data, same auth posture as GET api/track/{trackId} streaming. 404 when no profile is stored
@@ -611,15 +902,18 @@ public class TrackController : ControllerBase
[HttpPost("{trackId}/waveform")]
public async Task<ActionResult> GenerateWaveform(string trackId)
{
var audio = await _trackContentService.GetAudioBinaryAsync(trackId);
if (audio is null)
// Streaming compute (Wave 2): the WAV is read from the vault in bounded chunks, never buffered
// whole. Tri-state: null = no vault audio (404), false = present but uncomputable / write failed
// (500), true = stored.
var stored = await _waveformProfileService.ComputeAndStoreProfileStreamingAsync(
_ => _trackContentService.OpenAudioStreamAsync(trackId), trackId, HttpContext.RequestAborted);
if (stored is null)
{
_logger.LogWarning("GenerateWaveform: no audio in vault for {TrackId}", trackId);
return NotFound();
}
var stored = await _waveformProfileService.ComputeAndStoreAsync(audio.Buffer, trackId);
if (!stored)
if (stored is false)
{
_logger.LogError("GenerateWaveform: profile computation/storage failed for {TrackId}", trackId);
return StatusCode(500, "Failed to generate waveform profile.");
@@ -639,16 +933,27 @@ public class TrackController : ControllerBase
[HttpPost("{trackId}/waveform/high-res")]
public async Task<ActionResult> GenerateHighResWaveform(string trackId)
{
var audio = await _trackContentService.GetAudioBinaryAsync(trackId);
if (audio is null)
// The high-res bucket count is duration-derived. Read the duration from the vault index metadata
// (no body load); its absence means the track has no vault audio → 404.
var duration = await _trackContentService.GetAudioDurationAsync(trackId);
if (duration is null)
{
_logger.LogWarning("GenerateHighResWaveform: no audio in vault for {TrackId}", trackId);
return NotFound();
}
var stored = await _waveformProfileService.ComputeAndStoreHighResAsync(
audio.Buffer, trackId, audio.Duration);
if (!stored)
// Streaming compute (Wave 2): bounded read of the vault WAV. Tri-state mapping as in
// GenerateWaveform — null (entry vanished between the metadata read and the compute) → 404.
var stored = await _waveformProfileService.ComputeAndStoreHighResStreamingAsync(
_ => _trackContentService.OpenAudioStreamAsync(trackId), trackId, duration.Value,
HttpContext.RequestAborted);
if (stored is null)
{
_logger.LogWarning("GenerateHighResWaveform: no audio in vault for {TrackId}", trackId);
return NotFound();
}
if (stored is false)
{
_logger.LogError("GenerateHighResWaveform: computation/storage failed for {TrackId}", trackId);
return StatusCode(500, "Failed to generate high-res waveform datum.");
@@ -657,6 +962,32 @@ public class TrackController : ControllerBase
return Ok();
}
// POST api/track/{trackId}/opus ([ApiKeyAuthorize])
// Per-track Opus (re)derive trigger (18.5): schedule a single track's background transcode. Enqueue-only
// and non-blocking — the transcode runs on the shared serial worker; this returns as soon as it is
// scheduled. Re-runnable: overwrites any prior artifact in place. trackId is the EntryKey. 404 when the
// track id is unknown. The "opus" literal suffix keeps this distinct from the audio/waveform routes and
// from the parameterized PUT "{trackId}". Returns 202 Accepted — the work is queued, not done inline.
[ApiKeyAuthorize]
[HttpPost("{trackId}/opus")]
public async Task<ActionResult> GenerateOpus(string trackId, CancellationToken cancellationToken)
{
var result = await _unifiedService.EnqueueOpusAsync(trackId, cancellationToken);
if (result.Success)
{
return Accepted();
}
var error = result.Messages.FirstOrDefault()?.Message ?? "Unknown error";
if (string.Equals(error, UnifiedTrackService.TrackNotFoundMessage, StringComparison.Ordinal))
{
return NotFound();
}
_logger.LogError("GenerateOpus failed for {TrackId}: {Error}", trackId, error);
return StatusCode(500, error);
}
[ApiKeyAuthorize]
[HttpPut("{trackId}")]
public async Task<ActionResult> PutTrack(string trackId, [FromBody] AudioBinaryDto track)
+8 -1
View File
@@ -15,7 +15,13 @@
<PackageReference Include="Npgsql.EntityFrameworkCore.PostgreSQL" Version="10.0.1" />
<!-- AuthBlocks API host surface: AddAuthBlocks / MapAuthBlocks / UseAuthBlocksStartupAsync.
The Manager keeps only Cerebellum.AuthBlocks.Web (web-side auth, no signing secret). -->
<PackageReference Include="Cerebellum.AuthBlocks" Version="10.3.33" />
<PackageReference Include="Cerebellum.AuthBlocks" Version="10.3.39" />
</ItemGroup>
<ItemGroup>
<!-- Exposes the internal 409 markers (CardinalityViolationMarker / DuplicateReleaseMarker) to the
test suite so UploadDuplicateDetectionTests can assert the orchestrator's rejection contract. -->
<InternalsVisibleTo Include="DeepDrftTests" />
</ItemGroup>
<ItemGroup>
@@ -26,3 +32,4 @@
</Project>
+13
View File
@@ -0,0 +1,13 @@
namespace DeepDrftAPI.Models
{
/// <summary>
/// Non-secret upload tunables. <see cref="StagingPath"/> is the directory used to stage the raw
/// audio body during upload/replace-audio. It must live on the data disk, never the system temp
/// mount (on the Linux host <c>/tmp</c> is a small RAM-backed tmpfs that cannot hold a multi-hundred-MB
/// WAV). When null/empty it defaults to a "staging" subdirectory under the FileDatabase vault path.
/// </summary>
public class UploadSettings
{
public string? StagingPath { get; set; }
}
}
@@ -0,0 +1,10 @@
namespace DeepDrftAPI.Models
{
/// <summary>
/// The resolved, on-disk staging directory for upload/replace-audio bodies. Resolved once at
/// startup from <see cref="UploadSettings"/> (or the vault path default) and guaranteed to exist.
/// Injected into <c>TrackController</c> so the upload path never stages on the system temp mount.
/// A typed wrapper rather than a bare string so DI resolves it unambiguously.
/// </summary>
public sealed record UploadStagingDirectory(string Path);
}
+47 -2
View File
@@ -4,12 +4,15 @@ using DeepDrftAPI;
using DeepDrftAPI.Middleware;
using DeepDrftAPI.Models;
using DeepDrftAPI.Services;
using DeepDrftAPI.Services.Opus;
using DeepDrftData;
using DeepDrftData.Data;
using DeepDrftData.Repositories;
using Microsoft.AspNetCore.HttpOverrides;
using Microsoft.AspNetCore.RateLimiting;
using Microsoft.EntityFrameworkCore;
using NetBlocks.Utilities.Environment;
using System.Threading.RateLimiting;
// Required credential files — must exist before the app will start.
// Production secrets stay gitignored; the *.example.json templates at the project root show the shape.
@@ -64,6 +67,21 @@ builder.Services
.AddScoped<ITrackService>(sp => sp.GetRequiredService<TrackManager>());
builder.Services.AddScoped<UnifiedTrackService>();
// Background Opus transcode (Phase 18.1, OQ6). One singleton is both the enqueue seam
// (IOpusTranscodeQueue, injected into the scoped UnifiedTrackService) and the hosted drain loop
// (IHostedService). It resolves OpusTranscodeService — a singleton — so no scope is captured.
builder.Services.AddSingleton<OpusTranscodeBackgroundService>();
builder.Services.AddSingleton<IOpusTranscodeQueue>(sp => sp.GetRequiredService<OpusTranscodeBackgroundService>());
builder.Services.AddHostedService(sp => sp.GetRequiredService<OpusTranscodeBackgroundService>());
// Phase 16 anonymous telemetry — append-only event logs + incremental play-counter rollup (all SQL).
// EventManager is the IEventService boundary; EventRepository owns the EF writes and the
// release-resolution + counter-bump transaction.
builder.Services
.AddScoped<EventRepository>()
.AddScoped<EventManager>()
.AddScoped<IEventService>(sp => sp.GetRequiredService<EventManager>());
// Release domain — medium-aware read projection + satellite metadata writes. ReleaseManager is the
// IReleaseService implementation; UnifiedReleaseService orchestrates the vault + SQL satellite writes.
builder.Services
@@ -93,10 +111,13 @@ builder.Services.AddAuthBlocks(options =>
options.JwtSettings.Audience = builder.Configuration["AuthBlocks:Jwt:Audience"]
?? throw new InvalidOperationException("AuthBlocks:Jwt:Audience is required");
options.EmailConnection.Host = builder.Configuration["AuthBlocks:Email:Host"]
options.EmailConnection.Host = builder.Configuration["AuthBlocks:Email:Host"]
?? throw new InvalidOperationException("AuthBlocks:Email:Host is required");
options.EmailConnection.Token = builder.Configuration["AuthBlocks:Email:Token"]
options.EmailConnection.Token = builder.Configuration["AuthBlocks:Email:Token"]
?? throw new InvalidOperationException("AuthBlocks:Email:Token is required");
options.EmailConnection.FromAddress = builder.Configuration["AuthBlocks:Email:From"]
?? throw new InvalidOperationException("AuthBlocks:Email:From is required");
options.EmailConnection.TestInbox = builder.Configuration["AuthBlocks:Email:TestInbox"];
options.AdminUserSettings = new AdminUserSettings
{
@@ -118,6 +139,25 @@ builder.Services.Configure<ForwardedHeadersOptions>(options =>
options.KnownProxies.Clear();
});
// Per-IP rate limiting for the anonymous telemetry intake (Phase 16 §2.5). Coarse and stateless —
// a fixed window keyed by the (forwarded) remote IP. The substrate sits behind nginx, so the real
// client IP is the X-Forwarded-For value UseForwardedHeaders resolves into Connection.RemoteIpAddress.
// On limit, reject with 429 (the beacon ignores it; this only blunts casual inflation). The 30-window
// budget is generous for a real listening session and only bites on scripted spam.
builder.Services.AddRateLimiter(options =>
{
options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;
options.AddPolicy("events", httpContext =>
RateLimitPartition.GetFixedWindowLimiter(
partitionKey: httpContext.Connection.RemoteIpAddress?.ToString() ?? "unknown",
factory: _ => new FixedWindowRateLimiterOptions
{
PermitLimit = 30,
Window = TimeSpan.FromMinutes(1),
QueueLimit = 0,
}));
});
var app = builder.Build();
// Apply AuthBlocks EF migrations, seed system roles, seed admin user on first boot.
@@ -136,6 +176,11 @@ if (app.Environment.IsDevelopment())
app.UseCors("ContentApiPolicy");
// Rate limiter must sit in the pipeline for the [EnableRateLimiting("events")] attribute on
// EventController to take effect. Only the telemetry endpoints carry the policy; everything else is
// unaffected (no global limiter is set).
app.UseRateLimiter();
// ApiKey middleware only enforces on endpoints tagged [ApiKeyAuthorize] (the track surface); it
// passes all other endpoints through. JWT auth/authorization gate the AuthBlocks endpoints, which
// carry no [ApiKeyAuthorize] metadata — the two schemes are orthogonal and do not interfere.
@@ -0,0 +1,18 @@
namespace DeepDrftAPI.Services.Opus;
/// <summary>
/// The enqueue seam for the background Opus transcode (OQ6 / §3.1a). <see cref="UnifiedTrackService"/>
/// depends only on this thin interface — not on the worker — so adding the background derive to the
/// upload/replace paths costs one small dependency, not the whole transcode graph. Enqueuing is
/// non-blocking and best-effort: a freshly uploaded track is already persisted and playable losslessly
/// before anything is enqueued, and the transcode runs off the request thread.
/// </summary>
public interface IOpusTranscodeQueue
{
/// <summary>
/// Schedules a background Opus derive for the track identified by <paramref name="entryKey"/>. Returns
/// immediately. A dropped or failed enqueue must not affect the caller — the track remains
/// lossless-only and eligible for backfill.
/// </summary>
void Enqueue(string entryKey);
}
@@ -0,0 +1,72 @@
using System.Threading.Channels;
using DeepDrftContent.Processors.Opus;
namespace DeepDrftAPI.Services.Opus;
/// <summary>
/// The background worker behind <see cref="IOpusTranscodeQueue"/> (OQ6 / §3.1a). An unbounded in-process
/// channel buffers EntryKeys enqueued by the upload and replace-audio paths; a single hosted loop drains
/// them one at a time and runs <see cref="OpusTranscodeService.TranscodeAndStoreAsync"/> for each. Serial
/// by design — a transcode is CPU-heavy (§3.1), so running them concurrently would starve request
/// handling; one-at-a-time keeps the derive strictly off the hot path without saturating the host.
///
/// This worker IS the queue (implements <see cref="IOpusTranscodeQueue"/>) so enqueue and drain share one
/// channel with no extra indirection. It is registered as a singleton and surfaced under both the
/// interface and <see cref="IHostedService"/>.
/// </summary>
public sealed class OpusTranscodeBackgroundService : BackgroundService, IOpusTranscodeQueue
{
private readonly Channel<string> _channel =
Channel.CreateUnbounded<string>(new UnboundedChannelOptions { SingleReader = true });
private readonly OpusTranscodeService _transcodeService;
private readonly ILogger<OpusTranscodeBackgroundService> _logger;
public OpusTranscodeBackgroundService(
OpusTranscodeService transcodeService,
ILogger<OpusTranscodeBackgroundService> logger)
{
_transcodeService = transcodeService;
_logger = logger;
}
public void Enqueue(string entryKey)
{
if (string.IsNullOrWhiteSpace(entryKey))
return;
if (!_channel.Writer.TryWrite(entryKey))
{
// Unbounded writer only rejects after Complete(), i.e. during shutdown. The track stays
// lossless-only and is eligible for backfill, so a dropped enqueue is non-fatal — log it.
_logger.LogWarning("Opus transcode: could not enqueue {EntryKey} (queue closed).", entryKey);
}
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
await foreach (var entryKey in _channel.Reader.ReadAllAsync(stoppingToken))
{
try
{
await _transcodeService.TranscodeAndStoreAsync(entryKey, stoppingToken);
}
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
{
break; // host shutting down
}
catch (Exception ex)
{
// TranscodeAndStoreAsync already swallows expected failures; this guards the loop against
// anything unexpected so one bad track never kills the worker.
_logger.LogError(ex, "Opus transcode: unhandled failure draining {EntryKey}; worker continues.", entryKey);
}
}
}
public override Task StopAsync(CancellationToken cancellationToken)
{
_channel.Writer.TryComplete();
return base.StopAsync(cancellationToken);
}
}
+15 -6
View File
@@ -143,8 +143,9 @@ public class UnifiedReleaseService
return Result.CreateFailResult(MixHasNoTrackMessage);
}
var audio = await _trackContentService.GetAudioBinaryAsync(entryKey);
if (audio is null)
// Duration from the vault index metadata (no body load); its absence means no vault audio.
var duration = await _trackContentService.GetAudioDurationAsync(entryKey);
if (duration is null)
{
_logger.LogWarning("TriggerMixWaveform: no audio in vault for {EntryKey} (release {ReleaseId})", entryKey, releaseId);
return Result.CreateFailResult(MixTrackNoAudioMessage);
@@ -152,10 +153,18 @@ public class UnifiedReleaseService
// Duration-derived, constant-time-resolution capture (≈333 samples/sec) so long mixes are not
// under-sampled by a fixed bucket count — see WaveformResolution / spec §F. Same per-track
// high-res datum every track now carries (phase-12 §5).
var computed = await _waveformProfileService.ComputeAndStoreHighResAsync(
audio.Buffer, entryKey, audio.Duration);
if (!computed)
// high-res datum every track now carries (phase-12 §5). Streamed from the vault in bounded
// chunks (Wave 2): a ~GB mix is never buffered whole. Tri-state — null = entry vanished after
// the metadata read; false = uncomputable / write failed.
var computed = await _waveformProfileService.ComputeAndStoreHighResStreamingAsync(
_ => _trackContentService.OpenAudioStreamAsync(entryKey), entryKey, duration.Value, ct);
if (computed is null)
{
_logger.LogWarning("TriggerMixWaveform: no audio in vault for {EntryKey} (release {ReleaseId})", entryKey, releaseId);
return Result.CreateFailResult(MixTrackNoAudioMessage);
}
if (computed is false)
{
_logger.LogError("TriggerMixWaveform: waveform computation/storage failed for {EntryKey}", entryKey);
return Result.CreateFailResult("Failed to compute the Mix waveform.");
+303 -36
View File
@@ -1,6 +1,8 @@
using DeepDrftAPI.Services.Opus;
using DeepDrftContent;
using DeepDrftContent.Constants;
using DeepDrftContent.Processors;
using DeepDrftContent.Processors.Opus;
using DeepDrftData;
using DeepDrftModels.DTOs;
using DeepDrftModels.Enums;
@@ -25,10 +27,22 @@ public class UnifiedTrackService
/// follows the marker and is what the CMS surfaces to the admin.
/// </summary>
internal const string CardinalityViolationMarker = "CARDINALITY_VIOLATION: ";
/// <summary>
/// Stable marker prefixed onto a duplicate-release rejection so the controller can map it to 409
/// Conflict, the same way <see cref="CardinalityViolationMarker"/> is mapped. Fires when an upload
/// with no explicit releaseId would create a release whose (title, artist) already exists in the
/// catalogue — the upload form is a create-new tool, never an edit/append path. The human-readable
/// detail follows the marker and is what the CMS surfaces to the admin.
/// </summary>
internal const string DuplicateReleaseMarker = "DUPLICATE_RELEASE: ";
private readonly TrackContentService _contentTrackContentService;
private readonly ITrackService _sqlTrackService;
private readonly FileDb _fileDatabase;
private readonly WaveformProfileService _waveformProfileService;
private readonly IOpusTranscodeQueue _opusTranscodeQueue;
private readonly TrackFormatResolver _formatResolver;
private readonly ILogger<UnifiedTrackService> _logger;
public UnifiedTrackService(
@@ -36,12 +50,16 @@ public class UnifiedTrackService
ITrackService sqlTrackService,
FileDb fileDatabase,
WaveformProfileService waveformProfileService,
IOpusTranscodeQueue opusTranscodeQueue,
TrackFormatResolver formatResolver,
ILogger<UnifiedTrackService> logger)
{
_contentTrackContentService = contentTrackContentService;
_sqlTrackService = sqlTrackService;
_fileDatabase = fileDatabase;
_waveformProfileService = waveformProfileService;
_opusTranscodeQueue = opusTranscodeQueue;
_formatResolver = formatResolver;
_logger = logger;
}
@@ -64,38 +82,71 @@ public class UnifiedTrackService
ReleaseType releaseType,
ReleaseMedium medium,
int trackNumber,
long? releaseId,
CancellationToken ct)
{
// Cardinality pre-check — BEFORE the vault write so a rejected over-limit add never orphans
// audio in the tracks vault. This is a READ-only peek (no release is created for an upload we
// may reject); the real FindOrCreateRelease still runs below for the accepted path. Only the
// find path can violate: a release that does not yet exist has zero tracks and admits its
// first. The guard is the general form `(liveCount + 1) > Max`, not Session/Mix-hardcoded, so
// a future bounded medium is covered by the same line.
// Resolve which release this track lands on BEFORE the vault write, so a rejected upload never
// orphans audio. Two paths:
// - releaseId is null → CREATE path: this is the first row of a submit. (title, artist) must
// NOT already exist — the upload form creates new releases only. A pre-existing match is a
// duplicate and is blocked (409).
// - releaseId is set → ATTACH path: rows 2..N of a within-batch multi-track Cut, attaching
// to the release row 1 just created. No (title, artist) lookup — the release id is
// authoritative — so the within-batch build is never mistaken for a pre-existing duplicate.
// Both paths run the cardinality guard `(liveCount + 1) > Max` (not Session/Mix-hardcoded, so a
// future bounded medium is covered by the same line).
ResolvedRelease? resolved = null;
if (!string.IsNullOrWhiteSpace(album))
{
var peek = await _sqlTrackService.GetReleaseByTitleAndArtist(album, artist, ct);
if (!peek.Success)
if (releaseId is { } attachId)
{
var error = peek.Messages.FirstOrDefault()?.Message ?? "Unknown error";
_logger.LogError("UploadAsync: release peek failed for ({Album}, {Artist}): {Error}", album, artist, error);
return ResultContainer<TrackDto>.CreateFailResult($"Could not verify the release: {error}");
}
var attachPeek = await _sqlTrackService.GetReleaseByTitleAndArtist(album, artist, ct);
if (!attachPeek.Success)
{
var error = attachPeek.Messages.FirstOrDefault()?.Message ?? "Unknown error";
_logger.LogError("UploadAsync: release peek failed for ({Album}, {Artist}): {Error}", album, artist, error);
return ResultContainer<TrackDto>.CreateFailResult($"Could not verify the release: {error}");
}
if (peek.Value is { } existing)
{
var cardinality = MediumRules.CardinalityOf(existing.Medium);
if (existing.TrackCount + 1 > cardinality.Max)
// The attach target must be the same release the natural key resolves to — a guard against
// a stale/forged releaseId pointing at a different (title, artist) than this row carries.
if (attachPeek.Value is not { } target || target.Id != attachId)
{
return ResultContainer<TrackDto>.CreateFailResult(
$"{CardinalityViolationMarker}A {existing.Medium} release holds a single track; " +
$"'{existing.Title}' already has one — edit the existing track or choose a different release.");
$"{DuplicateReleaseMarker}The release this track should attach to could not be found. " +
"Start the upload again.");
}
var cardinalityCheck = CheckCardinality(target);
if (cardinalityCheck is { } violation)
return ResultContainer<TrackDto>.CreateFailResult(violation);
resolved = new ResolvedRelease(target.Id);
}
else
{
var peek = await _sqlTrackService.GetReleaseByTitleAndArtist(album, artist, ct);
if (!peek.Success)
{
var error = peek.Messages.FirstOrDefault()?.Message ?? "Unknown error";
_logger.LogError("UploadAsync: release peek failed for ({Album}, {Artist}): {Error}", album, artist, error);
return ResultContainer<TrackDto>.CreateFailResult($"Could not verify the release: {error}");
}
// CREATE path: a pre-existing (title, artist) is a duplicate. Block it — the form never
// edits or appends to an existing release.
if (peek.Value is { } existing)
{
return ResultContainer<TrackDto>.CreateFailResult(
$"{DuplicateReleaseMarker}A release titled '{existing.Title}' by {existing.Artist} already " +
"exists. The upload form creates new releases only — use the edit tools to change an existing one.");
}
// resolved stays null → FindOrCreateRelease below creates the release.
}
}
var unpersisted = await _contentTrackContentService.AddTrackAsync(
tempFilePath, trackName, artist, album, genre, releaseDate, originalFileName: originalFileName);
tempFilePath, trackName, artist, album, genre, releaseDate, originalFileName: originalFileName, cancellationToken: ct);
if (unpersisted is null)
{
@@ -109,9 +160,12 @@ public class UnifiedTrackService
// shared release (created on first sighting); an upload without one stays a loose track with
// a null ReleaseId. Release-cardinal metadata (artist/genre/description/releaseDate/type/uploader)
// rides on the release, not the track.
long? releaseId = null;
if (!string.IsNullOrWhiteSpace(album))
long? resolvedReleaseId = resolved?.Id;
if (!string.IsNullOrWhiteSpace(album) && resolvedReleaseId is null)
{
// CREATE path only: the duplicate guard above proved no (title, artist) match exists, so this
// mints the release. (The attach path already resolved the id from the pre-check above and
// skips FindOrCreateRelease entirely, so a within-batch row never re-runs the natural-key find.)
var releaseData = new ReleaseDto
{
Title = album,
@@ -124,13 +178,13 @@ public class UnifiedTrackService
CreatedByUserId = createdByUserId,
};
// Medium (like every other field in releaseData) applies only when this upload CREATES the
// release. FindOrCreateRelease returns an existing (title, artist) row untouched — the first
// upload's medium is authoritative. Do NOT "fix" this to overwrite the stored medium on a
// subsequent track add: medium is a release-level property, changed only via the edit path
// (PUT api/track/meta), never silently flipped by adding a track to an existing release.
// FindOrCreateRelease either creates a fresh release (WasCreated = true) or returns the
// row the concurrent winner just inserted (WasCreated = false). In the CREATE path the
// duplicate peek above already verified no pre-existing row exists — so WasCreated = false
// means we lost a concurrent-insert race. Treat that as the duplicate condition: reject
// rather than silently attaching, keeping the DB unique index as the final safety net.
var releaseResult = await _sqlTrackService.FindOrCreateRelease(album, artist, releaseData, ct);
if (!releaseResult.Success || releaseResult.Value is null)
if (!releaseResult.Success)
{
var error = releaseResult.Messages.FirstOrDefault()?.Message ?? "Unknown error";
_logger.LogError(
@@ -139,11 +193,21 @@ public class UnifiedTrackService
return ResultContainer<TrackDto>.CreateFailResult($"Track was uploaded but could not be saved: {error}");
}
releaseId = releaseResult.Value.Id;
var (resolvedRelease, wasCreated) = releaseResult.Value;
if (!wasCreated)
{
// The winning concurrent upload created this release between our peek and our insert.
// Reject with the same marker the pre-flight peek uses so the controller maps it to 409.
return ResultContainer<TrackDto>.CreateFailResult(
$"{DuplicateReleaseMarker}A release titled '{resolvedRelease.Title}' by {resolvedRelease.Artist} already " +
"exists. The upload form creates new releases only — use the edit tools to change an existing one.");
}
resolvedReleaseId = resolvedRelease.Id;
}
var trackDto = TrackConverter.Convert(unpersisted);
trackDto.ReleaseId = releaseId;
trackDto.ReleaseId = resolvedReleaseId;
trackDto.Release = null; // FK already resolved; Create must not re-resolve a detached graph.
var saveResult = await _sqlTrackService.Create(trackDto);
@@ -163,20 +227,109 @@ public class UnifiedTrackService
// frontend, so a failure here is logged and swallowed — never fails the upload.
await TryStoreWaveformDatumsAsync(unpersisted.EntryKey, ct);
// Schedule the low-data Opus derive (OQ6 / §3.1a): the track is persisted and lossless-playable
// NOW; the transcode + seek-index build run on a background worker. Non-blocking and best-effort
// — the upload response never waits on it, and a transcode failure leaves the track lossless-only.
_opusTranscodeQueue.Enqueue(unpersisted.EntryKey);
return saveResult;
}
// The release a track resolved onto before the vault write. A null Id is the create path (mint
// below); a non-null Id is the attach path (a within-batch multi-track Cut row 2..N).
private readonly record struct ResolvedRelease(long Id);
// The cardinality guard shared by the attach path and (historically) the create path: a release
// already at its medium's Max rejects a further track. Returns the marker-prefixed rejection
// message, or null when the add is within limits. The create path never trips this (a brand-new
// release has zero tracks and admits its first), so only the attach path calls it today.
private static string? CheckCardinality(ReleaseDto release)
{
var cardinality = MediumRules.CardinalityOf(release.Medium);
if (release.TrackCount + 1 > cardinality.Max)
{
return $"{CardinalityViolationMarker}A {release.Medium} release holds a single track; " +
$"'{release.Title}' already has one — edit the existing track or choose a different release.";
}
return null;
}
/// <summary>
/// Replace an existing track's audio in place: look up the SQL row, swap only the vault bytes
/// keyed by its EntryKey, regenerate both waveform datums from the new audio, then write the
/// new duration to SQL. Track id, EntryKey, release membership, track number, and all other
/// metadata are preserved. The waveform regen is best-effort (a missing datum renders as a flat
/// seekbar / blank visualizer downstream), so a datum failure is logged and swallowed rather than
/// failing the replace. The duration write is not best-effort — a failure is surfaced so derived
/// aggregates (e.g. MixRuntimeSeconds) do not silently go stale. No release-cardinality cascade
/// applies: the track count is unchanged, so the single-track-Mix case stays intact.
/// </summary>
public async Task<Result> ReplaceAudioAsync(long trackId, string tempFilePath, CancellationToken ct)
{
var lookup = await _sqlTrackService.GetById(trackId);
if (!lookup.Success)
{
var error = lookup.Messages.FirstOrDefault()?.Message ?? "unknown error";
_logger.LogError("ReplaceAudioAsync: GetById failed for track {TrackId}: {Error}", trackId, error);
return Result.CreateFailResult("Failed to load track.");
}
if (lookup.Value is null)
{
return Result.CreateFailResult(TrackNotFoundMessage);
}
var entryKey = lookup.Value.EntryKey;
var newDuration = await _contentTrackContentService.ReplaceTrackAudioAsync(entryKey, tempFilePath, ct);
if (newDuration is null)
{
_logger.LogWarning("ReplaceAudioAsync: content swap returned null for track {TrackId} ({EntryKey})", trackId, entryKey);
return Result.CreateFailResult("Failed to process and store the replacement audio.");
}
// The old waveform no longer matches the new bytes. Regenerate both datums in place, keyed by
// the same EntryKey (the re-run overwrites the stale data). The store path no longer hands back
// a buffer, so the waveform compute re-reads the freshly stored audio from the vault — the same
// path the upload uses. That re-read is now a bounded streaming pass (Wave 2); neither the store
// nor the compute holds the whole file. Best-effort throughout: a datum failure never fails the replace.
await TryStoreWaveformDatumsAsync(entryKey, ct);
// Write the new duration to SQL. The vault bytes are already swapped, so this is the
// authoritative metadata update for the replace. A failure here is surfaced (unlike the
// best-effort waveform regen above) because a stale DurationSeconds silently corrupts
// derived aggregates (e.g. MixRuntimeSeconds on the home stats endpoint).
var durationWrite = await _sqlTrackService.SetDuration(trackId, newDuration.Value, ct);
if (!durationWrite.Success)
{
var error = durationWrite.Messages.FirstOrDefault()?.Message ?? "Unknown error";
_logger.LogError(
"ReplaceAudioAsync: vault swap succeeded but SQL duration update failed for track {TrackId} ({EntryKey}): {Error}",
trackId, entryKey, error);
return Result.CreateFailResult("Audio replaced but duration metadata could not be updated.");
}
// The stale Opus artifact (if any) no longer matches the new source. Schedule a background
// regenerate — the transcode service overwrites the prior artifacts in place keyed by the same
// EntryKey. Best-effort, off the request thread, mirrors the waveform regen above.
_opusTranscodeQueue.Enqueue(entryKey);
return Result.CreatePassResult();
}
// Compute and store both waveform datums for a freshly uploaded track: the fixed 512-bucket profile
// the player-bar seeker consumes, and the duration-derived high-res datum the lava visualizer
// consumes (phase-12 §5 — every track now carries one, computed at upload). Both source the same
// audio: read it back from the vault once (the authoritative parsed duration + the stored buffer)
// rather than re-reading and re-parsing the temp file. Best-effort throughout — never fails upload.
// consumes (phase-12 §5 — every track now carries one, computed at upload). Both are reduced in a
// SINGLE streaming pass over the vault audio (Wave 2): the duration comes from the vault index
// metadata (no body load) and the PCM is streamed in bounded chunks through two accumulators, so a
// ~GB mix never lands its whole body in a managed byte[]. Best-effort throughout — never fails upload.
private async Task TryStoreWaveformDatumsAsync(string entryKey, CancellationToken ct)
{
try
{
var audio = await _contentTrackContentService.GetAudioBinaryAsync(entryKey);
if (audio is null)
var duration = await _contentTrackContentService.GetAudioDurationAsync(entryKey);
if (duration is null)
{
_logger.LogWarning(
"Waveform datum step: no audio in vault for {EntryKey} immediately after store; skipping.",
@@ -184,8 +337,8 @@ public class UnifiedTrackService
return;
}
await _waveformProfileService.ComputeAndStoreAsync(audio.Buffer, entryKey);
await _waveformProfileService.ComputeAndStoreHighResAsync(audio.Buffer, entryKey, audio.Duration);
await _waveformProfileService.ComputeAndStoreAllStreamingAsync(
_ => _contentTrackContentService.OpenAudioStreamAsync(entryKey), entryKey, duration.Value, ct);
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
@@ -193,6 +346,120 @@ public class UnifiedTrackService
}
}
/// <summary>
/// One-time backfill: for every non-deleted track whose SQL duration is still null, read the
/// processor-extracted runtime from the vault audio (by EntryKey) and write it to SQL. The migration
/// cannot read the vault, so this runs at runtime after deploy. Idempotent — a re-run only touches
/// rows still missing a duration. Returns (updated, skipped) counts. A per-track vault miss or SQL
/// failure is logged and skipped, never aborting the batch.
/// </summary>
public async Task<ResultContainer<(int Updated, int Skipped)>> BackfillDurationsAsync(CancellationToken ct)
{
var missing = await _sqlTrackService.GetTracksMissingDuration(ct);
if (!missing.Success || missing.Value is null)
{
var error = missing.Messages.FirstOrDefault()?.Message ?? "Unknown error";
_logger.LogError("BackfillDurationsAsync: failed to load tracks missing duration: {Error}", error);
return ResultContainer<(int, int)>.CreateFailResult($"Could not load tracks: {error}");
}
var updated = 0;
var skipped = 0;
foreach (var track in missing.Value)
{
ct.ThrowIfCancellationRequested();
// Read the duration from the vault index metadata (no audio body load) — the same value the
// processor wrote at upload. Bounds this admin path too (Wave 2): a backfill over a catalogue
// of long mixes no longer pulls each whole file into memory just to read its runtime.
var duration = await _contentTrackContentService.GetAudioDurationAsync(track.EntryKey);
if (duration is null)
{
_logger.LogWarning("BackfillDurationsAsync: no vault audio for {EntryKey} (track {Id}); skipping.",
track.EntryKey, track.Id);
skipped++;
continue;
}
var write = await _sqlTrackService.UpdateDuration(track.Id, duration.Value, ct);
if (!write.Success)
{
var error = write.Messages.FirstOrDefault()?.Message ?? "Unknown error";
_logger.LogWarning("BackfillDurationsAsync: SQL update failed for track {Id}: {Error}", track.Id, error);
skipped++;
continue;
}
updated++;
}
_logger.LogInformation("BackfillDurationsAsync complete: {Updated} updated, {Skipped} skipped.", updated, skipped);
return ResultContainer<(int, int)>.CreatePassResult((updated, skipped));
}
/// <summary>
/// Backfill-Opus (18.5, OQ4): enqueue a background Opus derive for every non-deleted track that lacks a
/// complete Opus artifact (missing audio OR missing sidecar — a half-derived track is treated as missing
/// and re-derived). Mirrors the duration-backfill posture: enumerate SQL rows, check each against the
/// <c>track-opus</c> vault, schedule the misses. Enqueue-only and non-blocking — the actual transcodes run
/// on the shared background worker, serially (the same queue the upload/replace paths feed), so this
/// returns as soon as the misses are scheduled rather than waiting on CPU-heavy transcodes. Idempotent:
/// a re-run only enqueues tracks still missing Opus, and already-queued/in-flight derives simply overwrite
/// in place. Returns (enqueued, skipped) — skipped = tracks that already have a complete Opus artifact.
/// </summary>
public async Task<ResultContainer<(int Enqueued, int Skipped)>> BackfillOpusAsync(CancellationToken ct)
{
var all = await _sqlTrackService.GetAll();
if (!all.Success || all.Value is null)
{
var error = all.Messages.FirstOrDefault()?.Message ?? "Unknown error";
_logger.LogError("BackfillOpusAsync: failed to load tracks: {Error}", error);
return ResultContainer<(int, int)>.CreateFailResult($"Could not load tracks: {error}");
}
var enqueued = 0;
var skipped = 0;
foreach (var track in all.Value)
{
ct.ThrowIfCancellationRequested();
if (await _formatResolver.HasOpusAsync(track.EntryKey))
{
skipped++;
continue;
}
_opusTranscodeQueue.Enqueue(track.EntryKey);
enqueued++;
}
_logger.LogInformation("BackfillOpusAsync complete: {Enqueued} enqueued, {Skipped} already had Opus.",
enqueued, skipped);
return ResultContainer<(int, int)>.CreatePassResult((enqueued, skipped));
}
/// <summary>
/// Per-track Opus (re)derive trigger (18.5): schedule a background transcode for one track. Returns false
/// only when the track id is unknown; the enqueue itself is non-blocking and best-effort, like the bulk
/// backfill. Re-runnable — overwrites any prior artifact in place.
/// </summary>
public async Task<Result> EnqueueOpusAsync(string entryKey, CancellationToken ct)
{
var lookup = await _sqlTrackService.GetByEntryKey(entryKey);
if (!lookup.Success)
{
var error = lookup.Messages.FirstOrDefault()?.Message ?? "Unknown error";
_logger.LogError("EnqueueOpusAsync: lookup failed for {EntryKey}: {Error}", entryKey, error);
return Result.CreateFailResult("Failed to load track.");
}
if (lookup.Value is null)
return Result.CreateFailResult(TrackNotFoundMessage);
_opusTranscodeQueue.Enqueue(entryKey);
return Result.CreatePassResult();
}
/// <summary>
/// Delete a track's SQL row, then its vault entry. SQL is the source of truth: a SQL delete
/// failure fails the operation (and leaves the vault untouched), but a subsequent vault delete
+59
View File
@@ -4,6 +4,7 @@ using DeepDrftContent.Constants;
using DeepDrftContent.FileDatabase.Models;
using DeepDrftContent.FileDatabase.Services;
using DeepDrftContent.Processors;
using DeepDrftContent.Processors.Opus;
using Microsoft.Extensions.Logging;
using NetBlocks.Utilities.Environment;
@@ -44,12 +45,60 @@ namespace DeepDrftAPI
InitializeTrackVault(db).GetAwaiter().GetResult();
InitializeImageVault(db).GetAwaiter().GetResult();
InitializeTrackWaveformsVault(db).GetAwaiter().GetResult();
InitializeTrackOpusVault(db).GetAwaiter().GetResult();
return db;
});
// Upload staging directory. Large audio bodies (multi-hundred-MB WAVs) must never stage on
// the system temp mount — on the Linux host /tmp is a small RAM-backed tmpfs. We move BOTH
// on-disk copies of an upload off /tmp onto the data disk:
// Layer 1 — the framework's multipart file-section buffer (FileBufferingReadStream), which
// reads its directory from the ASPNETCORE_TEMP env var (falling back to
// Path.GetTempPath()). Setting the var here, before the host runs, relocates it.
// Layer 2 — the controller's own staging file, via the injected UploadStagingDirectory.
// Default location is a "staging" subdirectory beside the vaults; override with
// Upload:StagingPath in appsettings.json.
var uploadSettings = builder.Configuration.GetSection("Upload").Get<UploadSettings>();
var stagingPath = ResolveStagingPath(uploadSettings?.StagingPath, vaultPath);
Directory.CreateDirectory(stagingPath);
// AspNetCoreTempDirectory caches this value on first read and throws if the directory is
// absent, so set it (and create the dir) before any request is served.
Environment.SetEnvironmentVariable("ASPNETCORE_TEMP", stagingPath);
builder.Services.AddSingleton(new UploadStagingDirectory(stagingPath));
// Opus low-data transcode (Phase 18.1). The domain service lives in DeepDrftContent; the host
// owns only the engine config and the background worker. Bitrate/ffmpeg-path come from the
// OpusTranscode config section; StagingPath is forced to the same data-disk staging directory
// the upload path uses so large transcode temp files never land on the /tmp tmpfs.
builder.Services.Configure<OpusTranscodeOptions>(
builder.Configuration.GetSection(nameof(OpusTranscodeOptions)));
builder.Services.PostConfigure<OpusTranscodeOptions>(o => o.StagingPath = stagingPath);
builder.Services.AddSingleton<FfmpegOpusEncoder>();
builder.Services.AddSingleton<OpusTranscodeService>();
// Opus delivery format resolution + sidecar lookup (Phase 18.2). The seam 18.3 calls behind
// the ?format= stream param and the sidecar path. Stateless over the FileDatabase + content
// service singletons; the lossless branch reuses the existing read path unchanged (C2).
builder.Services.AddSingleton<TrackFormatResolver>();
return Task.CompletedTask;
}
/// <summary>
/// Resolves the absolute upload-staging directory. An explicit <paramref name="configuredPath"/>
/// (from <c>Upload:StagingPath</c>) wins; otherwise it defaults to a <c>staging</c> subdirectory
/// under <paramref name="vaultPath"/> — on the data disk, never the system temp mount. Pure so
/// the "never <c>/tmp</c>" invariant is unit-testable without standing up the host.
/// </summary>
public static string ResolveStagingPath(string? configuredPath, string vaultPath)
{
var path = string.IsNullOrWhiteSpace(configuredPath)
? Path.Combine(vaultPath, "staging")
: configuredPath;
return Path.GetFullPath(path);
}
private static async Task InitializeTrackVault(FileDatabase fileDatabase)
{
if (!fileDatabase.HasVault(VaultConstants.Tracks))
@@ -75,5 +124,15 @@ namespace DeepDrftAPI
await fileDatabase.CreateVaultAsync(VaultConstants.TrackWaveforms, MediaVaultType.Media);
}
}
// Ensure the track-opus vault exists (Phase 18.1). Holds the derived low-data Ogg Opus artifacts
// — the Opus audio bytes and the setup-header + seek-index sidecar — keyed by the track's EntryKey.
private static async Task InitializeTrackOpusVault(FileDatabase fileDatabase)
{
if (!fileDatabase.HasVault(VaultConstants.TrackOpus))
{
await fileDatabase.CreateVaultAsync(VaultConstants.TrackOpus, MediaVaultType.Audio);
}
}
}
}
+6 -4
View File
@@ -1,19 +1,21 @@
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning",
"DeepDrftContent.Controllers.TrackController": "Information"
"Default": "Information"
}
},
"AllowedHosts": "*",
"Upload": {
"StagingPath": ""
},
"CorsSettings": {
"AllowedOrigins": [
"https://localhost:12778",
"https://localhost:5004",
"http://localhost:5003",
"https://deepdrft.com",
"https://www.deepdrft.com"
"https://www.deepdrft.com",
"https://app.deepdrft.com"
]
},
"ForwardedHeaders": {
@@ -8,7 +8,9 @@
},
"Email": {
"Host": "smtp.your-provider.com",
"Token": "your-email-token-here"
"Token": "your-email-token-here",
"From": "noreply@yourdomain.com",
"TestInbox": "<sandbox-id>"
},
"Admin": {
"UserName": "admin",
+55 -16
View File
@@ -18,7 +18,9 @@ DeepDrftContent.Services/
│ ├── Services/ # FileDatabase, MediaVault, IndexSystem, IndexWatcher
│ └── Utils/ # StructuralMap, StructuralSet, FileUtils
├── Processors/
── AudioProcessor.cs # WAV file parsing, metadata extraction
── AudioProcessor.cs # WAV file parsing, metadata extraction, streaming PCM header parse
│ ├── AudioStoreStream.cs # Bounded streaming copy/prefix helpers used by the processors
│ └── ProcessedAudio.cs # Store-path plan: metadata + streamed WriteToAsync callback
├── Constants/
│ └── VaultConstants.cs # Vault name definitions
├── TrackService.cs # Content-side orchestrator
@@ -47,6 +49,8 @@ FileBinary (base: byte buffer)
Each has a matching `*Dto` variant for base64 JSON transport (e.g., `AudioBinaryDto` with buffer encoded as base64).
**Read/load path**: vault reads (`LoadResourceAsync<AudioBinary>`) still return a full-buffer `AudioBinary`. **Write/store path** is streaming: audio processors return a `ProcessedAudio` plan (metadata + `WriteToAsync` callback); `RegisterResourceStreamingAsync` / `MediaVault.AddEntryStreamingAsync` write bytes to a temp file, then `File.Move` atomic-rename into place. The full `AudioBinary` buffer is never materialized on the store path.
### Index lifecycle
- **DirectoryIndex**: Root index file (at `{rootPath}/index`). Tracks which vaults exist.
@@ -70,19 +74,33 @@ public async Task<bool> RegisterResourceAsync(string vaultId, string entryId, Fi
try { /* store and update index */ }
catch { return false; } // Swallow, return false
}
// Streaming counterpart (Wave 1 OOM fix) — same bool contract:
public async Task<bool> RegisterResourceStreamingAsync(
string vaultId, string entryId, MetaData metaData,
Func<Stream, CancellationToken, Task> writeContent, CancellationToken ct)
{
try { /* stream to temp → atomic rename → update index */ }
catch { return false; } // Swallow (logs non-cancel exceptions), return false
}
```
`MediaVault.AddEntryStreamingAsync` (called internally) writes bytes to a temp file in the vault directory, then `File.Move(temp, final, overwrite: true)` (atomic POSIX rename on the Linux prod host), then updates the index. A cancel or I/O fault before the rename leaves any prior backing file intact; the temp file is cleaned up best-effort.
**Callers must check return values.** Do not change this without a deliberate design pass — it's embedded in all FileDatabase tests and client code.
## Audio processors
Multi-format support via router pattern. All processors live in `DeepDrftContent/Processors/`:
- `AudioProcessor.ProcessWavFileAsync(filePath)`: WAV-specific processor. Validates RIFF/WAVE structure and format code. Accepts standard PCM (audioFormat=1) and WAVE_FORMAT_EXTENSIBLE (audioFormat=0xFFFE) when the SubFormat GUID indicates PCM. Normalizes EXTENSIBLE-PCM uploads to standard 44-byte PCM WAV before storing. Parses fmt and data chunks; extracts duration and bitrate. Returns `AudioBinary` with metadata. On parse failure, logs warning and returns defaults (180s / 1411 kbps / 44.1 kHz / 16-bit stereo). Accepts standard PCM (audioFormat=1), WAVE_FORMAT_EXTENSIBLE with PCM SubFormat (0x0001), IEEE Float SubFormat (0x0003), and Padded 24-in-32 containers; normalizes Float and padded inputs to standard 24-bit PCM before storage.
- `Mp3AudioProcessor.ProcessMp3FileAsync(filePath)`: MP3 processor. Skips ID3v2 tag, finds first valid MPEG frame sync, decodes frame header (bitrate, sample rate, channels). Reads Xing/Info header for VBR total-frame count (accurate duration); VBRI header as fallback; CBR estimate from file size otherwise. Returns `AudioBinary` with original bytes and `.mp3` extension. On parse failure, falls back to defaults (180s / 320 kbps).
- `FlacAudioProcessor.ProcessFlacFileAsync(filePath)`: FLAC processor. Validates `fLaC` magic, reads STREAMINFO metadata block (20-bit sample rate, 3-bit channels, 5-bit bits-per-sample, 36-bit total samples — all bit-packed). Computes duration from `totalSamples / sampleRate`; average bitrate from file size. Returns `AudioBinary` with original bytes and `.flac` extension. On parse failure, falls back to defaults (180s / 1411 kbps).
- `AudioProcessorRouter.ProcessAudioFileAsync(filePath)`: Routes by extension — `.wav``AudioProcessor`, `.mp3``Mp3AudioProcessor`, `.flac``FlacAudioProcessor`. Throws `ArgumentException` for unsupported extensions.
- `WaveformProfileService.ComputeAndStoreHighResAsync(entryKey)`: The shared compute seam for the duration-derived high-res waveform datum (~333 samples/sec). Medium-neutral — computes for any track by `EntryKey`, stores in the `track-waveforms` vault. Called by the upload path (`UnifiedTrackService.UploadAsync` for every new track), the CMS per-row generate action, and the Mix release trigger (now a legacy delegate). Phase 12 generalization of the former Mix-only compute.
- `AudioProcessor.ProcessWavFileAsync(filePath)`: WAV-specific processor. Validates RIFF/WAVE structure and format code. Accepts standard PCM (audioFormat=1) and WAVE_FORMAT_EXTENSIBLE (audioFormat=0xFFFE) when the SubFormat GUID indicates PCM. Normalizes EXTENSIBLE-PCM uploads to standard 44-byte PCM WAV before storing. Parses fmt and data chunks; extracts duration and bitrate. **Returns `ProcessedAudio`** (metadata + streamed `WriteToAsync` callback — no whole-file buffer). Header window grows in 64 KB steps capped at 8 MB; the audio body is never read during processing. Standard PCM WAV is stored verbatim (passthrough via `AudioStoreStream.CopyFileAsync`); EXTENSIBLE-PCM / IEEE-float / padded-container WAVs stream their normalization to standard 24-bit PCM. On parse failure, falls back to defaults (180s / 1411 kbps). Also exposes `TryExtractPcm(ReadOnlySpan<byte>)` for the whole-buffer waveform parity oracle and `TryReadPcmStreamInfoAsync(stream, totalLength)` for the streaming waveform compute path (bounded header parse from a stream; returns `WavPcmStreamInfo?` with `DataStart`/`DataLength`/format fields).
- `Mp3AudioProcessor.ProcessMp3FileAsync(filePath)`: MP3 processor. Reads a bounded ≤8 MB prefix for header parsing; skips ID3v2 tag, finds first valid MPEG frame sync, decodes frame header (bitrate, sample rate, channels). Reads Xing/Info header for VBR total-frame count (accurate duration); VBRI header as fallback; CBR estimate from file size otherwise. **Returns `ProcessedAudio`** (passthrough plan — MP3 stored unmodified). On parse failure, falls back to defaults (180s / 320 kbps).
- `FlacAudioProcessor.ProcessFlacFileAsync(filePath)`: FLAC processor. Reads a bounded ≤64 KB prefix. Validates `fLaC` magic, reads STREAMINFO metadata block (20-bit sample rate, 3-bit channels, 5-bit bits-per-sample, 36-bit total samples — all bit-packed). Computes duration from `totalSamples / sampleRate`; average bitrate from file size. **Returns `ProcessedAudio`** (passthrough plan — FLAC stored unmodified). On parse failure, falls back to defaults (180s / 1411 kbps).
- `AudioProcessorRouter.ProcessAudioFileAsync(filePath)`: Routes by extension — `.wav``AudioProcessor`, `.mp3``Mp3AudioProcessor`, `.flac``FlacAudioProcessor`. **Returns `ProcessedAudio?`**. Throws `ArgumentException` for unsupported extensions.
- `ProcessedAudio`: Store-path plan returned by all processors. Carries `Extension`, `Duration`, `Bitrate`, `Size`, and a `WriteToAsync(destination, ct)` callback that streams the canonical vault bytes to the destination without materializing the whole file. `ProcessedAudio.Passthrough(sourcePath, ...)` builds a passthrough plan via `AudioStoreStream.CopyFileAsync`.
- `AudioStoreStream`: Internal bounded-buffer streaming helpers. `CopyFileAsync(sourcePath, destination, ct)` does a bounded 80 KB disk-to-disk copy; `ReadPrefixAsync(path, cap, ct)` reads at most `cap` bytes from the start of a file (used by processors for header parsing without loading the body).
- `ILoudnessAlgorithm` / `ILoudnessAccumulator`: The loudness strategy interface now exposes both `Compute(ReadOnlySpan<byte>, ...)` (whole-buffer, retained as the parity oracle for tests — no production callers) and `CreateAccumulator(pcmByteLength, ...)``ILoudnessAccumulator` (streaming: `Add(ReadOnlySpan<byte>)` / `Finish()``double[]`). `RmsLoudnessAlgorithm` implements both; `Compute` is defined in terms of the accumulator, so streaming and whole-buffer outputs are byte-identical.
- `WaveformProfileService`: Streaming loudness compute + vault store. The whole-buffer `ComputeAndStoreAsync` / `ComputeAndStoreHighResAsync` methods are **retained but have no production callers** — they exist as the byte-identity parity oracle for tests; do not delete them. The production paths are: `ComputeAndStoreProfileStreamingAsync` (512-bucket seeker profile, tri-state `bool?`), `ComputeAndStoreHighResStreamingAsync` (duration-derived high-res datum, tri-state `bool?`), and `ComputeAndStoreAllStreamingAsync` (both datums in a SINGLE streaming pass, used by the upload/replace hot path, tri-state `bool?`). Tri-state: `null` = no backing audio stream; `false` = audio present but not WAV-decodable or vault write failed; `true` = stored. Streaming reads the WAV in bounded ≤80 KB chunks through one accumulator per datum; peak memory is O(bucket arrays + read buffer), independent of file size.
- `WaveformResolution`: Enum / constants controlling bucket density for the high-res compute. Renamed from `MixWaveformResolution` in Phase 12.
Vault stores original bytes with correct extension and MIME type (inferred from file extension or content-type header at upload time).
@@ -101,20 +119,41 @@ The primary entry point is `TrackContentService.AddTrackAsync(filePath, mimeType
## Content-side TrackService (orchestrator)
### AddTrackFromWavAsync(filePath)
### AddTrackAsync(audioFilePath, ...)
1. Reads a WAV file from disk.
2. Calls `AudioProcessor.ProcessWavFileAsync``AudioBinary`.
3. Generates a GUID entry key (via `Guid.NewGuid().ToString()`).
4. Ensures the `tracks` vault exists (creates if missing).
5. Calls `FileDatabase.RegisterResourceAsync("tracks", entryKey, audioBinary)`.
6. Returns a populated `TrackEntity` (with `Id = 0` since it's not yet in SQL).
The primary upload entry point. Format-agnostic — routes by extension via `AudioProcessorRouter`.
**Note**: The caller (CLI or web) is responsible for then saving this entity to SQL via `DeepDrftWeb.Services.TrackService.Create`. If the vault write succeeds and SQL write fails, audio is orphaned (no compensating rollback).
1. Calls `AudioProcessorRouter.ProcessAudioFileAsync(filePath)``ProcessedAudio` plan (no whole-file buffer — Wave 1 OOM fix).
2. Generates a GUID entry key (via `Guid.NewGuid().ToString()`).
3. Ensures the `tracks` vault exists (creates if missing).
4. Builds `MetaData` from the plan's header-extracted fields and calls `FileDatabase.RegisterResourceStreamingAsync("tracks", entryKey, metaData, processed.WriteToAsync)` — bytes are streamed from the staging file to the vault via a bounded copy; `MediaVault.AddEntryStreamingAsync` writes to a temp file then atomic-renames into place.
5. Returns a populated `TrackEntity` (with `Id = 0` since it's not yet in SQL, and `DurationSeconds` populated from the header parse).
If the vault write succeeds and SQL write fails, audio is orphaned (no compensating rollback).
### AddTrackFromWavAsync(filePath, ...)
Backward-compatible shim — delegates to `AddTrackAsync`. The router accepts WAV alongside MP3 and FLAC, so this carries no WAV-specific logic of its own.
### ReplaceTrackAudioAsync(entryKey, audioFilePath)
Swaps the vault bytes for an existing track in place, under the same `entryKey`. Captures the old extension from the vault index metadata (not by loading the file) so a cross-format swap can clean up the stale backing file post-success. Streams the new audio via `ProcessedAudio` + `RegisterResourceStreamingAsync` (no whole-file buffer — Wave 1 OOM fix). Returns the new audio's duration on success, `null` on failure (original audio left intact).
### GetAudioBinaryAsync(entryKey)
Reads audio from the `tracks` vault via `FileDatabase.LoadResourceAsync<AudioBinary>("tracks", entryKey)`. Returns `null` if not found or read fails.
Reads audio from the `tracks` vault via `FileDatabase.LoadResourceAsync<AudioBinary>("tracks", entryKey)`. Returns a full-buffer `AudioBinary` or `null` if not found. This is a full-buffer convenience read — **not** on the audio-delivery hot path. The delivery path now uses `TrackFormatResolver` (Opus: `MediaVault.GetEntryStreamAsync`; lossless: `OpenAudioMediaStreamAsync`). `GetAudioBinaryAsync` is retained as a read-back oracle used by tests (`AudioStoreStreamingTests`, `TrackReplaceAudioTests`, `TrackFormatDeliveryTests`).
### OpenAudioStreamAsync(entryKey)
Returns a read-only, seekable `Stream` over a track's vault audio without buffering the whole file. Used by the streaming waveform compute path (`WaveformProfileService`). Caller owns and must dispose the stream. Returns `null` if the entry has no backing file.
### OpenAudioMediaStreamAsync(entryKey)
Returns a `MediaStream?` — a read-only, non-buffering stream over a track's vault audio together with its stored extension, or `null` if the entry has no backing file. Same non-buffering contract as `OpenAudioStreamAsync`, but exposes `.Extension` alongside `.Stream` so the caller can name a staging file by the original format (the Opus transcode stages the source with the correct extension so ffmpeg can detect the format). The caller owns the returned `MediaStream` and must dispose it. Follows the FileDatabase swallow-and-return-null contract.
### GetAudioDurationAsync(entryKey)
Reads the stored audio duration from the vault index metadata only — no audio body load. Used by the streaming waveform compute to derive the duration-based bucket count, and by `UnifiedTrackService.BackfillDurationsAsync`. Returns `null` if the entry is unknown or carries no audio metadata.
### InitializeTracksVaultAsync()
@@ -122,7 +161,7 @@ Safety call to ensure the `tracks` vault exists (creates if missing). Called on
## Vault constants
`VaultConstants.Tracks = "tracks"`, `VaultConstants.Images = "images"`, and `VaultConstants.TrackWaveforms = "track-waveforms"` — the vault names in production use. `TrackWaveforms` holds the per-track high-res waveform datum keyed by `TrackEntity.EntryKey` (Phase 12; renamed from the former `mix-waveforms`, which was Mix-only). New vault names go here when adding new vault types.
`VaultConstants.Tracks = "tracks"`, `VaultConstants.Images = "images"`, `VaultConstants.TrackWaveforms = "track-waveforms"`, and `VaultConstants.TrackOpus = "track-opus"` — the vault names in production use. `TrackWaveforms` holds the per-track high-res waveform datum keyed by `TrackEntity.EntryKey` (Phase 12; renamed from the former `mix-waveforms`, which was Mix-only). `TrackOpus` holds two entries per track: the derived Opus audio (keyed by `entryKey`, extension `.opus`) and the combined setup-header + seek-index sidecar (keyed by `{entryKey}-sidecar`, extension `.opusidx`). New vault names go here when adding new vault types.
## Service registration
@@ -28,4 +28,13 @@ public static class VaultConstants
/// The datum resolution is duration-derived (≈333 samples/sec, see <c>WaveformResolution</c>).
/// </summary>
public const string TrackWaveforms = "track-waveforms";
/// <summary>
/// Vault name for the derived low-data Ogg Opus artifacts, keyed by the track's EntryKey (Phase 18,
/// S2). Holds two entries per track: the Opus audio bytes (<c>.opus</c>) and the combined setup-header
/// + granule→byte seek-index sidecar (<c>.opusidx</c>). Both are best-effort derived artifacts —
/// regenerable, and a track without them still plays losslessly. Distinct from the source <c>tracks</c>
/// vault so the source means exactly one thing (mirrors the <c>track-waveforms</c> precedent).
/// </summary>
public const string TrackOpus = "track-opus";
}
@@ -206,6 +206,7 @@ public static class MimeTypeExtensions
{ ".flac", "audio/flac" },
{ ".aac", "audio/aac" },
{ ".ogg", "audio/ogg" },
{ ".opus", "audio/ogg" },
{ ".m4a", "audio/mp4" }
};
@@ -178,6 +178,46 @@ public class FileDatabase : DirectoryIndexDirectory, IDisposable
return false;
}
/// <summary>
/// Registers a resource by streaming its bytes into the vault, without materializing the whole
/// file in a managed <c>byte[]</c> (the store-path OOM fix). The caller supplies the index
/// <paramref name="metaData"/> and a <paramref name="writeContent"/> callback that emits bytes to
/// the backing stream. Swallows exceptions and returns false, matching
/// <see cref="RegisterResourceAsync"/>'s contract — callers check the bool.
/// </summary>
public async Task<bool> RegisterResourceStreamingAsync(
string vaultId,
string entryId,
MetaData metaData,
Func<Stream, CancellationToken, Task> writeContent,
CancellationToken cancellationToken = default)
{
try
{
var directoryVault = _vaults.Get(vaultId);
if (directoryVault != null)
{
var written = await directoryVault.AddEntryStreamingAsync(entryId, metaData, writeContent, cancellationToken);
_logger.LogInformation(
"Streamed {Bytes} bytes into vault {VaultId} entry {EntryId} (no whole-file buffer).",
written, vaultId, entryId);
return true;
}
}
catch (Exception ex)
{
// Swallow and return false, matching RegisterResourceAsync. Log at error for real failures
// only — a normal client cancel (OperationCanceledException) is not an error condition and
// would spam the error log on every client disconnect during a large upload or replace.
if (ex is not OperationCanceledException)
{
_logger.LogError(ex, "RegisterResourceStreamingAsync failed for vault {VaultId} entry {EntryId}", vaultId, entryId);
}
}
return false;
}
/// <summary>
/// Removes a resource from a specific vault. Returns null if the vault does not exist,
/// false if the entry was not found, true if the entry was removed. Distinguishing
@@ -56,6 +56,63 @@ public abstract class MediaVault : VaultIndexDirectory
await FileUtils.PutFileAsync(mediaPath, buffer);
}
/// <summary>
/// Streams an entry's bytes into the vault without ever materializing the whole file in memory.
/// The metadata is supplied by the caller (there is no in-memory <see cref="FileBinary"/> to infer
/// it from) — the store path (upload / replace-audio) sources its bytes from a staging file, not a
/// buffer. Returns the number of bytes written, for the caller to log.
///
/// Write ordering (atomic-replace guarantee): bytes are streamed to a temp file in the same vault
/// directory, the temp file is renamed over the final backing-file path (POSIX <c>rename(2)</c> —
/// atomic on the Linux prod host), and the index is updated only after the rename succeeds.
/// This ordering ensures: (a) the index never advertises a not-yet-present file; (b) a client
/// disconnect or I/O fault during the write leaves any prior backing file intact and the index
/// unchanged; (c) the temp file is cleaned up best-effort on any failure before re-throwing so the
/// vault directory stays tidy. The caller treats a thrown exception as a failed register.
/// </summary>
public async Task<long> AddEntryStreamingAsync(
string entryId,
MetaData metaData,
Func<Stream, CancellationToken, Task> writeContent,
CancellationToken cancellationToken = default)
{
var finalPath = GetMediaPathFromEntryKey(entryId, metaData.Extension);
var tempPath = Path.Combine(RootPath, Path.GetRandomFileName() + ".tmp");
try
{
long bytesWritten;
await using (var tempStream = new FileStream(
tempPath, FileMode.CreateNew, FileAccess.Write, FileShare.None,
bufferSize: 81920, useAsync: true))
{
await writeContent(tempStream, cancellationToken);
await tempStream.FlushAsync(cancellationToken);
bytesWritten = tempStream.Length;
}
// Rename into place — atomic on the Linux prod host (POSIX rename(2)); overwrites any
// existing same-extension backing file safely on the replace path.
File.Move(tempPath, finalPath, overwrite: true);
// Update the index only after the file is durably in place. A crash between Move and
// AddToIndexAsync leaves an unreferenced file on disk (a harmless orphan recoverable
// by a vault scan); a crash or cancel during the temp write leaves the original backing
// file and the index both unchanged.
await AddToIndexAsync(entryId, metaData);
return bytesWritten;
}
catch
{
// Best-effort temp-file cleanup. After a successful rename tempPath is gone and the
// delete is a no-op. After a write failure or cancel tempPath holds partial bytes that
// must be removed so the vault directory stays tidy.
try { if (File.Exists(tempPath)) File.Delete(tempPath); } catch { /* best-effort */ }
throw;
}
}
/// <summary>
/// Retrieves an entry from the vault (MediaVaultType inferred from T)
/// </summary>
+321 -63
View File
@@ -7,12 +7,22 @@ namespace DeepDrftContent.Processors;
/// </summary>
public class AudioProcessor
{
// Header parsing never needs the audio body. Read the file in 64 KB steps until the data-chunk
// header is locatable, capping the window so a pathological file with an enormous pre-data header
// cannot drive an unbounded allocation — such a file simply falls through to default metadata and
// passthrough storage, the same outcome as any unparseable WAV.
private const int HeaderWindowStep = 64 * 1024;
private const int HeaderWindowCap = 8 * 1024 * 1024;
/// <summary>
/// Processes a WAV file and creates an AudioBinary object
/// Processes a WAV file into a <see cref="ProcessedAudio"/> store plan: extracts metadata from a
/// bounded header window (never the whole file) and returns a streamed writer for the canonical
/// vault bytes. Standard PCM is stored verbatim (passthrough copy); EXTENSIBLE-PCM / IEEE-float /
/// padded-container WAVs are normalized to a plain 44-byte standard-PCM WAV, written progressively
/// so the vault only ever holds a format the streaming pipeline already handles.
/// </summary>
/// <param name="filePath">Path to the WAV file</param>
/// <returns>AudioBinary object with metadata</returns>
public async Task<AudioBinary?> ProcessWavFileAsync(string filePath)
public async Task<ProcessedAudio?> ProcessWavFileAsync(string filePath, CancellationToken cancellationToken = default)
{
if (!File.Exists(filePath))
{
@@ -26,30 +36,197 @@ public class AudioProcessor
try
{
var buffer = await File.ReadAllBytesAsync(filePath);
var wavInfo = ExtractWavMetadata(buffer);
var fileLength = new FileInfo(filePath).Length;
var window = await ReadWavHeaderWindowAsync(filePath, cancellationToken);
var wavInfo = ExtractWavMetadata(window);
// EXTENSIBLE-PCM is byte-compatible with standard PCM but carries a 40+ byte fmt chunk
// the streaming pipeline never expects. Normalize to a plain 44-byte PCM WAV at storage
// time so the vault only ever holds standard PCM and the client decode path stays unchanged.
var storedBuffer = wavInfo.IsExtensible ? NormalizeToStandardPcm(buffer, wavInfo) : buffer;
if (!wavInfo.IsExtensible)
{
// Standard PCM (or the default-fallback path, which reports IsExtensible = false):
// the source bytes are already a format the pipeline handles, so store them verbatim.
return ProcessedAudio.Passthrough(filePath, ".wav", wavInfo.Duration, wavInfo.Bitrate, fileLength);
}
var parameters = new AudioBinaryParams(
Buffer: storedBuffer,
Size: storedBuffer.Length,
Extension: ".wav",
Duration: wavInfo.Duration,
Bitrate: wavInfo.Bitrate
);
// EXTENSIBLE → streamed normalization. The output data size is derivable from the source
// data size alone (no body read needed): verbatim keeps it, float drops 1 byte per sample
// (4→3), padded keeps only the valid bytes per container sample.
var dataStart = (long)wavInfo.DataChunkPos + 8;
var available = fileLength - dataStart;
var srcDataSize = Math.Min((long)wavInfo.DataSize, available);
return new AudioBinary(parameters);
NormalizeMode mode;
int outBitsPerSample;
long outDataSize;
int containerBytes = 0;
int validBytes = 0;
if (wavInfo.IsFloat)
{
mode = NormalizeMode.Float;
outBitsPerSample = 24;
outDataSize = (srcDataSize / 4) * 3;
}
else if (wavInfo.IsPaddedContainer)
{
mode = NormalizeMode.Padded;
outBitsPerSample = wavInfo.BitsPerSample;
containerBytes = wavInfo.ContainerBitsPerSample / 8;
validBytes = wavInfo.BitsPerSample / 8;
outDataSize = (srcDataSize / containerBytes) * validBytes;
}
else
{
mode = NormalizeMode.Verbatim;
outBitsPerSample = wavInfo.BitsPerSample;
outDataSize = srcDataSize;
}
var channels = wavInfo.Channels;
var sampleRate = wavInfo.SampleRate;
return new ProcessedAudio(
".wav", wavInfo.Duration, wavInfo.Bitrate, 44 + outDataSize,
(destination, ct) => WriteNormalizedWavAsync(
filePath, dataStart, srcDataSize, channels, sampleRate, outBitsPerSample,
outDataSize, mode, containerBytes, validBytes, destination, ct));
}
catch (Exception ex)
catch (Exception ex) when (ex is not OperationCanceledException)
{
throw new InvalidOperationException($"Failed to process WAV file: {ex.Message}", ex);
}
}
/// <summary>
/// Reads only enough of the file to contain the fmt chunk and the data chunk's 8-byte header, so
/// metadata parsing never loads the (potentially ~GB) audio body. Grows the window in 64 KB steps
/// until the data chunk is locatable or EOF/<see cref="HeaderWindowCap"/> is hit.
/// </summary>
private static async Task<byte[]> ReadWavHeaderWindowAsync(string filePath, CancellationToken ct)
{
await using var fs = new FileStream(
filePath, FileMode.Open, FileAccess.Read, FileShare.Read,
bufferSize: HeaderWindowStep, useAsync: true);
using var ms = new MemoryStream();
var buffer = new byte[HeaderWindowStep];
while (ms.Length < HeaderWindowCap)
{
var read = await fs.ReadAsync(buffer, ct);
if (read == 0)
break;
ms.Write(buffer, 0, read);
// FindChunk returns -1 on a partial window (the data chunk isn't reachable yet), so keep
// reading until it is found or the cap/EOF is hit. On normal files the data chunk header
// sits within the first 64 KB, so this loop runs exactly once.
var soFar = ms.ToArray();
if (FindChunk(soFar, "data") >= 0)
return soFar;
}
return ms.ToArray();
}
/// <summary>
/// Writes a normalized standard-PCM WAV to <paramref name="destination"/>: the 44-byte header
/// followed by the data region streamed from the source in bounded, sample-aligned chunks. No
/// whole-file buffer is ever held — peak memory is O(chunk), independent of duration.
/// </summary>
private async Task WriteNormalizedWavAsync(
string sourcePath, long dataStart, long srcDataSize,
int channels, int sampleRate, int outBitsPerSample, long outDataSize,
NormalizeMode mode, int containerBytes, int validBytes,
Stream destination, CancellationToken ct)
{
var header = BuildStandardPcmHeader(channels, sampleRate, outBitsPerSample, outDataSize);
await destination.WriteAsync(header, ct);
await using var src = new FileStream(
sourcePath, FileMode.Open, FileAccess.Read, FileShare.Read,
bufferSize: 81920, useAsync: true);
src.Seek(dataStart, SeekOrigin.Begin);
switch (mode)
{
case NormalizeMode.Verbatim:
await CopyBoundedAsync(src, destination, srcDataSize, ct);
break;
case NormalizeMode.Float:
// Each 4-byte float sample becomes 3 bytes of 24-bit PCM.
await TransformBoundedAsync(src, destination, srcDataSize, unit: 4,
transform: (buf, len) => ConvertFloatTo24BitPcm(buf, 0, len), ct);
break;
case NormalizeMode.Padded:
await TransformBoundedAsync(src, destination, srcDataSize, unit: containerBytes,
transform: (buf, len) => RepackPaddedContainer(buf, 0, len, containerBytes * 8, validBytes * 8), ct);
break;
}
}
/// <summary>Bounded copy of exactly <paramref name="totalBytes"/> from src to dest.</summary>
private static async Task CopyBoundedAsync(Stream src, Stream dest, long totalBytes, CancellationToken ct)
{
var buffer = new byte[81920];
var remaining = totalBytes;
while (remaining > 0)
{
var want = (int)Math.Min(buffer.Length, remaining);
var read = await src.ReadAsync(buffer.AsMemory(0, want), ct);
if (read == 0)
break;
await dest.WriteAsync(buffer.AsMemory(0, read), ct);
remaining -= read;
}
}
/// <summary>
/// Streams <paramref name="totalBytes"/> of source data through <paramref name="transform"/> in
/// sample-aligned chunks, writing each transformed chunk to <paramref name="dest"/>. The read
/// buffer is a multiple of <paramref name="unit"/>; leftover bytes that do not complete a sample
/// are carried into the next read, and a final partial sample is dropped (matching the
/// whole-buffer transforms' integer-division behavior).
/// </summary>
private static async Task TransformBoundedAsync(
Stream src, Stream dest, long totalBytes, int unit,
Func<byte[], int, byte[]> transform, CancellationToken ct)
{
var bufLen = Math.Max(unit, (81920 / unit) * unit);
var buffer = new byte[bufLen];
var remaining = totalBytes;
var carried = 0;
while (remaining > 0)
{
var want = (int)Math.Min(bufLen - carried, remaining);
if (want == 0)
break;
var read = await src.ReadAsync(buffer.AsMemory(carried, want), ct);
if (read == 0)
break;
remaining -= read;
var filled = carried + read;
var whole = (filled / unit) * unit;
if (whole > 0)
{
var output = transform(buffer, whole);
await dest.WriteAsync(output, ct);
}
carried = filled - whole;
if (carried > 0)
Array.Copy(buffer, whole, buffer, 0, carried);
}
}
private enum NormalizeMode
{
/// <summary>Sample bytes already standard PCM (EXTENSIBLE-PCM, depth == container width).</summary>
Verbatim,
/// <summary>IEEE float samples converted to 24-bit PCM.</summary>
Float,
/// <summary>Padded container (e.g. 24-in-32) re-packed to the valid depth.</summary>
Padded
}
/// <summary>
/// Extracts the raw PCM data region and format parameters from a WAV buffer, reusing the
/// same chunk-walk and validation as metadata extraction. Returns null if the buffer is not
@@ -111,6 +288,105 @@ public class AudioProcessor
return new PcmData(pcm, metadata.Channels, metadata.SampleRate, metadata.BitsPerSample);
}
/// <summary>
/// Reads only the WAV header region from <paramref name="stream"/> (a bounded window, never the
/// audio body) and returns where the PCM data region begins, how long it is, and the format
/// parameters needed to decode it — the streaming counterpart of <see cref="TryExtractPcm"/>. The
/// data length is clamped against <paramref name="totalFileLength"/> (the true backing-file size),
/// so the caller streams exactly the present PCM. Returns null for the same inputs
/// <see cref="TryExtractPcm"/> rejects — non-WAV bytes (mp3/flac), float, and padded-container
/// EXTENSIBLE — so the caller treats null as "no profile computable" and continues gracefully.
///
/// <paramref name="stream"/> must be positioned at the start; on return its position is past the
/// header window (the caller seeks to <c>DataStart</c> before streaming the body). No whole-file
/// buffer is allocated — peak memory is the bounded header window.
/// </summary>
public async Task<WavPcmStreamInfo?> TryReadPcmStreamInfoAsync(
Stream stream, long totalFileLength, CancellationToken cancellationToken = default)
{
var window = await ReadWavHeaderWindowAsync(stream, cancellationToken);
if (window is null)
{
return null;
}
var validation = ValidateWavStructure(window);
if (!validation.IsValid || validation.IsFloat)
{
return null;
}
WavMetadata metadata;
try
{
metadata = ParseWavMetadata(window, validation);
ValidateAudioParameters(metadata);
if (metadata.IsPaddedContainer)
{
return null;
}
}
catch
{
return null;
}
long dataStart = validation.DataChunkPos + 8;
if (dataStart > totalFileLength)
{
return null;
}
var available = totalFileLength - dataStart;
var dataLength = Math.Min((long)metadata.DataSize, available);
if (dataLength <= 0)
{
return null;
}
return new WavPcmStreamInfo(
dataStart, dataLength, metadata.Channels, metadata.SampleRate, metadata.BitsPerSample);
}
/// <summary>
/// Reads enough of <paramref name="stream"/> to contain the fmt chunk and the data chunk's 8-byte
/// header, growing in 64 KB steps until the data chunk is locatable or EOF / the
/// <see cref="HeaderWindowCap"/> is reached. Bails after the first read when the bytes are not a
/// RIFF/WAVE container, so a non-WAV stream (mp3/flac) costs one read, not the full cap. Returns
/// null only when nothing could be read.
/// </summary>
private static async Task<byte[]?> ReadWavHeaderWindowAsync(Stream stream, CancellationToken ct)
{
using var ms = new MemoryStream();
var buffer = new byte[HeaderWindowStep];
while (ms.Length < HeaderWindowCap)
{
var read = await stream.ReadAsync(buffer, ct);
if (read == 0)
break;
ms.Write(buffer, 0, read);
var soFar = ms.ToArray();
// Early-out for non-WAV input: once at least the 12-byte RIFF/WAVE preamble is in hand,
// a missing signature means this will never be a WAV — stop rather than read to the cap.
if (soFar.Length >= 12 && !HasRiffWaveSignature(soFar))
return soFar;
// FindChunk returns -1 until the data chunk header is fully in the window; on a normal
// file it sits within the first 64 KB so this loop runs exactly once.
if (FindChunk(soFar, "data") >= 0)
return soFar;
}
return ms.Length > 0 ? ms.ToArray() : null;
}
private static bool HasRiffWaveSignature(byte[] buffer) =>
buffer.Length >= 12
&& System.Text.Encoding.ASCII.GetString(buffer, 0, 4) == "RIFF"
&& System.Text.Encoding.ASCII.GetString(buffer, 8, 4) == "WAVE";
/// <summary>
/// Extracts metadata from WAV file buffer with comprehensive validation
/// </summary>
@@ -317,50 +593,17 @@ public class AudioProcessor
}
/// <summary>
/// Rebuilds an EXTENSIBLE WAV as a canonical 44-byte-header standard PCM WAV (audioFormat = 1)
/// so the vault only ever holds a format the streaming pipeline already handles. Three source
/// shapes are normalized:
/// <list type="bullet">
/// <item>EXTENSIBLE-PCM (depth == container): sample bytes are byte-identical to standard PCM and
/// copied verbatim; only the header is replaced.</item>
/// <item>IEEE float: 32-bit float samples are converted to 24-bit signed integer PCM.</item>
/// <item>Padded container (e.g. 24-in-32): the padding/sign-extension bytes are stripped, keeping
/// the lowest valid bytes per sample.</item>
/// </list>
/// The output header always reports the valid bit depth (<see cref="WavMetadata.BitsPerSample"/>).
/// Builds the canonical 44-byte standard-PCM WAV header (audioFormat = 1) for a normalized stream.
/// The body is written separately so no whole-file buffer is allocated; this only emits the header
/// the streaming pipeline expects, reporting the valid (post-normalization) bit depth.
/// </summary>
private byte[] NormalizeToStandardPcm(byte[] buffer, WavMetadata metadata)
private static byte[] BuildStandardPcmHeader(int channels, int sampleRate, int outBitsPerSample, long dataSize)
{
// Clamp the declared data size to what is actually present; some encoders overshoot.
var dataStart = metadata.DataChunkPos + 8;
var available = buffer.Length - dataStart;
var srcDataSize = Math.Min(metadata.DataSize, available);
byte[] dataBytes;
int outBitsPerSample;
if (metadata.IsFloat)
{
dataBytes = ConvertFloatTo24BitPcm(buffer, dataStart, srcDataSize);
outBitsPerSample = 24;
}
else if (metadata.IsPaddedContainer)
{
dataBytes = RepackPaddedContainer(buffer, dataStart, srcDataSize, metadata.ContainerBitsPerSample, metadata.BitsPerSample);
outBitsPerSample = metadata.BitsPerSample;
}
else
{
dataBytes = new byte[srcDataSize];
Array.Copy(buffer, dataStart, dataBytes, 0, srcDataSize);
outBitsPerSample = metadata.BitsPerSample;
}
var dataSize = dataBytes.Length;
const int headerSize = 44;
var result = new byte[headerSize + dataSize];
var result = new byte[headerSize];
var blockAlign = (ushort)(metadata.Channels * (outBitsPerSample / 8));
var byteRate = (uint)(metadata.SampleRate * blockAlign);
var blockAlign = (ushort)(channels * (outBitsPerSample / 8));
var byteRate = (uint)(sampleRate * blockAlign);
// RIFF header
System.Text.Encoding.ASCII.GetBytes("RIFF").CopyTo(result, 0);
@@ -371,8 +614,8 @@ public class AudioProcessor
System.Text.Encoding.ASCII.GetBytes("fmt ").CopyTo(result, 12);
BitConverter.GetBytes((uint)16).CopyTo(result, 16);
BitConverter.GetBytes((ushort)1).CopyTo(result, 20); // audioFormat = PCM
BitConverter.GetBytes((ushort)metadata.Channels).CopyTo(result, 22);
BitConverter.GetBytes((uint)metadata.SampleRate).CopyTo(result, 24);
BitConverter.GetBytes((ushort)channels).CopyTo(result, 22);
BitConverter.GetBytes((uint)sampleRate).CopyTo(result, 24);
BitConverter.GetBytes(byteRate).CopyTo(result, 28);
BitConverter.GetBytes(blockAlign).CopyTo(result, 32);
BitConverter.GetBytes((ushort)outBitsPerSample).CopyTo(result, 34);
@@ -381,8 +624,6 @@ public class AudioProcessor
System.Text.Encoding.ASCII.GetBytes("data").CopyTo(result, 36);
BitConverter.GetBytes((uint)dataSize).CopyTo(result, 40);
Array.Copy(dataBytes, 0, result, headerSize, dataSize);
return result;
}
@@ -459,7 +700,7 @@ public class AudioProcessor
/// <summary>
/// Finds a chunk in the WAV file buffer with proper alignment handling
/// </summary>
private int FindChunk(byte[] buffer, string chunkId)
private static int FindChunk(byte[] buffer, string chunkId)
{
var chunkBytes = System.Text.Encoding.ASCII.GetBytes(chunkId);
int offset = 12; // Start after RIFF header
@@ -556,4 +797,21 @@ public readonly record struct PcmData(
ReadOnlyMemory<byte> Pcm,
int Channels,
int SampleRate,
int BitsPerSample);
/// <summary>
/// Where a WAV's PCM data region lives and how to decode it, without the bytes themselves — the
/// streaming counterpart of <see cref="PcmData"/>. The caller seeks to <see cref="DataStart"/> and
/// streams exactly <see cref="DataLength"/> bytes through a loudness accumulator.
/// </summary>
/// <param name="DataStart">Absolute byte offset of the first PCM sample (past the data chunk header).</param>
/// <param name="DataLength">PCM region length in bytes, clamped to what the backing file actually holds.</param>
/// <param name="Channels">Number of interleaved channels.</param>
/// <param name="SampleRate">Samples per second.</param>
/// <param name="BitsPerSample">Bit depth per sample (8, 16, 24, or 32).</param>
public readonly record struct WavPcmStreamInfo(
long DataStart,
long DataLength,
int Channels,
int SampleRate,
int BitsPerSample);
@@ -24,18 +24,18 @@ public class AudioProcessorRouter
}
/// <summary>
/// Processes <paramref name="filePath"/> with the processor matching its extension, returning an
/// <see cref="AudioBinary"/> carrying the stored bytes and extracted metadata. Throws
/// <see cref="ArgumentException"/> for unsupported extensions.
/// Processes <paramref name="filePath"/> with the processor matching its extension, returning a
/// <see cref="ProcessedAudio"/> store plan (extracted metadata plus a streamed writer for the
/// canonical vault bytes). Throws <see cref="ArgumentException"/> for unsupported extensions.
/// </summary>
public async Task<AudioBinary?> ProcessAudioFileAsync(string filePath)
public async Task<ProcessedAudio?> ProcessAudioFileAsync(string filePath, CancellationToken cancellationToken = default)
{
var ext = Path.GetExtension(filePath).ToLowerInvariant();
return ext switch
{
".wav" => await _wavProcessor.ProcessWavFileAsync(filePath),
".mp3" => await _mp3Processor.ProcessMp3FileAsync(filePath),
".flac" => await _flacProcessor.ProcessFlacFileAsync(filePath),
".wav" => await _wavProcessor.ProcessWavFileAsync(filePath, cancellationToken),
".mp3" => await _mp3Processor.ProcessMp3FileAsync(filePath, cancellationToken),
".flac" => await _flacProcessor.ProcessFlacFileAsync(filePath, cancellationToken),
_ => throw new ArgumentException($"Unsupported audio format: {ext}", nameof(filePath)),
};
}
@@ -0,0 +1,58 @@
namespace DeepDrftContent.Processors;
/// <summary>
/// Bounded-buffer streaming primitives shared by the audio processors on the store path. None of
/// these hold the whole file in memory: copies move a fixed window at a time, and the header read
/// caps its allocation regardless of file size.
/// </summary>
internal static class AudioStoreStream
{
private const int CopyBufferSize = 81920; // 80 KB — matches the controller staging copy.
/// <summary>
/// Bounded disk-to-disk copy of <paramref name="sourcePath"/> into <paramref name="destination"/>.
/// Used for passthrough formats whose stored bytes equal the source bytes. Hand-rolled rather than
/// <see cref="Stream.CopyToAsync(Stream)"/> because <c>FileStream</c>'s override writes in 128 KB
/// blocks; this keeps every write at or below <see cref="CopyBufferSize"/>, so peak managed memory
/// is provably O(buffer), never O(filesize).
/// </summary>
public static async Task CopyFileAsync(string sourcePath, Stream destination, CancellationToken ct)
{
await using var src = new FileStream(
sourcePath, FileMode.Open, FileAccess.Read, FileShare.Read,
bufferSize: CopyBufferSize, useAsync: true);
var buffer = new byte[CopyBufferSize];
int read;
while ((read = await src.ReadAsync(buffer, ct)) > 0)
{
await destination.WriteAsync(buffer.AsMemory(0, read), ct);
}
}
/// <summary>
/// Reads at most <paramref name="cap"/> bytes from the start of <paramref name="path"/> — enough
/// for header/metadata parsing without loading the (potentially ~GB) body. Bounds the allocation
/// at <c>min(cap, fileLength)</c>. Size-based metadata (e.g. average bitrate) must use the true
/// file length, supplied separately, not the prefix length.
/// </summary>
public static async Task<byte[]> ReadPrefixAsync(string path, long cap, CancellationToken ct)
{
await using var fs = new FileStream(
path, FileMode.Open, FileAccess.Read, FileShare.Read,
bufferSize: CopyBufferSize, useAsync: true);
var length = (int)Math.Min(cap, fs.Length);
var buffer = new byte[length];
var total = 0;
while (total < length)
{
var read = await fs.ReadAsync(buffer.AsMemory(total, length - total), ct);
if (read == 0)
break;
total += read;
}
return total == length ? buffer : buffer[..total];
}
}
@@ -12,7 +12,11 @@ public class FlacAudioProcessor
private const double FallbackDuration = 180.0;
private const int FallbackBitrate = 1411;
public async Task<AudioBinary?> ProcessFlacFileAsync(string filePath)
// STREAMINFO is mandatory and always the first metadata block, immediately after the 4-byte magic
// (data at offset 8, 34 bytes). A small prefix read covers it without loading the body.
private const long HeaderCap = 64 * 1024;
public async Task<ProcessedAudio?> ProcessFlacFileAsync(string filePath, CancellationToken cancellationToken = default)
{
if (!File.Exists(filePath))
{
@@ -24,25 +28,21 @@ public class FlacAudioProcessor
throw new ArgumentException("File must be a FLAC file", nameof(filePath));
}
var buffer = await File.ReadAllBytesAsync(filePath);
var meta = ExtractFlacMetadata(buffer);
var fileLength = new FileInfo(filePath).Length;
var window = await AudioStoreStream.ReadPrefixAsync(filePath, HeaderCap, cancellationToken);
var meta = ExtractFlacMetadata(window, fileLength);
var parameters = new AudioBinaryParams(
Buffer: buffer,
Size: buffer.Length,
Extension: ".flac",
Duration: meta.Duration,
Bitrate: meta.Bitrate);
return new AudioBinary(parameters);
// FLAC is stored unmodified — passthrough the original bytes via a streamed disk-to-disk copy.
return ProcessedAudio.Passthrough(filePath, ".flac", meta.Duration, meta.Bitrate, fileLength);
}
/// <summary>
/// Validates the <c>fLaC</c> magic and the leading STREAMINFO block, then computes duration from
/// total-samples / sample-rate and average bitrate from file size. On any parse failure, logs a
/// warning and returns synthetic defaults — never throws.
/// warning and returns synthetic defaults — never throws. <paramref name="fileLength"/> is the true
/// file size (the header window may be shorter), used for the average-bitrate computation.
/// </summary>
private static FlacMetadata ExtractFlacMetadata(byte[] buffer)
private static FlacMetadata ExtractFlacMetadata(byte[] buffer, long fileLength)
{
try
{
@@ -84,7 +84,7 @@ public class FlacAudioProcessor
var duration = (double)totalSamples / sampleRate;
var bitrate = duration > 0
? (int)(buffer.LongLength * 8L / (duration * 1000))
? (int)(fileLength * 8L / (duration * 1000))
: FallbackBitrate;
return new FlacMetadata { Duration = duration, Bitrate = bitrate };
@@ -20,4 +20,46 @@ public interface ILoudnessAlgorithm
/// is 1. All zeros when the signal is silent (peak is 0) or no samples are present.
/// </returns>
double[] Compute(ReadOnlySpan<byte> pcmData, int channels, int sampleRate, int bitsPerSample, int bucketCount);
/// <summary>
/// Creates a stateful accumulator that reduces the same loudness profile from PCM fed in bounded
/// chunks rather than from one contiguous buffer. The streaming waveform path uses this so a long
/// track's PCM is never materialized whole in a managed <c>byte[]</c>. The accumulator's output is
/// byte-identical to <see cref="Compute"/> for the same total PCM, because <see cref="Compute"/> is
/// itself defined in terms of one — the single source of truth for the loudness reduction.
/// </summary>
/// <param name="pcmByteLength">
/// Total length of the PCM data region in bytes. Required up front because the bucket each frame
/// lands in is derived from the frame's position relative to the total frame count.
/// </param>
/// <param name="channels">Number of interleaved channels; averaged to mono per frame.</param>
/// <param name="sampleRate">Samples per second (used for the envelope-smoothing time base).</param>
/// <param name="bitsPerSample">Bit depth (8 unsigned, 16/24/32 signed) used to decode samples.</param>
/// <param name="bucketCount">Number of equal time slices to reduce the signal to.</param>
ILoudnessAccumulator CreateAccumulator(
long pcmByteLength, int channels, int sampleRate, int bitsPerSample, int bucketCount);
}
/// <summary>
/// Stateful, single-pass reducer for one loudness profile. Frames are fed via <see cref="Add"/> in
/// arbitrary (non-frame-aligned) chunks — a partial frame straddling a chunk boundary is carried
/// internally — and <see cref="Finish"/> emits the peak-normalized <c>double[bucketCount]</c>. Not
/// thread-safe; feed one stream sequentially. Reusable across the same stream's chunks only, not
/// across streams.
/// </summary>
public interface ILoudnessAccumulator
{
/// <summary>
/// Feeds the next run of PCM bytes (interleaved, little-endian). Need not be frame-aligned; bytes
/// that do not complete a frame are retained until the next call. Bytes past the total frame count
/// declared at construction are ignored, matching the whole-buffer path's trailing-partial-frame drop.
/// </summary>
void Add(ReadOnlySpan<byte> pcmChunk);
/// <summary>
/// Finalizes and returns the peak-normalized loudness profile (<c>double[bucketCount]</c>, each in
/// [0, 1]). All zeros for silence or a degenerate (no-frame) input. Call once, after the last
/// <see cref="Add"/>.
/// </summary>
double[] Finish();
}
+20 -17
View File
@@ -25,7 +25,13 @@ public class Mp3AudioProcessor
private const double FallbackDuration = 180.0;
private const int FallbackBitrate = 320;
public async Task<AudioBinary?> ProcessMp3FileAsync(string filePath)
// Metadata lives in the leading ID3v2 tag plus the first MPEG frame. Cap the header read so a
// large MP3 is not pulled into memory whole just to read it; a tag larger than this (very large
// embedded art) simply falls back to the CBR/default estimate, never an OOM. The body is stored
// by streaming the original file, not from this window.
private const long HeaderCap = 8 * 1024 * 1024;
public async Task<ProcessedAudio?> ProcessMp3FileAsync(string filePath, CancellationToken cancellationToken = default)
{
if (!File.Exists(filePath))
{
@@ -37,24 +43,21 @@ public class Mp3AudioProcessor
throw new ArgumentException("File must be an MP3 file", nameof(filePath));
}
var buffer = await File.ReadAllBytesAsync(filePath);
var meta = ExtractMp3Metadata(buffer);
var fileLength = new FileInfo(filePath).Length;
var window = await AudioStoreStream.ReadPrefixAsync(filePath, HeaderCap, cancellationToken);
var meta = ExtractMp3Metadata(window, fileLength);
var parameters = new AudioBinaryParams(
Buffer: buffer,
Size: buffer.Length,
Extension: ".mp3",
Duration: meta.Duration,
Bitrate: meta.Bitrate);
return new AudioBinary(parameters);
// MP3 is stored unmodified — passthrough the original bytes via a streamed disk-to-disk copy.
return ProcessedAudio.Passthrough(filePath, ".mp3", meta.Duration, meta.Bitrate, fileLength);
}
/// <summary>
/// Parses the first valid MPEG frame (after any ID3v2 tag) and any Xing/VBRI tag inside it.
/// On any parse failure, logs a warning and returns synthetic defaults — never throws.
/// <paramref name="fileLength"/> is the true file size (the header window may be shorter), used
/// for the CBR duration estimate.
/// </summary>
private static Mp3Metadata ExtractMp3Metadata(byte[] buffer)
private static Mp3Metadata ExtractMp3Metadata(byte[] buffer, long fileLength)
{
try
{
@@ -65,7 +68,7 @@ public class Mp3AudioProcessor
}
var header = DecodeFrameHeader(buffer, frameStart);
var duration = ComputeDuration(buffer, frameStart, header);
var duration = ComputeDuration(buffer, frameStart, header, fileLength);
return new Mp3Metadata { Duration = duration, Bitrate = header.BitrateKbps };
}
@@ -202,7 +205,7 @@ public class Mp3AudioProcessor
/// Computes duration from a Xing/Info or VBRI tag (accurate for VBR) when present; otherwise
/// falls back to the CBR estimate fileSize / (bitrate_kbps * 125). Guards divide-by-zero.
/// </summary>
private static double ComputeDuration(byte[] buffer, int frameStart, FrameHeader header)
private static double ComputeDuration(byte[] buffer, int frameStart, FrameHeader header, long fileLength)
{
var xingFrames = ReadXingFrameCount(buffer, frameStart, header);
if (xingFrames > 0 && header.SampleRate > 0)
@@ -216,10 +219,10 @@ public class Mp3AudioProcessor
return (double)vbriFrames * header.SamplesPerFrame / header.SampleRate;
}
// CBR fallback: bitrate_kbps * 1000 / 8 bytes per second = bitrate_kbps * 125.
// Exclude the ID3v2 tag bytes (everything before frameStart) from the estimate.
// CBR fallback: bitrate_kbps * 1000 / 8 bytes per second = bitrate_kbps * 125. Uses the true
// file length (not the bounded header window), excluding the ID3v2 tag bytes before frameStart.
var bytesPerSecond = header.BitrateKbps * 125;
return bytesPerSecond > 0 ? (double)(buffer.Length - frameStart) / bytesPerSecond : FallbackDuration;
return bytesPerSecond > 0 ? (double)(fileLength - frameStart) / bytesPerSecond : FallbackDuration;
}
/// <summary>
@@ -0,0 +1,140 @@
using System.Diagnostics;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
namespace DeepDrftContent.Processors.Opus;
/// <summary>
/// Encodes a source audio file (any format the source vault holds — WAV/MP3/FLAC) to Ogg Opus fullband
/// 320 kbps by shelling out to FFmpeg (libopus). FFmpeg is chosen over a managed encoder because it
/// muxes a correct Ogg container with accurate granule positions across every input format — the page
/// structure the seek-index walk depends on — which a raw libopus binding does not provide. The external
/// <c>ffmpeg</c> binary is therefore a host runtime prerequisite (flagged in the wave handoff).
/// </summary>
public sealed class FfmpegOpusEncoder
{
private readonly OpusTranscodeOptions _options;
private readonly ILogger<FfmpegOpusEncoder> _logger;
public FfmpegOpusEncoder(IOptions<OpusTranscodeOptions> options, ILogger<FfmpegOpusEncoder> logger)
{
_options = options.Value;
_logger = logger;
}
/// <summary>
/// Transcodes <paramref name="sourcePath"/> to an Ogg Opus file at <paramref name="destinationPath"/>.
/// Returns true on a clean exit with a non-empty output. Returns false (logged) on a non-zero exit,
/// a timeout, a missing ffmpeg binary, or any process failure — a transcode failure must never throw
/// to the caller (C6); the background worker treats false as "leave the track lossless-only".
/// </summary>
public async Task<bool> EncodeAsync(string sourcePath, string destinationPath, CancellationToken ct)
{
var ffmpeg = string.IsNullOrWhiteSpace(_options.FfmpegPath) ? "ffmpeg" : _options.FfmpegPath;
// -vn drops any cover-art video stream; -map a:0 takes the first audio stream; -ar 48000 forces
// fullband (Opus internally resamples to 48 kHz anyway, but stating it keeps granulepos math
// unambiguous); libopus VBR at the target bitrate; -f ogg for an explicit Ogg container; -y
// overwrites the (pre-created, empty) destination temp file.
var args = new[]
{
"-hide_banner", "-nostdin", "-loglevel", "error",
"-i", sourcePath,
"-vn", "-map", "a:0",
"-c:a", "libopus", "-b:a", $"{_options.BitrateKbps}k",
"-ar", "48000",
"-f", "ogg",
"-y", destinationPath,
};
var psi = new ProcessStartInfo(ffmpeg)
{
RedirectStandardError = true,
RedirectStandardOutput = true,
UseShellExecute = false,
CreateNoWindow = true,
};
foreach (var arg in args)
psi.ArgumentList.Add(arg);
using var process = new Process { StartInfo = psi };
try
{
if (!process.Start())
{
_logger.LogError("Opus transcode: ffmpeg failed to start ({Ffmpeg}).", ffmpeg);
return false;
}
}
catch (Exception ex)
{
// Most commonly a missing binary (Win32Exception "file not found"). This is the ops
// prerequisite failing — log loudly so it is unmistakable in the deploy logs.
_logger.LogError(ex,
"Opus transcode: could not launch ffmpeg ({Ffmpeg}). Is the ffmpeg binary installed on the host?",
ffmpeg);
return false;
}
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(ct);
timeout.CancelAfter(TimeSpan.FromSeconds(_options.TimeoutSeconds));
// Drain stderr concurrently — ffmpeg can block writing diagnostics if the pipe is not read.
var stderrTask = process.StandardError.ReadToEndAsync(timeout.Token);
try
{
await process.WaitForExitAsync(timeout.Token);
}
catch (OperationCanceledException) when (ct.IsCancellationRequested)
{
TryKill(process);
await SafeStderr(stderrTask); // observe to avoid unobserved-task warnings
throw; // genuine shutdown cancellation — let it propagate
}
catch (OperationCanceledException)
{
TryKill(process);
await SafeStderr(stderrTask); // observe to avoid unobserved-task warnings
_logger.LogError("Opus transcode: ffmpeg exceeded the {Timeout}s timeout for {Source}.",
_options.TimeoutSeconds, sourcePath);
return false;
}
var stderr = await SafeStderr(stderrTask);
if (process.ExitCode != 0)
{
_logger.LogError("Opus transcode: ffmpeg exited {Code} for {Source}. stderr: {Stderr}",
process.ExitCode, sourcePath, stderr);
return false;
}
if (!File.Exists(destinationPath) || new FileInfo(destinationPath).Length == 0)
{
_logger.LogError("Opus transcode: ffmpeg exited 0 but produced no output for {Source}.", sourcePath);
return false;
}
return true;
}
private void TryKill(Process process)
{
try
{
if (!process.HasExited)
process.Kill(entireProcessTree: true);
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Opus transcode: failed to kill timed-out ffmpeg process.");
}
}
private static async Task<string> SafeStderr(Task<string> stderrTask)
{
try { return await stderrTask; }
catch { return "<stderr unavailable>"; }
}
}
@@ -0,0 +1,65 @@
namespace DeepDrftContent.Processors.Opus;
/// <summary>
/// Wire-format constants for the Ogg-Opus derived artifacts. Centralised so the seek-index codec,
/// the page walker, and the tests agree on one set of magic numbers.
/// </summary>
public static class OggOpusConstants
{
/// <summary>Opus granule positions are always sample counts at 48 kHz, regardless of input rate.</summary>
public const double OpusSampleRate = 48000.0;
/// <summary>One seek-index entry per this many seconds of audio (OQ7 — 0.5 s buckets).</summary>
public const double SeekBucketSeconds = 0.5;
/// <summary>The Ogg page capture pattern "OggS" — every page starts with these four bytes.</summary>
public static ReadOnlySpan<byte> CapturePattern => "OggS"u8;
/// <summary>Magic signature opening an OpusHead identification header packet.</summary>
public static ReadOnlySpan<byte> OpusHeadSignature => "OpusHead"u8;
/// <summary>Magic signature opening an OpusTags comment header packet.</summary>
public static ReadOnlySpan<byte> OpusTagsSignature => "OpusTags"u8;
/// <summary>
/// Fixed size of an Ogg page header before the segment table: capture(4) + version(1) +
/// header-type(1) + granulepos(8) + serial(4) + sequence(4) + checksum(4) + page-segments(1).
/// </summary>
public const int OggPageHeaderSize = 27;
/// <summary>Byte offset of the 64-bit granule position within an Ogg page header.</summary>
public const int GranulePositionOffset = 6;
/// <summary>Byte offset of the page-segment count (the segment-table length) within the header.</summary>
public const int PageSegmentCountOffset = 26;
/// <summary>Sentinel granule position for a page that ends mid-packet (no usable timestamp).</summary>
public const ulong NoGranulePosition = 0xFFFFFFFFFFFFFFFFUL;
/// <summary>
/// Minimum byte length of an <c>OpusHead</c> packet payload to safely read <c>pre_skip</c>.
/// RFC 7845 §5.1: "OpusHead"(8) + version(1) + channels(1) + pre_skip(2) = 12 bytes minimum.
/// </summary>
public const int OpusHeadMinSize = 12;
/// <summary>
/// Byte offset of <c>pre_skip</c> within the full <c>OpusHead</c> packet payload (including the
/// magic). RFC 7845 §5.1: "OpusHead"(8) + version(1) + channels(1) = 10 bytes before pre_skip.
/// </summary>
public const int OpusHeadPreSkipOffset = 10;
/// <summary>
/// Header size of the serialized seek-index blob:
/// totalBytes(8) + duration(8) + count(4) + preSkip(2) + reserved(2) = 24 bytes.
/// </summary>
public const int SeekIndexHeaderSize = 24;
/// <summary>Size of one serialized seek point: granulepos(8) + byteOffset(8).</summary>
public const int SeekPointSize = 16;
/// <summary>Vault-resource extension for the Opus audio bytes.</summary>
public const string OpusExtension = ".opus";
/// <summary>Vault-resource extension for the combined setup-header + seek-index sidecar.</summary>
public const string SidecarExtension = ".opusidx";
}
@@ -0,0 +1,310 @@
using System.Buffers.Binary;
namespace DeepDrftContent.Processors.Opus;
/// <summary>
/// The result of walking an encoded Ogg Opus stream once: the captured setup header (the leading
/// <c>OpusHead</c> + <c>OpusTags</c> pages, verbatim) and the bucketed granule→byte seek index. This
/// is everything the sidecar artifact carries (§3.4a) — built at transcode time so delivery never
/// re-walks the stream.
/// </summary>
/// <param name="SetupHeaderBytes">The leading setup pages (OpusHead + OpusTags), exactly as they
/// appear at the start of the stream, ready to prepend to any mid-stream page run before decode.</param>
/// <param name="SeekIndex">The accurate, 0.5 s-bucketed granule→byte transfer function.</param>
public sealed record OggOpusWalk(byte[] SetupHeaderBytes, OggOpusSeekIndex SeekIndex);
/// <summary>
/// Pure Ogg-Opus stream walker. Reads the page structure directly (the <c>OggS</c> capture pattern and
/// the 27-byte page header) to (1) capture the setup-header pages and (2) record, for every audio page,
/// its end granule position and exact byte offset — bucketed to 0.5 s with each bucket boundary snapped
/// to the nearest enclosing page start. No external dependency: the encoder (FFmpeg) produces the bytes;
/// this turns them into the seek artifact deterministically, so it is unit-testable without a codec.
/// </summary>
/// <remarks>
/// Two entry points share one <see cref="WalkState"/> page-processing core, so they produce byte-identical
/// output by construction (the project's parity-oracle convention, mirroring
/// <c>RmsLoudnessAlgorithm.Compute</c> over its accumulator):
/// <list type="bullet">
/// <item><see cref="Walk(ReadOnlySpan{byte})"/> — the whole-buffer overload, retained as the byte-identity
/// parity oracle for the streaming variant.</item>
/// <item><see cref="WalkAsync(System.IO.Stream,System.Threading.CancellationToken)"/> — the streaming
/// variant: walks the page structure from a forward stream in a bounded read buffer (one Ogg page at a
/// time), so peak managed memory is O(buffer + seek-index + setup-header), independent of file size.</item>
/// </list>
/// </remarks>
public static class OggOpusParser
{
/// <summary>
/// The largest a single Ogg page can be: header(27) + a full 255-entry segment table + the maximum
/// payload those segments can describe (255 × 255 bytes). The streaming read buffer is floored at this
/// so a complete page always fits, which means a short read on a page can only mean a truncated stream.
/// </summary>
private const int MaxOggPageSize = OggOpusConstants.OggPageHeaderSize + 255 + 255 * 255; // 65307
/// <summary>
/// Walks <paramref name="oggBytes"/> and produces the setup header + seek index, or null if the
/// bytes are not a recognisable Ogg Opus stream (no setup header, no audio pages, or truncated
/// structure). A null is the caller's signal to treat the transcode as failed and leave the track
/// lossless-only (C6) — it does not throw for malformed input.
/// </summary>
public static OggOpusWalk? Walk(ReadOnlySpan<byte> oggBytes)
{
var state = new WalkState();
var offset = 0;
while (offset + OggOpusConstants.OggPageHeaderSize <= oggBytes.Length)
{
var page = oggBytes.Slice(offset);
if (!page[..4].SequenceEqual(OggOpusConstants.CapturePattern))
{
// Not on a page boundary — the encoder writes contiguous pages, so this means the
// stream is malformed or we mis-stepped. Either way it is unrecoverable here.
return null;
}
var segmentCount = page[OggOpusConstants.PageSegmentCountOffset];
var segmentTableEnd = OggOpusConstants.OggPageHeaderSize + segmentCount;
if (segmentTableEnd > page.Length)
return null; // truncated header
var payloadSize = 0;
for (var i = 0; i < segmentCount; i++)
payloadSize += page[OggOpusConstants.OggPageHeaderSize + i];
var pageTotalSize = segmentTableEnd + payloadSize;
if (pageTotalSize > page.Length)
return null; // truncated payload
var payload = page.Slice(segmentTableEnd, payloadSize);
var granule = BinaryPrimitives.ReadUInt64LittleEndian(
page.Slice(OggOpusConstants.GranulePositionOffset, 8));
state.AddPage(granule, payload, page.Slice(0, pageTotalSize), offset);
offset += pageTotalSize;
}
return state.Finish((ulong)oggBytes.Length);
}
/// <summary>
/// Streaming counterpart of <see cref="Walk(ReadOnlySpan{byte})"/>: walks the Ogg page structure from
/// a forward <paramref name="stream"/> in a bounded read buffer (one Ogg page at a time), producing a
/// byte-identical <see cref="OggOpusWalk"/> without ever holding the whole encoded file in memory.
/// Returns null on the same malformed/truncated conditions as the buffer overload — it does not throw
/// for bad input (only <see cref="OperationCanceledException"/> propagates on cancellation).
/// </summary>
public static Task<OggOpusWalk?> WalkAsync(Stream stream, CancellationToken cancellationToken = default)
=> WalkAsync(stream, MaxOggPageSize, cancellationToken);
/// <summary>
/// Buffer-size-parameterised core. <paramref name="bufferSize"/> is floored at
/// <see cref="MaxOggPageSize"/> so any single page always fits in the buffer; the absolute byte
/// cursor (<c>absoluteOffset</c>) advances as the window compacts, so recorded seek offsets stay
/// absolute even though the buffer holds only a small window at any instant.
/// </summary>
internal static async Task<OggOpusWalk?> WalkAsync(Stream stream, int bufferSize, CancellationToken cancellationToken)
{
var state = new WalkState();
var buffer = new byte[Math.Max(bufferSize, MaxOggPageSize)];
var len = 0; // valid bytes held at buffer[0..len]
long absoluteOffset = 0; // absolute stream position of buffer[0]
while (true)
{
// The buffer overload's loop guard requires a full fixed header before parsing a page; once
// fewer than that remain (and the stream is drained) it is the natural end of the stream.
if (len < OggOpusConstants.OggPageHeaderSize)
{
len += await FillAsync(stream, buffer, len, cancellationToken);
if (len < OggOpusConstants.OggPageHeaderSize)
break;
}
if (!buffer.AsSpan(0, 4).SequenceEqual(OggOpusConstants.CapturePattern))
return null;
var segmentCount = buffer[OggOpusConstants.PageSegmentCountOffset];
var segmentTableEnd = OggOpusConstants.OggPageHeaderSize + segmentCount;
if (len < segmentTableEnd)
{
len += await FillAsync(stream, buffer, len, cancellationToken);
if (len < segmentTableEnd)
return null; // truncated header
}
var payloadSize = 0;
for (var i = 0; i < segmentCount; i++)
payloadSize += buffer[OggOpusConstants.OggPageHeaderSize + i];
var pageTotalSize = segmentTableEnd + payloadSize;
if (len < pageTotalSize)
{
len += await FillAsync(stream, buffer, len, cancellationToken);
if (len < pageTotalSize)
return null; // truncated payload (page never fully arrived before EOF)
}
var page = buffer.AsSpan(0, pageTotalSize);
var granule = BinaryPrimitives.ReadUInt64LittleEndian(
page.Slice(OggOpusConstants.GranulePositionOffset, 8));
var payload = page.Slice(segmentTableEnd, payloadSize);
state.AddPage(granule, payload, page, absoluteOffset);
// Compact the consumed page off the front of the window; the absolute cursor advances by the
// exact page size so every offset the state records remains an absolute stream position.
var remaining = len - pageTotalSize;
if (remaining > 0)
Buffer.BlockCopy(buffer, pageTotalSize, buffer, 0, remaining);
len = remaining;
absoluteOffset += pageTotalSize;
}
return state.Finish((ulong)absoluteOffset + (ulong)len);
}
/// <summary>
/// Fills <paramref name="buffer"/> from <paramref name="offset"/> to its end, issuing as many reads as
/// needed until the buffer is full or the stream is exhausted. Returns the number of bytes added.
/// </summary>
private static async Task<int> FillAsync(Stream stream, byte[] buffer, int offset, CancellationToken ct)
{
var added = 0;
while (offset + added < buffer.Length)
{
var read = await stream.ReadAsync(buffer.AsMemory(offset + added, buffer.Length - offset - added), ct);
if (read == 0)
break;
added += read;
}
return added;
}
private static bool StartsWith(ReadOnlySpan<byte> payload, ReadOnlySpan<byte> signature) =>
payload.Length >= signature.Length && payload[..signature.Length].SequenceEqual(signature);
/// <summary>
/// The single page-processing core both <see cref="Walk(ReadOnlySpan{byte})"/> and
/// <see cref="WalkAsync(Stream,int,CancellationToken)"/> drive, page by page, in stream order. Holding
/// the setup-header + seek-index accumulation here is what makes the two entry points byte-identical
/// by construction: there is exactly one copy of the OpusHead/OpusTags detection, pre-skip correction,
/// t=0 anchoring, and 0.5 s bucketing logic.
/// </summary>
private sealed class WalkState
{
// The real setup (OpusHead + OpusTags pages) is a few KB; this cap bounds the streaming capture so
// a malformed head-without-tags stream cannot grow it unboundedly. A stream that exceeds it has no
// OpusTags within the cap, so no audio points are ever recorded and Finish returns null either way
// — the cap never changes the output of a stream that produces a non-null result.
private const int MaxSetupHeaderBytes = 8 * 1024 * 1024;
private bool _sawOpusHead;
private bool _sawOpusTags;
private ushort _preSkip;
private int _setupHeaderEnd = -1;
private bool _capturingSetup = true;
private readonly List<byte> _setupHeader = new();
private readonly List<OpusSeekPoint> _points = new();
private ulong _lastGranule;
private double _nextBucketTime;
private bool _firstAudioPointTaken;
/// <summary>
/// Processes one fully-framed page. <paramref name="pageBytes"/> is the whole page (header +
/// segment table + payload) for verbatim setup capture; <paramref name="absoluteOffset"/> is the
/// page's absolute start in the stream — the value recorded in the seek index.
/// </summary>
public void AddPage(ulong granule, ReadOnlySpan<byte> payload, ReadOnlySpan<byte> pageBytes, long absoluteOffset)
{
if (_capturingSetup)
{
if (_setupHeader.Count + pageBytes.Length > MaxSetupHeaderBytes)
_capturingSetup = false; // malformed: give up capture (result will be null without tags)
else
_setupHeader.AddRange(pageBytes);
}
// The setup pages carry no audio granule (OpusHead has granulepos 0; OpusTags too). They
// are the leading pages whose payload opens with the Opus magic signatures.
if (!_sawOpusHead && StartsWith(payload, OggOpusConstants.OpusHeadSignature))
{
_sawOpusHead = true;
_setupHeaderEnd = (int)(absoluteOffset + pageBytes.Length);
// RFC 7845 §5.1 — OpusHead layout after the 8-byte "OpusHead" magic:
// [0] version (1 byte), [1] channel count (1 byte),
// [2-3] pre_skip (little-endian uint16) ← at packet bytes 10-11
// pre_skip is the number of decoder samples to discard before presenting audio;
// all granule→time conversions must subtract it (RFC 7845 §4.3).
if (payload.Length >= OggOpusConstants.OpusHeadMinSize)
_preSkip = BinaryPrimitives.ReadUInt16LittleEndian(
payload.Slice(OggOpusConstants.OpusHeadPreSkipOffset, 2));
}
else if (_sawOpusHead && !_sawOpusTags && StartsWith(payload, OggOpusConstants.OpusTagsSignature))
{
_sawOpusTags = true;
_setupHeaderEnd = (int)(absoluteOffset + pageBytes.Length);
// The setup header ends at the OpusTags page; stop capturing so audio pages never grow it.
_capturingSetup = false;
}
else if (_sawOpusHead && _sawOpusTags)
{
// Audio page. Record the first audio page unconditionally (the seek anchor at t=0),
// then one entry per 0.5 s bucket. A page with no end-granule (mid-packet continuation,
// granulepos == -1) is skipped for indexing — its time is unknown — but still advances
// the byte cursor.
if (granule != OggOpusConstants.NoGranulePosition)
{
// RFC 7845 §4.3: presentation time = max(0, granule preSkip) / 48000.
// Use this corrected time for bucketing so that a stream with pre-skip 3840 (~80 ms)
// does not systematically offset every indexed time by that amount.
var correctedTime = Math.Max(0.0,
(granule - (double)_preSkip) / OggOpusConstants.OpusSampleRate);
if (!_firstAudioPointTaken)
{
// Anchor the first seek point at corrected time = 0 by storing the granule as
// preSkip. This guarantees that a binary search for t=0 ("largest entry with
// corrected time ≤ 0") always resolves to the first audio page's byte offset —
// even when the real granule is slightly above preSkip due to encoder lead-in.
_points.Add(new OpusSeekPoint(_preSkip, (ulong)absoluteOffset));
_firstAudioPointTaken = true;
_nextBucketTime = OggOpusConstants.SeekBucketSeconds;
}
else if (correctedTime >= _nextBucketTime)
{
_points.Add(new OpusSeekPoint(granule, (ulong)absoluteOffset));
// Advance past every bucket this page crossed so a long page does not emit a
// backlog of entries; the next bucket is the first boundary strictly after it.
while (_nextBucketTime <= correctedTime)
_nextBucketTime += OggOpusConstants.SeekBucketSeconds;
}
_lastGranule = granule;
}
}
}
/// <summary>
/// Produces the final walk, or null on the same conditions the buffer overload rejected:
/// no OpusHead, no captured setup header, or no audio seek points. <paramref name="totalByteLength"/>
/// is the full stream length, recorded for end-of-stream seek clamping.
/// </summary>
public OggOpusWalk? Finish(ulong totalByteLength)
{
if (!_sawOpusHead || _setupHeaderEnd < 0 || _points.Count == 0)
return null;
var setupHeader = _setupHeader.ToArray();
// RFC 7845 §4.3: total duration is also pre-skip-corrected, matching the time a listener
// experiences (the last audio page's corrected time, clamped to ≥ 0).
var totalDuration = Math.Max(0.0,
(_lastGranule - (double)_preSkip) / OggOpusConstants.OpusSampleRate);
var index = new OggOpusSeekIndex(_points, totalDuration, totalByteLength, _preSkip);
return new OggOpusWalk(setupHeader, index);
}
}
}
@@ -0,0 +1,124 @@
using System.Buffers.Binary;
namespace DeepDrftContent.Processors.Opus;
/// <summary>
/// A single seek-index entry: an authoritative 48 kHz <see cref="GranulePosition"/> (Opus granule
/// positions are always sample counts at 48 kHz) paired with the exact byte offset of the Ogg page that
/// carries it. Every <see cref="ByteOffset"/> is a real page-start boundary, so a
/// <c>Range: bytes={ByteOffset}-</c> fetch lands the decoder Ogg-sync-aligned.
/// </summary>
/// <remarks>
/// Per RFC 7845 §4.3, the PCM presentation time is <c>(granulepos preSkip) / 48000</c>. The raw
/// <see cref="GranulePosition"/> is stored here as-is; callers should subtract the containing
/// <see cref="OggOpusSeekIndex.PreSkip"/> before converting to a presentation time. Use
/// <see cref="OggOpusSeekIndex.PresentationTimeSeconds"/> for the corrected value.
/// </remarks>
/// <param name="GranulePosition">The page's end granule position (48 kHz sample count).</param>
/// <param name="ByteOffset">The byte offset of the page start in the Opus file.</param>
public readonly record struct OpusSeekPoint(ulong GranulePosition, ulong ByteOffset)
{
/// <summary>
/// Raw granule-position-to-time conversion (granulepos / 48 kHz). Does NOT subtract pre-skip — use
/// <see cref="OggOpusSeekIndex.PresentationTimeSeconds"/> for the RFC 7845-correct presentation time.
/// </summary>
public double RawTimeSeconds => GranulePosition / OggOpusConstants.OpusSampleRate;
}
/// <summary>
/// The accurate, precomputed transfer function from seek-time to true file byte offset for one Ogg
/// Opus stream (§3.4a A). Built once at transcode time by walking the encoded stream; the client reads
/// it back and binary-searches <see cref="Points"/> instead of doing inaccurate VBR byte-rate math.
/// One entry per 0.5 s of audio (<see cref="OggOpusConstants.SeekBucketSeconds"/>), each snapped to the
/// nearest enclosing page start, plus the totals needed to clamp a seek to range.
/// </summary>
/// <param name="Points">Ordered (granulepos, byteOffset) entries, ascending. The first entry always
/// has <see cref="OpusSeekPoint.GranulePosition"/> == <paramref name="PreSkip"/> (corrected time = 0)
/// and points at the first audio page start, ensuring a seek to t=0 always resolves.</param>
/// <param name="TotalDurationSeconds">
/// Pre-skip-corrected total stream duration: <c>max(0, lastGranule preSkip) / 48000</c>.
/// </param>
/// <param name="TotalByteLength">Total Opus file byte length, for clamping a seek past the end.</param>
/// <param name="PreSkip">
/// The <c>pre_skip</c> value from the <c>OpusHead</c> identification header (RFC 7845 §5.1). Opus
/// decoders must discard this many samples from the decoded start before presenting audio. The client
/// (wave 18.4) needs this to trim the first decoded buffer; storing it here avoids a re-parse of the
/// Ogg stream at delivery time.
/// </param>
public sealed record OggOpusSeekIndex(
IReadOnlyList<OpusSeekPoint> Points,
double TotalDurationSeconds,
ulong TotalByteLength,
ushort PreSkip)
{
/// <summary>
/// Returns the RFC 7845-correct presentation time for a seek point: <c>max(0, granule preSkip) / 48000</c>.
/// Use this for all time comparisons; raw <see cref="OpusSeekPoint.RawTimeSeconds"/> omits the pre-skip.
/// </summary>
public double PresentationTimeSeconds(OpusSeekPoint point) =>
Math.Max(0.0, (point.GranulePosition - (double)PreSkip) / OggOpusConstants.OpusSampleRate);
/// <summary>
/// Serializes the index to the compact little-endian binary blob the sidecar stores. Layout:
/// <c>[uint64 totalByteLength][double totalDurationSeconds][uint32 pointCount][uint16 preSkip][uint16 reserved]</c>
/// then <c>pointCount × (uint64 granulepos, uint64 byteOffset)</c>. The four-byte preSkip+reserved
/// region pads the header to 24 bytes, keeping the point table 8-byte-aligned.
/// Fixed-width records keep the client parse to a single typed-array read.
/// </summary>
public byte[] ToBytes()
{
var size = OggOpusConstants.SeekIndexHeaderSize + Points.Count * OggOpusConstants.SeekPointSize;
var bytes = new byte[size];
var span = bytes.AsSpan();
BinaryPrimitives.WriteUInt64LittleEndian(span[..8], TotalByteLength);
BinaryPrimitives.WriteDoubleLittleEndian(span.Slice(8, 8), TotalDurationSeconds);
BinaryPrimitives.WriteUInt32LittleEndian(span.Slice(16, 4), (uint)Points.Count);
BinaryPrimitives.WriteUInt16LittleEndian(span.Slice(20, 2), PreSkip);
// bytes 22-23: reserved (zero-initialized by array allocation)
var cursor = OggOpusConstants.SeekIndexHeaderSize;
foreach (var point in Points)
{
BinaryPrimitives.WriteUInt64LittleEndian(span.Slice(cursor, 8), point.GranulePosition);
BinaryPrimitives.WriteUInt64LittleEndian(span.Slice(cursor + 8, 8), point.ByteOffset);
cursor += OggOpusConstants.SeekPointSize;
}
return bytes;
}
/// <summary>
/// Parses a blob produced by <see cref="ToBytes"/>. Returns null if the blob is too short or its
/// declared point count does not fit — the storage contract is exact, so a malformed blob is a
/// corruption signal, not a recoverable shape. (Provided so tests and any future server-side reader
/// share one codec with the writer.)
/// </summary>
public static OggOpusSeekIndex? FromBytes(ReadOnlySpan<byte> bytes)
{
if (bytes.Length < OggOpusConstants.SeekIndexHeaderSize)
return null;
var totalByteLength = BinaryPrimitives.ReadUInt64LittleEndian(bytes[..8]);
var totalDuration = BinaryPrimitives.ReadDoubleLittleEndian(bytes.Slice(8, 8));
var count = BinaryPrimitives.ReadUInt32LittleEndian(bytes.Slice(16, 4));
var preSkip = BinaryPrimitives.ReadUInt16LittleEndian(bytes.Slice(20, 2));
// bytes 22-23: reserved — ignored on read for forward-compatibility
var expected = OggOpusConstants.SeekIndexHeaderSize + (long)count * OggOpusConstants.SeekPointSize;
if (bytes.Length < expected)
return null;
var points = new OpusSeekPoint[count];
var cursor = OggOpusConstants.SeekIndexHeaderSize;
for (var i = 0; i < count; i++)
{
var granule = BinaryPrimitives.ReadUInt64LittleEndian(bytes.Slice(cursor, 8));
var offset = BinaryPrimitives.ReadUInt64LittleEndian(bytes.Slice(cursor + 8, 8));
points[i] = new OpusSeekPoint(granule, offset);
cursor += OggOpusConstants.SeekPointSize;
}
return new OggOpusSeekIndex(points, totalDuration, totalByteLength, preSkip);
}
}
@@ -0,0 +1,57 @@
using System.Buffers.Binary;
namespace DeepDrftContent.Processors.Opus;
/// <summary>
/// The single derived sidecar artifact per track (§3.4a B, recommended design): the Opus setup header
/// (<c>OpusHead</c> + <c>OpusTags</c>) followed by the granule→byte seek index. The client fetches this
/// once on track load and parses it into its <c>OpusSeekData</c>, so it always has both the setup bytes
/// (to prepend to any mid-stream slice) and the accurate seek transfer function before it ever issues a
/// Range fetch — including a window that opens away from byte 0 (UC9).
/// </summary>
/// <param name="SetupHeaderBytes">The verbatim OpusHead + OpusTags pages.</param>
/// <param name="SeekIndex">The bucketed granule→byte seek index.</param>
public sealed record OpusSidecar(byte[] SetupHeaderBytes, OggOpusSeekIndex SeekIndex)
{
/// <summary>
/// Serializes to <c>[uint32 setupHeaderLength][setup-header bytes][seek-index blob]</c>. The
/// length prefix lets the client split the two regions with one read; the seek-index blob carries
/// its own self-describing header (<see cref="OggOpusSeekIndex.ToBytes"/>), so it needs no trailing
/// length.
/// </summary>
public byte[] ToBytes()
{
var indexBytes = SeekIndex.ToBytes();
var bytes = new byte[4 + SetupHeaderBytes.Length + indexBytes.Length];
var span = bytes.AsSpan();
BinaryPrimitives.WriteUInt32LittleEndian(span[..4], (uint)SetupHeaderBytes.Length);
SetupHeaderBytes.CopyTo(span.Slice(4));
indexBytes.CopyTo(span.Slice(4 + SetupHeaderBytes.Length));
return bytes;
}
/// <summary>
/// Parses a blob produced by <see cref="ToBytes"/>. Returns null on any structural inconsistency
/// (short blob, length prefix that overruns, or an unparseable index) — the format is exact, so a
/// malformed blob is corruption.
/// </summary>
public static OpusSidecar? FromBytes(ReadOnlySpan<byte> bytes)
{
if (bytes.Length < 4)
return null;
var setupLength = BinaryPrimitives.ReadUInt32LittleEndian(bytes[..4]);
var indexStart = 4 + (long)setupLength;
if (bytes.Length < indexStart)
return null;
var setupHeader = bytes.Slice(4, (int)setupLength).ToArray();
var index = OggOpusSeekIndex.FromBytes(bytes.Slice((int)indexStart));
if (index is null)
return null;
return new OpusSidecar(setupHeader, index);
}
}
@@ -0,0 +1,33 @@
namespace DeepDrftContent.Processors.Opus;
/// <summary>
/// Host-supplied configuration for the Opus transcode. The only operationally significant knob is
/// <see cref="FfmpegPath"/> — the transcode shells out to FFmpeg (libopus), which must be present on the
/// DeepDrftAPI host (see the wave handoff notes). Defaults target Ogg Opus fullband (48 kHz) at 320 kbps,
/// the artifact the spec fixes (§1).
/// </summary>
public sealed class OpusTranscodeOptions
{
/// <summary>
/// Path to the ffmpeg executable. Empty/null resolves to <c>"ffmpeg"</c> (found on PATH). Override
/// with an absolute path when the binary is not on the host PATH.
/// </summary>
public string FfmpegPath { get; set; } = "ffmpeg";
/// <summary>Target Opus bitrate in kbps. 320 kbps fullband is the fixed artifact quality (§1).</summary>
public int BitrateKbps { get; set; } = 320;
/// <summary>
/// Directory for the transient source/output files the transcode stages. Defaults to the system
/// temp path; the host overrides it to the data-disk upload-staging directory so large files never
/// land on the small RAM-backed <c>/tmp</c> tmpfs (same constraint the upload path already honours).
/// </summary>
public string StagingPath { get; set; } = Path.GetTempPath();
/// <summary>
/// Hard ceiling on a single transcode, in seconds. A run that exceeds it is killed and the track
/// stays lossless-only (C6). Generous by default — a 1 GB mix is CPU-expensive (§3.1) — but bounded
/// so a hung ffmpeg never wedges the background worker.
/// </summary>
public int TimeoutSeconds { get; set; } = 3600;
}
@@ -0,0 +1,189 @@
using DeepDrftContent.Constants;
using DeepDrftContent.FileDatabase.Models;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using FileDb = DeepDrftContent.FileDatabase.Services.FileDatabase;
namespace DeepDrftContent.Processors.Opus;
/// <summary>
/// Derives and persists a track's low-data Ogg Opus artifacts (Phase 18.1). Mirrors
/// <see cref="WaveformProfileService"/>'s derived-artifact lifecycle: compute from the stored source,
/// store in a dedicated vault keyed by <c>EntryKey</c>, regenerable, failure-tolerant. For one track it
/// produces two entries in the <see cref="VaultConstants.TrackOpus"/> vault — the Opus audio bytes and a
/// combined setup-header + seek-index sidecar (§3.4a). Strictly additive: the source <c>tracks</c> vault
/// is never touched, and a failure here leaves the track lossless-only and eligible for backfill (C2/C6).
/// </summary>
public sealed class OpusTranscodeService
{
private readonly FileDb _fileDatabase;
private readonly TrackContentService _trackContent;
private readonly FfmpegOpusEncoder _encoder;
private readonly OpusTranscodeOptions _options;
private readonly ILogger<OpusTranscodeService> _logger;
public OpusTranscodeService(
FileDb fileDatabase,
TrackContentService trackContent,
FfmpegOpusEncoder encoder,
IOptions<OpusTranscodeOptions> options,
ILogger<OpusTranscodeService> logger)
{
_fileDatabase = fileDatabase;
_trackContent = trackContent;
_encoder = encoder;
_options = options.Value;
_logger = logger;
}
/// <summary>
/// Reads the source audio for <paramref name="entryKey"/> from the <c>tracks</c> vault, transcodes it
/// to Ogg Opus 320, walks the encoded stream to build the seek index + capture the setup header, and
/// stores the Opus bytes and the sidecar in the <see cref="VaultConstants.TrackOpus"/> vault under the
/// same key. Re-runnable — a second call overwrites the prior artifacts (backfill / replace-audio).
/// Returns false (logged) on any failure; never throws for expected failure modes (C6). The only
/// propagated exception is <see cref="OperationCanceledException"/> on genuine shutdown.
/// </summary>
public async Task<bool> TranscodeAndStoreAsync(string entryKey, CancellationToken ct)
{
// Read the source extension + duration from the vault index (no body load) and open a streamed
// read over the source bytes — never the whole-buffer AudioBinary. A 92-min mix source is ~970 MB;
// buffering it (and the encoded output below) was the last unconverted store-path OOM violation.
var trackDuration = await _trackContent.GetAudioDurationAsync(entryKey) ?? 0.0;
var sourceMedia = await _trackContent.OpenAudioMediaStreamAsync(entryKey);
if (sourceMedia is null)
{
_logger.LogWarning("Opus transcode: no source audio in vault for {EntryKey}; skipping.", entryKey);
return false;
}
string? sourcePath = null;
string? opusPath = null;
try
{
// Stage the source to disk in bounded chunks so ffmpeg can read it by file path/extension.
// The inner finally disposes the source stream as soon as the copy is done — the read handle
// is not held across the (long) encode — and guarantees disposal even if staging setup throws.
try
{
Directory.CreateDirectory(_options.StagingPath);
sourcePath = Path.Combine(_options.StagingPath, $"opus-src-{Guid.NewGuid():N}{sourceMedia.Extension}");
opusPath = Path.Combine(_options.StagingPath, $"opus-out-{Guid.NewGuid():N}{OggOpusConstants.OpusExtension}");
await using var staging = new FileStream(
sourcePath, FileMode.Create, FileAccess.Write, FileShare.None,
bufferSize: 81920, useAsync: true);
await sourceMedia.Stream.CopyToAsync(staging, bufferSize: 81920, ct);
}
finally
{
await sourceMedia.DisposeAsync();
}
if (!await _encoder.EncodeAsync(sourcePath, opusPath, ct))
return false; // encoder already logged the cause
// Walk the encoded output from a streamed read in a bounded buffer (no whole-file load). The
// seek index and setup header are byte-identical to the buffer walk (parity-tested).
OggOpusWalk? walk;
await using (var opusIn = new FileStream(
opusPath, FileMode.Open, FileAccess.Read, FileShare.Read,
bufferSize: 81920, useAsync: true))
{
walk = await OggOpusParser.WalkAsync(opusIn, ct);
}
if (walk is null)
{
_logger.LogError(
"Opus transcode: ffmpeg produced output but the Ogg stream could not be walked for {EntryKey}; " +
"no artifacts stored.", entryKey);
return false;
}
await EnsureVaultAsync();
// Bitrate from the output file length + duration — both available without buffering the bytes.
var opusLength = new FileInfo(opusPath).Length;
var opusBitrate = trackDuration > 0
? (int)(opusLength * 8 / trackDuration / 1000)
: _options.BitrateKbps;
// Store the audio first, then the sidecar. If the sidecar write fails the Opus bytes are
// present but unseekable — treat that as a failed derive (return false) so a backfill re-runs
// it; do not leave a half-derived track that the delivery layer would treat as complete.
var audioMeta = MetaDataFactory.CreateAudioMetaData(
OpusAudioKey(entryKey), OggOpusConstants.OpusExtension, trackDuration, opusBitrate);
var stagedOpusPath = opusPath;
var audioStored = await _fileDatabase.RegisterResourceStreamingAsync(
VaultConstants.TrackOpus, OpusAudioKey(entryKey), audioMeta,
(destination, token) => AudioStoreStream.CopyFileAsync(stagedOpusPath, destination, token), ct);
if (!audioStored)
{
_logger.LogError("Opus transcode: vault write of Opus audio failed for {EntryKey}.", entryKey);
return false;
}
// The sidecar is the setup header (a few KB) plus the seek index (~16 bytes per 0.5 s bucket);
// it is inherently bounded and already in managed memory, so the whole-buffer write is correct.
var sidecar = new OpusSidecar(walk.SetupHeaderBytes, walk.SeekIndex).ToBytes();
var sidecarBinary = new MediaBinary(new MediaBinaryParams(
sidecar, sidecar.Length, OggOpusConstants.SidecarExtension));
var sidecarStored = await _fileDatabase.RegisterResourceAsync(
VaultConstants.TrackOpus, OpusSidecarKey(entryKey), sidecarBinary);
if (!sidecarStored)
{
_logger.LogError("Opus transcode: vault write of sidecar failed for {EntryKey}.", entryKey);
return false;
}
_logger.LogInformation(
"Opus transcode complete for {EntryKey}: {OpusBytes} bytes, {Points} seek points, {Duration:F1}s.",
entryKey, opusLength, walk.SeekIndex.Points.Count, walk.SeekIndex.TotalDurationSeconds);
return true;
}
catch (OperationCanceledException) when (ct.IsCancellationRequested)
{
throw;
}
catch (Exception ex)
{
_logger.LogError(ex, "Opus transcode failed for {EntryKey}; track stays lossless-only.", entryKey);
return false;
}
finally
{
if (sourcePath is not null)
TryDelete(sourcePath);
if (opusPath is not null)
TryDelete(opusPath);
}
}
/// <summary>The vault entry key under which a track's Opus audio bytes are stored.</summary>
public static string OpusAudioKey(string entryKey) => entryKey;
/// <summary>The vault entry key under which a track's setup-header + seek-index sidecar is stored.</summary>
public static string OpusSidecarKey(string entryKey) => $"{entryKey}-sidecar";
private async Task EnsureVaultAsync()
{
// The TrackOpus vault is created at host startup (Startup.cs), so this guard is normally a
// no-op for the upload path. It is retained for the backfill path, which may run via a
// standalone CLI or a host that skips vault pre-creation, where the vault might not exist.
if (!_fileDatabase.HasVault(VaultConstants.TrackOpus))
await _fileDatabase.CreateVaultAsync(VaultConstants.TrackOpus, MediaVaultType.Audio);
}
private void TryDelete(string path)
{
try
{
if (File.Exists(path))
File.Delete(path);
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Opus transcode: failed to delete staging file {Path}.", path);
}
}
}
@@ -0,0 +1,36 @@
using DeepDrftModels.Enums;
namespace DeepDrftContent.Processors.Opus;
/// <summary>
/// The outcome of resolving a track + requested <see cref="AudioFormat"/> to a concrete artifact
/// (Phase 18.2; read-path streaming). Carries an <em>open, seekable, disk-backed</em> <see cref="Stream"/>
/// over the artifact's bytes — never a buffered <c>byte[]</c>, so a ~220 MB Opus file or ~970 MB lossless
/// source is never materialized in a managed array per request. Also carries the content-type that matches
/// <em>what was actually returned</em>, and the format actually served — which may differ from the requested
/// one when the C2 fallback fires (Opus requested, no Opus artifact → the lossless artifact + its
/// content-type). The delivery layer (18.3) sets the response <c>Content-Type</c> from
/// <see cref="ContentType"/> so the eventual decoder picks the right decoder for the bytes it receives.
/// <para>
/// Ownership: the resolver opens the stream; the caller takes ownership. On the success path the caller hands
/// <see cref="Stream"/> to <c>File(..., enableRangeProcessing: true)</c>, which disposes it after the
/// response. On any pre-handoff throw the caller disposes this instance (which disposes the stream) so the
/// underlying <see cref="FileStream"/> never leaks — mirroring the lossless disk-stream path's catch-path
/// disposal.
/// </para>
/// </summary>
/// <param name="Stream">An open, seekable, disk-backed stream over the resolved artifact. The caller owns it.</param>
/// <param name="ContentType">The MIME type of the bytes in <paramref name="Stream"/> (e.g. <c>audio/ogg</c>
/// for Opus, or the source's real MIME for lossless).</param>
/// <param name="ResolvedFormat">The format actually returned. Equal to the requested format on a direct
/// hit; <see cref="AudioFormat.Lossless"/> when an Opus request fell back.</param>
public sealed record ResolvedAudio(Stream Stream, string ContentType, AudioFormat ResolvedFormat)
: IDisposable, IAsyncDisposable
{
/// <summary>True when an Opus request was served the lossless artifact because no Opus existed (C2).</summary>
public bool DidFallBack(AudioFormat requested) => requested != ResolvedFormat;
public void Dispose() => Stream.Dispose();
public ValueTask DisposeAsync() => Stream.DisposeAsync();
}
@@ -0,0 +1,129 @@
using DeepDrftContent.Constants;
using DeepDrftContent.FileDatabase.Models;
using DeepDrftModels.Enums;
using Microsoft.Extensions.Logging;
using FileDb = DeepDrftContent.FileDatabase.Services.FileDatabase;
namespace DeepDrftContent.Processors.Opus;
/// <summary>
/// The server-side format resolution + sidecar lookup seam (Phase 18.2). Given a track's
/// <c>EntryKey</c> and a requested <see cref="AudioFormat"/>, returns the correct audio artifact and the
/// content-type that matches it; given an <c>EntryKey</c>, returns the Opus seek/setup sidecar bytes.
/// Downstream waves call this — 18.3 wires it behind the <c>?format=</c> stream param and serves the
/// sidecar over HTTP; this wave delivers only the seam, not the HTTP surface.
/// <para>
/// Additive and non-breaking (C2): the lossless branch streams the source exactly as the existing read
/// path does (via <see cref="TrackContentService.OpenAudioMediaStreamAsync"/>, a non-buffering disk
/// stream), and an Opus request for a track with no Opus artifact falls back to lossless rather than
/// failing. Mirrors the <see cref="WaveformProfileService"/> derived-artifact lookup precedent: read from
/// the dedicated vault, swallow misses to null (FileDatabase convention), let the caller decide.
/// </para>
/// <para>
/// Read-path streaming: artifacts are resolved as open, seekable, disk-backed <see cref="ResolvedAudio"/>
/// handles — never whole-file <c>byte[]</c> loads — so the delivery layer streams them straight to the
/// response (Range/206 honoured by the seekable <c>FileStream</c>) without buffering a ~220 MB Opus file
/// or a ~970 MB lossless source per request.
/// </para>
/// </summary>
public sealed class TrackFormatResolver
{
private readonly FileDb _fileDatabase;
private readonly TrackContentService _trackContentService;
private readonly ILogger<TrackFormatResolver> _logger;
public TrackFormatResolver(
FileDb fileDatabase,
TrackContentService trackContentService,
ILogger<TrackFormatResolver> logger)
{
_fileDatabase = fileDatabase;
_trackContentService = trackContentService;
_logger = logger;
}
/// <summary>
/// Resolves <paramref name="entryKey"/> + <paramref name="requestedFormat"/> to the audio artifact to
/// serve plus its content-type. <see cref="AudioFormat.Lossless"/> resolves the source artifact in the
/// <c>tracks</c> vault with its real MIME (WAV/MP3/FLAC). <see cref="AudioFormat.Opus"/> resolves the
/// derived Opus artifact (<c>audio/ogg</c>) when present, and <strong>falls back to lossless</strong>
/// when it is not (C2). Returns null only when even the lossless source is missing — i.e. the track has
/// no audio at all (an unknown key or a genuinely empty vault), the one case the caller treats as 404.
/// </summary>
public async Task<ResolvedAudio?> ResolveAsync(string entryKey, AudioFormat requestedFormat)
{
if (requestedFormat == AudioFormat.Opus)
{
var opusVault = _fileDatabase.GetVault(VaultConstants.TrackOpus);
if (opusVault is not null)
{
// Disk-backed, seekable stream over the Opus artifact — no whole-file buffer. The caller
// owns the stream (hands it to File(...) on success, disposes on a pre-handoff throw).
var opus = await opusVault.GetEntryStreamAsync(OpusTranscodeService.OpusAudioKey(entryKey));
if (opus is not null)
return new ResolvedAudio(
opus.Stream, MimeTypeExtensions.GetMimeType(opus.Extension), AudioFormat.Opus);
}
// C2 fallback: no Opus artifact yet (legacy row, not backfilled, or transcode failed). Degrade
// to lossless rather than 404 — Opus is strictly additive; its absence never means "no audio".
_logger.LogInformation(
"Opus requested for {EntryKey} but no Opus artifact exists; falling back to lossless.", entryKey);
}
return await ResolveLosslessAsync(entryKey);
}
/// <summary>
/// Resolves the lossless source artifact and its real MIME as a non-buffering disk stream — the existing
/// read path. Shared by the explicit-lossless branch and the Opus fallback so both produce identical
/// bytes + content-type. The returned stream is seekable, so the delivery layer's Range→206 still works.
/// </summary>
private async Task<ResolvedAudio?> ResolveLosslessAsync(string entryKey)
{
var source = await _trackContentService.OpenAudioMediaStreamAsync(entryKey);
if (source is null)
return null;
return new ResolvedAudio(
source.Stream, MimeTypeExtensions.GetMimeType(source.Extension), AudioFormat.Lossless);
}
/// <summary>
/// Returns the Opus setup-header + seek-index sidecar bytes for <paramref name="entryKey"/>, or null
/// when no sidecar is stored (no Opus artifact yet, or an older derive predating the sidecar). 18.3
/// serves these on their own path; 18.4 fetches them once on track load and parses them into the
/// client's <c>OpusSeekData</c>. The bytes are the raw <see cref="OpusSidecar"/> blob
/// (<c>[uint32 setupHeaderLength][setup-header][seek-index]</c>) exactly as 18.1 stored them.
/// </summary>
public async Task<byte[]?> GetOpusSidecarAsync(string entryKey)
{
var sidecar = await _fileDatabase.LoadResourceAsync<MediaBinary>(
VaultConstants.TrackOpus, OpusTranscodeService.OpusSidecarKey(entryKey));
return sidecar?.Buffer;
}
/// <summary>
/// Reports whether <paramref name="entryKey"/> already has a complete Opus derive — both the audio bytes
/// AND the seek/setup sidecar present in the <c>track-opus</c> vault. The Backfill-Opus pass (18.5) uses
/// this to enqueue only tracks that are missing or half-derived (audio without sidecar = unseekable, so
/// treated as incomplete and re-derived). Both halves are required because the transcode stores them in
/// sequence and a sidecar-write failure leaves a track the delivery layer must not treat as Opus-ready.
/// </summary>
public async Task<bool> HasOpusAsync(string entryKey)
{
// Index-only existence — never read a file body. The opus-status admin endpoint calls this in a loop
// over the entire catalogue, so a body load here would stream the whole library's audio sequentially.
// HasIndexEntry is a pure in-memory index lookup (no disk read, no allocation per track).
var opusVault = _fileDatabase.GetVault(VaultConstants.TrackOpus);
if (opusVault is null)
return false;
if (!await opusVault.HasIndexEntry(OpusTranscodeService.OpusAudioKey(entryKey)))
return false;
// Both halves required: audio without the seek/setup sidecar is unseekable, so a half-derived track
// counts as not-having-Opus (the same completeness rule the Backfill-Opus pass enqueues against).
return await opusVault.HasIndexEntry(OpusTranscodeService.OpusSidecarKey(entryKey));
}
}
@@ -0,0 +1,68 @@
namespace DeepDrftContent.Processors;
/// <summary>
/// The product of processing an uploaded audio file on the store path: the metadata SQL and the
/// vault index need, plus a streamed writer that emits the canonical vault bytes to a destination
/// stream without ever materializing the whole file in a managed <c>byte[]</c>.
///
/// This replaces the former whole-file <c>AudioBinary</c> as the processor output for upload /
/// replace-audio (Wave 1 OOM fix): passthrough formats (standard-PCM WAV, MP3, FLAC) stream the
/// source file straight to the destination, and EXTENSIBLE WAVs stream their normalization to
/// standard PCM. The vault <em>load</em> path still uses <c>AudioBinary</c> (a full buffer) — that
/// is the Wave 2 read path and is out of scope here.
///
/// <see cref="WriteToAsync"/> is invoked exactly once by the streaming vault register, against the
/// freshly opened backing <see cref="System.IO.FileStream"/>. The writer re-opens the source file
/// itself, so the source (a staging file) must still exist when the register runs — it does, because
/// processing and registration are sequential within the store call, before the staging-file
/// <c>finally</c> cleanup.
/// </summary>
public sealed class ProcessedAudio
{
/// <summary>The stored file extension (e.g. <c>.wav</c>, <c>.mp3</c>, <c>.flac</c>).</summary>
public string Extension { get; }
/// <summary>Audio duration in seconds, extracted from the header.</summary>
public double Duration { get; }
/// <summary>Audio bitrate in kbps, extracted from (or estimated for) the header.</summary>
public int Bitrate { get; }
/// <summary>
/// The canonical stored byte count — computed from the header and file length, never by
/// buffering the body. Used only for diagnostics (confirming the streamed path was taken).
/// </summary>
public long Size { get; }
private readonly Func<Stream, CancellationToken, Task> _writeTo;
public ProcessedAudio(
string extension,
double duration,
int bitrate,
long size,
Func<Stream, CancellationToken, Task> writeTo)
{
Extension = extension;
Duration = duration;
Bitrate = bitrate;
Size = size;
_writeTo = writeTo;
}
/// <summary>
/// Streams the canonical vault bytes to <paramref name="destination"/>. Bounded-buffer — peak
/// managed memory is O(buffer), not O(filesize).
/// </summary>
public Task WriteToAsync(Stream destination, CancellationToken cancellationToken = default)
=> _writeTo(destination, cancellationToken);
/// <summary>
/// Builds a passthrough plan: the stored bytes are byte-identical to the source file (standard
/// PCM WAV, MP3, FLAC — no transcoding). The writer is a bounded disk-to-disk copy.
/// </summary>
public static ProcessedAudio Passthrough(
string sourcePath, string extension, double duration, int bitrate, long sourceLength)
=> new(extension, duration, bitrate, sourceLength,
(destination, ct) => AudioStoreStream.CopyFileAsync(sourcePath, destination, ct));
}
@@ -18,100 +18,27 @@ public class RmsLoudnessAlgorithm : ILoudnessAlgorithm
/// </summary>
public const double SmoothingTimeConstantSeconds = 0.005;
/// <summary>
/// Whole-buffer reduction. Defined in terms of <see cref="CreateAccumulator"/> so the streaming and
/// whole-buffer paths share one decode + finalize implementation — byte-identical output by
/// construction, not by parallel maintenance.
/// </summary>
public double[] Compute(ReadOnlySpan<byte> pcmData, int channels, int sampleRate, int bitsPerSample, int bucketCount)
{
var accumulator = CreateAccumulator(pcmData.Length, channels, sampleRate, bitsPerSample, bucketCount);
accumulator.Add(pcmData);
return accumulator.Finish();
}
public ILoudnessAccumulator CreateAccumulator(
long pcmByteLength, int channels, int sampleRate, int bitsPerSample, int bucketCount)
{
if (bucketCount <= 0)
{
throw new ArgumentOutOfRangeException(nameof(bucketCount), "Bucket count must be positive.");
}
var result = new double[bucketCount];
if (channels <= 0)
{
return result;
}
var bytesPerSample = bitsPerSample / 8;
if (bytesPerSample <= 0)
{
return result;
}
var bytesPerFrame = bytesPerSample * channels;
var frameCount = pcmData.Length / bytesPerFrame;
if (frameCount == 0)
{
return result;
}
// Sum of squared mono amplitudes and the frame count, per bucket. A frame's bucket is
// determined by its position in the timeline so buckets are equal-duration slices.
var sumSquares = new double[bucketCount];
var counts = new long[bucketCount];
for (var frame = 0; frame < frameCount; frame++)
{
var frameStart = frame * bytesPerFrame;
double channelSum = 0;
for (var ch = 0; ch < channels; ch++)
{
var sampleStart = frameStart + ch * bytesPerSample;
channelSum += ReadSampleNormalized(pcmData, sampleStart, bitsPerSample);
}
var mono = channelSum / channels;
// long math avoids overflow on large files before the divide back into bucket index.
var bucket = (int)((long)frame * bucketCount / frameCount);
if (bucket >= bucketCount)
{
bucket = bucketCount - 1;
}
sumSquares[bucket] += mono * mono;
counts[bucket]++;
}
for (var i = 0; i < bucketCount; i++)
{
if (counts[i] > 0)
{
result[i] = Math.Sqrt(sumSquares[i] / counts[i]);
}
}
// Envelope smoothing (~15 ms): round the spikey per-bucket RMS into a smooth contour before
// peak-normalization, so the rendered ribbon reads as a continuous curve, not faceted polygons.
// Each bucket spans (totalSeconds / bucketCount) of audio; the filter coefficient is derived
// from that against the time constant so the smoothing is duration-aware, not a fixed window.
var totalSeconds = (double)frameCount / sampleRate;
var bucketSeconds = totalSeconds / bucketCount;
SmoothEnvelope(result, bucketSeconds);
var peak = 0.0;
for (var i = 0; i < bucketCount; i++)
{
if (result[i] > peak)
{
peak = result[i];
}
}
if (peak <= 0)
{
// Silence — return all zeros (Array is already zero-initialized).
Array.Clear(result);
return result;
}
for (var i = 0; i < bucketCount; i++)
{
result[i] /= peak;
}
return result;
return new RmsLoudnessAccumulator(pcmByteLength, channels, sampleRate, bitsPerSample, bucketCount);
}
/// <summary>
@@ -122,7 +49,7 @@ public class RmsLoudnessAlgorithm : ILoudnessAlgorithm
/// each bucket blends <c>(1 a)</c> of itself with <c>a</c> of the running envelope. A near-zero
/// or non-finite bucket duration leaves the data untouched (nothing to smooth meaningfully).
/// </summary>
private static void SmoothEnvelope(double[] data, double bucketSeconds)
internal static void SmoothEnvelope(double[] data, double bucketSeconds)
{
if (data.Length < 2 || bucketSeconds <= 0 || !double.IsFinite(bucketSeconds))
{
@@ -154,7 +81,7 @@ public class RmsLoudnessAlgorithm : ILoudnessAlgorithm
/// Decodes one PCM sample at <paramref name="offset"/> to a normalized amplitude in [-1, 1].
/// 8-bit is unsigned (0..255, centered at 128); 16/24/32-bit are signed little-endian.
/// </summary>
private static double ReadSampleNormalized(ReadOnlySpan<byte> data, int offset, int bitsPerSample)
internal static double ReadSampleNormalized(ReadOnlySpan<byte> data, int offset, int bitsPerSample)
{
switch (bitsPerSample)
{
@@ -194,3 +121,167 @@ public class RmsLoudnessAlgorithm : ILoudnessAlgorithm
}
}
}
/// <summary>
/// Single-pass RMS accumulator backing <see cref="RmsLoudnessAlgorithm"/>. Frames are fed via
/// <see cref="Add"/> in arbitrary chunks; a partial frame straddling a chunk boundary is carried in a
/// one-frame buffer. The per-frame decode, bucket assignment, and per-bucket accumulation are the exact
/// arithmetic the former whole-buffer loop used, in the same frame order, so the floating-point result
/// is bit-identical whether the PCM arrives in one span or many. <see cref="Finish"/> applies the same
/// envelope smoothing and peak-normalization as before. Memory is O(bucketCount + one frame).
/// </summary>
public sealed class RmsLoudnessAccumulator : ILoudnessAccumulator
{
private readonly int _channels;
private readonly int _sampleRate;
private readonly int _bitsPerSample;
private readonly int _bucketCount;
private readonly int _bytesPerSample;
private readonly int _bytesPerFrame;
private readonly long _frameCount;
private readonly double[] _sumSquares;
private readonly long[] _counts;
private readonly byte[] _carry;
private int _carryLen;
private long _frameIndex;
internal RmsLoudnessAccumulator(long pcmByteLength, int channels, int sampleRate, int bitsPerSample, int bucketCount)
{
_channels = channels;
_sampleRate = sampleRate;
_bitsPerSample = bitsPerSample;
_bucketCount = bucketCount;
_sumSquares = new double[bucketCount];
_counts = new long[bucketCount];
// Guards mirror the former whole-buffer Compute exactly: any degenerate parameter leaves
// _frameCount at 0, so Add is a no-op and Finish returns the zero-initialized profile.
_bytesPerSample = bitsPerSample / 8;
if (channels <= 0 || _bytesPerSample <= 0)
{
_bytesPerFrame = 0;
_frameCount = 0;
_carry = [];
return;
}
_bytesPerFrame = _bytesPerSample * channels;
_frameCount = pcmByteLength / _bytesPerFrame;
_carry = new byte[_bytesPerFrame];
}
public void Add(ReadOnlySpan<byte> pcmChunk)
{
if (_frameIndex >= _frameCount)
{
return; // degenerate input, or every expected frame already consumed
}
var pos = 0;
// Complete a frame carried from the previous chunk first.
if (_carryLen > 0)
{
var need = _bytesPerFrame - _carryLen;
var take = Math.Min(need, pcmChunk.Length);
pcmChunk.Slice(0, take).CopyTo(_carry.AsSpan(_carryLen));
_carryLen += take;
pos += take;
if (_carryLen < _bytesPerFrame)
{
return; // still not a full frame
}
ProcessFrame(_carry);
_carryLen = 0;
if (_frameIndex >= _frameCount)
{
return;
}
}
// Whole frames directly from the chunk.
while (pos + _bytesPerFrame <= pcmChunk.Length && _frameIndex < _frameCount)
{
ProcessFrame(pcmChunk.Slice(pos, _bytesPerFrame));
pos += _bytesPerFrame;
}
// Stash a trailing partial frame for the next chunk — but only while frames are still expected.
// A trailing partial frame on the final chunk is dropped, matching the whole-buffer path.
if (_frameIndex < _frameCount && pos < pcmChunk.Length)
{
var remainder = pcmChunk.Slice(pos);
remainder.CopyTo(_carry);
_carryLen = remainder.Length;
}
}
private void ProcessFrame(ReadOnlySpan<byte> frame)
{
double channelSum = 0;
for (var ch = 0; ch < _channels; ch++)
{
channelSum += RmsLoudnessAlgorithm.ReadSampleNormalized(frame, ch * _bytesPerSample, _bitsPerSample);
}
var mono = channelSum / _channels;
// long math avoids overflow on large files before the divide back into bucket index.
var bucket = (int)(_frameIndex * _bucketCount / _frameCount);
if (bucket >= _bucketCount)
{
bucket = _bucketCount - 1;
}
_sumSquares[bucket] += mono * mono;
_counts[bucket]++;
_frameIndex++;
}
public double[] Finish()
{
var result = new double[_bucketCount];
if (_frameCount == 0)
{
return result; // degenerate input — all zeros, as the whole-buffer guards returned
}
for (var i = 0; i < _bucketCount; i++)
{
if (_counts[i] > 0)
{
result[i] = Math.Sqrt(_sumSquares[i] / _counts[i]);
}
}
// Envelope smoothing (~15 ms) then peak-normalization — identical to the whole-buffer finalize.
var totalSeconds = (double)_frameCount / _sampleRate;
var bucketSeconds = totalSeconds / _bucketCount;
RmsLoudnessAlgorithm.SmoothEnvelope(result, bucketSeconds);
var peak = 0.0;
for (var i = 0; i < _bucketCount; i++)
{
if (result[i] > peak)
{
peak = result[i];
}
}
if (peak <= 0)
{
Array.Clear(result);
return result;
}
for (var i = 0; i < _bucketCount; i++)
{
result[i] /= peak;
}
return result;
}
}
@@ -17,6 +17,10 @@ public class WaveformProfileService
{
private const string ProfileExtension = ".wfp";
/// <summary>Bounded read-buffer size for the streaming PCM pass — the only filesize-independent
/// allocation on the streaming path (matches the store path's 80 KB copy buffer).</summary>
private const int StreamReadBufferSize = 81920;
private readonly FileDb _fileDatabase;
private readonly AudioProcessor _audioProcessor;
private readonly ILoudnessAlgorithm _loudnessAlgorithm;
@@ -117,6 +121,161 @@ public class WaveformProfileService
return ComputeAndStoreAsync(wavBytes, entryKey, bucketCount, VaultConstants.TrackWaveforms);
}
/// <summary>
/// Streaming counterpart of <see cref="ComputeAndStoreAsync"/>: computes and stores the fixed
/// 512-bucket player-bar profile by reading the WAV from <paramref name="openWavStream"/> in bounded
/// chunks, never materializing the whole file in a managed <c>byte[]</c>. Tri-state result matches
/// the <c>RemoveResourceAsync</c> idiom so callers can map outcomes precisely: <c>null</c> = no audio
/// stream available (the entry has no backing audio); <c>false</c> = audio present but no profile
/// computable (non-WAV / float / padded) or the vault write failed; <c>true</c> = stored. Output is
/// byte-identical to the whole-buffer path for the same WAV.
/// </summary>
public Task<bool?> ComputeAndStoreProfileStreamingAsync(
Func<CancellationToken, Task<Stream?>> openWavStream,
string entryKey,
CancellationToken ct = default) =>
RunStreamingAsync(
openWavStream, entryKey,
[(_options.BucketCount, VaultConstants.WaveformProfiles)], ct);
/// <summary>
/// Streaming counterpart of <see cref="ComputeAndStoreHighResAsync"/>: computes and stores the
/// duration-derived high-res datum (<see cref="VaultConstants.TrackWaveforms"/>) by streaming the WAV
/// from <paramref name="openWavStream"/>. <paramref name="durationSeconds"/> drives the bucket count
/// exactly as the whole-buffer path's <c>audio.Duration</c> did — pass the same vault-metadata
/// duration to keep the stored bytes identical. Tri-state result as in
/// <see cref="ComputeAndStoreProfileStreamingAsync"/>.
/// </summary>
public Task<bool?> ComputeAndStoreHighResStreamingAsync(
Func<CancellationToken, Task<Stream?>> openWavStream,
string entryKey,
double durationSeconds,
CancellationToken ct = default) =>
RunStreamingAsync(
openWavStream, entryKey,
[(WaveformResolution.BucketCountForDuration(durationSeconds), VaultConstants.TrackWaveforms)], ct);
/// <summary>
/// Computes and stores BOTH datums a track carries — the 512-bucket profile and the duration-derived
/// high-res datum — from a SINGLE streaming pass over the WAV. One sequential read of the (possibly
/// ~GB) audio feeds two independent accumulators, so memory stays O(bucket arrays + read buffer) and
/// disk I/O is halved versus two separate passes. This is the upload / replace-audio hot path. Each
/// datum's stored bytes are byte-identical to its whole-buffer counterpart. Tri-state: <c>null</c> =
/// no audio stream; <c>false</c> = not WAV-decodable or a vault write failed; <c>true</c> = both
/// datums stored. Best-effort callers ignore the result.
/// </summary>
public Task<bool?> ComputeAndStoreAllStreamingAsync(
Func<CancellationToken, Task<Stream?>> openWavStream,
string entryKey,
double durationSeconds,
CancellationToken ct = default) =>
RunStreamingAsync(
openWavStream, entryKey,
[
(_options.BucketCount, VaultConstants.WaveformProfiles),
(WaveformResolution.BucketCountForDuration(durationSeconds), VaultConstants.TrackWaveforms),
],
ct);
/// <summary>
/// Core streaming reduction: opens the WAV once, parses its header (bounded), then streams the PCM
/// data region through one loudness accumulator per requested target, storing each datum. All
/// targets are computed in the single pass. See the tri-state contract on the public wrappers.
/// </summary>
private async Task<bool?> RunStreamingAsync(
Func<CancellationToken, Task<Stream?>> openWavStream,
string entryKey,
IReadOnlyList<(int BucketCount, string VaultName)> targets,
CancellationToken ct)
{
try
{
await using var stream = await openWavStream(ct);
if (stream is null)
{
// No backing audio for this entry — distinct from "present but undecodable".
return null;
}
var info = await _audioProcessor.TryReadPcmStreamInfoAsync(stream, stream.Length, ct);
if (info is null)
{
_logger.LogWarning(
"Waveform profile not computed for {EntryKey}: WAV PCM could not be extracted (streaming).",
entryKey);
return false;
}
var v = info.Value;
var accumulators = new ILoudnessAccumulator[targets.Count];
for (var i = 0; i < targets.Count; i++)
{
accumulators[i] = _loudnessAlgorithm.CreateAccumulator(
v.DataLength, v.Channels, v.SampleRate, v.BitsPerSample, targets[i].BucketCount);
}
await StreamPcmThroughAsync(stream, v.DataStart, v.DataLength, accumulators, ct);
_logger.LogInformation(
"Streaming waveform compute for {EntryKey}: {DataLength} PCM bytes, {TargetCount} datum(s), " +
"{BufferSize}B read buffer — no whole-file load.",
entryKey, v.DataLength, targets.Count, StreamReadBufferSize);
var allStored = true;
for (var i = 0; i < targets.Count; i++)
{
var profile = accumulators[i].Finish();
var quantized = Quantize(profile);
await EnsureVaultAsync(targets[i].VaultName);
var binary = new MediaBinary(new MediaBinaryParams(quantized, quantized.Length, ProfileExtension));
var stored = await _fileDatabase.RegisterResourceAsync(targets[i].VaultName, entryKey, binary);
if (!stored)
{
_logger.LogWarning(
"Waveform vault write failed for {EntryKey} in {VaultName}.", entryKey, targets[i].VaultName);
allStored = false;
}
}
return allStored;
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
_logger.LogError(ex, "Streaming waveform computation failed for {EntryKey}.", entryKey);
return false;
}
}
/// <summary>
/// Seeks to the PCM data region and streams exactly <paramref name="dataLength"/> bytes through each
/// accumulator in bounded reads. The accumulators carry partial frames internally, so the read
/// boundaries need not align to frames. Peak memory is one read buffer — independent of file size.
/// </summary>
private static async Task StreamPcmThroughAsync(
Stream stream, long dataStart, long dataLength, ILoudnessAccumulator[] accumulators, CancellationToken ct)
{
stream.Seek(dataStart, SeekOrigin.Begin);
var buffer = new byte[StreamReadBufferSize];
var remaining = dataLength;
while (remaining > 0)
{
var want = (int)Math.Min(buffer.Length, remaining);
var read = await stream.ReadAsync(buffer.AsMemory(0, want), ct);
if (read == 0)
break;
var span = buffer.AsSpan(0, read);
foreach (var accumulator in accumulators)
{
accumulator.Add(span);
}
remaining -= read;
}
}
/// <summary>
/// Returns the stored quantized profile bytes for a track from <paramref name="vaultName"/>
/// (defaults to <see cref="VaultConstants.WaveformProfiles"/> when null), or null if no profile
+159 -9
View File
@@ -40,13 +40,15 @@ public class TrackContentService
string? album = null,
string? genre = null,
DateOnly? releaseDate = null,
string? originalFileName = null)
string? originalFileName = null,
CancellationToken cancellationToken = default)
{
try
{
// Process the audio file (routed by extension)
var audioBinary = await _audioProcessorRouter.ProcessAudioFileAsync(audioFilePath);
if (audioBinary == null)
// Process the audio file (routed by extension). The returned plan carries metadata plus a
// streamed writer — no whole-file buffer (the store-path OOM fix).
var processed = await _audioProcessorRouter.ProcessAudioFileAsync(audioFilePath, cancellationToken);
if (processed == null)
{
throw new InvalidOperationException("Failed to process audio file");
}
@@ -60,8 +62,11 @@ public class TrackContentService
await _fileDatabase.CreateVaultAsync(VaultConstants.Tracks, MediaVaultType.Audio);
}
// Store the audio in FileDatabase
var success = await _fileDatabase.RegisterResourceAsync(VaultConstants.Tracks, trackId, audioBinary);
// Stream the audio into the vault. The metadata is supplied directly (there is no in-memory
// AudioBinary on this path), and the bytes are written progressively from the staging file.
var metaData = MetaDataFactory.CreateAudioMetaData(trackId, processed.Extension, processed.Duration, processed.Bitrate);
var success = await _fileDatabase.RegisterResourceStreamingAsync(
VaultConstants.Tracks, trackId, metaData, processed.WriteToAsync, cancellationToken);
if (!success)
{
throw new InvalidOperationException("Failed to store audio in FileDatabase");
@@ -74,7 +79,10 @@ public class TrackContentService
{
EntryKey = trackId, // FileDatabase entry ID
TrackName = trackName,
OriginalFileName = originalFileName
OriginalFileName = originalFileName,
// Persist the processor-extracted runtime to SQL so aggregate queries (total mix runtime)
// need not touch the vault. Same value the high-res waveform compute reads downstream.
DurationSeconds = processed.Duration
};
return trackEntity;
@@ -97,8 +105,91 @@ public class TrackContentService
string? album = null,
string? genre = null,
DateOnly? releaseDate = null,
string? originalFileName = null) =>
AddTrackAsync(wavFilePath, trackName, artist, album, genre, releaseDate, originalFileName);
string? originalFileName = null,
CancellationToken cancellationToken = default) =>
AddTrackAsync(wavFilePath, trackName, artist, album, genre, releaseDate, originalFileName, cancellationToken);
/// <summary>
/// Swaps the audio bytes for an existing track in place: processes a new audio file and
/// re-registers it under the SAME <paramref name="entryKey"/> in the tracks vault. The track's
/// vault key — and therefore its SQL link, release membership, position, and metadata — is
/// untouched; only the binary changes. The new audio is streamed to the vault first; only on
/// confirmed success is a stale old backing file cleaned up. A cross-format replacement (e.g.
/// .wav → .flac) leaves the old file on disk under its former filename once the index is updated;
/// the post-success cleanup removes it. For a same-extension overwrite the register alone suffices.
/// If the register fails the original audio is left intact and null is returned, so the track
/// remains playable. Returns the freshly stored audio's <b>duration</b> on success (the caller
/// re-reads the vault for waveform regen and uses this for the SQL duration write) — matching the
/// FileDatabase swallow-and-return-null contract. The new bytes are never materialized in memory.
/// </summary>
public async Task<double?> ReplaceTrackAudioAsync(string entryKey, string audioFilePath, CancellationToken cancellationToken = default)
{
try
{
// Capture the old extension from the index metadata (not by loading the file — that would
// pull the whole old audio into memory). After register the index points to the new
// extension, so we need the old value now to detect a cross-format swap and clean up the
// stale file post-success.
var trackVault = _fileDatabase.GetVault(VaultConstants.Tracks);
var existingMeta = trackVault is null ? null : await trackVault.GetEntryMetadata(entryKey);
var oldExtension = existingMeta?.Extension;
var processed = await _audioProcessorRouter.ProcessAudioFileAsync(audioFilePath, cancellationToken);
if (processed == null)
{
Console.WriteLine($"TrackContentService.ReplaceTrackAudioAsync: processing returned null for {entryKey}");
return null;
}
if (!_fileDatabase.HasVault(VaultConstants.Tracks))
{
await _fileDatabase.CreateVaultAsync(VaultConstants.Tracks, MediaVaultType.Audio);
}
// Stream the new audio in. This upserts the index entry (new extension recorded) and writes
// the new file to disk. If this fails the original entry and file are untouched.
var metaData = MetaDataFactory.CreateAudioMetaData(entryKey, processed.Extension, processed.Duration, processed.Bitrate);
var success = await _fileDatabase.RegisterResourceStreamingAsync(
VaultConstants.Tracks, entryKey, metaData, processed.WriteToAsync, cancellationToken);
if (!success)
{
Console.WriteLine($"TrackContentService.ReplaceTrackAudioAsync: vault write failed for {entryKey}; original audio preserved");
return null;
}
// Post-success stale-file cleanup for cross-format swaps. The register wrote the new
// file (e.g. .flac) and updated the index to the new extension, but the old backing
// file (e.g. .wav) is now unreferenced on disk. Delete it directly by constructing the
// old path — RemoveResourceAsync would now resolve to the new extension and delete the
// wrong file. Non-fatal: an orphaned old file is a disk-hygiene concern, not a
// playback issue (the index no longer references it).
if (oldExtension != null && oldExtension != processed.Extension)
{
var vault = _fileDatabase.GetVault(VaultConstants.Tracks);
if (vault != null)
{
var sanitizedKey = System.Text.RegularExpressions.Regex.Replace(entryKey, @"[^a-zA-Z0-9]", "-");
var staleFilePath = Path.Combine(vault.RootPath, $"{sanitizedKey}{oldExtension}");
try
{
if (File.Exists(staleFilePath))
File.Delete(staleFilePath);
}
catch (Exception ex)
{
Console.WriteLine($"TrackContentService.ReplaceTrackAudioAsync: stale backing-file removal failed for {entryKey} ({staleFilePath}): {ex.Message} — new audio is live; orphaned file may remain on disk");
}
}
}
return processed.Duration;
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
Console.WriteLine($"TrackContentService.ReplaceTrackAudioAsync failed: {ex.Message}");
return null;
}
}
/// <summary>
/// Retrieves audio binary from FileDatabase
@@ -110,6 +201,65 @@ public class TrackContentService
return await _fileDatabase.LoadResourceAsync<AudioBinary>(VaultConstants.Tracks, trackId);
}
/// <summary>
/// Opens a read-only, seekable stream over a track's vault audio, or null if the entry has no
/// backing file. The caller owns the stream and must dispose it. Unlike <see cref="GetAudioBinaryAsync"/>
/// this never buffers the whole file — it is the source for the streaming waveform compute. Follows
/// the FileDatabase swallow-and-return-null contract.
/// </summary>
/// <param name="trackId">Track ID (EntryKey)</param>
public async Task<Stream?> OpenAudioStreamAsync(string trackId)
{
var vault = _fileDatabase.GetVault(VaultConstants.Tracks);
if (vault is null)
{
return null;
}
var media = await vault.GetEntryStreamAsync(trackId);
return media?.Stream;
}
/// <summary>
/// Opens a read-only stream over a track's vault audio together with its stored extension, or null if
/// the entry has no backing file. Same non-buffering contract as <see cref="OpenAudioStreamAsync"/>,
/// but keeps the <see cref="MediaStream.Extension"/> the caller needs to name a staging file for a
/// format-detecting consumer (the Opus transcode reopens the source by extension for ffmpeg). The
/// caller owns the returned <see cref="MediaStream"/> and must dispose it. Follows the FileDatabase
/// swallow-and-return-null contract.
/// </summary>
/// <param name="trackId">Track ID (EntryKey)</param>
public async Task<MediaStream?> OpenAudioMediaStreamAsync(string trackId)
{
var vault = _fileDatabase.GetVault(VaultConstants.Tracks);
if (vault is null)
{
return null;
}
return await vault.GetEntryStreamAsync(trackId);
}
/// <summary>
/// Reads a track's stored audio duration from the vault index metadata WITHOUT loading the audio
/// body — the cheap counterpart of <c>GetAudioBinaryAsync(...).Duration</c>. Returns null if the
/// entry is unknown or carries no audio metadata. The streaming high-res waveform path uses this to
/// derive the duration-based bucket count, matching the value the whole-buffer path read off
/// <see cref="AudioBinary.Duration"/> so the stored datum is byte-identical.
/// </summary>
/// <param name="trackId">Track ID (EntryKey)</param>
public async Task<double?> GetAudioDurationAsync(string trackId)
{
var vault = _fileDatabase.GetVault(VaultConstants.Tracks);
if (vault is null)
{
return null;
}
var metaData = await vault.GetEntryMetadata(trackId);
return metaData is AudioMetaData audio ? audio.Duration : null;
}
/// <summary>
/// Checks if FileDatabase is available and tracks vault exists
/// </summary>
+22 -7
View File
@@ -20,7 +20,7 @@ Separating domain logic from hosts so DeepDrftAPI can reuse `TrackManager` / `Tr
DeepDrftData/
├── Data/
│ ├── DeepDrftContext.cs # EF DbContext
│ ├── DeepDrftContextFactory.cs # Design-time factory (hard-codes ../Database/deepdrft.db)
│ ├── DeepDrftContextFactory.cs # Design-time factory (reads environment/connections.json; Npgsql dummy fallback)
│ └── Configurations/
│ └── TrackConfiguration.cs # EF fluent configuration for TrackEntity
├── Migrations/ # EF-generated migrations (namespace DeepDrftData.Migrations)
@@ -32,7 +32,7 @@ DeepDrftData/
## EF DbContext and configuration
`DeepDrftContext` targets SQLite, connection string from `appsettings.json` (`ConnectionStrings:DefaultConnection`). The design-time factory (`DeepDrftContextFactory`) hard-codes `../Database/deepdrft.db` for `dotnet ef` commands, so you can run migrations locally without a full app context.
`DeepDrftContext` targets **PostgreSQL** (Npgsql), connection string from `environment/connections.json` (loaded at runtime via `CredentialTools.ResolvePathOrThrow("connections", ...)` in `DeepDrftAPI/Program.cs`, key `ConnectionStrings:DefaultConnection`). The design-time factory (`DeepDrftContextFactory`) reads the same `environment/connections.json` when present and falls back to a Npgsql dummy connection string (`Host=localhost;Database=deepdrft-design-time;Username=dummy`) for CI or environments without the file, so `dotnet ef` commands work without a live database.
`TrackConfiguration` uses EF fluent API:
- Table name: `track` (singular)
@@ -42,6 +42,7 @@ DeepDrftData/
- `Album`, `Genre`: optional, max 200 / 100
- `ReleaseDate`: optional `DateOnly`
- `ImagePath`: optional, max 500 (currently a free-form URL string; points to images vault in future)
- `DurationSeconds`: optional `double?` (nullable; populated at upload from vault audio; backfillable via `POST api/track/duration/backfill`; used for aggregate mix-runtime queries). Column: `duration_seconds`. Migration: `20260618155002_AddTrackDuration`.
## Service → Repository → DbContext shape
@@ -49,6 +50,20 @@ DeepDrftData/
- **Repository** (`TrackRepository`): Internal data access. Queries the DbContext. Throws on error (service catches).
- **DbContext** (`DeepDrftContext`): EF Core. Directly accessed by repository, never by service (pattern isolation).
Notable repository / service methods beyond the standard CRUD:
- `TrackRepository.GetHomeStatsAsync` / `ITrackService.GetHomeStats`: Returns `HomeStatsDto` — cut track count, per-`ReleaseType` cut release counts (zero-suppressed), mix release count, total mix runtime seconds (null durations counted as 0; tracks under a soft-deleted release excluded). Used by `StatsController`.
- `TrackRepository.UpdateDurationAsync` / `ITrackService.UpdateDuration`: Null-guarded duration write — skips rows where `DurationSeconds` is already set. Used by the one-time backfill (`POST api/track/duration/backfill`).
- `TrackRepository.SetDurationAsync` / `ITrackService.SetDuration`: Unconditional duration overwrite — no null guard, always stamps the new value. Used by the replace-audio path (`POST api/track/{id:long}/replace-audio`) where the existing non-null duration must be overwritten with the new audio's value. Returns a fail result when zero rows are affected (track removed between lookup and write).
- `ITrackService.FindOrCreateRelease` / `TrackManager.FindOrCreateRelease`: Finds the release row matching (title, artist) or creates one if none exists. Returns `ResultContainer<(ReleaseDto Release, bool WasCreated)>` — the `WasCreated` flag lets the upload CREATE path distinguish a freshly minted release from one returned because a concurrent upload won the insert race (the latter is treated as a duplicate and rejected with 409, not silently attached). `ITrackService.GetReleaseByTitleAndArtist` is the read-only counterpart used for the upload pre-flight check and the ATTACH-path validation.
## Phase 16 — anonymous telemetry domain (EventRepository / EventManager)
`EventRepository` and `EventManager` (with `IEventService` boundary) are the SQL-side domain for anonymous play/share telemetry (Phase 16 waves 16.1 + 16.3). Unlike `TrackRepository`, these entities have no soft-delete lifecycle and are not `BaseEntity`/`IEntity``EventRepository` is a plain context-backed repository against the same scoped `DeepDrftContext`.
- **`EventRepository`** (`Repositories/EventRepository.cs`): append-only writes to the `play_event` and `share_event` tables; incremental-on-write bump of the `play_counter` rollup (D6); server-side track→release resolution at write time (D4) — the client sends only the track `EntryKey`, the repository joins track→release and stamps the `release_id` on the row. Also owns the three distinct-listener aggregation queries added in wave 16.3: `CountDistinctListenersAsync()` (site-wide), `CountDistinctListenersForTrackAsync(trackEntryKey)`, `CountDistinctListenersForReleaseAsync(releaseId)` — each excludes null `anon_id` rows.
- **`EventManager` / `IEventService`** (`EventManager.cs`): `RecordPlay(trackEntryKey, bucket, anonId, ct)` and `RecordShare(targetType, targetKey, channel, anonId, ct)` return NetBlocks `Result`. Wave 16.3 added three distinct-count members returning `ResultContainer<int>`: `GetDistinctListenerCount()`, `GetDistinctListenerCountForTrack(trackEntryKey)`, `GetDistinctListenerCountForRelease(releaseId)`. Registered scoped in `DeepDrftAPI/Program.cs`. Migration: `20260619155610_AddPlayShareTelemetry` (authored; not yet applied — Daniel-gated). The `anon_id` columns and covering indexes on `play_event`/`share_event` are part of this migration — no additional migration was needed for 16.3.
Example:
```csharp
@@ -117,10 +132,10 @@ Run from the solution root:
```bash
# Add a migration
dotnet ef migrations add MigrationName --project DeepDrftData --startup-project DeepDrftPublic
dotnet ef migrations add MigrationName --project DeepDrftData --startup-project DeepDrftAPI
# Apply to database
dotnet ef database update --project DeepDrftData --startup-project DeepDrftPublic
dotnet ef database update --project DeepDrftData --startup-project DeepDrftAPI
```
The design-time factory means you can also run `dotnet ef ... --project DeepDrftData` standalone for local development (it doesn't need the startup project).
@@ -132,9 +147,9 @@ Migrations live in the `DeepDrftData.Migrations` namespace. Migration files are
## Connection string
- **DeepDrftAPI**: `environment/connections.json``ConnectionStrings:DefaultConnection`
- Points at the same database (PostgreSQL in production, SQLite for local development).
- Always PostgreSQL (Npgsql) — both production and local development.
The design-time factory hard-codes the local path for `dotnet ef` commands.
The design-time factory reads `environment/connections.json` when present; falls back to a Npgsql dummy for CI.
## Service registration
@@ -142,7 +157,7 @@ In `DeepDrftAPI/Program.cs`:
```csharp
services.AddDbContext<DeepDrftContext>(options =>
options.UseNpgsql(configuration.GetConnectionString("DefaultConnection"))); // or UseSqlite for dev
options.UseNpgsql(configuration.GetConnectionString("DefaultConnection")));
services.AddScoped<TrackRepository>();
services.AddScoped<TrackManager>();
services.AddScoped<ITrackService>(sp => sp.GetRequiredService<TrackManager>());
@@ -0,0 +1,47 @@
using DeepDrftModels.Entities;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Metadata.Builders;
namespace DeepDrftData.Data.Configurations;
/// <summary>
/// EF configuration for the <c>play_counter</c> rollup (Phase 16 §4.1 / D6). One row per track, unique
/// on track_id so the incremental-on-write bump is an upsert against a single row. <c>TotalPlays</c> is
/// a computed C# property (sum of the three bucket columns) and is not mapped — it is derived on read.
/// </summary>
public class PlayCounterConfiguration : IEntityTypeConfiguration<PlayCounter>
{
public void Configure(EntityTypeBuilder<PlayCounter> builder)
{
builder.ToTable("play_counter");
builder.HasKey(e => e.Id);
builder.Property(e => e.Id).HasColumnName("id");
builder.Property(e => e.TrackId)
.IsRequired()
.HasColumnName("track_id");
builder.Property(e => e.PartialCount)
.IsRequired()
.HasDefaultValue(0L)
.HasColumnName("partial_count");
builder.Property(e => e.SampledCount)
.IsRequired()
.HasDefaultValue(0L)
.HasColumnName("sampled_count");
builder.Property(e => e.CompleteCount)
.IsRequired()
.HasDefaultValue(0L)
.HasColumnName("complete_count");
// Derived headline figure — never a column.
builder.Ignore(e => e.TotalPlays);
builder.HasIndex(e => e.TrackId)
.IsUnique()
.HasDatabaseName("IX_play_counter_track_id");
}
}
@@ -0,0 +1,49 @@
using DeepDrftModels.Entities;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Metadata.Builders;
namespace DeepDrftData.Data.Configurations;
/// <summary>
/// EF configuration for the append-only <c>play_event</c> log (Phase 16 §4.2). Plain entity, not a
/// <c>BaseEntity</c> — no soft-delete or updated_at, just an immutable fact with a created_at stamp.
/// Indexed on track key, release id, and anon id (the last reserved for the wave-16.3 distinct-listener
/// query) so the aggregation paths stay cheap as the log grows.
/// </summary>
public class PlayEventConfiguration : IEntityTypeConfiguration<PlayEvent>
{
public void Configure(EntityTypeBuilder<PlayEvent> builder)
{
builder.ToTable("play_event");
builder.HasKey(e => e.Id);
builder.Property(e => e.Id).HasColumnName("id");
builder.Property(e => e.TrackEntryKey)
.IsRequired()
.HasMaxLength(100)
.HasColumnName("track_entry_key");
builder.Property(e => e.ReleaseId)
.HasColumnName("release_id");
builder.Property(e => e.Bucket)
.IsRequired()
.HasConversion<string>() // Store the readable bucket name, mirroring ReleaseMedium.
.HasMaxLength(20)
.HasColumnName("bucket");
// Reserved nullable token (wave 16.3). Same width as a stringified GUID.
builder.Property(e => e.AnonId)
.HasMaxLength(64)
.HasColumnName("anon_id");
builder.Property(e => e.CreatedAt)
.IsRequired()
.HasColumnName("created_at");
builder.HasIndex(e => e.TrackEntryKey).HasDatabaseName("IX_play_event_track_entry_key");
builder.HasIndex(e => e.ReleaseId).HasDatabaseName("IX_play_event_release_id");
builder.HasIndex(e => e.AnonId).HasDatabaseName("IX_play_event_anon_id");
}
}
@@ -0,0 +1,48 @@
using DeepDrftModels.Entities;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Metadata.Builders;
namespace DeepDrftData.Data.Configurations;
/// <summary>
/// EF configuration for the append-only <c>share_event</c> log (Phase 16 §4.2). Plain immutable-fact
/// entity. Indexed on the target key so per-target share tallies stay cheap.
/// </summary>
public class ShareEventConfiguration : IEntityTypeConfiguration<ShareEvent>
{
public void Configure(EntityTypeBuilder<ShareEvent> builder)
{
builder.ToTable("share_event");
builder.HasKey(e => e.Id);
builder.Property(e => e.Id).HasColumnName("id");
builder.Property(e => e.TargetType)
.IsRequired()
.HasConversion<string>()
.HasMaxLength(20)
.HasColumnName("target_type");
builder.Property(e => e.TargetKey)
.IsRequired()
.HasMaxLength(100)
.HasColumnName("target_key");
builder.Property(e => e.Channel)
.IsRequired()
.HasConversion<string>()
.HasMaxLength(20)
.HasColumnName("channel");
builder.Property(e => e.AnonId)
.HasMaxLength(64)
.HasColumnName("anon_id");
builder.Property(e => e.CreatedAt)
.IsRequired()
.HasColumnName("created_at");
builder.HasIndex(e => e.TargetKey).HasDatabaseName("IX_share_event_target_key");
builder.HasIndex(e => e.AnonId).HasDatabaseName("IX_share_event_anon_id");
}
}
@@ -39,6 +39,10 @@ public class TrackConfiguration : BaseEntityConfiguration<TrackEntity>
.HasColumnName("track_number")
.HasDefaultValue(1);
// Nullable: existing rows carry NULL until the one-time duration backfill populates them.
builder.Property(e => e.DurationSeconds)
.HasColumnName("duration_seconds");
builder.Property(e => e.ReleaseId)
.HasColumnName("release_id");
+9
View File
@@ -15,6 +15,12 @@ public class DeepDrftContext : DbContext
public DbSet<SessionMetadata> SessionMetadata { get; set; }
public DbSet<MixMetadata> MixMetadata { get; set; }
// Phase 16 anonymous telemetry: append-only event logs + incremental play rollup. All SQL — the
// FileDatabase vault is not involved.
public DbSet<PlayEvent> PlayEvents { get; set; }
public DbSet<ShareEvent> ShareEvents { get; set; }
public DbSet<PlayCounter> PlayCounters { get; set; }
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
@@ -23,5 +29,8 @@ public class DeepDrftContext : DbContext
modelBuilder.ApplyConfiguration(new ReleaseConfiguration());
modelBuilder.ApplyConfiguration(new SessionMetadataConfiguration());
modelBuilder.ApplyConfiguration(new MixMetadataConfiguration());
modelBuilder.ApplyConfiguration(new PlayEventConfiguration());
modelBuilder.ApplyConfiguration(new ShareEventConfiguration());
modelBuilder.ApplyConfiguration(new PlayCounterConfiguration());
}
}
+3 -3
View File
@@ -18,9 +18,9 @@
</PackageReference>
<!-- Npgsql 10.0.1 requires Microsoft.EntityFrameworkCore >= 10.0.4; keep in sync -->
<PackageReference Include="Npgsql.EntityFrameworkCore.PostgreSQL" Version="10.0.1" />
<PackageReference Include="Cerebellum.NetBlocks" Version="10.3.30" />
<PackageReference Include="Cerebellum.BlazorBlocks.Data" Version="10.3.30" />
<PackageReference Include="Cerebellum.BlazorBlocks.Data.Postgres" Version="10.3.30" />
<PackageReference Include="Cerebellum.NetBlocks" Version="10.3.32" />
<PackageReference Include="Cerebellum.BlazorBlocks.Data" Version="10.3.35" />
<PackageReference Include="Cerebellum.BlazorBlocks.Data.Postgres" Version="10.3.35" />
</ItemGroup>
<ItemGroup>
+114
View File
@@ -0,0 +1,114 @@
using DeepDrftData.Repositories;
using DeepDrftModels.Enums;
using Microsoft.Extensions.Logging;
using NetBlocks.Models;
namespace DeepDrftData;
/// <summary>
/// <see cref="IEventService"/> implementation over <see cref="EventRepository"/>. The layer boundary
/// matches the rest of DeepDrftData: the repository owns the EF constructs and the write transaction;
/// this service catches at the boundary and returns a NetBlocks <see cref="Result"/>. Telemetry is
/// best-effort by design (§2.2) — a failed write is logged and surfaced as a fail result, never thrown
/// at the caller, so a telemetry hiccup can never reach a listener.
/// </summary>
public class EventManager : IEventService
{
private readonly EventRepository _repository;
private readonly ILogger<EventManager> _logger;
public EventManager(EventRepository repository, ILogger<EventManager> logger)
{
_repository = repository;
_logger = logger;
}
public async Task<Result> RecordPlay(
string trackEntryKey, PlayBucket bucket, string? anonId = null, CancellationToken cancellationToken = default)
{
try
{
await _repository.RecordPlayAsync(trackEntryKey, bucket, anonId, cancellationToken);
return Result.CreatePassResult();
}
catch (Exception e)
{
_logger.LogError(e, "Failed to record play event for track {TrackEntryKey}", trackEntryKey);
return Result.CreateFailResult(e.Message);
}
}
public async Task<Result> RecordShare(
ShareTargetType targetType, string targetKey, ShareChannel channel, string? anonId = null,
CancellationToken cancellationToken = default)
{
try
{
await _repository.RecordShareAsync(targetType, targetKey, channel, anonId, cancellationToken);
return Result.CreatePassResult();
}
catch (Exception e)
{
_logger.LogError(e, "Failed to record share event for {TargetType} {TargetKey}", targetType, targetKey);
return Result.CreateFailResult(e.Message);
}
}
public async Task<ResultContainer<long>> GetTotalPlayCount(CancellationToken cancellationToken = default)
{
try
{
var count = await _repository.CountTotalPlaysAsync(cancellationToken);
return ResultContainer<long>.CreatePassResult(count);
}
catch (Exception e)
{
_logger.LogError(e, "Failed to count total plays");
return ResultContainer<long>.CreateFailResult(e.Message);
}
}
public async Task<ResultContainer<int>> GetDistinctListenerCount(CancellationToken cancellationToken = default)
{
try
{
var count = await _repository.CountDistinctListenersAsync(cancellationToken);
return ResultContainer<int>.CreatePassResult(count);
}
catch (Exception e)
{
_logger.LogError(e, "Failed to count distinct listeners");
return ResultContainer<int>.CreateFailResult(e.Message);
}
}
public async Task<ResultContainer<int>> GetDistinctListenerCountForTrack(
string trackEntryKey, CancellationToken cancellationToken = default)
{
try
{
var count = await _repository.CountDistinctListenersForTrackAsync(trackEntryKey, cancellationToken);
return ResultContainer<int>.CreatePassResult(count);
}
catch (Exception e)
{
_logger.LogError(e, "Failed to count distinct listeners for track {TrackEntryKey}", trackEntryKey);
return ResultContainer<int>.CreateFailResult(e.Message);
}
}
public async Task<ResultContainer<int>> GetDistinctListenerCountForRelease(
long releaseId, CancellationToken cancellationToken = default)
{
try
{
var count = await _repository.CountDistinctListenersForReleaseAsync(releaseId, cancellationToken);
return ResultContainer<int>.CreatePassResult(count);
}
catch (Exception e)
{
_logger.LogError(e, "Failed to count distinct listeners for release {ReleaseId}", releaseId);
return ResultContainer<int>.CreateFailResult(e.Message);
}
}
}
+46
View File
@@ -0,0 +1,46 @@
using DeepDrftModels.Enums;
using NetBlocks.Models;
namespace DeepDrftData;
/// <summary>
/// SQL-side anonymous telemetry service (Phase 16). Records play and share events to the append-only
/// logs and maintains the incremental play-counter rollup. The release dimension on a play is resolved
/// server-side from the track key (§2.3 / D4) — callers pass only what the client cheaply knows.
/// Returns NetBlocks <see cref="Result"/> at the boundary; the controller maps that to 202/4xx/5xx.
/// </summary>
public interface IEventService
{
/// <summary>
/// Record one play: append a <c>play_event</c> row (release resolved from the track key) and bump
/// the track's <c>play_counter</c> in the same transaction. A play of an unknown/removed track key
/// still logs (with a null release and no counter bump) rather than failing.
/// </summary>
Task<Result> RecordPlay(string trackEntryKey, PlayBucket bucket, string? anonId = null, CancellationToken cancellationToken = default);
/// <summary>Record one share: append a <c>share_event</c> row. Target and channel come straight from the client.</summary>
Task<Result> RecordShare(ShareTargetType targetType, string targetKey, ShareChannel channel, string? anonId = null, CancellationToken cancellationToken = default);
/// <summary>
/// Site-wide total play count (Phase 16 §5 — all-time): the sum of every <c>play_counter</c> row's
/// three bucket columns. Zero until the telemetry migration is applied. The home Plays card's primary
/// figure; the controller composes it onto <c>HomeStatsDto</c> alongside the track-domain figures.
/// </summary>
Task<ResultContainer<long>> GetTotalPlayCount(CancellationToken cancellationToken = default);
/// <summary>
/// Site-wide distinct-listener count (Phase 16 §3, D3 — all-time): distinct non-null <c>anon_id</c>
/// values across all play events. Null tokens are excluded (not a known listener). The capability for
/// wave 16.5's "N listeners" card; nothing surfaces it via API or UI in wave 16.3.
/// </summary>
Task<ResultContainer<int>> GetDistinctListenerCount(CancellationToken cancellationToken = default);
/// <summary>Distinct listeners who played the given track (by vault entry key). Null tokens excluded.</summary>
Task<ResultContainer<int>> GetDistinctListenerCountForTrack(string trackEntryKey, CancellationToken cancellationToken = default);
/// <summary>
/// Distinct listeners across the release's tracks (derived, D4) — a listener who played any track in
/// the release counts once. Null tokens excluded.
/// </summary>
Task<ResultContainer<int>> GetDistinctListenerCountForRelease(long releaseId, CancellationToken cancellationToken = default);
}
+32 -1
View File
@@ -28,12 +28,43 @@ public interface ITrackService
/// <summary>Distinct non-null genres with track counts, genre-ascending.</summary>
Task<ResultContainer<List<GenreSummaryDto>>> GetDistinctGenres(CancellationToken cancellationToken = default);
/// <summary>
/// Aggregate figures behind the public home hero stat row: Cut track count + per-ReleaseType Cut
/// release breakdown, Mix release count, and total Mix runtime in seconds. One read for all three cards.
/// </summary>
Task<ResultContainer<HomeStatsDto>> GetHomeStats(CancellationToken cancellationToken = default);
/// <summary>
/// Non-deleted tracks whose SQL duration is still null — the work list for the one-time duration
/// backfill. The backfill reads each track's stored duration from the vault and writes it via
/// <see cref="UpdateDuration"/>.
/// </summary>
Task<ResultContainer<List<TrackDto>>> GetTracksMissingDuration(CancellationToken cancellationToken = default);
/// <summary>
/// Set the SQL duration for one track. Idempotent: a track whose duration is already set is left
/// untouched. Backs the duration backfill. Returns the number of rows updated (0 or 1).
/// </summary>
Task<ResultContainer<int>> UpdateDuration(long id, double durationSeconds, CancellationToken cancellationToken = default);
/// <summary>
/// Unconditionally overwrite the SQL duration for one track. Unlike <see cref="UpdateDuration"/>,
/// this carries no null guard — it is for the replace-audio path where the track already has a
/// non-null duration that must be overwritten with the new audio's value. Returns a fail Result
/// when zero rows are affected (track removed between lookup and write).
/// </summary>
Task<ResultContainer<int>> SetDuration(long id, double durationSeconds, CancellationToken cancellationToken = default);
/// <summary>
/// Resolve the release matching <paramref name="title"/> + <paramref name="artist"/>, creating
/// one from <paramref name="releaseData"/> when none exists. Backs the upload flow's FK
/// resolution so a track lands on a shared release rather than duplicating release-cardinal data.
/// The <c>WasCreated</c> flag in the result is <see langword="true"/> when a new row was inserted
/// and <see langword="false"/> when an existing row was found (including after a lost concurrent-insert
/// race). The CREATE path in <c>UnifiedTrackService.UploadAsync</c> uses this to turn a
/// "found existing" outcome into a duplicate rejection rather than a silent attach.
/// </summary>
Task<ResultContainer<ReleaseDto>> FindOrCreateRelease(
Task<ResultContainer<(ReleaseDto Release, bool WasCreated)>> FindOrCreateRelease(
string title, string artist, ReleaseDto releaseData, CancellationToken cancellationToken = default);
/// <summary>
@@ -0,0 +1,322 @@
// <auto-generated />
using System;
using DeepDrftData.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using Npgsql.EntityFrameworkCore.PostgreSQL.Metadata;
#nullable disable
namespace DeepDrftData.Migrations
{
[DbContext(typeof(DeepDrftContext))]
[Migration("20260618155002_AddTrackDuration")]
partial class AddTrackDuration
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 63);
NpgsqlModelBuilderExtensions.UseIdentityByDefaultColumns(modelBuilder);
modelBuilder.Entity("DeepDrftModels.Entities.MixMetadata", b =>
{
b.Property<long>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<long>("Id"));
b.Property<DateTime>("CreatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("created_at");
b.Property<bool>("IsDeleted")
.ValueGeneratedOnAdd()
.HasColumnType("boolean")
.HasDefaultValue(false)
.HasColumnName("is_deleted");
b.Property<long>("ReleaseId")
.HasColumnType("bigint")
.HasColumnName("release_id");
b.Property<DateTime>("UpdatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("updated_at");
b.Property<string>("WaveformEntryKey")
.IsRequired()
.HasMaxLength(500)
.HasColumnType("character varying(500)")
.HasColumnName("waveform_entry_key");
b.HasKey("Id");
b.HasIndex("IsDeleted")
.HasDatabaseName("IX_mix_metadata_is_deleted");
b.HasIndex("ReleaseId")
.IsUnique()
.HasDatabaseName("IX_mix_metadata_release_id");
b.ToTable("mix_metadata", (string)null);
});
modelBuilder.Entity("DeepDrftModels.Entities.ReleaseEntity", b =>
{
b.Property<long>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<long>("Id"));
b.Property<string>("Artist")
.IsRequired()
.HasMaxLength(200)
.HasColumnType("character varying(200)")
.HasColumnName("artist");
b.Property<DateTime>("CreatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("created_at");
b.Property<long?>("CreatedByUserId")
.HasColumnType("bigint")
.HasColumnName("created_by_user_id");
b.Property<string>("Description")
.HasMaxLength(4000)
.HasColumnType("character varying(4000)")
.HasColumnName("description");
b.Property<string>("EntryKey")
.IsRequired()
.HasMaxLength(100)
.HasColumnType("character varying(100)")
.HasColumnName("entry_key");
b.Property<string>("Genre")
.HasMaxLength(100)
.HasColumnType("character varying(100)")
.HasColumnName("genre");
b.Property<string>("ImagePath")
.HasMaxLength(500)
.HasColumnType("character varying(500)")
.HasColumnName("image_path");
b.Property<bool>("IsDeleted")
.ValueGeneratedOnAdd()
.HasColumnType("boolean")
.HasDefaultValue(false)
.HasColumnName("is_deleted");
b.Property<string>("Medium")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(20)
.HasColumnType("character varying(20)")
.HasDefaultValue("Cut")
.HasColumnName("medium");
b.Property<DateOnly?>("ReleaseDate")
.HasColumnType("date")
.HasColumnName("release_date");
b.Property<string>("ReleaseType")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(20)
.HasColumnType("character varying(20)")
.HasDefaultValue("Single")
.HasColumnName("release_type");
b.Property<string>("Title")
.IsRequired()
.HasMaxLength(200)
.HasColumnType("character varying(200)")
.HasColumnName("title");
b.Property<DateTime>("UpdatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("updated_at");
b.HasKey("Id");
b.HasIndex("EntryKey")
.IsUnique()
.HasDatabaseName("IX_release_entry_key");
b.HasIndex("IsDeleted")
.HasDatabaseName("IX_release_is_deleted");
b.HasIndex("Title", "Artist")
.IsUnique()
.HasDatabaseName("IX_release_title_artist")
.HasFilter("\"is_deleted\" = false");
b.ToTable("release", (string)null);
});
modelBuilder.Entity("DeepDrftModels.Entities.SessionMetadata", b =>
{
b.Property<long>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<long>("Id"));
b.Property<DateTime>("CreatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("created_at");
b.Property<string>("HeroImageEntryKey")
.IsRequired()
.HasMaxLength(500)
.HasColumnType("character varying(500)")
.HasColumnName("hero_image_entry_key");
b.Property<bool>("IsDeleted")
.ValueGeneratedOnAdd()
.HasColumnType("boolean")
.HasDefaultValue(false)
.HasColumnName("is_deleted");
b.Property<long>("ReleaseId")
.HasColumnType("bigint")
.HasColumnName("release_id");
b.Property<DateTime>("UpdatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("updated_at");
b.HasKey("Id");
b.HasIndex("IsDeleted")
.HasDatabaseName("IX_session_metadata_is_deleted");
b.HasIndex("ReleaseId")
.IsUnique()
.HasDatabaseName("IX_session_metadata_release_id");
b.ToTable("session_metadata", (string)null);
});
modelBuilder.Entity("DeepDrftModels.Entities.TrackEntity", b =>
{
b.Property<long>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<long>("Id"));
b.Property<DateTime>("CreatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("created_at");
b.Property<double?>("DurationSeconds")
.HasColumnType("double precision")
.HasColumnName("duration_seconds");
b.Property<string>("EntryKey")
.IsRequired()
.HasMaxLength(100)
.HasColumnType("character varying(100)")
.HasColumnName("entry_key");
b.Property<bool>("IsDeleted")
.ValueGeneratedOnAdd()
.HasColumnType("boolean")
.HasDefaultValue(false)
.HasColumnName("is_deleted");
b.Property<string>("OriginalFileName")
.HasMaxLength(500)
.HasColumnType("character varying(500)")
.HasColumnName("original_file_name");
b.Property<long?>("ReleaseId")
.HasColumnType("bigint")
.HasColumnName("release_id");
b.Property<string>("TrackName")
.IsRequired()
.HasMaxLength(200)
.HasColumnType("character varying(200)")
.HasColumnName("track_name");
b.Property<int>("TrackNumber")
.ValueGeneratedOnAdd()
.HasColumnType("integer")
.HasDefaultValue(1)
.HasColumnName("track_number");
b.Property<DateTime>("UpdatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("updated_at");
b.HasKey("Id");
b.HasIndex("IsDeleted")
.HasDatabaseName("IX_track_is_deleted");
b.HasIndex("ReleaseId");
b.ToTable("track", (string)null);
});
modelBuilder.Entity("DeepDrftModels.Entities.MixMetadata", b =>
{
b.HasOne("DeepDrftModels.Entities.ReleaseEntity", "Release")
.WithOne("MixMetadata")
.HasForeignKey("DeepDrftModels.Entities.MixMetadata", "ReleaseId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
b.Navigation("Release");
});
modelBuilder.Entity("DeepDrftModels.Entities.SessionMetadata", b =>
{
b.HasOne("DeepDrftModels.Entities.ReleaseEntity", "Release")
.WithOne("SessionMetadata")
.HasForeignKey("DeepDrftModels.Entities.SessionMetadata", "ReleaseId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
b.Navigation("Release");
});
modelBuilder.Entity("DeepDrftModels.Entities.TrackEntity", b =>
{
b.HasOne("DeepDrftModels.Entities.ReleaseEntity", "Release")
.WithMany("Tracks")
.HasForeignKey("ReleaseId")
.OnDelete(DeleteBehavior.SetNull);
b.Navigation("Release");
});
modelBuilder.Entity("DeepDrftModels.Entities.ReleaseEntity", b =>
{
b.Navigation("MixMetadata");
b.Navigation("SessionMetadata");
b.Navigation("Tracks");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,28 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace DeepDrftData.Migrations
{
/// <inheritdoc />
public partial class AddTrackDuration : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<double>(
name: "duration_seconds",
table: "track",
type: "double precision",
nullable: true);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropColumn(
name: "duration_seconds",
table: "track");
}
}
}
@@ -0,0 +1,457 @@
// <auto-generated />
using System;
using DeepDrftData.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using Npgsql.EntityFrameworkCore.PostgreSQL.Metadata;
#nullable disable
namespace DeepDrftData.Migrations
{
[DbContext(typeof(DeepDrftContext))]
[Migration("20260619155610_AddPlayShareTelemetry")]
partial class AddPlayShareTelemetry
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 63);
NpgsqlModelBuilderExtensions.UseIdentityByDefaultColumns(modelBuilder);
modelBuilder.Entity("DeepDrftModels.Entities.MixMetadata", b =>
{
b.Property<long>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<long>("Id"));
b.Property<DateTime>("CreatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("created_at");
b.Property<bool>("IsDeleted")
.ValueGeneratedOnAdd()
.HasColumnType("boolean")
.HasDefaultValue(false)
.HasColumnName("is_deleted");
b.Property<long>("ReleaseId")
.HasColumnType("bigint")
.HasColumnName("release_id");
b.Property<DateTime>("UpdatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("updated_at");
b.Property<string>("WaveformEntryKey")
.IsRequired()
.HasMaxLength(500)
.HasColumnType("character varying(500)")
.HasColumnName("waveform_entry_key");
b.HasKey("Id");
b.HasIndex("IsDeleted")
.HasDatabaseName("IX_mix_metadata_is_deleted");
b.HasIndex("ReleaseId")
.IsUnique()
.HasDatabaseName("IX_mix_metadata_release_id");
b.ToTable("mix_metadata", (string)null);
});
modelBuilder.Entity("DeepDrftModels.Entities.PlayCounter", b =>
{
b.Property<long>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<long>("Id"));
b.Property<long>("CompleteCount")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasDefaultValue(0L)
.HasColumnName("complete_count");
b.Property<long>("PartialCount")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasDefaultValue(0L)
.HasColumnName("partial_count");
b.Property<long>("SampledCount")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasDefaultValue(0L)
.HasColumnName("sampled_count");
b.Property<long>("TrackId")
.HasColumnType("bigint")
.HasColumnName("track_id");
b.HasKey("Id");
b.HasIndex("TrackId")
.IsUnique()
.HasDatabaseName("IX_play_counter_track_id");
b.ToTable("play_counter", (string)null);
});
modelBuilder.Entity("DeepDrftModels.Entities.PlayEvent", b =>
{
b.Property<long>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<long>("Id"));
b.Property<string>("AnonId")
.HasMaxLength(64)
.HasColumnType("character varying(64)")
.HasColumnName("anon_id");
b.Property<string>("Bucket")
.IsRequired()
.HasMaxLength(20)
.HasColumnType("character varying(20)")
.HasColumnName("bucket");
b.Property<DateTime>("CreatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("created_at");
b.Property<long?>("ReleaseId")
.HasColumnType("bigint")
.HasColumnName("release_id");
b.Property<string>("TrackEntryKey")
.IsRequired()
.HasMaxLength(100)
.HasColumnType("character varying(100)")
.HasColumnName("track_entry_key");
b.HasKey("Id");
b.HasIndex("AnonId")
.HasDatabaseName("IX_play_event_anon_id");
b.HasIndex("ReleaseId")
.HasDatabaseName("IX_play_event_release_id");
b.HasIndex("TrackEntryKey")
.HasDatabaseName("IX_play_event_track_entry_key");
b.ToTable("play_event", (string)null);
});
modelBuilder.Entity("DeepDrftModels.Entities.ReleaseEntity", b =>
{
b.Property<long>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<long>("Id"));
b.Property<string>("Artist")
.IsRequired()
.HasMaxLength(200)
.HasColumnType("character varying(200)")
.HasColumnName("artist");
b.Property<DateTime>("CreatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("created_at");
b.Property<long?>("CreatedByUserId")
.HasColumnType("bigint")
.HasColumnName("created_by_user_id");
b.Property<string>("Description")
.HasMaxLength(4000)
.HasColumnType("character varying(4000)")
.HasColumnName("description");
b.Property<string>("EntryKey")
.IsRequired()
.HasMaxLength(100)
.HasColumnType("character varying(100)")
.HasColumnName("entry_key");
b.Property<string>("Genre")
.HasMaxLength(100)
.HasColumnType("character varying(100)")
.HasColumnName("genre");
b.Property<string>("ImagePath")
.HasMaxLength(500)
.HasColumnType("character varying(500)")
.HasColumnName("image_path");
b.Property<bool>("IsDeleted")
.ValueGeneratedOnAdd()
.HasColumnType("boolean")
.HasDefaultValue(false)
.HasColumnName("is_deleted");
b.Property<string>("Medium")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(20)
.HasColumnType("character varying(20)")
.HasDefaultValue("Cut")
.HasColumnName("medium");
b.Property<DateOnly?>("ReleaseDate")
.HasColumnType("date")
.HasColumnName("release_date");
b.Property<string>("ReleaseType")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(20)
.HasColumnType("character varying(20)")
.HasDefaultValue("Single")
.HasColumnName("release_type");
b.Property<string>("Title")
.IsRequired()
.HasMaxLength(200)
.HasColumnType("character varying(200)")
.HasColumnName("title");
b.Property<DateTime>("UpdatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("updated_at");
b.HasKey("Id");
b.HasIndex("EntryKey")
.IsUnique()
.HasDatabaseName("IX_release_entry_key");
b.HasIndex("IsDeleted")
.HasDatabaseName("IX_release_is_deleted");
b.HasIndex("Title", "Artist")
.IsUnique()
.HasDatabaseName("IX_release_title_artist")
.HasFilter("\"is_deleted\" = false");
b.ToTable("release", (string)null);
});
modelBuilder.Entity("DeepDrftModels.Entities.SessionMetadata", b =>
{
b.Property<long>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<long>("Id"));
b.Property<DateTime>("CreatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("created_at");
b.Property<string>("HeroImageEntryKey")
.IsRequired()
.HasMaxLength(500)
.HasColumnType("character varying(500)")
.HasColumnName("hero_image_entry_key");
b.Property<bool>("IsDeleted")
.ValueGeneratedOnAdd()
.HasColumnType("boolean")
.HasDefaultValue(false)
.HasColumnName("is_deleted");
b.Property<long>("ReleaseId")
.HasColumnType("bigint")
.HasColumnName("release_id");
b.Property<DateTime>("UpdatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("updated_at");
b.HasKey("Id");
b.HasIndex("IsDeleted")
.HasDatabaseName("IX_session_metadata_is_deleted");
b.HasIndex("ReleaseId")
.IsUnique()
.HasDatabaseName("IX_session_metadata_release_id");
b.ToTable("session_metadata", (string)null);
});
modelBuilder.Entity("DeepDrftModels.Entities.ShareEvent", b =>
{
b.Property<long>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<long>("Id"));
b.Property<string>("AnonId")
.HasMaxLength(64)
.HasColumnType("character varying(64)")
.HasColumnName("anon_id");
b.Property<string>("Channel")
.IsRequired()
.HasMaxLength(20)
.HasColumnType("character varying(20)")
.HasColumnName("channel");
b.Property<DateTime>("CreatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("created_at");
b.Property<string>("TargetKey")
.IsRequired()
.HasMaxLength(100)
.HasColumnType("character varying(100)")
.HasColumnName("target_key");
b.Property<string>("TargetType")
.IsRequired()
.HasMaxLength(20)
.HasColumnType("character varying(20)")
.HasColumnName("target_type");
b.HasKey("Id");
b.HasIndex("AnonId")
.HasDatabaseName("IX_share_event_anon_id");
b.HasIndex("TargetKey")
.HasDatabaseName("IX_share_event_target_key");
b.ToTable("share_event", (string)null);
});
modelBuilder.Entity("DeepDrftModels.Entities.TrackEntity", b =>
{
b.Property<long>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<long>("Id"));
b.Property<DateTime>("CreatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("created_at");
b.Property<double?>("DurationSeconds")
.HasColumnType("double precision")
.HasColumnName("duration_seconds");
b.Property<string>("EntryKey")
.IsRequired()
.HasMaxLength(100)
.HasColumnType("character varying(100)")
.HasColumnName("entry_key");
b.Property<bool>("IsDeleted")
.ValueGeneratedOnAdd()
.HasColumnType("boolean")
.HasDefaultValue(false)
.HasColumnName("is_deleted");
b.Property<string>("OriginalFileName")
.HasMaxLength(500)
.HasColumnType("character varying(500)")
.HasColumnName("original_file_name");
b.Property<long?>("ReleaseId")
.HasColumnType("bigint")
.HasColumnName("release_id");
b.Property<string>("TrackName")
.IsRequired()
.HasMaxLength(200)
.HasColumnType("character varying(200)")
.HasColumnName("track_name");
b.Property<int>("TrackNumber")
.ValueGeneratedOnAdd()
.HasColumnType("integer")
.HasDefaultValue(1)
.HasColumnName("track_number");
b.Property<DateTime>("UpdatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("updated_at");
b.HasKey("Id");
b.HasIndex("IsDeleted")
.HasDatabaseName("IX_track_is_deleted");
b.HasIndex("ReleaseId");
b.ToTable("track", (string)null);
});
modelBuilder.Entity("DeepDrftModels.Entities.MixMetadata", b =>
{
b.HasOne("DeepDrftModels.Entities.ReleaseEntity", "Release")
.WithOne("MixMetadata")
.HasForeignKey("DeepDrftModels.Entities.MixMetadata", "ReleaseId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
b.Navigation("Release");
});
modelBuilder.Entity("DeepDrftModels.Entities.SessionMetadata", b =>
{
b.HasOne("DeepDrftModels.Entities.ReleaseEntity", "Release")
.WithOne("SessionMetadata")
.HasForeignKey("DeepDrftModels.Entities.SessionMetadata", "ReleaseId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
b.Navigation("Release");
});
modelBuilder.Entity("DeepDrftModels.Entities.TrackEntity", b =>
{
b.HasOne("DeepDrftModels.Entities.ReleaseEntity", "Release")
.WithMany("Tracks")
.HasForeignKey("ReleaseId")
.OnDelete(DeleteBehavior.SetNull);
b.Navigation("Release");
});
modelBuilder.Entity("DeepDrftModels.Entities.ReleaseEntity", b =>
{
b.Navigation("MixMetadata");
b.Navigation("SessionMetadata");
b.Navigation("Tracks");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,110 @@
using System;
using Microsoft.EntityFrameworkCore.Migrations;
using Npgsql.EntityFrameworkCore.PostgreSQL.Metadata;
#nullable disable
namespace DeepDrftData.Migrations
{
/// <inheritdoc />
public partial class AddPlayShareTelemetry : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateTable(
name: "play_counter",
columns: table => new
{
id = table.Column<long>(type: "bigint", nullable: false)
.Annotation("Npgsql:ValueGenerationStrategy", NpgsqlValueGenerationStrategy.IdentityByDefaultColumn),
track_id = table.Column<long>(type: "bigint", nullable: false),
partial_count = table.Column<long>(type: "bigint", nullable: false, defaultValue: 0L),
sampled_count = table.Column<long>(type: "bigint", nullable: false, defaultValue: 0L),
complete_count = table.Column<long>(type: "bigint", nullable: false, defaultValue: 0L)
},
constraints: table =>
{
table.PrimaryKey("PK_play_counter", x => x.id);
});
migrationBuilder.CreateTable(
name: "play_event",
columns: table => new
{
id = table.Column<long>(type: "bigint", nullable: false)
.Annotation("Npgsql:ValueGenerationStrategy", NpgsqlValueGenerationStrategy.IdentityByDefaultColumn),
track_entry_key = table.Column<string>(type: "character varying(100)", maxLength: 100, nullable: false),
release_id = table.Column<long>(type: "bigint", nullable: true),
bucket = table.Column<string>(type: "character varying(20)", maxLength: 20, nullable: false),
anon_id = table.Column<string>(type: "character varying(64)", maxLength: 64, nullable: true),
created_at = table.Column<DateTime>(type: "timestamp with time zone", nullable: false)
},
constraints: table =>
{
table.PrimaryKey("PK_play_event", x => x.id);
});
migrationBuilder.CreateTable(
name: "share_event",
columns: table => new
{
id = table.Column<long>(type: "bigint", nullable: false)
.Annotation("Npgsql:ValueGenerationStrategy", NpgsqlValueGenerationStrategy.IdentityByDefaultColumn),
target_type = table.Column<string>(type: "character varying(20)", maxLength: 20, nullable: false),
target_key = table.Column<string>(type: "character varying(100)", maxLength: 100, nullable: false),
channel = table.Column<string>(type: "character varying(20)", maxLength: 20, nullable: false),
anon_id = table.Column<string>(type: "character varying(64)", maxLength: 64, nullable: true),
created_at = table.Column<DateTime>(type: "timestamp with time zone", nullable: false)
},
constraints: table =>
{
table.PrimaryKey("PK_share_event", x => x.id);
});
migrationBuilder.CreateIndex(
name: "IX_play_counter_track_id",
table: "play_counter",
column: "track_id",
unique: true);
migrationBuilder.CreateIndex(
name: "IX_play_event_anon_id",
table: "play_event",
column: "anon_id");
migrationBuilder.CreateIndex(
name: "IX_play_event_release_id",
table: "play_event",
column: "release_id");
migrationBuilder.CreateIndex(
name: "IX_play_event_track_entry_key",
table: "play_event",
column: "track_entry_key");
migrationBuilder.CreateIndex(
name: "IX_share_event_anon_id",
table: "share_event",
column: "anon_id");
migrationBuilder.CreateIndex(
name: "IX_share_event_target_key",
table: "share_event",
column: "target_key");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropTable(
name: "play_counter");
migrationBuilder.DropTable(
name: "play_event");
migrationBuilder.DropTable(
name: "share_event");
}
}
}
@@ -67,6 +67,94 @@ namespace DeepDrftData.Migrations
b.ToTable("mix_metadata", (string)null);
});
modelBuilder.Entity("DeepDrftModels.Entities.PlayCounter", b =>
{
b.Property<long>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<long>("Id"));
b.Property<long>("CompleteCount")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasDefaultValue(0L)
.HasColumnName("complete_count");
b.Property<long>("PartialCount")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasDefaultValue(0L)
.HasColumnName("partial_count");
b.Property<long>("SampledCount")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasDefaultValue(0L)
.HasColumnName("sampled_count");
b.Property<long>("TrackId")
.HasColumnType("bigint")
.HasColumnName("track_id");
b.HasKey("Id");
b.HasIndex("TrackId")
.IsUnique()
.HasDatabaseName("IX_play_counter_track_id");
b.ToTable("play_counter", (string)null);
});
modelBuilder.Entity("DeepDrftModels.Entities.PlayEvent", b =>
{
b.Property<long>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<long>("Id"));
b.Property<string>("AnonId")
.HasMaxLength(64)
.HasColumnType("character varying(64)")
.HasColumnName("anon_id");
b.Property<string>("Bucket")
.IsRequired()
.HasMaxLength(20)
.HasColumnType("character varying(20)")
.HasColumnName("bucket");
b.Property<DateTime>("CreatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("created_at");
b.Property<long?>("ReleaseId")
.HasColumnType("bigint")
.HasColumnName("release_id");
b.Property<string>("TrackEntryKey")
.IsRequired()
.HasMaxLength(100)
.HasColumnType("character varying(100)")
.HasColumnName("track_entry_key");
b.HasKey("Id");
b.HasIndex("AnonId")
.HasDatabaseName("IX_play_event_anon_id");
b.HasIndex("ReleaseId")
.HasDatabaseName("IX_play_event_release_id");
b.HasIndex("TrackEntryKey")
.HasDatabaseName("IX_play_event_track_entry_key");
b.ToTable("play_event", (string)null);
});
modelBuilder.Entity("DeepDrftModels.Entities.ReleaseEntity", b =>
{
b.Property<long>("Id")
@@ -209,6 +297,53 @@ namespace DeepDrftData.Migrations
b.ToTable("session_metadata", (string)null);
});
modelBuilder.Entity("DeepDrftModels.Entities.ShareEvent", b =>
{
b.Property<long>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<long>("Id"));
b.Property<string>("AnonId")
.HasMaxLength(64)
.HasColumnType("character varying(64)")
.HasColumnName("anon_id");
b.Property<string>("Channel")
.IsRequired()
.HasMaxLength(20)
.HasColumnType("character varying(20)")
.HasColumnName("channel");
b.Property<DateTime>("CreatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("created_at");
b.Property<string>("TargetKey")
.IsRequired()
.HasMaxLength(100)
.HasColumnType("character varying(100)")
.HasColumnName("target_key");
b.Property<string>("TargetType")
.IsRequired()
.HasMaxLength(20)
.HasColumnType("character varying(20)")
.HasColumnName("target_type");
b.HasKey("Id");
b.HasIndex("AnonId")
.HasDatabaseName("IX_share_event_anon_id");
b.HasIndex("TargetKey")
.HasDatabaseName("IX_share_event_target_key");
b.ToTable("share_event", (string)null);
});
modelBuilder.Entity("DeepDrftModels.Entities.TrackEntity", b =>
{
b.Property<long>("Id")
@@ -222,6 +357,10 @@ namespace DeepDrftData.Migrations
.HasColumnType("timestamp with time zone")
.HasColumnName("created_at");
b.Property<double?>("DurationSeconds")
.HasColumnType("double precision")
.HasColumnName("duration_seconds");
b.Property<string>("EntryKey")
.IsRequired()
.HasMaxLength(100)
@@ -0,0 +1,175 @@
using DeepDrftData.Data;
using DeepDrftModels.Entities;
using DeepDrftModels.Enums;
using Microsoft.EntityFrameworkCore;
namespace DeepDrftData.Repositories;
/// <summary>
/// Data access for the Phase 16 anonymous telemetry tables (all SQL — the FileDatabase vault is not
/// involved). Owns the append-only writes to <c>play_event</c> / <c>share_event</c> and the
/// incremental-on-write bump of the <c>play_counter</c> rollup (D6). Server-side release resolution
/// (§2.3 / D4) lives here: a play event carries only the track key, and this repository joins
/// track→release at write time and stamps the release id on the row.
///
/// <para>
/// Unlike <see cref="TrackRepository"/> these entities are not <c>BaseEntity</c>/<c>IEntity</c> (no
/// soft-delete lifecycle), so this is a plain context-backed repository rather than an extension of the
/// BlazorBlocks <c>Repository&lt;&gt;</c> base. It holds the same scoped <see cref="DeepDrftContext"/>
/// the rest of the SQL layer uses, never a service locator.
/// </para>
/// </summary>
public class EventRepository
{
private readonly DeepDrftContext _context;
public EventRepository(DeepDrftContext context)
{
_context = context;
}
/// <summary>
/// Append one play event and bump the track's counter in a single transaction (D6). The release id
/// is resolved here from the track key (§2.3 / D4): a live track contributes its release id (null
/// for a loose track); an unknown key records the event with a null release and no counter bump
/// (there is no track to roll up against). Returns true when the event was written.
/// </summary>
public async Task<bool> RecordPlayAsync(
string trackEntryKey, PlayBucket bucket, string? anonId, CancellationToken ct = default)
{
// Resolve the track→release link server-side. Soft-deleted tracks resolve to null so a play of
// a since-removed track still logs (with no counter bump) rather than throwing.
var track = await _context.Tracks
.Where(t => t.EntryKey == trackEntryKey && !t.IsDeleted)
.Select(t => new { t.Id, t.ReleaseId })
.FirstOrDefaultAsync(ct);
// The append and the counter bump must commit together — wrap them in one transaction so a
// counter that drifts from the log is impossible. Reuse an ambient transaction if the caller
// already opened one.
var ownsTransaction = _context.Database.CurrentTransaction is null;
var transaction = ownsTransaction
? await _context.Database.BeginTransactionAsync(ct)
: null;
try
{
_context.PlayEvents.Add(new PlayEvent
{
TrackEntryKey = trackEntryKey,
ReleaseId = track?.ReleaseId,
Bucket = bucket,
AnonId = anonId,
CreatedAt = DateTime.UtcNow,
});
if (track is not null)
await BumpCounterAsync(track.Id, bucket, ct);
await _context.SaveChangesAsync(ct);
if (transaction is not null)
await transaction.CommitAsync(ct);
return true;
}
catch
{
if (transaction is not null)
await transaction.RollbackAsync(ct);
throw;
}
finally
{
if (transaction is not null)
await transaction.DisposeAsync();
}
}
/// <summary>
/// Site-wide total plays: the sum of every counter's three bucket columns across all rows (Phase 16
/// §5). Sums the mapped columns directly rather than <see cref="PlayCounter.TotalPlays"/>, which is an
/// EF-ignored computed property and so not translatable. An empty counter table sums to 0 (the home
/// card's expected reading until the telemetry migration is applied).
/// </summary>
public Task<long> CountTotalPlaysAsync(CancellationToken ct = default)
=> _context.PlayCounters
.SumAsync(c => c.PartialCount + c.SampledCount + c.CompleteCount, ct);
/// <summary>
/// Count distinct non-null anon ids across every play event (Phase 16 §3 / §4.2 — the all-time
/// unique-listener metric, D3). Null anon ids (events where the listener sent no token, or storage
/// was unavailable) are excluded — they are not a known listener and must not inflate the count. This
/// is the site-wide listener reach figure; the per-track / per-release overloads scope it.
/// </summary>
public Task<int> CountDistinctListenersAsync(CancellationToken ct = default)
=> _context.PlayEvents
.Where(e => e.AnonId != null)
.Select(e => e.AnonId)
.Distinct()
.CountAsync(ct);
/// <summary>
/// Distinct listeners for one track, keyed by its vault entry key (the same key the play event
/// stamps). Null anon ids excluded. Per-track scope of <see cref="CountDistinctListenersAsync()"/>.
/// </summary>
public Task<int> CountDistinctListenersForTrackAsync(string trackEntryKey, CancellationToken ct = default)
=> _context.PlayEvents
.Where(e => e.TrackEntryKey == trackEntryKey && e.AnonId != null)
.Select(e => e.AnonId)
.Distinct()
.CountAsync(ct);
/// <summary>
/// Distinct listeners for one release, derived across the release's tracks (D4): the play event
/// stamps the resolved release id at write time, so a distinct count over <c>anon_id</c> filtered by
/// <c>release_id</c> is exactly "distinct listeners who played any track in this release." Null anon
/// ids excluded. A listener who heard two tracks of the release counts once (it is a distinct count
/// over the union, not a sum of per-track counts).
/// </summary>
public Task<int> CountDistinctListenersForReleaseAsync(long releaseId, CancellationToken ct = default)
=> _context.PlayEvents
.Where(e => e.ReleaseId == releaseId && e.AnonId != null)
.Select(e => e.AnonId)
.Distinct()
.CountAsync(ct);
/// <summary>Append one share event. No rollup table for shares in wave 16.1 — a plain insert.</summary>
public async Task RecordShareAsync(
ShareTargetType targetType, string targetKey, ShareChannel channel, string? anonId,
CancellationToken ct = default)
{
_context.ShareEvents.Add(new ShareEvent
{
TargetType = targetType,
TargetKey = targetKey,
Channel = channel,
AnonId = anonId,
CreatedAt = DateTime.UtcNow,
});
await _context.SaveChangesAsync(ct);
}
// Bump the matching bucket column on the track's counter row, creating the row on first play. The
// row is added to the change tracker but not saved here — the caller's SaveChanges/commit persists
// it inside the same transaction as the event append.
//
// Race note: two concurrent first-plays of the same track can both reach this method, find no
// counter row, and both Add a new PlayCounter. The second SaveChanges will hit the unique index on
// (track_id) and throw, causing the outer transaction to roll back and the event to be dropped —
// no crash, no counter corruption. At the expected play volume this is an acceptable loss; the
// unique index is the integrity backstop.
private async Task BumpCounterAsync(long trackId, PlayBucket bucket, CancellationToken ct)
{
var counter = await _context.PlayCounters.FirstOrDefaultAsync(c => c.TrackId == trackId, ct);
if (counter is null)
{
counter = new PlayCounter { TrackId = trackId };
_context.PlayCounters.Add(counter);
}
switch (bucket)
{
case PlayBucket.Partial: counter.PartialCount++; break;
case PlayBucket.Sampled: counter.SampledCount++; break;
case PlayBucket.Complete: counter.CompleteCount++; break;
}
}
}
@@ -3,6 +3,7 @@ using Data.Errors;
using DeepDrftData.Data;
using DeepDrftModels.DTOs;
using DeepDrftModels.Entities;
using DeepDrftModels.Enums;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
using Models.Common;
@@ -157,6 +158,70 @@ public class TrackRepository : Repository<DeepDrftContext, TrackEntity>
.Select(g => new { ReleaseId = g.Key, Count = g.Count() })
.ToDictionaryAsync(x => x.ReleaseId, x => x.Count, ct);
// Aggregate figures for the public home hero stat row, assembled in as few round-trips as is clean.
// All counts go through Query (!t.IsDeleted) plus an explicit !t.Release.IsDeleted guard so tracks
// under a directly-deleted release are also excluded. Mix runtime sums DurationSeconds with a
// null-coalesce to 0 so not-yet-backfilled rows contribute zero rather than throwing or skewing the
// total. The cut release-type breakdown is grouped here so a zero-count type is simply absent from
// the result (no present-with-zero row).
public async Task<HomeStatsDto> GetHomeStatsAsync(CancellationToken ct = default)
{
var releases = _context.Set<ReleaseEntity>().Where(r => !r.IsDeleted);
var cutTrackCount = await Query
.CountAsync(t => t.Release != null && !t.Release.IsDeleted && t.Release.Medium == ReleaseMedium.Cut, ct);
var cutReleaseTypeCounts = await releases
.Where(r => r.Medium == ReleaseMedium.Cut)
.GroupBy(r => r.ReleaseType)
.Select(g => new CutReleaseTypeCount { ReleaseType = g.Key, Count = g.Count() })
.ToListAsync(ct);
var mixReleaseCount = await releases
.CountAsync(r => r.Medium == ReleaseMedium.Mix, ct);
var mixRuntimeSeconds = await Query
.Where(t => t.Release != null && !t.Release.IsDeleted && t.Release.Medium == ReleaseMedium.Mix)
.SumAsync(t => t.DurationSeconds ?? 0d, ct);
return new HomeStatsDto
{
CutTrackCount = cutTrackCount,
CutReleaseTypeCounts = cutReleaseTypeCounts,
MixReleaseCount = mixReleaseCount,
MixRuntimeSeconds = mixRuntimeSeconds,
};
}
// EntryKey + stored duration for non-deleted tracks whose SQL duration is still null — the work list
// the one-time duration backfill iterates. The migration cannot read the vault, so duration is filled
// at runtime: this lists which rows still need it, the backfill reads each from the vault and writes
// it back via UpdateDurationAsync.
public async Task<List<TrackEntity>> GetTracksMissingDurationAsync(CancellationToken ct = default)
=> await Query.Where(t => t.DurationSeconds == null).ToListAsync(ct);
// Set-based duration write for one track (no load round-trip), used by the backfill. The
// DurationSeconds == null guard keeps a re-run from re-stamping updated_at on an already-filled row
// and from clobbering a value the upload path may have set in the meantime.
public async Task<int> UpdateDurationAsync(long id, double durationSeconds, CancellationToken ct = default)
=> await Query
.Where(t => t.Id == id && t.DurationSeconds == null)
.ExecuteUpdateAsync(s => s
.SetProperty(t => t.DurationSeconds, durationSeconds)
.SetProperty(t => t.UpdatedAt, DateTime.UtcNow), ct);
// Unconditional duration overwrite for one track (no load round-trip), used by the replace-audio
// path. Unlike UpdateDurationAsync, there is no null guard — replace always overwrites the
// existing value because a normally-uploaded track already has a non-null DurationSeconds and the
// null-guarded backfill query would match zero rows and silently leave it stale. Returns the count
// of rows affected; zero means the track was removed between the GetById lookup and this write.
public async Task<int> SetDurationAsync(long id, double durationSeconds, CancellationToken ct = default)
=> await Query
.Where(t => t.Id == id)
.ExecuteUpdateAsync(s => s
.SetProperty(t => t.DurationSeconds, durationSeconds)
.SetProperty(t => t.UpdatedAt, DateTime.UtcNow), ct);
// Resolve an existing release by its natural key (title + artist). Returns null when no match,
// signalling the manager to create one. Soft-deleted releases never match.
public async Task<ReleaseEntity?> GetReleaseByTitleAndArtistAsync(
@@ -211,6 +276,7 @@ public class TrackRepository : Repository<DeepDrftContext, TrackEntity>
target.TrackName = source.TrackName;
target.TrackNumber = source.TrackNumber;
target.OriginalFileName = source.OriginalFileName;
target.DurationSeconds = source.DurationSeconds;
target.ReleaseId = source.ReleaseId;
}
}
+2
View File
@@ -80,6 +80,7 @@ public class TrackConverter : IEntityToModelConverter<TrackEntity, TrackDto>
TrackName = entity.TrackName,
OriginalFileName = entity.OriginalFileName,
TrackNumber = entity.TrackNumber,
DurationSeconds = entity.DurationSeconds,
ReleaseId = entity.ReleaseId,
Release = entity.Release is null ? null : Convert(entity.Release)
};
@@ -96,6 +97,7 @@ public class TrackConverter : IEntityToModelConverter<TrackEntity, TrackDto>
TrackName = model.TrackName,
OriginalFileName = model.OriginalFileName,
TrackNumber = model.TrackNumber,
DurationSeconds = model.DurationSeconds,
ReleaseId = model.ReleaseId
};
}
+64 -9
View File
@@ -164,14 +164,14 @@ public class TrackManager
}
}
public async Task<ResultContainer<ReleaseDto>> FindOrCreateRelease(
public async Task<ResultContainer<(ReleaseDto Release, bool WasCreated)>> FindOrCreateRelease(
string title, string artist, ReleaseDto releaseData, CancellationToken cancellationToken = default)
{
try
{
var existing = await Repository.GetReleaseByTitleAndArtistAsync(title, artist, cancellationToken);
if (existing is not null)
return ResultContainer<ReleaseDto>.CreatePassResult(TrackConverter.Convert(existing));
return ResultContainer<(ReleaseDto, bool)>.CreatePassResult((TrackConverter.Convert(existing), false));
// The natural key (title + artist) is authoritative — override whatever the caller put
// in releaseData so a typo upstream cannot create a release that won't be found again.
@@ -186,21 +186,21 @@ public class TrackManager
try
{
var added = await Repository.AddReleaseAsync(entity, cancellationToken);
return ResultContainer<ReleaseDto>.CreatePassResult(TrackConverter.Convert(added));
return ResultContainer<(ReleaseDto, bool)>.CreatePassResult((TrackConverter.Convert(added), true));
}
catch (ClassifiedDbException ex) when (ex.Error.Category == DbErrorCategory.UniqueViolation)
{
// Concurrent upload inserted the same (title, artist) between our read and write.
// Re-query and return the winning row. Should not return null here since the
// constraint just fired, but re-throw if it does so the caller sees an error.
// Re-query and return the winning row as WasCreated=false so the caller (UploadAsync
// CREATE path) treats the lost race as a duplicate rather than silently attaching.
var race = await Repository.GetReleaseByTitleAndArtistAsync(title, artist, cancellationToken);
if (race is null) throw;
return ResultContainer<ReleaseDto>.CreatePassResult(TrackConverter.Convert(race));
return ResultContainer<(ReleaseDto, bool)>.CreatePassResult((TrackConverter.Convert(race), false));
}
}
catch (Exception e)
{
return ResultContainer<ReleaseDto>.CreateFailResult(e.Message);
return ResultContainer<(ReleaseDto, bool)>.CreateFailResult(e.Message);
}
}
@@ -236,6 +236,61 @@ public class TrackManager
}
}
public async Task<ResultContainer<HomeStatsDto>> GetHomeStats(CancellationToken cancellationToken = default)
{
try
{
var stats = await Repository.GetHomeStatsAsync(cancellationToken);
return ResultContainer<HomeStatsDto>.CreatePassResult(stats);
}
catch (Exception e)
{
return ResultContainer<HomeStatsDto>.CreateFailResult(e.Message);
}
}
public async Task<ResultContainer<List<TrackDto>>> GetTracksMissingDuration(CancellationToken cancellationToken = default)
{
try
{
var entities = await Repository.GetTracksMissingDurationAsync(cancellationToken);
return ResultContainer<List<TrackDto>>.CreatePassResult(
entities.Select(TrackConverter.Convert).ToList());
}
catch (Exception e)
{
return ResultContainer<List<TrackDto>>.CreateFailResult(e.Message);
}
}
public async Task<ResultContainer<int>> UpdateDuration(long id, double durationSeconds, CancellationToken cancellationToken = default)
{
try
{
var updated = await Repository.UpdateDurationAsync(id, durationSeconds, cancellationToken);
return ResultContainer<int>.CreatePassResult(updated);
}
catch (Exception e)
{
return ResultContainer<int>.CreateFailResult(e.Message);
}
}
public async Task<ResultContainer<int>> SetDuration(long id, double durationSeconds, CancellationToken cancellationToken = default)
{
try
{
var affected = await Repository.SetDurationAsync(id, durationSeconds, cancellationToken);
if (affected == 0)
return ResultContainer<int>.CreateFailResult($"Duration write matched no rows for track {id}.");
return ResultContainer<int>.CreatePassResult(affected);
}
catch (Exception e)
{
return ResultContainer<int>.CreateFailResult(e.Message);
}
}
public async Task<ResultContainer<TrackDto>> Create(TrackDto newTrack)
{
try
@@ -247,13 +302,13 @@ public class TrackManager
if (newTrack.Release is { } release && !string.IsNullOrWhiteSpace(release.Title))
{
var resolved = await FindOrCreateRelease(release.Title, release.Artist, release);
if (!resolved.Success || resolved.Value is null)
if (!resolved.Success)
{
var error = resolved.Messages.FirstOrDefault()?.Message ?? "Failed to resolve release.";
return ResultContainer<TrackDto>.CreateFailResult(error);
}
newTrack.ReleaseId = resolved.Value.Id;
newTrack.ReleaseId = resolved.Value.Release.Id;
}
var added = await Repository.AddAsync(TrackConverter.Convert(newTrack));
+1
View File
@@ -13,6 +13,7 @@
<link rel="stylesheet" href="@Assets["_content/DeepDrftShared.Client/styles/deepdrft-tokens.css"]" />
<ImportMap />
<link rel="icon" type="image/ico" href="deepdrft-logo.ico" />
<meta name="robots" content="noindex,nofollow" />
<HeadOutlet @rendermode="ServerMode" />
</head>
@@ -6,9 +6,23 @@
<MudLayout>
<MudAppBar Dense="true" Elevation="1" Color="Color.Primary">
<MudText Typo="Typo.h6" Class="ml-3" Style="font-family: 'DM Sans', sans-serif; letter-spacing: 0.05em;">
Deep Drft — Admin
</MudText>
<a href="/" class="mx-2">
<MudStack Row AlignItems="AlignItems.Center">
<MudImage Src="img/deepdrft-logo-l.webp"
Alt="Deep Drft Ornamental Logo Left"
Width="24"
Height="24 "
Style="filter: invert(1);"/>
<MudText Typo="Typo.button" Style="color: var(--deepdrft-white);">Deep DRFT Management</MudText>
<MudImage Src="img/deepdrft-logo-r.webp"
Alt="Deep Drft Ornamental Logo Right"
Width="24"
Height="24"
Style="filter: invert(1);"/>
</MudStack>
</a>
</MudAppBar>
<MudMainContent>
<MudContainer MaxWidth="MaxWidth.Small"
@@ -1,5 +1,6 @@
@inherits LayoutComponentBase
@using DeepDrftShared.Client.Common
@using AuthBlocksWeb.Components.Layout
<MudThemeProvider IsDarkMode="false" Theme="@DeepDrftPalettes.Cms" />
<MudPopoverProvider />
@@ -8,9 +9,27 @@
<MudLayout>
<MudAppBar Dense="true" Elevation="1" Color="Color.Primary">
<MudText Typo="Typo.h6" Class="ml-3" Style="font-family: 'DM Sans', sans-serif; letter-spacing: 0.05em;">
Deep Drft — Admin
</MudText>
<MudIconButton Icon="@Icons.Material.Filled.Menu"
Color="Color.Inherit"
Edge="Edge.Start"
OnClick="ToggleDrawer" />
<a href="/" class="mx-2">
<MudStack Row AlignItems="AlignItems.Center">
<MudImage Src="img/deepdrft-logo-l.webp"
Alt="Deep Drft Ornamental Logo Left"
Width="24"
Height="24 "
Style="filter: invert(1);"/>
<MudText Typo="Typo.button" Style="color: var(--deepdrft-white);">Deep DRFT Management</MudText>
<MudImage Src="img/deepdrft-logo-r.webp"
Alt="Deep Drft Ornamental Logo Right"
Width="24"
Height="24"
Style="filter: invert(1);"/>
</MudStack>
</a>
<MudSpacer />
<MudTooltip Text="Catalogue">
<MudIconButton Icon="@Icons.Material.Filled.Home"
@@ -18,6 +37,20 @@
Color="Color.Inherit" />
</MudTooltip>
</MudAppBar>
<MudDrawer @bind-Open="_drawerOpen" Elevation="2" Variant="DrawerVariant.Responsive" ClipMode="DrawerClipMode.Always">
<MudNavMenu>
<MudNavLink Href="/catalogue" Match="NavLinkMatch.All" Icon="@Icons.Material.Filled.Dashboard">Catalogue</MudNavLink>
<MudNavLink Href="/releases" Match="NavLinkMatch.Prefix" Icon="@Icons.Material.Filled.LibraryMusic">Releases</MudNavLink>
<MudNavLink Href="/tracks/upload" Match="NavLinkMatch.Prefix" Icon="@Icons.Material.Filled.UploadFile">Upload</MudNavLink>
<UserAdminMenu />
<HierarchicalRoleAuthorizeView RolesList="@([SystemRoleConstants.UserAdmin])">
<Authorized>
<MudNavLink Href="/useradmin/users/new" Match="NavLinkMatch.Prefix" Icon="@Icons.Material.Filled.PersonAdd">Provision User</MudNavLink>
</Authorized>
</HierarchicalRoleAuthorizeView>
<AccountNavMenu />
</MudNavMenu>
</MudDrawer>
<MudMainContent Class="pt-14 pb-8">
<MudContainer MaxWidth="MaxWidth.False" Class="pa-4">
@Body
@@ -25,6 +58,12 @@
</MudMainContent>
</MudLayout>
@code {
private bool _drawerOpen = true;
private void ToggleDrawer() => _drawerOpen = !_drawerOpen;
}
<div id="blazor-error-ui" data-nosnippet>
An unhandled error has occurred.
<a href="." class="reload">Reload</a>
+1 -1
View File
@@ -1,6 +1,6 @@
@page "/404"
<PageTitle>SkipperHaven - Page Not Found</PageTitle>
<PageTitle>Deep DRFT Management - Page Not Found</PageTitle>
<MudText Typo="Typo.h1" Color="Color.Primary">
404 - Resource Not Found
+2 -3
View File
@@ -1,7 +1,7 @@
@page "/"
@layout Layout.CmsHomeLayout
<PageTitle>Deep Drft — Admin</PageTitle>
<PageTitle>Deep DRFT Management</PageTitle>
<HierarchicalRoleAuthorizeView>
<Authorized>
@@ -9,8 +9,7 @@
</Authorized>
<NotAuthorized>
<MudStack AlignItems="AlignItems.Center" Spacing="4" Class="my-8">
<MudImage Fluid="true" Src="img/cms-hero.png" Alt="Deep Drft" />
<MudText Typo="Typo.h2" Align="Align.Center">Deep Drft</MudText>
<MudImage Fluid="true" Src="img/cms-hero.webp" Alt="Deep Drft" />
<MudText Typo="Typo.subtitle1" Align="Align.Center" Class="text-uppercase mud-text-secondary">
Catalogue Management
</MudText>
+1 -1
View File
@@ -7,7 +7,7 @@
@inject ICmsReleaseService CmsReleaseService
@inject ILogger<Index> Logger
<PageTitle>DeepDrft CMS</PageTitle>
<PageTitle>Deep DRFT Management - Catalogue</PageTitle>
<MudContainer MaxWidth="MaxWidth.Large" Class="mt-8">
<MudText Typo="Typo.h3" Class="mb-6">Catalogue</MudText>
@@ -14,7 +14,7 @@
@inject IDialogService DialogService
@inject ILogger<BatchEdit> Logger
<PageTitle>Edit Release — DeepDrft CMS</PageTitle>
<PageTitle>Edit Release — Deep DRFT Management</PageTitle>
<MudContainer MaxWidth="MaxWidth.Large" Class="mt-8">
<MudText Typo="Typo.h4" GutterBottom="true">Edit Release</MudText>
@@ -59,14 +59,19 @@
single-track medium, mirroring BatchUpload's same-named collapse. Cut keeps the full list. *@
<MudGrid>
<MudItem xs="12" md="5">
@* ExistingTrackCount counts edit-session persisted rows (Id.HasValue), not authoritative
live release count — acceptable because this gate only hides a UI control; the
TrySoftDeleteEmptyReleaseAsync backstop remains the authoritative guard. *@
<BatchTrackList Tracks="_tracks"
@bind-SelectedIndex="_selectedIndex"
Disabled="_saving"
AllowNewTracks="@(_medium == ReleaseMedium.Cut)"
ExistingTrackCount="@_tracks.Count(t => t.Id.HasValue)"
OnWavFilesSelected="HandleWavFilesSelected"
OnMoveUp="MoveUp"
OnMoveDown="MoveDown"
OnRemove="RemoveRow" />
OnRemove="RemoveRow"
OnReplaceFileSelected="HandleReplaceFileSelected" />
</MudItem>
<MudItem xs="12" md="7">
@@ -109,8 +114,8 @@
</MudContainer>
@code {
// 1 GB ceiling matches DeepDrftAPI's per-request limit on api/track/upload.
private const long MaxUploadBytes = 1_073_741_824L;
// ~1.86 GB ceiling matches DeepDrftAPI's per-request limit on api/track/upload.
private const long MaxUploadBytes = 2_000_000_000L;
// Release-title addressing (Album-mode batch Edit): loads the whole release by title.
[Parameter] public string AlbumName { get; set; } = string.Empty;
@@ -141,6 +146,9 @@
private string _releaseDate = string.Empty;
private ReleaseType _releaseType = ReleaseType.Single;
private ReleaseMedium _medium = ReleaseMedium.Cut;
// The id of the release being edited. New tracks added in this session attach to it via the upload
// service's releaseId (ATTACH) path, so they are not rejected as a pre-existing-(title,artist) duplicate.
private long? _releaseId;
// The medium selector drives ReleaseType visibility and is persisted on save: every UpdateAsync /
// UploadTrackAsync call below passes _medium, and PUT api/track/meta resets ReleaseType to its
@@ -209,6 +217,10 @@
}
var release = tracks[0].Release;
// The release being edited already exists, so any new track added here ATTACHES to it (the upload
// service's releaseId path) rather than taking the CREATE path, which would reject it as a
// duplicate (title, artist). Fall back to the track's own ReleaseId if the nav is not populated.
_releaseId = release?.Id ?? tracks[0].ReleaseId;
_albumName = albumName;
_artist = release?.Artist ?? string.Empty;
_genre = release?.Genre ?? string.Empty;
@@ -322,6 +334,85 @@
if (_selectedIndex >= _tracks.Count) _selectedIndex = _tracks.Count - 1;
}
private async Task HandleReplaceFileSelected((int Index, IBrowserFile File) picked)
{
var (index, file) = picked;
if (index < 0 || index >= _tracks.Count) return;
var row = _tracks[index];
if (!row.Id.HasValue)
{
// Defensive: replace is only offered on persisted rows. A new row would have no track to
// swap against — it takes the upload path on save instead.
return;
}
if (!file.Name.EndsWith(".wav", StringComparison.OrdinalIgnoreCase))
{
Snackbar.Add($"'{file.Name}' is not a .wav file.", Severity.Warning);
return;
}
var confirmed = await DialogService.ShowMessageBox(
"Replace audio",
$"Replace the audio for '{row.TrackName}' with '{file.Name}'? " +
"Metadata stays the same; the waveform is regenerated for the new audio.",
yesText: "Replace", cancelText: "Cancel");
if (confirmed != true) return;
row.Status = BatchRowStatus.Uploading;
row.UploadedBytes = 0;
row.TotalBytes = file.Size;
row.ErrorMessage = null;
StateHasChanged();
try
{
await using var wavStream = file.OpenReadStream(MaxUploadBytes);
var lastPercent = -1;
var progress = new Progress<long>(written =>
{
row.UploadedBytes = written;
if (row.UploadPercent != lastPercent)
{
lastPercent = row.UploadPercent;
StateHasChanged();
}
});
var result = await CmsTrackService.ReplaceTrackAudioAsync(
row.Id.Value, wavStream, file.Size, file.Name, file.ContentType, progress);
if (!result.Success)
{
var error = result.Messages.FirstOrDefault()?.Message ?? "Unknown error";
row.Status = BatchRowStatus.Failed;
row.ErrorMessage = error;
Snackbar.Add($"Replace failed: {error}", Severity.Error);
}
else
{
// Reset to Queued (not Done): a Done row is skipped by SaveAsync, but the admin may
// still want to save pending metadata edits. The audio swap is already persisted.
row.Status = BatchRowStatus.Queued;
row.OriginalFileName = file.Name;
Snackbar.Add($"Replaced audio for '{row.TrackName}'.", Severity.Success);
}
}
catch (Exception ex)
{
Logger.LogError(ex, "Replace audio failed for track id {Id}", row.Id);
row.Status = BatchRowStatus.Failed;
row.ErrorMessage = "Replace failed — please try again.";
Snackbar.Add("Replace failed — please try again.", Severity.Error);
}
finally
{
StateHasChanged();
}
}
private void RemoveCover()
{
// Defer the actual clear to save: pass "" to UpdateAsync's tri-state imagePath. Nulling
@@ -427,7 +518,10 @@
{
var row = _tracks[i];
if (row.Status == BatchRowStatus.Done)
// Skip rows already processed in a prior submit attempt. PostProcessing counts as processed:
// the track persisted successfully (only its background Opus derive is still settling), so a
// re-submit after a partial failure must NOT re-upload it and mint a duplicate.
if (row.Status is BatchRowStatus.Done or BatchRowStatus.PostProcessing)
{
_processedCount++;
continue;
@@ -508,6 +602,7 @@
_releaseType,
trackNumber,
_medium,
_releaseId,
progress);
if (!uploadResult.Success || uploadResult.Value is null)
@@ -546,7 +641,12 @@
}
}
row.Status = BatchRowStatus.Done;
// §3.1a: a new-track upload persists the track (live + lossless) and the server
// derives Opus in the background, so the row enters the visible Post-Processing
// phase — same as BatchUpload. (The metadata-only update path above stays Done: it
// changes no audio, so it triggers no transcode.) Non-blocking; the Releases view
// polls the durable Opus status to settle it.
row.Status = BatchRowStatus.PostProcessing;
succeeded++;
}
}
@@ -40,4 +40,9 @@ public class BatchRowModel
: 0;
}
public enum BatchRowStatus { Queued, Uploading, Done, Failed }
// Done is the terminal success state (track persisted + playable losslessly). PostProcessing is the
// visible third upload phase (§3.1a): the byte transfer and server persist are finished and the track is
// live, but the server-side background Opus transcode is still running. It is NOT a failure and never
// blocks completion — the form may navigate away while a row sits in PostProcessing; the Releases browse
// view polls the durable Opus status from there.
public enum BatchRowStatus { Queued, Uploading, PostProcessing, Done, Failed }
@@ -21,7 +21,7 @@ else
{
<MudField Label="Original File" Variant="Variant.Outlined" InnerPadding="false">
<MudText Typo="Typo.body2">@(string.IsNullOrEmpty(SelectedTrack.OriginalFileName) ? "—" : SelectedTrack.OriginalFileName)</MudText>
<MudText Typo="Typo.caption" Color="Color.Default">Existing track — audio is not editable.</MudText>
<MudText Typo="Typo.caption" Color="Color.Default">Use the Replace audio action in the list to swap this track's audio.</MudText>
</MudField>
}
else
@@ -34,12 +34,37 @@
Disabled="@(index == Tracks.Count - 1 || Disabled)"
OnClick="@(() => OnMoveDown.InvokeAsync(index))"
aria-label="Move track down" />
<MudIconButton Icon="@Icons.Material.Filled.Delete"
Color="Color.Error"
Size="Size.Small"
Disabled="@Disabled"
OnClick="@(() => OnRemove.InvokeAsync(index))"
aria-label="Remove track" />
@* Replace audio: existing (persisted) rows only. New rows still pick their WAV
via the file input above, so a replace control there would be redundant. A
native <label for> drives a per-row hidden InputFile — clicking the icon
opens that row's picker with zero JS (no eval, no programmatic .click()). *@
@if (row.Id.HasValue)
{
<label for="@ReplaceInputId(index)" @onclick:stopPropagation="true"
style="display: inline-flex; @(Disabled ? "pointer-events: none; opacity: 0.5;" : "cursor: pointer;")">
<MudIcon Icon="@Icons.Material.Filled.SwapHoriz"
Color="Color.Primary"
Size="Size.Small"
aria-label="Replace audio" />
</label>
<InputFile id="@ReplaceInputId(index)"
OnChange="@(e => OnReplaceFileSelected.InvokeAsync((index, e.File)))"
accept=".wav,audio/wav,audio/x-wav"
disabled="@Disabled"
style="display: none;" />
}
@* Remove: hidden for the sole remaining persisted track so a release can never
be track-deleted down to zero (that path soft-deletes the whole release). New
rows are always removable — dropping one only discards a pending upload. *@
@if (!(row.Id.HasValue && ExistingTrackCount <= 1))
{
<MudIconButton Icon="@Icons.Material.Filled.Delete"
Color="Color.Error"
Size="Size.Small"
Disabled="@Disabled"
OnClick="@(() => OnRemove.InvokeAsync(index))"
aria-label="Remove track" />
}
</MudStack>
@if (row.Status == BatchRowStatus.Uploading)
{
@@ -60,11 +85,28 @@
[Parameter] public EventCallback<int> SelectedIndexChanged { get; set; }
[Parameter] public bool Disabled { get; set; }
[Parameter] public bool AllowNewTracks { get; set; } = true;
/// <summary>
/// Count of existing (persisted, Id-bearing) tracks in the list. When this is 1, the remove
/// control on the sole persisted row is suppressed so a release cannot be track-deleted to zero
/// (replace + release-level delete remain). New unsaved rows are excluded from this count.
/// </summary>
[Parameter] public int ExistingTrackCount { get; set; }
[Parameter] public EventCallback<IReadOnlyList<IBrowserFile>> OnWavFilesSelected { get; set; }
[Parameter] public EventCallback<int> OnMoveUp { get; set; }
[Parameter] public EventCallback<int> OnMoveDown { get; set; }
[Parameter] public EventCallback<int> OnRemove { get; set; }
/// <summary>
/// Raised when the admin picks a replacement WAV for an existing row, carrying the list index and
/// the chosen file. Only fired for persisted (Id-bearing) rows.
/// </summary>
[Parameter] public EventCallback<(int Index, IBrowserFile File)> OnReplaceFileSelected { get; set; }
// Stable per-row DOM id linking the swap-icon <label> to its hidden InputFile.
private static string ReplaceInputId(int index) => $"replace-audio-input-{index}";
private const int MaxFilesPerPick = 50;
private Task SelectRow(int index) => SelectedIndexChanged.InvokeAsync(index);
@@ -84,6 +126,8 @@
{
BatchRowStatus.Uploading => @<MudChip T="string" Size="Size.Small" Color="Color.Info" Variant="Variant.Text">
<MudProgressCircular Indeterminate="true" Size="Size.Small" Class="mr-1" />Uploading</MudChip>,
BatchRowStatus.PostProcessing => @<MudChip T="string" Size="Size.Small" Color="Color.Info" Variant="Variant.Text">
<MudProgressCircular Indeterminate="true" Size="Size.Small" Class="mr-1" />Post-Processing</MudChip>,
BatchRowStatus.Done => @<MudChip T="string" Size="Size.Small" Color="Color.Success" Variant="Variant.Text" Icon="@Icons.Material.Filled.CheckCircle">Done</MudChip>,
BatchRowStatus.Failed => @<MudChip T="string" Size="Size.Small" Color="Color.Error" Variant="Variant.Text" Icon="@Icons.Material.Filled.Error">Failed</MudChip>,
_ => @<MudChip T="string" Size="Size.Small" Color="Color.Default" Variant="Variant.Text">Queued</MudChip>
@@ -12,7 +12,7 @@
@inject ISnackbar Snackbar
@inject ILogger<BatchUpload> Logger
<PageTitle>Upload Release — DeepDrft CMS</PageTitle>
<PageTitle>Upload Release — Deep DRFT Management</PageTitle>
<MudContainer MaxWidth="MaxWidth.Large" Class="mt-8">
<MudText Typo="Typo.h4" GutterBottom="true">Upload Release</MudText>
@@ -69,6 +69,15 @@
Value="@_tracks[0].UploadPercent"
aria-label="Uploading track" />
}
else if (_tracks[0].Status == BatchRowStatus.PostProcessing)
{
@* §3.1a: track is live + plays lossless; the Opus transcode runs in the background.
Indeterminate (no client-side progress for a server-side job) and non-blocking. *@
<MudStack Row="true" AlignItems="AlignItems.Center" Spacing="2">
<MudProgressCircular Indeterminate="true" Size="Size.Small" />
<MudText Typo="Typo.caption">Post-Processing (deriving Opus)…</MudText>
</MudStack>
}
}
</MudStack>
</MudPaper>
@@ -108,9 +117,9 @@
</MudContainer>
@code {
// 1 GB ceiling matches DeepDrftAPI's per-request limit on api/track/upload; the
// ~1.86 GB ceiling matches DeepDrftAPI's per-request limit on api/track/upload; the
// streaming path means the limit caps the request, not in-memory buffering.
private const long MaxUploadBytes = 1_073_741_824L;
private const long MaxUploadBytes = 2_000_000_000L;
private List<BatchRowModel> _tracks = new();
private int _selectedIndex = -1;
@@ -129,6 +138,11 @@
// Set true once the admin has acknowledged the missing-hero warning, so a second submit proceeds.
private bool _heroWarningAcknowledged;
// Captured once at component initialization on the live interactive circuit, while the token
// is known-good, so a mid-session token expiry at submit time cannot discard a long-composed
// release. Only assigned when the id parses successfully.
private long? _createdByUserId;
private string _albumName = string.Empty;
private string _artist = string.Empty;
private string _genre = string.Empty;
@@ -156,6 +170,19 @@
}
}
protected override async Task OnInitializedAsync()
{
// Capture the user id once at load, while the token is known-good. The CMS host runs with
// prerender: false (InteractiveServer), so this is the single init pass — auth state is
// fully available. The page is [Authorize]-gated, so the parse should always succeed.
var authState = await AuthStateProvider.GetAuthenticationStateAsync();
var userIdValue = authState.User.FindFirstValue(ClaimTypes.NameIdentifier);
if (long.TryParse(userIdValue, out var userId))
{
_createdByUserId = userId;
}
}
// Switching to a single-track medium collapses any multi-track selection to the first row so the
// single-track invariant holds before submit. The predicate reads the same MediumRules cardinality
// declaration the upload service enforces, so the form and the domain cannot drift.
@@ -275,13 +302,12 @@
}
}
var authState = await AuthStateProvider.GetAuthenticationStateAsync();
var userIdValue = authState.User.FindFirstValue(ClaimTypes.NameIdentifier);
if (!long.TryParse(userIdValue, out var createdByUserId))
if (_createdByUserId is not long createdByUserId)
{
// The page is gated by [Authorize] under the Admin role, so a missing or
// unparseable id here is a configuration bug, not normal client state.
Logger.LogError("Authenticated user has no parseable NameIdentifier claim: {Value}", userIdValue);
// _createdByUserId is set at component initialization from the authenticated principal.
// A null here means the id was unavailable even at load — a genuine configuration bug,
// since the page is [Authorize]-gated.
Logger.LogError("User id was not captured at initialization — NameIdentifier claim missing or unparseable.");
_errorMessage = "Your session is missing a valid identifier. Please sign in again.";
return;
}
@@ -298,6 +324,29 @@
return;
}
// Pre-flight duplicate guard (primary block): the upload form creates new releases only, so a
// (title, artist) that already exists in the catalogue is refused BEFORE any bytes transfer —
// the admin is not surprised at the end of a long upload. The server backstops this on the
// create path, but checking here keeps the failure fast and visible. The values passed match
// exactly what the upload sends (untrimmed _albumName/_artist) so the pre-flight and the server
// agree on the match. A check failure (API unreachable) blocks rather than proceeding blind.
var duplicateCheck = await CmsTrackService.GetExistingReleaseAsync(_albumName, _artist);
if (!duplicateCheck.Success)
{
var checkError = duplicateCheck.Messages.FirstOrDefault()?.Message ?? "Unknown error";
_errorMessage = $"Could not verify the release name: {checkError}";
Snackbar.Add(_errorMessage, Severity.Error);
return;
}
if (duplicateCheck.Value is { } existing)
{
_errorMessage = $"A release titled '{existing.Title}' by {existing.Artist} already exists. "
+ "The upload form creates new releases only — use the edit tools to change an existing one.";
Snackbar.Add(_errorMessage, Severity.Error);
return;
}
// For single-track media (Session/Mix) the track name is derived from the Release Name —
// no separate Track Name input is shown. Sync here so the stored name always matches.
if (MediumRules.CardinalityOf(_medium).IsSingleTrack && _tracks.Count > 0)
@@ -327,6 +376,11 @@
}
int succeeded = 0, failed = 0;
// Within-batch attach: row 1 creates the release (no releaseId → CREATE path); once it
// succeeds we carry its ReleaseId into rows 2..N so they ATTACH to the just-created release
// rather than tripping the server's pre-existing-duplicate block. Only a multi-track Cut
// reaches row 2 (single-track media collapse to one row).
long? batchReleaseId = null;
for (int i = 0; i < _tracks.Count; i++)
{
var row = _tracks[i];
@@ -375,6 +429,7 @@
_releaseType,
trackNumber,
_medium,
batchReleaseId,
progress);
if (!result.Success || result.Value is null)
@@ -387,6 +442,15 @@
}
else
{
// Capture the release id created by the first successful row so subsequent rows
// attach to it (the within-batch multi-track Cut path). Only set once — later
// rows must not overwrite it. A null ReleaseId here (loose track) leaves it null,
// which is correct: a release-less upload has no within-batch release to attach to.
if (batchReleaseId is null && result.Value.ReleaseId is { } createdReleaseId)
{
batchReleaseId = createdReleaseId;
}
// The upload endpoint does not accept an imagePath, so link the cover art with
// a follow-up metadata update — same two-step pattern BatchEdit uses.
if (_imagePath is { } imgPath)
@@ -464,7 +528,13 @@
}
}
row.Status = BatchRowStatus.Done;
// §3.1a: the byte transfer + server persist are done and the track is live and plays
// losslessly — the upload is successful HERE. The server then derives Opus on a
// background worker, so the row enters the visible Post-Processing phase rather than
// jumping straight to Done. This never blocks: the loop continues and the form may
// navigate away while rows sit in Post-Processing; the Releases view polls the durable
// Opus status to settle each one.
row.Status = BatchRowStatus.PostProcessing;
succeeded++;
}
}
@@ -487,7 +557,13 @@
}
else
{
Snackbar.Add($"Uploaded {succeeded} track(s); {failed} failed — review errors below.", Severity.Warning);
// Surface the actual reason, not just counts — a server rejection (duplicate, cardinality)
// relays a human-readable message via row.ErrorMessage. Show the first failure's reason so
// the admin sees WHY without scanning the rows; the per-row errors remain as detail.
var firstError = _tracks.FirstOrDefault(t => t.Status == BatchRowStatus.Failed)?.ErrorMessage;
var reason = string.IsNullOrWhiteSpace(firstError) ? "review errors below" : firstError;
_errorMessage = succeeded == 0 ? reason : $"{succeeded} uploaded; {failed} failed: {reason}";
Snackbar.Add($"Uploaded {succeeded} track(s); {failed} failed — {reason}", Severity.Warning);
// Stay on page so the admin can see the failed rows.
}
}
@@ -18,7 +18,7 @@
}
else
{
<PageTitle>Mixes — DeepDrft CMS</PageTitle>
<PageTitle>Mixes — Deep DRFT Management</PageTitle>
<MudContainer MaxWidth="MaxWidth.Large" Class="mt-8">
<MudButton Variant="Variant.Text"
@@ -19,7 +19,7 @@
}
else
{
<PageTitle>Sessions — DeepDrft CMS</PageTitle>
<PageTitle>Sessions — Deep DRFT Management</PageTitle>
<MudContainer MaxWidth="MaxWidth.Large" Class="mt-8">
<MudButton Variant="Variant.Text"
@@ -9,9 +9,10 @@
@inject ISnackbar Snackbar
@inject ILogger<Releases> Logger
@inject NavigationManager NavigationManager
@implements IDisposable
@attribute [Authorize]
<PageTitle>Releases — DeepDrft CMS</PageTitle>
<PageTitle>Releases — Deep DRFT Management</PageTitle>
<MudContainer MaxWidth="MaxWidth.Large" Class="mt-8">
<MudStack Row="true" AlignItems="AlignItems.Center" Justify="Justify.SpaceBetween" Class="mb-4">
@@ -51,6 +52,28 @@
<span>Backfill High-res (@MissingHighResCount)</span>
}
</MudButton>
@* Backfill-Opus (Phase 18.5 + 18.6 badge). The Opus derive runs on a server-side background
worker: pressing the button enqueues every track lacking a complete Opus artifact and reports
the (enqueued / skipped) outcome. The "missing N" badge (18.6) reads the same opus-status map
the page polls, giving visual parity with the two waveform backfill buttons — but unlike them,
the count is informational, not a per-track client loop (the work is scheduled, not driven from
here). The button stays pressable at N=0 (a no-op re-run is harmless); it only disables while a
press is in flight or another bulk run holds the page. *@
<MudButton Variant="Variant.Outlined"
Color="Color.Primary"
StartIcon="@Icons.Material.Filled.GraphicEq"
Disabled="@(_bulkRunning || _highResBulkRunning || _opusBackfillRunning)"
OnClick="BackfillOpusAsync">
@if (_opusBackfillRunning)
{
<MudProgressCircular Class="mr-2" Size="Size.Small" Indeterminate="true" />
<span>Scheduling…</span>
}
else
{
<span>Backfill Opus (@MissingOpusCount)</span>
}
</MudButton>
</MudStack>
</MudStack>
@@ -140,6 +163,22 @@
private int MissingProfileCount => _waveformStatus.Count(s => !s.HasProfile);
private int MissingHighResCount => _waveformStatus.Count(s => !s.HasHighRes);
// EntryKey → HasOpus (a complete audio+sidecar derive). Loaded alongside the waveform status on init and
// re-read after a Backfill-Opus run so the "missing N" badge settles. Also the source the Post-Processing
// poll watches: a freshly uploaded track lands here with HasOpus=false and flips to true once the
// server-side background transcode finishes — the durable surface for the upload meter's Post-Processing
// phase after the form returns the admin to this view (§3.1a).
private IReadOnlyList<OpusStatusDto> _opusStatus = Array.Empty<OpusStatusDto>();
private int MissingOpusCount => _opusStatus.Count(s => !s.HasOpus);
// Post-Processing poll: while any track is still missing Opus (a transcode in flight or not yet
// backfilled), re-read the opus-status map on an interval so the "missing N" badge and any per-track
// Post-Processing indicator settle without a manual refresh. Stops itself once nothing is missing, and is
// torn down on dispose. Non-blocking — it never gates an upload or a button; it only refreshes a count.
private const int OpusPollIntervalMs = 4000;
private CancellationTokenSource? _opusPollCts;
// Local state for the parent-owned "Generate All Profiles" bulk run.
private bool _bulkRunning;
private int _bulkTotal;
@@ -150,6 +189,11 @@
private int _highResBulkTotal;
private int _highResBulkDone;
// Local state for the "Backfill Opus" action. The Opus derive is server-side and background-queued, so
// there is no client-side per-track loop or progress total — this flag only guards the button while the
// single scheduling call is in flight.
private bool _opusBackfillRunning;
protected override async Task OnInitializedAsync()
{
// Seed the active tab from ?medium= so a catalogue card deep-links straight to its medium. Panel 0
@@ -170,9 +214,81 @@
_waveformStatus = result.Success && result.Value is not null
? result.Value
: Array.Empty<WaveformStatusDto>();
await RefreshOpusStatusAsync();
StateHasChanged();
}
/// <summary>
/// Re-read the per-track Opus derive status and (re)arm the Post-Processing poll when work is still
/// pending. Called on init, after a Backfill-Opus run, and from the poll itself. Best-effort: a failed
/// fetch leaves the previous map in place rather than zeroing the badge on a transient API blip. Does not
/// call StateHasChanged itself — callers batch it with their own render (the poll path renders explicitly).
/// </summary>
private async Task RefreshOpusStatusAsync()
{
var opusResult = await CmsTrackService.GetOpusStatusAsync();
if (opusResult.Success && opusResult.Value is not null)
{
_opusStatus = opusResult.Value;
}
if (MissingOpusCount > 0)
{
EnsureOpusPollRunning();
}
}
// Start the Post-Processing poll if it is not already running. The loop re-reads opus-status every
// OpusPollIntervalMs and renders; it exits as soon as nothing is missing (or on cancel/dispose). Guarded
// by a non-null CTS so overlapping callers (init + backfill) cannot start two loops.
private void EnsureOpusPollRunning()
{
if (_opusPollCts is not null)
{
return;
}
_opusPollCts = new CancellationTokenSource();
_ = PollOpusStatusAsync(_opusPollCts.Token);
}
private async Task PollOpusStatusAsync(CancellationToken ct)
{
try
{
while (!ct.IsCancellationRequested && MissingOpusCount > 0)
{
await Task.Delay(OpusPollIntervalMs, ct);
var result = await CmsTrackService.GetOpusStatusAsync();
if (result.Success && result.Value is not null)
{
_opusStatus = result.Value;
await InvokeAsync(StateHasChanged);
}
}
}
catch (OperationCanceledException)
{
// Expected on dispose / navigation away — nothing to do.
}
catch (Exception ex)
{
Logger.LogWarning(ex, "Opus Post-Processing poll stopped on an unexpected error.");
}
finally
{
_opusPollCts?.Dispose();
_opusPollCts = null;
}
}
public void Dispose()
{
_opusPollCts?.Cancel();
}
// Invalidates the cached per-track waveform status on all embedded grids so the next row expand
// re-fetches fresh data. Called after each catalogue-wide bulk run so already-expanded rows
// reflect the new waveform state on the next expand interaction.
@@ -291,4 +407,54 @@
Snackbar.Add($"Backfilled {succeeded} high-res datum(s); {failures} failed.", Severity.Warning);
}
}
/// <summary>
/// Kick off the catalogue-wide Backfill-Opus pass. The API enumerates the tracks lacking a complete Opus
/// artifact, enqueues a background derive for each, and returns the (enqueued, skipped) counts. This is a
/// single scheduling call — the transcodes run server-side afterward — so there is no per-track progress
/// to render here, just a busy flag and a result snackbar. Re-runnable: a second press only schedules
/// tracks still missing Opus.
/// </summary>
private async Task BackfillOpusAsync()
{
_opusBackfillRunning = true;
StateHasChanged();
try
{
var result = await CmsTrackService.BackfillOpusAsync();
if (!result.Success)
{
var error = result.Messages.FirstOrDefault()?.Message ?? "Failed to start the Opus backfill.";
Snackbar.Add(error, Severity.Error);
return;
}
// Re-read the status map so the "missing N" badge reflects the just-enqueued work and the
// Post-Processing poll arms to watch the transcodes settle from N→0 as each finishes.
await RefreshOpusStatusAsync();
var (enqueued, skipped) = (result.Value.Enqueued, result.Value.Skipped);
if (enqueued == 0)
{
Snackbar.Add($"All {skipped} track(s) already have Opus — nothing to backfill.", Severity.Info);
}
else
{
Snackbar.Add(
$"Scheduled {enqueued} Opus transcode(s) in the background ({skipped} already had Opus). " +
"They will appear as each finishes.",
Severity.Success);
}
}
catch (Exception ex)
{
Logger.LogError(ex, "Opus backfill failed to start");
Snackbar.Add("Failed to start the Opus backfill.", Severity.Error);
}
finally
{
_opusBackfillRunning = false;
StateHasChanged();
}
}
}
+19 -2
View File
@@ -2,8 +2,8 @@
AdditionalAssemblies="new[] { typeof(AuthBlocksWeb._Imports).Assembly }"
NotFoundPage="typeof(NotFound)">
<Found Context="routeData">
<AuthorizeRouteView RouteData="routeData"
DefaultLayout="typeof(Layout.CmsLayout)">
<AuthorizeRouteView RouteData="routeData"
DefaultLayout="@_currentLayout">
<NotAuthorized Context="authState">
@if (authState.User.Identity?.IsAuthenticated == true)
{
@@ -18,3 +18,20 @@
<FocusOnNavigate RouteData="routeData" Selector="h1" />
</Found>
</Router>
@code {
[CascadingParameter] private Task<AuthenticationState>? AuthenticationState { get; set; }
private Type _currentLayout = typeof(Layout.CmsHomeLayout);
protected override async Task OnParametersSetAsync()
{
if (AuthenticationState is not null)
{
var authState = await AuthenticationState;
_currentLayout = authState.User.Identity?.IsAuthenticated == true
? typeof(Layout.CmsLayout)
: typeof(Layout.CmsHomeLayout);
}
}
}
+2 -1
View File
@@ -8,7 +8,7 @@
<ItemGroup>
<PackageReference Include="MudBlazor" Version="8.15.0" />
<PackageReference Include="Cerebellum.AuthBlocks.Web" Version="10.3.33" />
<PackageReference Include="Cerebellum.AuthBlocks.Web" Version="10.3.39" />
</ItemGroup>
<ItemGroup>
@@ -17,3 +17,4 @@
</ItemGroup>
</Project>
+308 -77
View File
@@ -28,11 +28,11 @@ public class CmsTrackService : ICmsTrackService
private const int DefaultIdleTimeoutSeconds = 90;
// Response-wait budget: once the request body is fully on the wire the server runs AudioProcessor
// decode → vault write → SQL persist. For a several-hundred-MB WAV this can take many minutes.
// decode → vault write → SQL persist. For a multi-GB WAV this can exceed 10 minutes.
// The idle heartbeat goes silent after the last byte, so a separate, larger deadline governs the
// response-wait phase so a fully-uploaded file is never killed mid-persist.
// Operator-tunable via Upload:ResponseTimeoutSeconds.
private const int DefaultResponseTimeoutSeconds = 600; // 10 minutes
private const int DefaultResponseTimeoutSeconds = 1200; // 20 minutes
private readonly IHttpClientFactory _httpClientFactory;
private readonly ILogger<CmsTrackService> _logger;
@@ -68,57 +68,15 @@ public class CmsTrackService : ICmsTrackService
ReleaseType releaseType,
int trackNumber,
ReleaseMedium medium = ReleaseMedium.Cut,
long? releaseId = null,
IProgress<long>? progress = null,
CancellationToken ct = default)
{
// Two-phase cancellation for the upload send:
//
// BODY-STREAMING phase (while bytes are on the wire):
// idleCts fires if no progress tick arrives within the idle window. Each
// ProgressStreamContent chunk resets CancelAfter(idle), so a slow-but-moving
// upload never trips it; a genuinely stalled socket does.
//
// RESPONSE-WAIT phase (after the last byte, while the server persists):
// The idle heartbeat goes silent once the body is fully sent. responseCts is
// armed at that moment with a larger budget so a fully-uploaded file is never
// killed mid-persist. idleCts is simultaneously disarmed (CancelAfter(Infinite))
// so it cannot misfire during the response-wait.
//
// sendCts links both so either deadline — plus the caller's ct — cancels the send.
using var idleCts = CancellationTokenSource.CreateLinkedTokenSource(ct);
idleCts.CancelAfter(_uploadIdleTimeout);
// Build the WAV part once; the two-phase send helper owns the cancellation plumbing.
using var phase = new UploadPhase(this, ct);
var wavContent = phase.WrapContent(wavStream, contentLength, contentType, progress);
// responseCts starts disarmed; the body-complete callback below arms it.
using var responseCts = CancellationTokenSource.CreateLinkedTokenSource(ct);
// Umbrella token passed to SendAsync — either phase token (or the caller) can cancel.
using var sendCts = CancellationTokenSource.CreateLinkedTokenSource(idleCts.Token, responseCts.Token);
// Rebuild the multipart container so the boundary is owned by HttpClient and the
// caller-supplied stream (already buffered by the SignalR upload) is the source.
using var multipart = new MultipartFormDataContent();
var wavContent = new ProgressStreamContent(
wavStream,
contentLength,
written =>
{
// One mechanism, three consumers: advance the UI meter, reset the idle heartbeat,
// and on body-complete transition to the response-wait budget.
progress?.Report(written);
if (written < contentLength)
{
// Body still in flight — keep the idle heartbeat alive.
idleCts.CancelAfter(_uploadIdleTimeout);
}
else
{
// Last byte on the wire. Disarm the idle timer and start the response budget.
idleCts.CancelAfter(Timeout.InfiniteTimeSpan);
responseCts.CancelAfter(_uploadResponseTimeout);
}
});
wavContent.Headers.ContentType = new MediaTypeHeaderValue(
string.IsNullOrWhiteSpace(contentType) ? "audio/wav" : contentType);
multipart.Add(wavContent, "audioFile", fileName);
multipart.Add(new StringContent(trackName), "trackName");
multipart.Add(new StringContent(artist), "artist");
@@ -134,37 +92,14 @@ public class CmsTrackService : ICmsTrackService
// The upload endpoint binds "medium" to the created release's ReleaseMedium (defaulting to Cut
// for an unrecognised value). Authoritative only when this upload creates the release.
multipart.Add(new StringContent(medium.ToString()), "medium");
// releaseId present → ATTACH (rows 2..N of a within-batch Cut); absent → CREATE (server rejects a
// pre-existing (title, artist) as a duplicate). Only sent when set so the form omits it on row 1.
if (releaseId is { } rid) multipart.Add(new StringContent(rid.ToString()), "releaseId");
// Use the dedicated upload client (InfiniteTimeSpan) so the two-phase CTS logic above is the
// sole timeout authority. Non-upload operations use the bounded "DeepDrft.Content.Cms" client.
var client = _httpClientFactory.CreateClient(UploadClientName);
using var request = new HttpRequestMessage(HttpMethod.Post, UploadPath) { Content = multipart };
HttpResponseMessage response;
try
var send = await phase.SendAsync(UploadPath, multipart, $"upload of {trackName}");
if (send.Response is not { } response)
{
response = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, sendCts.Token);
}
catch (OperationCanceledException) when (!ct.IsCancellationRequested)
{
// Either idle window (body-streaming stall) or response-wait budget (server persist too slow).
if (idleCts.IsCancellationRequested)
{
_logger.LogWarning("Upload of {TrackName} stalled — no progress for {IdleSeconds}s; aborting.",
trackName, _uploadIdleTimeout.TotalSeconds);
return ResultContainer<TrackDto>.CreateFailResult(
$"Upload stalled — no data transferred for {_uploadIdleTimeout.TotalSeconds:0}s. Please retry.");
}
// responseCts fired: body reached the server but persist timed out.
_logger.LogWarning("Upload of {TrackName} timed out waiting for server response after {ResponseSeconds}s.",
trackName, _uploadResponseTimeout.TotalSeconds);
return ResultContainer<TrackDto>.CreateFailResult(
$"Upload timed out waiting for the server to respond after {_uploadResponseTimeout.TotalSeconds:0}s. Please retry.");
}
catch (Exception ex)
{
_logger.LogError(ex, "Content API call failed for upload of {TrackName}", trackName);
return ResultContainer<TrackDto>.CreateFailResult("Content API is unreachable.");
return ResultContainer<TrackDto>.CreateFailResult(send.FailureMessage!);
}
using (response)
@@ -208,6 +143,172 @@ public class CmsTrackService : ICmsTrackService
}
}
public async Task<Result> ReplaceTrackAudioAsync(
long id,
Stream wavStream,
long contentLength,
string fileName,
string contentType,
IProgress<long>? progress = null,
CancellationToken ct = default)
{
// Same two-phase send plumbing as UploadTrackAsync — a WAV replace is an equally large body.
// The request carries only the audio part; the server resolves the track by route id and
// preserves its metadata, so no metadata fields ride along.
using var phase = new UploadPhase(this, ct);
var wavContent = phase.WrapContent(wavStream, contentLength, contentType, progress);
using var multipart = new MultipartFormDataContent();
multipart.Add(wavContent, "audioFile", fileName);
var send = await phase.SendAsync($"api/track/{id}/replace-audio", multipart, $"replace of track {id}");
if (send.Response is not { } response)
{
return Result.CreateFailResult(send.FailureMessage!);
}
using (response)
{
if (response.IsSuccessStatusCode)
{
return Result.CreatePassResult();
}
if (response.StatusCode == HttpStatusCode.NotFound)
{
return Result.CreateFailResult("Track not found.");
}
var body = await response.Content.ReadAsStringAsync(ct);
var statusCode = (int)response.StatusCode;
if (statusCode >= 500)
{
_logger.LogError("Content API returned {Status} for replace of track {TrackId}: {Body}", statusCode, id, body);
return Result.CreateFailResult("Replace failed on the content server. Please try again.");
}
// 4xx: body is user-friendly validation text from DeepDrftAPI — relay as-is.
_logger.LogWarning("Content API rejected replace of track {TrackId}: {Status} {Body}", id, statusCode, body);
return Result.CreateFailResult(
string.IsNullOrWhiteSpace(body) ? $"Replace rejected ({statusCode})." : body);
}
}
/// <summary>
/// Owns the two-phase cancellation for a large-body multipart send (the original upload and the
/// audio replace share it identically):
///
/// BODY-STREAMING phase (while bytes are on the wire): the idle CTS fires if no progress tick
/// arrives within the idle window. Each <see cref="ProgressStreamContent"/> chunk resets it, so a
/// slow-but-moving body never trips it; a genuinely stalled socket does.
///
/// RESPONSE-WAIT phase (after the last byte, while the server persists): the idle heartbeat goes
/// silent, so a separate, larger budget is armed at body-complete and the idle timer is disarmed,
/// guaranteeing a fully-sent body is never killed mid-persist.
///
/// The send CTS links both phase tokens plus the caller's token. Single-sourcing this here keeps
/// the idle/response-wait behaviour identical across every large-body call.
/// </summary>
private sealed class UploadPhase : IDisposable
{
private readonly CmsTrackService _owner;
private readonly CancellationToken _callerToken;
private readonly CancellationTokenSource _idleCts;
private readonly CancellationTokenSource _responseCts;
private readonly CancellationTokenSource _sendCts;
public UploadPhase(CmsTrackService owner, CancellationToken callerToken)
{
_owner = owner;
_callerToken = callerToken;
_idleCts = CancellationTokenSource.CreateLinkedTokenSource(callerToken);
_idleCts.CancelAfter(owner._uploadIdleTimeout);
// responseCts starts disarmed; the body-complete callback arms it.
_responseCts = CancellationTokenSource.CreateLinkedTokenSource(callerToken);
_sendCts = CancellationTokenSource.CreateLinkedTokenSource(_idleCts.Token, _responseCts.Token);
}
public ProgressStreamContent WrapContent(
Stream wavStream, long contentLength, string contentType, IProgress<long>? progress)
{
var content = new ProgressStreamContent(
wavStream,
contentLength,
written =>
{
// One mechanism, three consumers: advance the UI meter, reset the idle heartbeat,
// and on body-complete transition to the response-wait budget.
progress?.Report(written);
if (written < contentLength)
{
_idleCts.CancelAfter(_owner._uploadIdleTimeout);
}
else
{
// Last byte on the wire. Disarm the idle timer and start the response budget.
_idleCts.CancelAfter(Timeout.InfiniteTimeSpan);
_responseCts.CancelAfter(_owner._uploadResponseTimeout);
}
});
content.Headers.ContentType = new MediaTypeHeaderValue(
string.IsNullOrWhiteSpace(contentType) ? "audio/wav" : contentType);
return content;
}
public async Task<LargeBodySendResult> SendAsync(
string path, HttpContent content, string operationLabel)
{
// Dedicated upload client (InfiniteTimeSpan) so the two-phase CTS logic is the sole timeout
// authority. Non-upload operations use the bounded "DeepDrft.Content.Cms" client.
var client = _owner._httpClientFactory.CreateClient(UploadClientName);
using var request = new HttpRequestMessage(HttpMethod.Post, path) { Content = content };
try
{
var response = await client.SendAsync(
request, HttpCompletionOption.ResponseHeadersRead, _sendCts.Token);
return LargeBodySendResult.Ok(response);
}
catch (OperationCanceledException) when (!_callerToken.IsCancellationRequested)
{
if (_idleCts.IsCancellationRequested)
{
_owner._logger.LogWarning("{Operation} stalled — no progress for {IdleSeconds}s; aborting.",
operationLabel, _owner._uploadIdleTimeout.TotalSeconds);
return LargeBodySendResult.Fail(
$"{operationLabel} stalled — no data transferred for {_owner._uploadIdleTimeout.TotalSeconds:0}s. Please retry.");
}
_owner._logger.LogWarning("{Operation} timed out waiting for server response after {ResponseSeconds}s.",
operationLabel, _owner._uploadResponseTimeout.TotalSeconds);
return LargeBodySendResult.Fail(
$"{operationLabel} timed out waiting for the server to respond after {_owner._uploadResponseTimeout.TotalSeconds:0}s. Please retry.");
}
catch (Exception ex)
{
_owner._logger.LogError(ex, "Content API call failed for {Operation}", operationLabel);
return LargeBodySendResult.Fail("Content API is unreachable.");
}
}
public void Dispose()
{
_sendCts.Dispose();
_responseCts.Dispose();
_idleCts.Dispose();
}
}
// Outcome of a two-phase send: either a live response the caller must dispose, or a user-facing
// failure message. Exactly one is non-null.
private readonly struct LargeBodySendResult
{
public HttpResponseMessage? Response { get; private init; }
public string? FailureMessage { get; private init; }
public static LargeBodySendResult Ok(HttpResponseMessage response) => new() { Response = response };
public static LargeBodySendResult Fail(string message) => new() { FailureMessage = message };
}
public async Task<Result> DeleteTrackAsync(long id, CancellationToken ct = default)
{
var client = _httpClientFactory.CreateClient(ContentCmsClientName);
@@ -377,6 +478,53 @@ public class CmsTrackService : ICmsTrackService
}
}
public async Task<ResultContainer<ReleaseDto?>> GetExistingReleaseAsync(
string title, string artist, CancellationToken ct = default)
{
var client = _httpClientFactory.CreateClient(ContentCmsClientName);
var query = $"api/track/release/exists?title={Uri.EscapeDataString(title)}&artist={Uri.EscapeDataString(artist)}";
HttpResponseMessage response;
try
{
response = await client.GetAsync(query, ct);
}
catch (Exception ex)
{
_logger.LogError(ex, "Content API call failed for release existence check ({Title}, {Artist})", title, artist);
return ResultContainer<ReleaseDto?>.CreateFailResult("Content API is unreachable.");
}
using (response)
{
// 404 is the not-found (null) case, not a failure — no release matches this (title, artist).
if (response.StatusCode == HttpStatusCode.NotFound)
{
return ResultContainer<ReleaseDto?>.CreatePassResult(null);
}
if (!response.IsSuccessStatusCode)
{
_logger.LogError("Content API release existence check failed for ({Title}, {Artist}): {Status}",
title, artist, (int)response.StatusCode);
return ResultContainer<ReleaseDto?>.CreateFailResult("Failed to check for an existing release.");
}
ReleaseDto? release;
try
{
release = await response.Content.ReadFromJsonAsync<ReleaseDto>(ct);
}
catch (Exception ex)
{
_logger.LogError(ex, "Failed to deserialize ReleaseDto from release existence check");
return ResultContainer<ReleaseDto?>.CreateFailResult("Content API returned an unexpected response.");
}
return ResultContainer<ReleaseDto?>.CreatePassResult(release);
}
}
private static readonly HashSet<string> KnownImageMimeTypes = new(StringComparer.OrdinalIgnoreCase)
{
"image/jpeg", "image/png", "image/gif", "image/webp", "image/svg+xml", "image/bmp"
@@ -617,6 +765,89 @@ public class CmsTrackService : ICmsTrackService
}
}
public async Task<ResultContainer<OpusBackfillResult>> BackfillOpusAsync(CancellationToken ct = default)
{
var client = _httpClientFactory.CreateClient(ContentCmsClientName);
HttpResponseMessage response;
try
{
response = await client.PostAsync("api/track/opus/backfill", null, ct);
}
catch (Exception ex)
{
_logger.LogError(ex, "Content API call failed for Opus backfill");
return ResultContainer<OpusBackfillResult>.CreateFailResult("Content API is unreachable.");
}
using (response)
{
if (!response.IsSuccessStatusCode)
{
var body = await response.Content.ReadAsStringAsync(ct);
_logger.LogError("Content API Opus backfill failed: {Status} {Body}", (int)response.StatusCode, body);
return ResultContainer<OpusBackfillResult>.CreateFailResult("Failed to start the Opus backfill.");
}
OpusBackfillResult payload;
try
{
payload = await response.Content.ReadFromJsonAsync<OpusBackfillResult>(ct);
}
catch (Exception ex)
{
_logger.LogError(ex, "Failed to deserialize Opus backfill response from Content API");
return ResultContainer<OpusBackfillResult>.CreateFailResult("Content API returned an unexpected response.");
}
return ResultContainer<OpusBackfillResult>.CreatePassResult(payload);
}
}
public async Task<ResultContainer<OpusStatusDto[]>> GetOpusStatusAsync(CancellationToken ct = default)
{
var client = _httpClientFactory.CreateClient(ContentCmsClientName);
HttpResponseMessage response;
try
{
response = await client.GetAsync("api/track/opus-status", ct);
}
catch (Exception ex)
{
_logger.LogError(ex, "Content API call failed for Opus status");
return ResultContainer<OpusStatusDto[]>.CreateFailResult("Content API is unreachable.");
}
using (response)
{
if (!response.IsSuccessStatusCode)
{
_logger.LogError("Content API Opus status failed: {Status}", (int)response.StatusCode);
return ResultContainer<OpusStatusDto[]>.CreateFailResult("Failed to load Opus status.");
}
OpusStatusDto[]? status;
try
{
status = await response.Content.ReadFromJsonAsync<OpusStatusDto[]>(ct);
}
catch (Exception ex)
{
_logger.LogError(ex, "Failed to deserialize Opus status from Content API response");
return ResultContainer<OpusStatusDto[]>.CreateFailResult("Content API returned an unexpected response.");
}
if (status is null)
{
_logger.LogError("Content API returned a null Opus status list");
return ResultContainer<OpusStatusDto[]>.CreateFailResult("Content API returned an empty response.");
}
return ResultContainer<OpusStatusDto[]>.CreatePassResult(status);
}
}
public async Task<ResultContainer<List<ReleaseDto>>> GetReleasesAsync(CancellationToken ct = default)
{
var client = _httpClientFactory.CreateClient(ContentCmsClientName);
@@ -25,6 +25,10 @@ public interface ICmsTrackService
/// sets Content-Length and is the denominator for <paramref name="progress"/>, which reports cumulative
/// bytes pushed to the wire. Each progress tick also resets the idle/heartbeat upload timeout, so a
/// stalled connection aborts without a fixed total-duration cap.
/// <paramref name="releaseId"/> distinguishes the two rows of a within-batch multi-track Cut: null on
/// the first row (CREATE — the server rejects a pre-existing (title, artist) as a duplicate) and the
/// id returned by that first row on rows 2..N (ATTACH — the server skips the duplicate check and adds
/// the track to the release the batch just created).
/// </summary>
Task<ResultContainer<TrackDto>> UploadTrackAsync(
Stream wavStream,
@@ -42,15 +46,44 @@ public interface ICmsTrackService
ReleaseType releaseType,
int trackNumber,
ReleaseMedium medium = ReleaseMedium.Cut,
long? releaseId = null,
IProgress<long>? progress = null,
CancellationToken ct = default);
/// <summary>
/// Upload-form pre-flight: returns the existing release whose exact (title, artist) matches, or null
/// when none exists. Backs the duplicate block the form runs BEFORE transferring bytes, so the admin
/// is not surprised at the end of a long upload. A 404 from the API is the not-found (null) case, not
/// a failure. The match semantics are the API's <c>GetReleaseByTitleAndArtist</c> — the same read the
/// server backstop uses — so the pre-flight and the backstop agree.
/// </summary>
Task<ResultContainer<ReleaseDto?>> GetExistingReleaseAsync(
string title, string artist, CancellationToken ct = default);
/// <summary>
/// Delete a track via the Content API, which removes the SQL row then the vault entry.
/// Maps a 404 to a "Track not found." failure.
/// </summary>
Task<Result> DeleteTrackAsync(long id, CancellationToken ct = default);
/// <summary>
/// Replace an existing track's audio via <c>POST api/track/{id}/replace-audio</c>. Swaps only the
/// vault bytes and regenerates the track's waveform data server-side; the track id, vault key,
/// release membership, position, and metadata are preserved. Uses the dedicated upload client and
/// the same two-phase (idle / response-wait) cancellation as <see cref="UploadTrackAsync"/>, since
/// a WAV replace is a large-body upload. <paramref name="contentLength"/> sets Content-Length and is
/// the denominator for <paramref name="progress"/>; each progress tick resets the idle heartbeat.
/// Maps a 404 to a "Track not found." failure.
/// </summary>
Task<Result> ReplaceTrackAudioAsync(
long id,
Stream wavStream,
long contentLength,
string fileName,
string contentType,
IProgress<long>? progress = null,
CancellationToken ct = default);
/// <summary>
/// Soft-delete a release record via DELETE api/track/release/{id}. Use when a release
/// has no live tracks and needs to be removed from the albums browser.
@@ -119,6 +152,23 @@ public interface ICmsTrackService
/// </summary>
Task<Result> GenerateHighResWaveformAsync(string entryKey, CancellationToken ct = default);
/// <summary>
/// Trigger the catalogue-wide Backfill-Opus pass via <c>POST api/track/opus/backfill</c> (Phase 18.5).
/// The API enqueues a background Opus derive for every track lacking a complete Opus artifact and returns
/// the (enqueued, skipped) counts. Enqueue-only — the transcodes run server-side on a serial background
/// worker, so this call returns as soon as the work is scheduled, not when transcoding finishes. The
/// <c>Enqueued</c> count is how many derives were scheduled; <c>Skipped</c> is how many already had Opus.
/// </summary>
Task<ResultContainer<OpusBackfillResult>> BackfillOpusAsync(CancellationToken ct = default);
/// <summary>
/// Fetch per-track Opus derive status from <c>GET api/track/opus-status</c> (Phase 18.6) for the CMS
/// Post-Processing surfaces. Unpaged — the admin catalogue is small. Each row's <c>HasOpus</c> is true only
/// when the track carries a complete Opus artifact (audio + sidecar). Drives the Backfill-Opus "missing N"
/// badge and the post-upload Post-Processing poll. Idempotent read — safe to poll on an interval.
/// </summary>
Task<ResultContainer<OpusStatusDto[]>> GetOpusStatusAsync(CancellationToken ct = default);
/// <summary>Returns all releases with track counts from GET api/track/albums.</summary>
Task<ResultContainer<List<ReleaseDto>>> GetReleasesAsync(CancellationToken ct = default);
@@ -127,3 +177,11 @@ public interface ICmsTrackService
/// </summary>
Task<ResultContainer<int>> GetTrackCountAsync(CancellationToken ct = default);
}
/// <summary>
/// Outcome of a Backfill-Opus pass (Phase 18.5): how many tracks had a background derive scheduled
/// (<paramref name="Enqueued"/>) and how many were skipped because they already carry a complete Opus
/// artifact (<paramref name="Skipped"/>). Both are counts of tracks, not finished transcodes — the work
/// runs asynchronously on the API's background worker after this returns.
/// </summary>
public readonly record struct OpusBackfillResult(int Enqueued, int Skipped);
+5 -2
View File
@@ -1,12 +1,15 @@
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning"
"Default": "Warning"
}
},
"AllowedHosts": "*",
"ForwardedHeaders": {
"DisableHttpsRedirection": false
},
"Upload": {
"IdleTimeoutSeconds": 90,
"ResponseTimeoutSeconds": 1200
}
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 39 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 39 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 56 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.9 KiB

+2
View File
@@ -0,0 +1,2 @@
User-agent: *
Disallow: /
+16 -1
View File
@@ -15,7 +15,8 @@ DeepDrftModels/
├── Entities/
│ └── TrackEntity.cs # Database entity for tracks
├── DTOs/
── TrackDto.cs # DTO mirror of TrackEntity
── TrackDto.cs # DTO mirror of TrackEntity
│ └── HomeStatsDto.cs # Aggregate figures for the public home hero stat row
├── Models/
│ ├── PagingParameters.cs # Pagination configuration (base + generic)
│ └── PagedResult.cs # Paginated result wrapper
@@ -35,6 +36,7 @@ public string? Album { get; set; } // Optional album (max 200)
public string? Genre { get; set; } // Optional genre (max 100)
public DateOnly? ReleaseDate { get; set; } // Optional release date
public string? ImagePath { get; set; } // Optional image URL (max 500)
public double? DurationSeconds { get; set; } // Audio runtime in seconds (nullable; populated at upload, backfilled for older rows)
```
**No `MediaPath` field exists.** That was a legacy name. The field is `EntryKey`.
@@ -47,6 +49,19 @@ Convention: required reference fields use `required` modifier; optional referenc
Mirrors `TrackEntity` structure (identical fields, same nullability). Used where DTO/entity separation is needed for serialisation. In practice, both flow over the wire today, but the separation is available if APIs need to diverge (e.g., hide `Id` in responses).
## HomeStatsDto
Aggregate figures behind the public home hero stat row (`NowPlayingStats`). A single round-trip returns everything the three cards need. Fields:
- `CutTrackCount` (int): total non-deleted tracks on Cut-medium releases.
- `CutReleaseTypeCounts` (`List<CutReleaseTypeCount>`): per-`ReleaseType` Cut release counts; zero-count types are absent (suppressed server-side).
- `MixReleaseCount` (int): total non-deleted Mix-medium releases.
- `MixRuntimeSeconds` (double): sum of `DurationSeconds` across all non-deleted tracks on Mix releases (null durations count as 0). Rendered as `hh:mm` by `RuntimeFormat` on the client.
- `TotalPlays` (long): site-wide total plays — sum of every `play_counter` row's `PartialCount + SampledCount + CompleteCount`, all-time (Phase 16 §5). Zero until the play-telemetry migration is applied; that is expected, not an error. The Plays card's primary odometer figure.
- `UniqueListeners` (int): site-wide distinct anonymous listeners — distinct non-null `anon_id` values across all play events, all-time (Phase 16 §3 / D7). Zero until the migration is applied. The Plays card's secondary line ("N listeners").
`CutReleaseTypeCount` is a nested type (`ReleaseType`, `Count` int) defined in the same file.
## Pagination system
### PagingParameters (base)
+51
View File
@@ -0,0 +1,51 @@
using DeepDrftModels.Enums;
namespace DeepDrftModels.DTOs;
/// <summary>
/// Aggregate figures behind the public home hero stat row (NowPlayingStats). A single read returns
/// everything the three cards need so the client makes one round-trip. The track-domain counts exclude
/// soft-deleted rows; the play-domain figures (Phase 16) come from the event domain.
/// </summary>
public class HomeStatsDto
{
/// <summary>Total non-deleted tracks whose release is the Cut medium. The Studio Cuts card's primary figure.</summary>
public int CutTrackCount { get; set; }
/// <summary>
/// Per-ReleaseType counts of non-deleted Cut releases. Only types with at least one release are
/// present — a zero-count type is absent from the list (the card suppresses it). The Studio Cuts
/// card's secondary breakdown.
/// </summary>
public List<CutReleaseTypeCount> CutReleaseTypeCounts { get; set; } = [];
/// <summary>Total non-deleted releases of the Mix medium. The Mixes card's primary figure ("N Sets").</summary>
public int MixReleaseCount { get; set; }
/// <summary>
/// Sum of DurationSeconds across all non-deleted tracks on Mix releases. Tracks with a null
/// duration (not yet backfilled) contribute 0. The Mixes card's secondary figure, rendered hh:mm.
/// </summary>
public double MixRuntimeSeconds { get; set; }
/// <summary>
/// Site-wide total plays across all tracks — the sum of every play_counter's bucket columns
/// (partial + sampled + complete), all-time (Phase 16 §5). The Plays card's primary odometer figure.
/// Reads zero until the play-telemetry migration is applied; that is expected, not an error.
/// </summary>
public long TotalPlays { get; set; }
/// <summary>
/// Site-wide distinct anonymous listeners — distinct non-null anon_id across all play events,
/// all-time (Phase 16 §3 / D7). The Plays card's secondary line ("N listeners"). Over-counts by
/// design (one token per browser-install, honestly labelled "listeners").
/// </summary>
public int UniqueListeners { get; set; }
}
/// <summary>One row of the Cut release-type breakdown: a ReleaseType and how many Cut releases have it.</summary>
public class CutReleaseTypeCount
{
public ReleaseType ReleaseType { get; set; }
public int Count { get; set; }
}
+19
View File
@@ -0,0 +1,19 @@
namespace DeepDrftModels.DTOs;
/// <summary>
/// Per-track Opus derive status for the CMS Post-Processing surfaces (Phase 18.6). Mirrors
/// <see cref="WaveformStatusDto"/>: one row per track, flagging whether the track already carries a
/// <strong>complete</strong> Opus artifact. "Complete" means BOTH the Opus audio bytes AND the seek/setup
/// sidecar are present in the <c>track-opus</c> vault — a half-derived track (audio without sidecar) is
/// unseekable and counts as missing, so the Backfill-Opus pass re-derives it. <see cref="EntryKey"/> is the
/// vault key the per-track enqueue trigger and the polling Post-Processing affordance key on.
/// </summary>
public class OpusStatusDto
{
public long TrackId { get; set; }
public string EntryKey { get; set; } = string.Empty;
public string TrackName { get; set; } = string.Empty;
/// <summary>True only when both the Opus audio and the seek/setup sidecar are stored (a complete derive).</summary>
public bool HasOpus { get; set; }
}
+18
View File
@@ -0,0 +1,18 @@
using DeepDrftModels.Enums;
namespace DeepDrftModels.DTOs;
/// <summary>
/// Wire payload for <c>POST api/event/play</c> (Phase 16 §2.2 / §4.3). The client sends only what it
/// cheaply knows — the track key and the client-computed completion bucket; the server resolves the
/// release. No duration or raw position is transmitted (a privacy plus — only a coarse bucket leaves
/// the browser). <see cref="AnonId"/> is reserved for wave 16.3 and stays null in wave 16.1.
/// </summary>
public class PlayEventDto
{
public string? TrackEntryKey { get; set; }
public PlayBucket Bucket { get; set; }
public string? AnonId { get; set; }
}
+19
View File
@@ -0,0 +1,19 @@
using DeepDrftModels.Enums;
namespace DeepDrftModels.DTOs;
/// <summary>
/// Wire payload for <c>POST api/event/share</c> (Phase 16 §2.2 / §4.3). The popover knows the target
/// and channel at the point of the action, so the payload is self-describing — no server-side resolution.
/// <see cref="AnonId"/> is reserved for wave 16.3 and stays null in wave 16.1.
/// </summary>
public class ShareEventDto
{
public ShareTargetType TargetType { get; set; }
public string? TargetKey { get; set; }
public ShareChannel Channel { get; set; }
public string? AnonId { get; set; }
}
+1
View File
@@ -16,6 +16,7 @@ public class TrackDto : BaseModel
public string TrackName { get; set; } = string.Empty;
public string? OriginalFileName { get; set; }
public int TrackNumber { get; set; } = 1;
public double? DurationSeconds { get; set; }
public long? ReleaseId { get; set; }
public ReleaseDto? Release { get; set; }
}
+2 -2
View File
@@ -7,8 +7,8 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Cerebellum.NetBlocks" Version="10.3.30" />
<PackageReference Include="Cerebellum.BlazorBlocks.Models" Version="10.3.30" />
<PackageReference Include="Cerebellum.NetBlocks" Version="10.3.32" />
<PackageReference Include="Cerebellum.BlazorBlocks.Models" Version="10.3.35" />
</ItemGroup>
</Project>
+28
View File
@@ -0,0 +1,28 @@
namespace DeepDrftModels.Entities;
/// <summary>
/// Incremental rollup of play counts per track (Phase 16 §4.1 / D6). One row per track, bumped inside
/// the same transaction that appends the <see cref="PlayEvent"/> — no background aggregation job. The
/// home card and per-target reads sum these instead of <c>COUNT(*)</c>-ing the event log on every
/// landing. Release totals are <em>derived</em> (D4) by summing the counters of the release's tracks,
/// so there is no separate release-counter row — this keeps the rollup normalized at one row per track.
/// </summary>
public class PlayCounter
{
public long Id { get; set; }
/// <summary>The track these counts belong to (SQL id). Unique — one counter row per track.</summary>
public long TrackId { get; set; }
/// <summary>Count of plays that ended in the <c>Partial</c> bucket (&lt; 30%).</summary>
public long PartialCount { get; set; }
/// <summary>Count of plays that ended in the <c>Sampled</c> bucket (30%80%).</summary>
public long SampledCount { get; set; }
/// <summary>Count of plays that ended in the <c>Complete</c> bucket (&gt; 80%).</summary>
public long CompleteCount { get; set; }
/// <summary>Total plays for the track — the sum of the three bucket counts (headline figure).</summary>
public long TotalPlays => PartialCount + SampledCount + CompleteCount;
}
+36
View File
@@ -0,0 +1,36 @@
using DeepDrftModels.Enums;
namespace DeepDrftModels.Entities;
/// <summary>
/// Append-only log row for one recorded play (Phase 16 §4.2). Written once at session close, after the
/// engagement floor is crossed; never updated or deleted. Deliberately NOT a <c>BaseEntity</c>: events
/// have no soft-delete lifecycle, no <c>UpdatedAt</c> — they are immutable facts. The release link is
/// resolved server-side from the track key at write time (§2.3 / D4) and stored here so release-total
/// plays are a cheap sum over this column.
/// </summary>
public class PlayEvent
{
public long Id { get; set; }
/// <summary>The played track's vault entry key (the only target the client sends).</summary>
public required string TrackEntryKey { get; set; }
/// <summary>
/// The owning release's SQL id, resolved from <see cref="TrackEntryKey"/> at write time. Null when
/// the track is loose (no release) or the key did not resolve to a live track at write time.
/// </summary>
public long? ReleaseId { get; set; }
/// <summary>The completion bucket computed client-side from the high-water fraction (§1a / D1).</summary>
public PlayBucket Bucket { get; set; }
/// <summary>
/// Anonymous listener token (Phase 16 §3, wave 16.3). Reserved nullable; nothing writes it in wave
/// 16.1 — the client sends none and the column stays NULL. Wave 16.3 lights it up for the
/// distinct-listener count.
/// </summary>
public string? AnonId { get; set; }
public DateTime CreatedAt { get; set; }
}
+34
View File
@@ -0,0 +1,34 @@
using DeepDrftModels.Enums;
namespace DeepDrftModels.Entities;
/// <summary>
/// Append-only log row for one recorded share (Phase 16 §4.2). Written once per share action that
/// survives the per-(target,channel) client debounce; never updated or deleted. Like <see cref="PlayEvent"/>
/// it is deliberately NOT a <c>BaseEntity</c> — an immutable fact with no soft-delete lifecycle. Shares
/// carry their target directly (the popover knows track vs. release), so no server-side resolution step.
/// </summary>
public class ShareEvent
{
public long Id { get; set; }
/// <summary>Whether the share targets a track or a release.</summary>
public ShareTargetType TargetType { get; set; }
/// <summary>
/// The shared target's key: a track's vault <c>EntryKey</c> or a release's public <c>EntryKey</c>,
/// selected by <see cref="TargetType"/>. Stored as the opaque key, not resolved to a SQL id — the
/// share metric is a simple per-target tally and needs no join in wave 16.1.
/// </summary>
public required string TargetKey { get; set; }
/// <summary>The channel the share was performed through (link vs. embed).</summary>
public ShareChannel Channel { get; set; }
/// <summary>
/// Anonymous listener token (Phase 16 §3, wave 16.3). Reserved nullable; unused in wave 16.1.
/// </summary>
public string? AnonId { get; set; }
public DateTime CreatedAt { get; set; }
}
+4
View File
@@ -15,6 +15,10 @@ public class TrackEntity : BaseEntity, IEntity
public required string TrackName { get; set; }
public string? OriginalFileName { get; set; }
public int TrackNumber { get; set; } = 1;
// Audio runtime in seconds, extracted by the processor at upload (AudioBinary.Duration) and
// persisted here so aggregate queries (e.g. total mix runtime) read it from SQL rather than the
// vault. Nullable: rows that predate this column are valid until the one-time backfill populates them.
public double? DurationSeconds { get; set; }
public long? ReleaseId { get; set; }
public ReleaseEntity? Release { get; set; }
}

Some files were not shown because too many files have changed in this diff Show More