From ca44979b08c53ac0a0d26d4be23a8b56c2ee3577 Mon Sep 17 00:00:00 2001 From: daniel-c-harvey Date: Fri, 26 Jun 2026 15:32:18 -0400 Subject: [PATCH] docs: record Opus/derived read-path streaming and index-only opus-status --- CLAUDE.md | 2 +- DeepDrftAPI/CLAUDE.md | 17 +++++++++++++---- DeepDrftContent/CLAUDE.md | 4 ++-- 3 files changed, 16 insertions(+), 7 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 6bb9471..0a5030e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -42,7 +42,7 @@ Server-side (SSR): Both clients point directly at DeepDrftAPI (server-to-server, - Root contains typed **MediaVaults** (Media, Image, Audio) - Each vault has a JSON `index` file listing entries + per-entry metadata - Entries are user-supplied strings sanitized to `[a-zA-Z0-9-]` + file extension - - Binary hierarchy (**read/load path**): `FileBinary` → `MediaBinary` (+ Extension/MIME) → `AudioBinary` (+ Duration/Bitrate) | `ImageBinary` (+ AspectRatio). The **write/store path** is streaming: audio processors return a `ProcessedAudio` plan (metadata + streamed `WriteToAsync` callback); `RegisterResourceStreamingAsync` / `MediaVault.AddEntryStreamingAsync` write bytes to a temp file then `File.Move` atomic-rename into place — the full `AudioBinary` buffer is never materialized on this path. + - Binary hierarchy (model shapes): `FileBinary` → `MediaBinary` (+ Extension/MIME) → `AudioBinary` (+ Duration/Bitrate) | `ImageBinary` (+ AspectRatio). **Non-delivery read path** (`LoadResourceAsync`) returns a full-buffer `AudioBinary` — still used for non-delivery operations (e.g., duration backfill). **Audio delivery path** streams via `GetEntryStreamAsync` (Opus artifact, `track-opus` vault) or `OpenAudioMediaStreamAsync` (lossless source, `tracks` vault) — a seekable, disk-backed `Stream` per request, never a whole-file `byte[]` (read-side OOM fix, parallel to the store-side). The **write/store path** is streaming: audio processors return a `ProcessedAudio` plan (metadata + streamed `WriteToAsync` callback); `RegisterResourceStreamingAsync` / `MediaVault.AddEntryStreamingAsync` write bytes to a temp file then `File.Move` atomic-rename into place — the full `AudioBinary` buffer is never materialized on this path. - **Error-handling philosophy**: public operations swallow exceptions and return `null`/`false` — callers must check return values, not catch. ## Key Architectural Decisions diff --git a/DeepDrftAPI/CLAUDE.md b/DeepDrftAPI/CLAUDE.md index 64e84f2..f029065 100644 --- a/DeepDrftAPI/CLAUDE.md +++ b/DeepDrftAPI/CLAUDE.md @@ -32,12 +32,12 @@ Dual-database authority for tracks (SQL metadata + FileDatabase binary), release ### GET api/track/{trackId} (unauthenticated) -Returns the WAV bytes from the `tracks` vault with HTTP Range support. +Streams the track's audio bytes from disk with HTTP Range support. An optional `?format=opus` query parameter selects between the derived Opus artifact and the lossless source. - **Route parameter `trackId`** (string): the entry id inside the `tracks` vault (i.e. `TrackEntity.EntryKey`). -- **Range header** (optional): HTTP Range header for byte-range requests (e.g., `Range: bytes=1000-`). Server responds with `206 Partial Content` and streams from the requested offset. -- Streams the file directly from disk with `enableRangeProcessing: true`, supporting both full-file and partial-range requests without synthesizing WAV headers or buffering. -- Returns 200 for full-file requests, 206 for Range requests, 404 if track not found, 500 if vault operations fail (with error swallowing — the vault returns `null`). +- **Query parameter `format`** (optional): `opus` requests the derived Ogg Opus artifact from the `track-opus` vault when present, falling back to lossless when it is not (C2 — never 404 if any audio exists); omitted or any other value delivers the lossless source in its stored format (WAV/MP3/FLAC). Both arms stream from a seekable disk `FileStream` via `File(stream, contentType, enableRangeProcessing: true)` — no whole-file `byte[]`; `Content-Type` reflects what was actually served (e.g., `audio/ogg` on an Opus hit, the source's real MIME on a lossless request or C2 fallback). +- **Range header** (optional): HTTP Range header for byte-range requests (e.g., `Range: bytes=1000-`). Server responds with `206 Partial Content` and streams from the requested offset. Honoured on both arms (seekable `FileStream` in both cases). +- Returns 200 for full-file requests, 206 for Range requests, 404 if no audio artifact exists for the track, 500 if vault operations fail. ### GET api/track/albums (unauthenticated) @@ -110,6 +110,15 @@ Admin backfill view: returns every track with flags indicating whether each wave - **Response**: `List` with `TrackId`, `EntryKey`, `TrackName`, `HasProfile` (bool — 512-bucket player-bar seeker profile in `waveform-profiles` vault), and `HasHighRes` (bool — duration-derived high-res visualizer datum in `track-waveforms` vault). - Returns 200 on success. Returns 500 on query error. +### GET api/track/opus-status ([ApiKeyAuthorize]) + +Admin Post-Processing view: returns every track with a flag indicating whether it has a complete Opus artifact (both audio AND seek/setup sidecar present in the `track-opus` vault). Used by the CMS to show the Backfill-Opus badge and to poll per-track Post-Processing status after an upload. Mirrors the shape and auth posture of `GET api/track/waveform-status`. + +- **Header `ApiKey`**: required. Validated by `ApiKeyAuthenticationMiddleware`. +- **Response**: `List` with `TrackId`, `EntryKey`, `TrackName`, and `HasOpus` (bool — true only when both the Opus audio entry and the seek/setup sidecar entry are present in `track-opus`; a half-derived track counts as incomplete). +- `HasOpus` is resolved via `TrackFormatResolver.HasOpusAsync` — an **index-only** existence check (`MediaVault.HasIndexEntry` for both entries; no file-body load). The endpoint loops over the whole catalogue, so a body load per track would stream the full library sequentially; the index lookup costs zero disk reads per track. +- Returns 200 on success. Returns 500 on query error. + ### POST api/track/duration/backfill ([ApiKeyAuthorize]) Admin backfill: for every track whose `DurationSeconds` SQL column is still null, reads the `AudioBinary.Duration` from the vault and writes it to SQL. Idempotent — a re-run only touches still-null rows; rows that already have a value are skipped. diff --git a/DeepDrftContent/CLAUDE.md b/DeepDrftContent/CLAUDE.md index 8230aab..7f2a1d2 100644 --- a/DeepDrftContent/CLAUDE.md +++ b/DeepDrftContent/CLAUDE.md @@ -141,7 +141,7 @@ Swaps the vault bytes for an existing track in place, under the same `entryKey`. ### GetAudioBinaryAsync(entryKey) -Reads audio from the `tracks` vault via `FileDatabase.LoadResourceAsync("tracks", entryKey)`. Returns a full-buffer `AudioBinary` or `null` if not found. This is the **read/playback path** — unchanged by the streaming store fix. +Reads audio from the `tracks` vault via `FileDatabase.LoadResourceAsync("tracks", entryKey)`. Returns a full-buffer `AudioBinary` or `null` if not found. This is a full-buffer convenience read — **not** on the audio-delivery hot path. The delivery path now uses `TrackFormatResolver` (Opus: `MediaVault.GetEntryStreamAsync`; lossless: `OpenAudioMediaStreamAsync`). `GetAudioBinaryAsync` is retained as a read-back oracle used by tests (`AudioStoreStreamingTests`, `TrackReplaceAudioTests`, `TrackFormatDeliveryTests`). ### OpenAudioStreamAsync(entryKey) @@ -161,7 +161,7 @@ Safety call to ensure the `tracks` vault exists (creates if missing). Called on ## Vault constants -`VaultConstants.Tracks = "tracks"`, `VaultConstants.Images = "images"`, and `VaultConstants.TrackWaveforms = "track-waveforms"` — the vault names in production use. `TrackWaveforms` holds the per-track high-res waveform datum keyed by `TrackEntity.EntryKey` (Phase 12; renamed from the former `mix-waveforms`, which was Mix-only). New vault names go here when adding new vault types. +`VaultConstants.Tracks = "tracks"`, `VaultConstants.Images = "images"`, `VaultConstants.TrackWaveforms = "track-waveforms"`, and `VaultConstants.TrackOpus = "track-opus"` — the vault names in production use. `TrackWaveforms` holds the per-track high-res waveform datum keyed by `TrackEntity.EntryKey` (Phase 12; renamed from the former `mix-waveforms`, which was Mix-only). `TrackOpus` holds two entries per track: the derived Opus audio (keyed by `entryKey`, extension `.opus`) and the combined setup-header + seek-index sidecar (keyed by `{entryKey}-sidecar`, extension `.opusidx`). New vault names go here when adding new vault types. ## Service registration