docs: record Opus/derived read-path streaming and index-only opus-status
Deploy DeepDrftAPI / Build, Publish & Bundle (push) Successful in 2m25s
Deploy DeepDrftManager / Build & Publish (push) Successful in 1m28s
Deploy DeepDrftPublic / Build & Publish (push) Successful in 4m2s
Package install tarball / package (push) Successful in 6s
Deploy DeepDrftAPI / Deploy (push) Successful in 1m33s
Deploy DeepDrftManager / Deploy (push) Successful in 1m59s
Deploy DeepDrftPublic / Deploy (push) Successful in 1m22s

This commit is contained in:
daniel-c-harvey
2026-06-26 15:32:18 -04:00
parent bf5b314aed
commit ca44979b08
3 changed files with 16 additions and 7 deletions
+1 -1
View File
@@ -42,7 +42,7 @@ Server-side (SSR): Both clients point directly at DeepDrftAPI (server-to-server,
- Root contains typed **MediaVaults** (Media, Image, Audio) - Root contains typed **MediaVaults** (Media, Image, Audio)
- Each vault has a JSON `index` file listing entries + per-entry metadata - Each vault has a JSON `index` file listing entries + per-entry metadata
- Entries are user-supplied strings sanitized to `[a-zA-Z0-9-]` + file extension - Entries are user-supplied strings sanitized to `[a-zA-Z0-9-]` + file extension
- Binary hierarchy (**read/load path**): `FileBinary``MediaBinary` (+ Extension/MIME) → `AudioBinary` (+ Duration/Bitrate) | `ImageBinary` (+ AspectRatio). The **write/store path** is streaming: audio processors return a `ProcessedAudio` plan (metadata + streamed `WriteToAsync` callback); `RegisterResourceStreamingAsync` / `MediaVault.AddEntryStreamingAsync` write bytes to a temp file then `File.Move` atomic-rename into place — the full `AudioBinary` buffer is never materialized on this path. - Binary hierarchy (model shapes): `FileBinary``MediaBinary` (+ Extension/MIME) → `AudioBinary` (+ Duration/Bitrate) | `ImageBinary` (+ AspectRatio). **Non-delivery read path** (`LoadResourceAsync<AudioBinary>`) returns a full-buffer `AudioBinary` — still used for non-delivery operations (e.g., duration backfill). **Audio delivery path** streams via `GetEntryStreamAsync` (Opus artifact, `track-opus` vault) or `OpenAudioMediaStreamAsync` (lossless source, `tracks` vault) — a seekable, disk-backed `Stream` per request, never a whole-file `byte[]` (read-side OOM fix, parallel to the store-side). The **write/store path** is streaming: audio processors return a `ProcessedAudio` plan (metadata + streamed `WriteToAsync` callback); `RegisterResourceStreamingAsync` / `MediaVault.AddEntryStreamingAsync` write bytes to a temp file then `File.Move` atomic-rename into place — the full `AudioBinary` buffer is never materialized on this path.
- **Error-handling philosophy**: public operations swallow exceptions and return `null`/`false` — callers must check return values, not catch. - **Error-handling philosophy**: public operations swallow exceptions and return `null`/`false` — callers must check return values, not catch.
## Key Architectural Decisions ## Key Architectural Decisions
+13 -4
View File
@@ -32,12 +32,12 @@ Dual-database authority for tracks (SQL metadata + FileDatabase binary), release
### GET api/track/{trackId} (unauthenticated) ### GET api/track/{trackId} (unauthenticated)
Returns the WAV bytes from the `tracks` vault with HTTP Range support. Streams the track's audio bytes from disk with HTTP Range support. An optional `?format=opus` query parameter selects between the derived Opus artifact and the lossless source.
- **Route parameter `trackId`** (string): the entry id inside the `tracks` vault (i.e. `TrackEntity.EntryKey`). - **Route parameter `trackId`** (string): the entry id inside the `tracks` vault (i.e. `TrackEntity.EntryKey`).
- **Range header** (optional): HTTP Range header for byte-range requests (e.g., `Range: bytes=1000-`). Server responds with `206 Partial Content` and streams from the requested offset. - **Query parameter `format`** (optional): `opus` requests the derived Ogg Opus artifact from the `track-opus` vault when present, falling back to lossless when it is not (C2 — never 404 if any audio exists); omitted or any other value delivers the lossless source in its stored format (WAV/MP3/FLAC). Both arms stream from a seekable disk `FileStream` via `File(stream, contentType, enableRangeProcessing: true)` — no whole-file `byte[]`; `Content-Type` reflects what was actually served (e.g., `audio/ogg` on an Opus hit, the source's real MIME on a lossless request or C2 fallback).
- Streams the file directly from disk with `enableRangeProcessing: true`, supporting both full-file and partial-range requests without synthesizing WAV headers or buffering. - **Range header** (optional): HTTP Range header for byte-range requests (e.g., `Range: bytes=1000-`). Server responds with `206 Partial Content` and streams from the requested offset. Honoured on both arms (seekable `FileStream` in both cases).
- Returns 200 for full-file requests, 206 for Range requests, 404 if track not found, 500 if vault operations fail (with error swallowing — the vault returns `null`). - Returns 200 for full-file requests, 206 for Range requests, 404 if no audio artifact exists for the track, 500 if vault operations fail.
### GET api/track/albums (unauthenticated) ### GET api/track/albums (unauthenticated)
@@ -110,6 +110,15 @@ Admin backfill view: returns every track with flags indicating whether each wave
- **Response**: `List<WaveformStatusDto>` with `TrackId`, `EntryKey`, `TrackName`, `HasProfile` (bool — 512-bucket player-bar seeker profile in `waveform-profiles` vault), and `HasHighRes` (bool — duration-derived high-res visualizer datum in `track-waveforms` vault). - **Response**: `List<WaveformStatusDto>` with `TrackId`, `EntryKey`, `TrackName`, `HasProfile` (bool — 512-bucket player-bar seeker profile in `waveform-profiles` vault), and `HasHighRes` (bool — duration-derived high-res visualizer datum in `track-waveforms` vault).
- Returns 200 on success. Returns 500 on query error. - Returns 200 on success. Returns 500 on query error.
### GET api/track/opus-status ([ApiKeyAuthorize])
Admin Post-Processing view: returns every track with a flag indicating whether it has a complete Opus artifact (both audio AND seek/setup sidecar present in the `track-opus` vault). Used by the CMS to show the Backfill-Opus badge and to poll per-track Post-Processing status after an upload. Mirrors the shape and auth posture of `GET api/track/waveform-status`.
- **Header `ApiKey`**: required. Validated by `ApiKeyAuthenticationMiddleware`.
- **Response**: `List<OpusStatusDto>` with `TrackId`, `EntryKey`, `TrackName`, and `HasOpus` (bool — true only when both the Opus audio entry and the seek/setup sidecar entry are present in `track-opus`; a half-derived track counts as incomplete).
- `HasOpus` is resolved via `TrackFormatResolver.HasOpusAsync` — an **index-only** existence check (`MediaVault.HasIndexEntry` for both entries; no file-body load). The endpoint loops over the whole catalogue, so a body load per track would stream the full library sequentially; the index lookup costs zero disk reads per track.
- Returns 200 on success. Returns 500 on query error.
### POST api/track/duration/backfill ([ApiKeyAuthorize]) ### POST api/track/duration/backfill ([ApiKeyAuthorize])
Admin backfill: for every track whose `DurationSeconds` SQL column is still null, reads the `AudioBinary.Duration` from the vault and writes it to SQL. Idempotent — a re-run only touches still-null rows; rows that already have a value are skipped. Admin backfill: for every track whose `DurationSeconds` SQL column is still null, reads the `AudioBinary.Duration` from the vault and writes it to SQL. Idempotent — a re-run only touches still-null rows; rows that already have a value are skipped.
+2 -2
View File
@@ -141,7 +141,7 @@ Swaps the vault bytes for an existing track in place, under the same `entryKey`.
### GetAudioBinaryAsync(entryKey) ### GetAudioBinaryAsync(entryKey)
Reads audio from the `tracks` vault via `FileDatabase.LoadResourceAsync<AudioBinary>("tracks", entryKey)`. Returns a full-buffer `AudioBinary` or `null` if not found. This is the **read/playback path** — unchanged by the streaming store fix. Reads audio from the `tracks` vault via `FileDatabase.LoadResourceAsync<AudioBinary>("tracks", entryKey)`. Returns a full-buffer `AudioBinary` or `null` if not found. This is a full-buffer convenience read — **not** on the audio-delivery hot path. The delivery path now uses `TrackFormatResolver` (Opus: `MediaVault.GetEntryStreamAsync`; lossless: `OpenAudioMediaStreamAsync`). `GetAudioBinaryAsync` is retained as a read-back oracle used by tests (`AudioStoreStreamingTests`, `TrackReplaceAudioTests`, `TrackFormatDeliveryTests`).
### OpenAudioStreamAsync(entryKey) ### OpenAudioStreamAsync(entryKey)
@@ -161,7 +161,7 @@ Safety call to ensure the `tracks` vault exists (creates if missing). Called on
## Vault constants ## Vault constants
`VaultConstants.Tracks = "tracks"`, `VaultConstants.Images = "images"`, and `VaultConstants.TrackWaveforms = "track-waveforms"` — the vault names in production use. `TrackWaveforms` holds the per-track high-res waveform datum keyed by `TrackEntity.EntryKey` (Phase 12; renamed from the former `mix-waveforms`, which was Mix-only). New vault names go here when adding new vault types. `VaultConstants.Tracks = "tracks"`, `VaultConstants.Images = "images"`, `VaultConstants.TrackWaveforms = "track-waveforms"`, and `VaultConstants.TrackOpus = "track-opus"` — the vault names in production use. `TrackWaveforms` holds the per-track high-res waveform datum keyed by `TrackEntity.EntryKey` (Phase 12; renamed from the former `mix-waveforms`, which was Mix-only). `TrackOpus` holds two entries per track: the derived Opus audio (keyed by `entryKey`, extension `.opus`) and the combined setup-header + seek-index sidecar (keyed by `{entryKey}-sidecar`, extension `.opusidx`). New vault names go here when adding new vault types.
## Service registration ## Service registration