daniel/deepdrft
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 13 Wiki Activity

docs: reflect streaming upload store path and waveform compute (OOM fix)

Browse Source
This commit is contained in:
daniel-c-harvey
2026-06-26 05:55:54 -04:00
parent 5d9f410cd8
commit 9bfa921703
3 changed files with 58 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files
DeepDrftAPI/CLAUDE.md
+6 -6
View File
@@ -11,7 +11,7 @@ Dual-database authority for tracks (SQL metadata + FileDatabase binary), release
## What lives here now (only)
- `Program.cs`, `Startup.cs`: HTTP host config, DI wiring, middleware setup, port binding. AuthBlocks startup: `AddAuthBlocks`, `UseAuthBlocksStartupAsync`, `MapAuthBlocks`, authentication/authorization middleware.
- `Services/UnifiedTrackService.cs`: Host-internal orchestrator. Coordinates vault write + SQL persist for upload (`UploadAsync`), and SQL delete + vault remove for delete (`DeleteAsync`).
- `Services/UnifiedTrackService.cs`: Host-internal orchestrator. Coordinates streaming vault write + SQL persist for upload (`UploadAsync`), and SQL delete + vault remove for delete (`DeleteAsync`). The upload/replace hot path streams audio into the vault via the `ProcessedAudio` plan (Wave 1 OOM fix) and then computes both waveform datums in a single bounded streaming pass via `TryStoreWaveformDatumsAsync` (Wave 2 OOM fix) — neither path buffers the whole audio file in a managed `byte[]`.
- `Services/UnifiedReleaseService.cs`: Host-internal orchestrator. Coordinates release mutations (mix waveform compute + store, session hero-image upload + link).
- `Controllers/TrackController.cs`: Track endpoints (see below).
- `Controllers/ReleaseController.cs`: Release endpoints (see below).
@@ -74,7 +74,7 @@ Admin backfill: computes and stores a waveform profile for an existing track fro
- **Header `ApiKey`**: required. Validated by `ApiKeyAuthenticationMiddleware`.
- **Route parameter `trackId`** (string): the entry id (TrackEntity.EntryKey).
- Fetches audio from vault, decodes it, computes a loudness profile, and stores the profile in the `waveform-profiles` vault.
- Streams the vault audio in bounded ≤80 KB chunks (no whole-file load) via `WaveformProfileService.ComputeAndStoreProfileStreamingAsync`. Tri-state result: `null` = no vault audio → 404; `false` = audio present but not WAV-decodable or vault write failed → 500; `true` = stored → 200.
- Returns 200 on success. Returns 404 if no audio is stored under that key. Returns 500 if WAV decoding or vault write fails.
### GET api/track/{trackId}/waveform/high-res (unauthenticated)
@@ -91,7 +91,7 @@ Server-side trigger: compute and store the per-track high-res datum for any trac
- **Header `ApiKey`**: required. Validated by `ApiKeyAuthenticationMiddleware`.
- **Route parameter `trackId`** (string): the entry id (TrackEntity.EntryKey).
- Calls `WaveformProfileService.ComputeAndStoreHighResAsync` via `UnifiedTrackService`.
- Reads the duration from vault index metadata (no audio body load), then streams the vault audio in bounded chunks via `WaveformProfileService.ComputeAndStoreHighResStreamingAsync`. Tri-state result: `null` = no vault audio → 404; `false` = audio present but not WAV-decodable or vault write failed → 500; `true` = stored → 200.
- Returns 200 on success. Returns 404 if no audio stored under that key. Returns 500 on compute/storage failure.
### GET api/track/meta/by-key/{entryKey} (unauthenticated)
@@ -170,7 +170,7 @@ Soft-delete a release row. Used by the albums browser to remove an orphaned rele
- `releaseId` (long?, optional): the SQL release ID to attach this track to. Omit (null) on the first row of a submit — this is the **CREATE path**, which mints a new release and blocks a pre-existing (title, artist) with 409. Set to the release id returned by row 1 for rows 2..N of a within-batch multi-track Cut — this is the **ATTACH path**, which skips the (title, artist) pre-existing check and attaches directly to the already-created release after validating the id matches the natural key. The upload form is create-only; appending to a pre-existing release must go through the edit tools.
- The upload stream is copied to a staging file under the **upload staging directory** (resolved from `Upload:StagingPath`, defaulting to a `staging` subdirectory under the FileDatabase vault path — on the data disk, **never** `Path.GetTempPath()`) with the appropriate extension (`.wav`, `.mp3`, or `.flac`). The audio processor reads from disk and requires the correct extension for format detection. The staging file is always deleted in a `finally` block — success or failure. The framework's own multipart file-section buffer is relocated off the system temp mount too: `Startup.ConfigureDomainServices` sets the `ASPNETCORE_TEMP` env var to the same staging directory, so neither on-disk copy of a large body lands on `/tmp` (a small RAM-backed tmpfs on the Linux host).
- `[RequestSizeLimit(~1.86 GB / 2_000_000_000)]` + `[RequestFormLimits(MultipartBodyLengthLimit = 2_000_000_000)]` lift the per-request ceiling above the framework default (~28 MB) so production-sized files are accepted. The body is streamed to the staging file, not buffered in memory.
- `UnifiedTrackService.UploadAsync` orchestrates: release resolution (CREATE or ATTACH, see above) → `TrackContentService.AddTrackAsync` (format-agnostic vault write via router) → `TrackManager` (SQL persist with `createdByUserId`). Release resolution runs the cardinality guard on both paths and, on the CREATE path, calls `ITrackService.FindOrCreateRelease` (returns `(ReleaseDto Release, bool WasCreated)`); if `WasCreated` is false, a concurrent upload won the race and the request is rejected as a duplicate rather than silently attaching.
- `UnifiedTrackService.UploadAsync` orchestrates: release resolution (CREATE or ATTACH, see above) → `TrackContentService.AddTrackAsync` (format-agnostic streaming vault write via router — audio is streamed to the vault via the `ProcessedAudio` plan, no whole-file buffer — Wave 1 OOM fix) → `TrackManager` (SQL persist with `createdByUserId`) → `TryStoreWaveformDatumsAsync` (best-effort: reads duration from vault index metadata, then computes both waveform datums from a single bounded streaming pass over the stored audio — Wave 2 OOM fix). Release resolution runs the cardinality guard on both paths and, on the CREATE path, calls `ITrackService.FindOrCreateRelease` (returns `(ReleaseDto Release, bool WasCreated)`); if `WasCreated` is false, a concurrent upload won the race and the request is rejected as a duplicate rather than silently attaching.
- Returns 200 with the **persisted** `TrackDto` JSON (Id populated) on success. Returns 400 for missing/invalid form fields or unsupported audio format. Returns 409 for two distinct domain conditions: a pre-existing (title, artist) duplicate on the CREATE path (`DUPLICATE_RELEASE:` marker → 409 Conflict), or a track-number conflict within the release (`CARDINALITY_VIOLATION:` marker → 409 Conflict). Returns 500 if processing fails.
### DELETE api/track/{id:long} ([ApiKeyAuthorize])
@@ -190,7 +190,7 @@ Soft-delete a release row. Used by the albums browser to remove an orphaned rele
- **Route parameter `id`** (long): the SQL track ID.
- **Form field `audioFile`** (`IFormFile`, required): the replacement audio bytes. File name must end in `.wav`, `.mp3`, or `.flac`.
- `[RequestSizeLimit(~1.86 GB / 2_000_000_000)]` + `[RequestFormLimits(MultipartBodyLengthLimit = 2_000_000_000)]` mirror the upload ceiling. The body is streamed to a staging file under the upload staging directory (the same off-`/tmp` data-disk location as the upload path; correct extension preserved for the audio processor), always deleted in a `finally` block.
- Calls `UnifiedTrackService.ReplaceAudioAsync`, which: looks up SQL row by id → calls `TrackContentService.ReplaceTrackAudioAsync(entryKey, tempFilePath)` (registers new audio under the existing `EntryKey`; removes the stale backing file only on a cross-format swap, after the new write succeeds) → regenerates both waveform datums (best-effort; a datum failure is logged and swallowed) → writes the new audio's duration to `DurationSeconds` via `ITrackService.SetDuration` (unconditional overwrite; a failure is surfaced, not swallowed, to prevent derived aggregates like `MixRuntimeSeconds` from silently going stale).
- Calls `UnifiedTrackService.ReplaceAudioAsync`, which: looks up SQL row by id → calls `TrackContentService.ReplaceTrackAudioAsync(entryKey, tempFilePath)` (streams new audio into the vault under the existing `EntryKey` via the `ProcessedAudio` plan, no whole-file buffer — Wave 1 OOM fix; removes the stale backing file only on a cross-format swap, after the new write succeeds) → regenerates both waveform datums from a single bounded streaming pass over the freshly stored audio via `TryStoreWaveformDatumsAsync` (best-effort; a datum failure is logged and swallowed — Wave 2 OOM fix) → writes the new audio's duration to `DurationSeconds` via `ITrackService.SetDuration` (unconditional overwrite; a failure is surfaced, not swallowed, to prevent derived aggregates like `MixRuntimeSeconds` from silently going stale).
- Returns 200 on success. Returns 400 if the file is missing or the format is unsupported. Returns 404 if the track id is not found. Returns 500 if vault processing fails.
### GET api/track/page (unauthenticated)
@@ -288,7 +288,7 @@ Legacy endpoint: formerly served the high-res waveform datum for a Mix release f
### POST api/release/{id:long}/mix/waveform ([ApiKeyAuthorize])
Server-side trigger: fetch the Mix's track audio from the vault, compute the duration-derived high-res datum, store it in the `track-waveforms` vault under the track's `EntryKey`, and link it via `MixMetadata.WaveformEntryKey`. Delegates to `WaveformProfileService.ComputeAndStoreHighResAsync` — the same shared seam used by the upload path and the generalized CMS generate action. No request body.
Server-side trigger: fetch the Mix's track audio from the vault, compute the duration-derived high-res datum, store it in the `track-waveforms` vault under the track's `EntryKey`, and link it via `MixMetadata.WaveformEntryKey`. Delegates to `WaveformProfileService.ComputeAndStoreHighResStreamingAsync` (streams the vault audio in bounded chunks, no whole-file buffer) — the same shared seam used by the upload path and the generalized CMS generate action. No request body.
- **Header `ApiKey`**: required. Validated by `ApiKeyAuthenticationMiddleware`.
- **Route parameter `id`** (long): the SQL release ID.