deepdrft/DeepDrftContent/CLAUDE.md

# CLAUDE.md - DeepDrftContent.Services

Guidance for working in the DeepDrftContent.Services project (the binary-content domain logic).

See the root `CLAUDE.md` for full architecture overview. This file covers what is specific to this project.

## One-line purpose

Binary-content domain logic. The FileDatabase implementation in full (Models, Services, Utils, Abstractions, Constants), audio processing, and the content-side track service. Consumed by `DeepDrftContent` (the host) and `DeepDrftCli` (the admin CLI).

## Layout

```
DeepDrftContent.Services/
├── FileDatabase/                       # The subsystem (port of TypeScript system)
│   ├── Abstractions/                   # Interfaces
│   ├── Models/                         # Data models, DTOs, enums
│   ├── Services/                       # FileDatabase, MediaVault, IndexSystem, IndexWatcher
│   └── Utils/                          # StructuralMap, StructuralSet, FileUtils
├── Processors/
│   ├── AudioProcessor.cs               # WAV file parsing, metadata extraction, streaming PCM header parse
│   ├── AudioStoreStream.cs             # Bounded streaming copy/prefix helpers used by the processors
│   └── ProcessedAudio.cs               # Store-path plan: metadata + streamed WriteToAsync callback
├── Constants/
│   └── VaultConstants.cs               # Vault name definitions
├── TrackService.cs                     # Content-side orchestrator
└── DeepDrftContent.Services.csproj
```

## FileDatabase model (high-level)

See `FileDatabase/README.md` for the long-form design discussion — it's a port of a TypeScript system and has deep rationale. This section covers the essentials for an agent walking in cold.

### Core structure

- **FileDatabase**: Root object. Created via `FileDatabase.FromAsync(rootPath)`. Holds a collection of `MediaVault` instances and an `IndexWatcher`. Implements `IDisposable`. Singleton in the host.
- **MediaVault**: A subdirectory under the FileDatabase root. Each vault has its own JSON `index` file listing entries and per-entry metadata. Typed via `MediaVaultType` enum (`Media | Image | Audio`).
  - Concrete implementations: `ImageVault` (for images), `AudioVault` (for audio). Do not use `ImageDirectoryVault` (that's stale docs) — the type is `ImageVault`.
- **Entry filenames**: `{sanitized-key}{extension}`, where sanitisation is `Regex.Replace(entryKey, @"[^a-zA-Z0-9]", "-")`. So entry id `"my-song"` with extension `.wav` → filename `my-song.wav`.

### Binary hierarchy

```
FileBinary (base: byte buffer)
  └── MediaBinary (+ Extension: string, MIME type inferred via MimeTypeExtensions)
      ├── AudioBinary (+ Duration: double, Bitrate: int)
      └── ImageBinary (+ AspectRatio: double)
```

Each has a matching `*Dto` variant for base64 JSON transport (e.g., `AudioBinaryDto` with buffer encoded as base64).

**Read/load path**: vault reads (`LoadResourceAsync<AudioBinary>`) still return a full-buffer `AudioBinary`. **Write/store path** is streaming: audio processors return a `ProcessedAudio` plan (metadata + `WriteToAsync` callback); `RegisterResourceStreamingAsync` / `MediaVault.AddEntryStreamingAsync` write bytes to a temp file, then `File.Move` atomic-rename into place. The full `AudioBinary` buffer is never materialized on the store path.

### Index lifecycle

- **DirectoryIndex**: Root index file (at `{rootPath}/index`). Tracks which vaults exist.
- **VaultIndex**: Per-vault index (at `{vaultPath}/index`). Records `MediaVaultType` and lists all entries in that vault.
- Both are JSON files. Created/loaded via `IndexFactoryService`.
- When a file is written externally (e.g., the CLI calls `FileDatabase.RegisterResourceAsync` directly), the **IndexWatcher** detects the write to the vault's `index` file and triggers `MediaVault.ReloadIndexAsync`, so a long-running web host stays consistent without restart.

## Error-handling philosophy (load-bearing)

Public `Load*` / `Register*` operations **swallow exceptions and return `null` / `false`** to match the TypeScript original.

```csharp
public async Task<T?> LoadResourceAsync<T>(string vaultId, string entryId) where T : FileBinary
{
    try { /* load and deserialize */ }
    catch { return null; }  // Swallow, return null
}

public async Task<bool> RegisterResourceAsync(string vaultId, string entryId, FileBinary media)
{
    try { /* store and update index */ }
    catch { return false; }  // Swallow, return false
}

// Streaming counterpart (Wave 1 OOM fix) — same bool contract:
public async Task<bool> RegisterResourceStreamingAsync(
    string vaultId, string entryId, MetaData metaData,
    Func<Stream, CancellationToken, Task> writeContent, CancellationToken ct)
{
    try { /* stream to temp → atomic rename → update index */ }
    catch { return false; }  // Swallow (logs non-cancel exceptions), return false
}
```

`MediaVault.AddEntryStreamingAsync` (called internally) writes bytes to a temp file in the vault directory, then `File.Move(temp, final, overwrite: true)` (atomic POSIX rename on the Linux prod host), then updates the index. A cancel or I/O fault before the rename leaves any prior backing file intact; the temp file is cleaned up best-effort.

**Callers must check return values.** Do not change this without a deliberate design pass — it's embedded in all FileDatabase tests and client code.

## Audio processors

Multi-format support via router pattern. All processors live in `DeepDrftContent/Processors/`:

- `AudioProcessor.ProcessWavFileAsync(filePath)`: WAV-specific processor. Validates RIFF/WAVE structure and format code. Accepts standard PCM (audioFormat=1) and WAVE_FORMAT_EXTENSIBLE (audioFormat=0xFFFE) when the SubFormat GUID indicates PCM. Normalizes EXTENSIBLE-PCM uploads to standard 44-byte PCM WAV before storing. Parses fmt and data chunks; extracts duration and bitrate. **Returns `ProcessedAudio`** (metadata + streamed `WriteToAsync` callback — no whole-file buffer). Header window grows in 64 KB steps capped at 8 MB; the audio body is never read during processing. Standard PCM WAV is stored verbatim (passthrough via `AudioStoreStream.CopyFileAsync`); EXTENSIBLE-PCM / IEEE-float / padded-container WAVs stream their normalization to standard 24-bit PCM. On parse failure, falls back to defaults (180s / 1411 kbps). Also exposes `TryExtractPcm(ReadOnlySpan<byte>)` for the whole-buffer waveform parity oracle and `TryReadPcmStreamInfoAsync(stream, totalLength)` for the streaming waveform compute path (bounded header parse from a stream; returns `WavPcmStreamInfo?` with `DataStart`/`DataLength`/format fields).
- `Mp3AudioProcessor.ProcessMp3FileAsync(filePath)`: MP3 processor. Reads a bounded ≤8 MB prefix for header parsing; skips ID3v2 tag, finds first valid MPEG frame sync, decodes frame header (bitrate, sample rate, channels). Reads Xing/Info header for VBR total-frame count (accurate duration); VBRI header as fallback; CBR estimate from file size otherwise. **Returns `ProcessedAudio`** (passthrough plan — MP3 stored unmodified). On parse failure, falls back to defaults (180s / 320 kbps).
- `FlacAudioProcessor.ProcessFlacFileAsync(filePath)`: FLAC processor. Reads a bounded ≤64 KB prefix. Validates `fLaC` magic, reads STREAMINFO metadata block (20-bit sample rate, 3-bit channels, 5-bit bits-per-sample, 36-bit total samples — all bit-packed). Computes duration from `totalSamples / sampleRate`; average bitrate from file size. **Returns `ProcessedAudio`** (passthrough plan — FLAC stored unmodified). On parse failure, falls back to defaults (180s / 1411 kbps).
- `AudioProcessorRouter.ProcessAudioFileAsync(filePath)`: Routes by extension — `.wav` → `AudioProcessor`, `.mp3` → `Mp3AudioProcessor`, `.flac` → `FlacAudioProcessor`. **Returns `ProcessedAudio?`**. Throws `ArgumentException` for unsupported extensions.
- `ProcessedAudio`: Store-path plan returned by all processors. Carries `Extension`, `Duration`, `Bitrate`, `Size`, and a `WriteToAsync(destination, ct)` callback that streams the canonical vault bytes to the destination without materializing the whole file. `ProcessedAudio.Passthrough(sourcePath, ...)` builds a passthrough plan via `AudioStoreStream.CopyFileAsync`.
- `AudioStoreStream`: Internal bounded-buffer streaming helpers. `CopyFileAsync(sourcePath, destination, ct)` does a bounded 80 KB disk-to-disk copy; `ReadPrefixAsync(path, cap, ct)` reads at most `cap` bytes from the start of a file (used by processors for header parsing without loading the body).
- `ILoudnessAlgorithm` / `ILoudnessAccumulator`: The loudness strategy interface now exposes both `Compute(ReadOnlySpan<byte>, ...)` (whole-buffer, retained as the parity oracle for tests — no production callers) and `CreateAccumulator(pcmByteLength, ...)` → `ILoudnessAccumulator` (streaming: `Add(ReadOnlySpan<byte>)` / `Finish()` → `double[]`). `RmsLoudnessAlgorithm` implements both; `Compute` is defined in terms of the accumulator, so streaming and whole-buffer outputs are byte-identical.
- `WaveformProfileService`: Streaming loudness compute + vault store. The whole-buffer `ComputeAndStoreAsync` / `ComputeAndStoreHighResAsync` methods are **retained but have no production callers** — they exist as the byte-identity parity oracle for tests; do not delete them. The production paths are: `ComputeAndStoreProfileStreamingAsync` (512-bucket seeker profile, tri-state `bool?`), `ComputeAndStoreHighResStreamingAsync` (duration-derived high-res datum, tri-state `bool?`), and `ComputeAndStoreAllStreamingAsync` (both datums in a SINGLE streaming pass, used by the upload/replace hot path, tri-state `bool?`). Tri-state: `null` = no backing audio stream; `false` = audio present but not WAV-decodable or vault write failed; `true` = stored. Streaming reads the WAV in bounded ≤80 KB chunks through one accumulator per datum; peak memory is O(bucket arrays + read buffer), independent of file size.
- `WaveformResolution`: Enum / constants controlling bucket density for the high-res compute. Renamed from `MixWaveformResolution` in Phase 12.

Vault stores original bytes with correct extension and MIME type (inferred from file extension or content-type header at upload time).

The primary entry point is `TrackContentService.AddTrackAsync(filePath, mimeType)` — format-agnostic. It selects the right processor via `AudioProcessorRouter`, processes the file, generates an entry GUID, stores in vault, returns unpersisted `TrackEntity`. Legacy `AddTrackFromWavAsync(filePath)` is now a shim over `AddTrackAsync` for backward compatibility.

## Image processor

`ImageProcessor.ProcessImageAsync(buffer, mimeType)`:

1. Accepts raw image bytes and MIME type (e.g., `image/png`, `image/jpeg`).
2. Parses PNG or JPEG headers to extract image dimensions.
3. Computes aspect ratio (width / height). Defaults to 1.0 if parsing fails or format is unsupported.
4. Returns `ImageBinary` with MIME type and aspect ratio metadata.
5. **No disk I/O**: operates on `byte[]` only — no file reading required.

## Content-side TrackService (orchestrator)

### AddTrackAsync(audioFilePath, ...)

The primary upload entry point. Format-agnostic — routes by extension via `AudioProcessorRouter`.

1. Calls `AudioProcessorRouter.ProcessAudioFileAsync(filePath)` → `ProcessedAudio` plan (no whole-file buffer — Wave 1 OOM fix).
2. Generates a GUID entry key (via `Guid.NewGuid().ToString()`).
3. Ensures the `tracks` vault exists (creates if missing).
4. Builds `MetaData` from the plan's header-extracted fields and calls `FileDatabase.RegisterResourceStreamingAsync("tracks", entryKey, metaData, processed.WriteToAsync)` — bytes are streamed from the staging file to the vault via a bounded copy; `MediaVault.AddEntryStreamingAsync` writes to a temp file then atomic-renames into place.
5. Returns a populated `TrackEntity` (with `Id = 0` since it's not yet in SQL, and `DurationSeconds` populated from the header parse).

If the vault write succeeds and SQL write fails, audio is orphaned (no compensating rollback).

### AddTrackFromWavAsync(filePath, ...)

Backward-compatible shim — delegates to `AddTrackAsync`. The router accepts WAV alongside MP3 and FLAC, so this carries no WAV-specific logic of its own.

### ReplaceTrackAudioAsync(entryKey, audioFilePath)

Swaps the vault bytes for an existing track in place, under the same `entryKey`. Captures the old extension from the vault index metadata (not by loading the file) so a cross-format swap can clean up the stale backing file post-success. Streams the new audio via `ProcessedAudio` + `RegisterResourceStreamingAsync` (no whole-file buffer — Wave 1 OOM fix). Returns the new audio's duration on success, `null` on failure (original audio left intact).

### GetAudioBinaryAsync(entryKey)

Reads audio from the `tracks` vault via `FileDatabase.LoadResourceAsync<AudioBinary>("tracks", entryKey)`. Returns a full-buffer `AudioBinary` or `null` if not found. This is a full-buffer convenience read — **not** on the audio-delivery hot path. The delivery path now uses `TrackFormatResolver` (Opus: `MediaVault.GetEntryStreamAsync`; lossless: `OpenAudioMediaStreamAsync`). `GetAudioBinaryAsync` is retained as a read-back oracle used by tests (`AudioStoreStreamingTests`, `TrackReplaceAudioTests`, `TrackFormatDeliveryTests`).

### OpenAudioStreamAsync(entryKey)

Returns a read-only, seekable `Stream` over a track's vault audio without buffering the whole file. Used by the streaming waveform compute path (`WaveformProfileService`). Caller owns and must dispose the stream. Returns `null` if the entry has no backing file.

### OpenAudioMediaStreamAsync(entryKey)

Returns a `MediaStream?` — a read-only, non-buffering stream over a track's vault audio together with its stored extension, or `null` if the entry has no backing file. Same non-buffering contract as `OpenAudioStreamAsync`, but exposes `.Extension` alongside `.Stream` so the caller can name a staging file by the original format (the Opus transcode stages the source with the correct extension so ffmpeg can detect the format). The caller owns the returned `MediaStream` and must dispose it. Follows the FileDatabase swallow-and-return-null contract.

### GetAudioDurationAsync(entryKey)

Reads the stored audio duration from the vault index metadata only — no audio body load. Used by the streaming waveform compute to derive the duration-based bucket count, and by `UnifiedTrackService.BackfillDurationsAsync`. Returns `null` if the entry is unknown or carries no audio metadata.

### InitializeTracksVaultAsync()

Safety call to ensure the `tracks` vault exists (creates if missing). Called on host startup.

## Vault constants

`VaultConstants.Tracks = "tracks"`, `VaultConstants.Images = "images"`, `VaultConstants.TrackWaveforms = "track-waveforms"`, and `VaultConstants.TrackOpus = "track-opus"` — the vault names in production use. `TrackWaveforms` holds the per-track high-res waveform datum keyed by `TrackEntity.EntryKey` (Phase 12; renamed from the former `mix-waveforms`, which was Mix-only). `TrackOpus` holds two entries per track: the derived Opus audio (keyed by `entryKey`, extension `.opus`) and the combined setup-header + seek-index sidecar (keyed by `{entryKey}-sidecar`, extension `.opusidx`). New vault names go here when adding new vault types.

## Service registration

In `DeepDrftContent/Startup.ConfigureDomainServices()` and `DeepDrftCli/Program.cs`:

```csharp
services.AddSingleton<FileDatabase>(/* from FileDatabase.FromAsync */);
services.AddScoped<AudioProcessor>();
services.AddScoped<TrackService>();  // DeepDrftContent.Services.TrackService
```

## Development commands

```bash
# Build
dotnet build DeepDrftContent.Services

# Run tests (FileDatabase tests cover vault/index/factory/utilities thoroughly)
dotnet test DeepDrftTests/

# Run CLI (which consumes this service)
dotnet run --project DeepDrftCli -- add myfile.wav "Track Name" "Artist"
```

## Important patterns

- **Async/await**: All FileDatabase operations are async. No sync methods.
- **Type safety**: Generic `LoadResourceAsync<T>` ensures callers know what they're loading.
- **Vault lifecycle**: Vaults are created on first boot, then reused. The `FileDatabase` singleton holds them in memory with live `IndexWatcher`es.
- **Entry sanitisation**: Keys are sanitised client-side (in the CLI and web host) *and* by `MediaVault` (defensive). Always sanitise before registering — it's the only way to ensure safe filenames.
- **Metadata hierarchy**: Use the appropriate media type (`AudioBinary`, `ImageBinary`) so downstream code can rely on the metadata. Don't store an audio file as a generic `MediaBinary` — use `AudioBinary` with duration/bitrate.

## What does NOT live here

- HTTP controllers or middleware — that's `DeepDrftContent`.
- SQL database code — that's `DeepDrftWeb.Services`.
- Blazor components or UI logic — that's `DeepDrftWeb.Client`.
- Configuration files (`appsettings.json`, `filedatabase.json`) — those are in the host project.

When working with this project, focus on the FileDatabase subsystem (the most complex piece of the codebase), audio processing, and the orchestration logic that bridges binary and SQL databases. The tests (in `DeepDrftTests/`) are the load-bearing documentation of FileDatabase behaviour — consult them when FileDatabase semantics are unclear.