Files

T

daniel-c-harvey f8186fb7c7 docs: move Phase 1.1 to COMPLETED.md; update DeepDrftContent CLAUDE.md for float and padded WAV support

2026-06-10 20:42:58 -04:00

25 KiB

Raw Blame History

PLAN.md — DeepDrftHome forward roadmap

Forward-looking roadmap. Sits alongside CONTEXT.md (architecture orientation) and COMPLETED.md (history). Per CONTEXT.md §6, items move from here to COMPLETED.md when work lands; do not delete completed entries.

Organised by theme, not by date. Themes are roughly ordered by current product weight, not commitment. Nothing here carries a timeline unless it explicitly says so.

0. Baseline — what just landed

A two-part audit (design + streaming) ran on 2026-05-17 and the fixes for Critical, Major, and Minor findings are now on dev. The remainder of this plan assumes that baseline. In summary the audit-pass fixed:

Index concurrency — VaultIndexDirectory no longer drops the lock before its async disk write; the index file can no longer be clobbered by interleaved writers.
Repository semantics — TrackRepository.Update now fails-fast when an Id is not found instead of silently issuing an INSERT.
Streaming Criticals — concurrent-seek race in the client, dirty trailing bytes leaking out of the ArrayPool-rented buffer, final-tail audio dropped at EOF below the minimum decode frame, and the assumption that the first network chunk contains the whole WAV header.
17 design and streaming Majors/Minors across all eight projects — format-validation alignment between processor/offset/decoder, IAsyncDisposable on the player provider, cancellation tokens threaded through the HTTP path, structured logging into the FileDatabase subsystem, sort-sentinel cleanup, sundry DRY/SRP tightenings.

What this means for the roadmap: the streaming substrate is solid. Future work can build on top of it rather than around it. The remaining items in TODO-V2.md that did not land are deferred as features, not bugs — they are captured below under Phase 1.

Phase 1 — Streaming features deferred from the audit

These were flagged during the audit but classified as feature work, not defect fixes. They are listed in rough order of user-visible impact.

1.2 Audio format diversity

What: Today AudioProcessor, WavOffsetService, and the JS decoder are PCM/WAV-only. MimeTypeExtensions already maps MP3, FLAC, Ogg, AAC, M4A — none are wired.
Why it matters: WAV-only is a real ceiling for any non-internal release. Distribution-grade formats (MP3, FLAC at minimum) are table stakes for a music site.
Shape: Two seams need a strategy pattern.
- Server side: replace AudioProcessor.ProcessWavFileAsync with a format-router that selects a per-format processor; replace WavOffsetService with a per-format offset strategy (some formats — MP3, Ogg — have natural frame boundaries; FLAC has block headers; AAC has ADTS).
- Client side: the JS decoder is currently a WAV byte-walker. For non-WAV, the simplest path is decodeAudioData over the full payload (loses streaming-start). The richer path is per-format chunked decoders. Worth a design pass before committing.
Prerequisite: None functionally, but consider settling Phase 4 (HTTP Range) first — native range/cache is much more important for large MP3s than for WAVs.
Constraint: Spectrum FFT tap currently relies on raw AudioBuffers through decodeAudioData. If a future path uses MediaElementAudioSourceNode (see 4.1), the FFT tap still works but the early-playback story changes.

1.3 Preload / prefetch of the next track

What: No mechanism to begin the next track's stream during the tail of the current. Each play is a cold fetch.
Why it matters: Prerequisite for both crossfade (1.4) and gapless (1.5). Also a perceived-latency win on its own — track-change feels instant when the bytes are already in flight.
Shape: A second HttpClient request kicked off when the current track passes a configurable threshold (e.g. last 10 seconds). Bytes accumulate into a staged StreamDecoder instance rather than the live one. Promotion to "current" happens at end-of-stream or on user-selected next.
Prerequisite: Requires a notion of "next track" — today the player only knows the current one. That implies either a playlist/queue model in IPlayerService or a passive "what was the next row in the gallery" inference.
Open question: Does a queue model belong in IPlayerService, or is the player a single-slot device that a future PlaylistService orchestrates above? Worth a design note before implementation. Capture in product notes when picked up.

1.4 Crossfade

What: Smooth A→B transition with overlapping fade-out / fade-in.
Why it matters: DJ/mix aesthetic that fits the DeepDrft collective's electronic-music context. Distinguishing UX from generic "next track."
Shape: Architecturally two simultaneous PlaybackScheduler instances suffice — each owns its own gain node, crossfaded via GainNode.gain.linearRampToValueAtTime. The wiring is the work, not the audio graph itself.
Prerequisite: 1.3 (Preload) — there is nothing to fade into without prefetch.

1.5 Gapless playback

What: Eliminate the inter-track silence that exists today.
Why it matters: Important for live-set rips, mix tapes, anything authored to flow continuously.
Shape: The decoder must be able to start the next track's first buffer scheduled exactly at the end of the current one's last buffer (sample-accurate, not wall-clock). With PlaybackScheduler's existing 500 ms lookahead this is mechanically achievable — the next track's first AudioBufferSourceNode.start(t) is set to the previous track's end time.
Prerequisite: 1.3 (Preload). Also needs to play nicely with 1.2 because gapless across formats is hard (encoder padding/priming on MP3 in particular).
Constraint: Truly sample-accurate gapless requires knowing the priming/padding sample counts of the source format. Out of scope for WAV-only; revisit when format diversity lands.

1.6 Track-skip on error

What: A failed processStreamingChunk aborts the entire load with no recovery path.
Why it matters: One corrupt frame at byte 4M of a 100 MB stream currently means the listener loses the entire track. Should at minimum surface a clear error and (optionally) skip past the bad region.
Shape: Two-level response.
- Cheap: catch in the streaming loop, surface a user-visible error, advance the gallery to the next track if a queue exists.
- Richer: byte-scan forward to the next valid frame header for the format and resume. Format-dependent — only worth doing once 1.2 lands.

1.7 Safari compatibility

What: Two known Safari edge cases.
- webkitAudioContext.close() is async-but-not-Promise on older Safari (≤ ~14); await resolves immediately and the next initialize() can run against a not-yet-closed context.
- iOS Safari < 15 had streaming-fetch quirks; HttpCompletionOption.ResponseHeadersRead behaviour is not guaranteed there.
Why it matters: Real listener share. iOS in particular is a primary listening surface for music.
Shape: For the close() race — detect webkitAudioContext and poll state === "closed" with a short timeout instead of trusting the await. For the fetch quirks — first decide the minimum supported iOS version; if pre-15 is in scope, fall back to a non-streaming fetch path and accept the latency.
Open question: What's the floor? Decide before designing the fallback. iOS 15+ as the floor would let us drop the second concern entirely.

Phase 2 — Product surface: gallery, browsing, ingestion

These follow from CONTEXT.md §5. Direction is strongly implied but no specific UI has been committed.

Phase 6 — CMS Enhancements

Two CMS-side features Daniel has committed to. Both live in DeepDrftManager (InteractiveServer render mode throughout, MudBlazor UI). All track reads/writes route through ICmsTrackService / CmsTrackService — no in-process data layer in Blazor components. Follow the existing patterns in TrackList.razor / TrackNew.razor (MudContainer, MudPaper, MudStack, server-data MudTable).

6.1 CMS Home Page — catalogue summary dashboard

What: Replace the redirect-to-/tracks at Index.razor (route /) with a real dashboard showing a grid of summary cards: total tracks, distinct albums, distinct genres.
Why: Quick orientation for the CMS admin — at-a-glance catalogue health on landing, instead of dropping straight into the table. First thing the admin sees, so it carries the bold DeepDrft palette rather than a conservative admin look.
Shape:
- Route / component: Keep Index.razor at /; remove the OnInitialized redirect and render the dashboard. The CMS nav lands here; /tracks remains reachable from the nav and from the cards.
- UI: A responsive MudGrid of three MudCards (Tracks / Albums / Genres). Each card: an icon (LibraryMusic, Album, Category or similar), the metric as a large Typo.h2/h3 number, and a label. Cards are clickable (@onclick → Nav.NavigateTo). Lean into the active MudBlazor palette — Color.Primary/Color.Secondary fills or accent borders, generous elevation — this is the visual-punch surface, not a muted KPI strip. Loading state: skeleton or per-card MudProgressCircular while the three fetches resolve. Each card fetches independently so one slow/failed call doesn't blank the others; a failed card shows a "—" with a retry affordance rather than collapsing the grid.
- Card navigation (Phase 6 scope): All three cards navigate to /tracks (the track maintenance page). Per-album / per-genre pre-filtering is deferred — see 6.2. Ship the cards as plain links to /tracks now.
- Data model: No entity changes. AlbumSummaryDto and GenreSummaryDto already exist in DeepDrftModels.
- API surface: No new API endpoints. The three numbers are already available:
  - Albums count = length of GET api/track/albums (exists, unauthenticated, returns List<AlbumSummaryDto>).
  - Genres count = length of GET api/track/genres (exists, unauthenticated, returns List<GenreSummaryDto>).
  - Tracks count = TotalCount from GET api/track/page (exists) requested with pageSize=1 (cheapest paged call that still returns the total).
- CmsTrackService surface (new methods): ICmsTrackService does not currently expose albums/genres. Add three thin proxy methods mirroring the existing pattern (e.g. GetAlbumSummariesAsync, GetGenreSummariesAsync, and a GetTrackCountAsync that calls page?pageSize=1 and returns TotalCount). These are the only new code on the service. No controller work.
- Components: Index.razor (dashboard host) plus, optionally, a small SummaryCard.razor for the repeated card — worth extracting given three near-identical cards, but staff-engineer's call.
Prerequisites: None. All backing endpoints and DTOs exist.
Open questions:
- (Daniel) Should the dashboard show only the three counts, or is there appetite for a fourth/fifth metric now (e.g. "tracks missing cover art", "tracks missing waveform profile" — the latter is already computable from GetWaveformStatusAsync)? Counts-only is the committed scope; flag any additions before build.

6.2 Card-contextual filtering of the Tracks page — `[deferred]`

What: Make the Album and Genre dashboard cards navigate into a filtered /tracks view (e.g. clicking an album card shows only that album's tracks), rather than the unfiltered table.
Why: Turns the dashboard from a read-only summary into a navigation hub — the natural next step once the cards exist.
Why deferred: The dashboard cards aggregate across all albums/genres — there is no single album/genre to filter to from a top-level count card. Meaningful per-album/per-genre navigation needs an intermediate browse surface (a list of albums, a list of genres) for the admin to pick from — i.e. it's really a CMS analogue of the public AlbumsView/GenresView, not a property of the summary cards. That's a larger surface than the dashboard itself and shouldn't be smuggled in. The GET api/track/page endpoint already accepts album= and genre= query filters, so the API substrate is ready; the missing piece is the CMS browse UI and the filter plumbing in TrackList.
Shape (sketch, not committed): CMS album/genre browse pages (or tabs on /tracks) backed by the existing albums/genres endpoints; rows link to /tracks?album=… / /tracks?genre=…; TrackList.LoadServerData reads the query param and passes it to GetPagedAsync. Revisit as its own item when Daniel wants it.

6.3 Batch Upload Page

What: Replace the single-track form at /tracks/new with a two-panel batch upload page that uploads many WAVs in one session under a shared album header.
Why: Uploading an album one track at a time is the current reality — re-entering album, genre, release date, cover art, and artist on every track. Batch upload makes "add a release" a single operation: set the shared header once, queue the tracks, submit. This is the dominant ingestion shape for the collective (releases, not loose singles).
Shape:
- Route: New page at /tracks/upload. Justification: /tracks/new reads as "new single track" and the edit route is /tracks/{id}; /tracks/upload names the operation (batch ingestion) without colliding with the id-parameterised edit route. Repoint the "Add Track" button in TrackList.razor (currently Href="/tracks/new") to /tracks/upload. Whether /tracks/new is retired or left as a redirect is staff-engineer's call; the committed change is that the button goes to the batch page.
- Data model change — ReleaseType: Add a ReleaseType enum to DeepDrftModels (enum ReleaseType { Single, EP, Album }). Enum over string: three fixed values, and it gates UI (selector) and future grouping logic — a free-text column invites typos. Add a ReleaseType property to TrackEntity and TrackDto. Decide nullability: recommend non-null with a default of Single so existing rows backfill cleanly to a sensible value (a release of one track is a single) and the column is never null. This ripples to TrackConfiguration (EF mapping — store as string via HasConversion<string>() for readable DB values, or as int; recommend string for legibility), TrackConverter (assign on round-trip), and the upload/update service signatures. An EF migration is required — author it via dotnet ef migrations add, never by hand.
- Data model change — TrackNumber: Add a TrackNumber property (type int, 1-based, non-null) to TrackEntity and TrackDto to store per-track ordinal position within a release. This ripples through TrackConfiguration (EF mapping) and TrackConverter (assign on round-trip) the same way ReleaseType does. A second EF migration is required — author it via dotnet ef migrations add, never by hand. May be combined into a single migration with the ReleaseType change — staff-engineer's call on whether to combine or keep separate.
- Shared-vs-per-track field split:
  - Shared (header strip, applied to every track in the batch): album name, artist, album cover image (single upload), genre, release date, and ReleaseType. One album per batch — the entire batch is one release, and all release-level fields live in the header.
  - Per-track (right detail panel): track name, the individual WAV file, and that row's upload status.
- Layout (two-panel under a header strip):
  - Header strip (full width, top): album name, artist MudTextField, single cover-art InputFile (reuse the MudField cover-art pattern from TrackNew, including the upload-on-submit behaviour), genre MudTextField, release-date field, and ReleaseType MudSelect. These bind to a single batch-header model.
  - Left panel (track queue): an ordered list of queued tracks; the row order is the release track order and reflects each track's TrackNumber. Each row shows track name, a reorder affordance (up/down MudIconButtons are the low-risk choice; drag-and-drop is a nice-to-have — see open questions), a remove button, and a per-row status indicator (queued / uploading / done / failed). A +/InputFile (with multiple) at the top or bottom of the list adds WAV files; each added file becomes a row with track name defaulted from the filename (sans extension). On submit, each track is assigned its TrackNumber (1-based) from its position in the list.
  - Right panel (selected-track detail): when a row is selected, show its editable fields — track name and the WAV file name/size/status. Selecting a different row swaps the detail.
- Add-files behaviour: InputFile multiple → append a row per file. Default track name = filename without extension. New rows append to the end of the list, taking the next ordinal position. Keep the 1 GB per-file ceiling and the .wav validation from TrackNew.
- Submit behaviour: Sequential, one request at a time — reuse the existing single-track upload path (CmsTrackService.UploadTrackAsync) in a loop. This mirrors the deliberately-sequential waveform backfill in TrackList.GenerateAllMissing ("one request at a time so a large backfill does not flood the API"). Per-track progress: each left-panel row reflects its state as the loop advances (StateHasChanged between rows). Cover-art upload happens once before the loop (upload the image, get the entry key, then pass/link it to every track) — do not re-upload the cover per track. On completion, snackbar a summary (uploaded N, M failed) and navigate to /tracks. Partial failure: completed tracks stay persisted; failed rows remain visible with their error so the admin can retry just those — do not roll back the batch.
- CmsTrackService surface: No new method strictly required — the loop calls the existing UploadTrackAsync per track and the existing image upload/link path per batch. UploadTrackAsync's signature gains releaseType and trackNumber parameters (ripples from the data-model change). If the cover-link follow-up (the UpdateAsync step TrackNew does today) is kept per track, that's existing surface too.
- API surface: No new endpoints. Existing POST api/track/upload (per track) and POST api/image/upload (once per batch) cover it. api/track/upload and the metadata update endpoints gain releaseType and trackNumber in their payloads as a consequence of the entity change.
- Components: BatchUpload.razor (page + header strip + orchestration), and reasonably a BatchTrackRow model class plus left-panel/right-panel as child components or inline sections — staff-engineer's structural call.
- Constraint — dual-write orphan risk: Each track inherits the existing dual-write hazard (audio lands in the vault, SQL persist may fail → orphaned audio, no rollback). Batch upload multiplies the exposure (N tracks per session instead of one). The mitigation is Phase 4.3 (dual-write rollback / dead-letter log) — not a blocker for this feature, but this is the strongest argument yet for landing 4.3. Flag it as a known constraint; do not attempt per-batch transactional rollback (the dual-database split can't give it).
Prerequisites:
- ReleaseType enum + TrackNumber field + TrackEntity/TrackDto changes + EF migration(s) must land first (it's the data-model floor for the whole feature, and ripples through TrackConfiguration/TrackConverter/service signatures). Could be a separate prep commit before the page work.
- Not blocked by Phase 4.3, but 4.3 is the right mitigation for the amplified orphan risk and is worth sequencing alongside.
Resolved (no longer open):
- One album per batch. The whole batch is one release; album name and all release-level fields (artist, genre, release date, ReleaseType, cover art) live in the shared header strip. A batch never mixes albums.
- Track ordinals are persistent — TrackNumber (int, 1-based, non-null) stores per-track position within a release. The left-panel row order reflects TrackNumber, and each track is assigned its ordinal from its list position on submit.
Open questions (need Daniel before build):
- (Daniel) Drag-and-drop vs. up/down arrows for reorder — up/down MudIconButtons are the low-risk default and ship first. Drag-and-drop (MudBlazor MudDropContainer) is a polish upgrade. Confirm whether arrows are acceptable for v1.

Phase 3 — New content kinds

3.1 Live / session content

What: The home page advertises "Live Sessions" and "Video Content (coming soon)". No data model exists for these.
Why it matters: Honour the home page copy. Also differentiates the site from a generic track gallery — live sessions and video are the collective's authored output.
Shape: Speculative; no commitment yet.
- Likely new entity table(s) sibling to TrackEntity (SessionEntity, VideoEntity?) — or a polymorphic MediaEntity with discriminator. The choice affects how much code in TrackService / TrackController can be reused.
- New vault type(s). MediaVaultType.Media exists and is the obvious home for video; sessions are probably still Audio.
- New routes, new UI surfaces, new player considerations (video has its own playback element and does not go through the WAV decoder).
Prerequisite: Probably 2.1 (vault wiring proof) and a decision on the entity model before any code lands.
[speculative] — direction inferred from home-page copy, not a Daniel-confirmed commitment.

Phase 4 — Infrastructure / delivery

4.3 Dual-write rollback / dead-letter log

What: If content-side write succeeds and SQL-side write fails, audio is orphaned in the vault. No compensating mechanism exists.
Why it matters: A latent data-integrity issue. Materially riskier once web upload (2.4) exists.
Shape: Audit suggested a DeadLetterLog recording orphaned entryKeys for a periodic maintenance pass. Lighter than full transactional rollback (which the dual-database split fundamentally cannot give us).
Prerequisite: None. Worth landing alongside or just before 2.4.

Phase 5 — Documentation backlog

5.1 Folder-level CLAUDE.md sweep

What: Eight folder-level CLAUDE.md files need writing/rewriting per the brief in DOC_PLAN.md. Five are rewrites (drift from the .NET 10 upgrade and structural moves); three are new (DeepDrftWeb.Services, DeepDrftContent.Services — the two libraries where most domain logic now lives — plus the open question on DeepDrftContent.Services/FileDatabase/README.md).
Why it matters: The agent guidance files are how every future implementer (human or agent) gets oriented in a directory. They are currently misleading in ways that will cause wrong assumptions on first contact — claiming .NET 9, referencing MediaPath that has been EntryKey for two migrations, describing a FileDatabase/ tree inside DeepDrftContent that has moved out, and missing entirely for the two *.Services libraries.
Shape: Doc-keeper executes against DOC_PLAN.md. Order of operations and the per-folder briefs are already specified there.
Prerequisite: None. Can run fully in parallel with any feature work.
Constraint: Wait on Daniel for the DeepDrftContent.Services/FileDatabase/README.md judgement call before that file changes (retire, keep + refresh, or replace with a CLAUDE.md). The other seven can proceed without that decision.

Cross-cutting / not yet themed

A small set of items that are real but don't fit a phase yet. Surface them when they become relevant rather than committing now.

Identity / accounts. Currently no user concept. Needed before web upload (2.4); also a precondition for favourites, listening history, per-user playlists. Decide the shape before any of those lands. [speculative] until Daniel signals interest.
ITrackService interface. Audit-suggested. Low value today (one consumer pair); higher value when the test surface expands beyond FileDatabase.
Test coverage outside FileDatabase. Tests today cover the FileDatabase subsystem comprehensively and nothing else. As features in Phases 1–4 land, test scope should expand — at minimum WavOffsetService, AudioProcessor, TrackService (both sides), and the streaming player services. Not a phase of its own; an attached cost to feature work.

Working with this file

Add items by extending an existing phase first; only create a new phase when the addition genuinely doesn't fit any of 1–5. Phase numbers are organisational, not sequencing.
When something lands, move it to COMPLETED.md rather than deleting it. Keep the original "What / Why / Shape" body intact so the history reads as a record of the decision, not just the outcome.
Mark genuinely uncertain items [speculative] so future readers can tell what is direction vs. commitment.
Open questions belong in the item that raises them, not in a separate "questions" list — they expire when the item does.

25 KiB Raw Blame History Unescape Escape