feat: add audio track support with WAV/MP3 playback and looping

Introduce KGAudioTrack and KGAudioRegion as new track/region types
alongside existing MIDI tracks. Users can import WAV, MP3, OGG, FLAC,
and AAC files into audio tracks with full playback, looping, and
solo/mute/volume support.

New core models:
- KGAudioTrack (extends KGTrack with TrackType.Wave)
- KGAudioRegion (extends KGRegion with audio file reference)
- KGAudioPlayerBus (Tone.Gain + ToneBufferSource-based playback)
- KGAudioFileStorage (OPFS media/ directory for binary audio files)

New commands:
- AddAudioTrackCommand (create audio track with player bus)
- ImportAudioCommand (import audio file, create region at playhead,
auto-expand maxBars)

Playback features:
- Audio regions scheduled via Tone.Transport alongside MIDI events
- Loop support with duration capping at loop boundaries
- Resume safety offset to prevent ToneBufferSource timing misses
- Cross-track buffer copy when moving audio regions between tracks
- Proper buffer cleanup on track deletion and project switch

UI changes:
- Separate "+ MIDI" and "+ Audio" buttons in top-left spacer
- Audio track shows upload button instead of instrument selector
- FileImportModal reused for audio file drag-and-drop upload
- Real waveform rendering on audio region canvas
- Green color scheme for audio regions (vs blue for MIDI)
- Pencil icon and resize handles hidden for audio regions
- Cross-type region moves blocked (MIDI ↔ Audio)
- Piano roll blocked for audio regions
- Audio region deletion support

Infrastructure:
- Project structure version bumped to 4 (no-op migration)
- class-transformer discriminators updated for new subtypes
- RemoveTrackCommand updated to clean up audio player buses
This commit is contained in:
Xiaohan-Tian
2026-04-09 21:54:54 -07:00
parent de2979178e
commit fc84edfc59
23 changed files with 1406 additions and 100 deletions
+21 -6
View File
@@ -1,6 +1,8 @@
import React, { useEffect, useState, useRef } from 'react';
import { KGTrack } from '../../core/track/KGTrack';
import { KGMidiRegion } from '../../core/region/KGMidiRegion';
import { KGAudioRegion } from '../../core/region/KGAudioRegion';
import { KGAudioInterface } from '../../core/audio-interface/KGAudioInterface';
import RegionItem from './RegionItem';
import type { RegionUI, ResizeAction } from '../interfaces';
import { REGION_CONSTANTS, DEBUG_MODE } from '../../constants';
@@ -506,11 +508,22 @@ const TrackGridItem: React.FC<TrackGridItemProps> = ({
>
{/* Render regions for this track */}
{trackRegions.map(region => {
// Find the corresponding KGMidiRegion in the track
const midiRegion = track.getRegions().find(r => r.getId() === region.id) as KGMidiRegion | undefined;
// Find the corresponding region in the track
const coreRegion = track.getRegions().find(r => r.getId() === region.id);
const midiRegion = coreRegion?.getCurrentType() === 'KGMidiRegion' ? coreRegion as unknown as KGMidiRegion : undefined;
const audioRegion = coreRegion?.getCurrentType() === 'KGAudioRegion' ? coreRegion as unknown as KGAudioRegion : undefined;
// Get audio buffer for waveform rendering
let audioBuffer: AudioBuffer | undefined;
if (audioRegion) {
audioBuffer = KGAudioInterface.instance().getAudioBuffer(
track.getId().toString(),
audioRegion.getAudioFileId()
);
}
return (
<RegionItem
<RegionItem
key={region.id}
id={region.id}
name={region.name}
@@ -526,8 +539,8 @@ const TrackGridItem: React.FC<TrackGridItemProps> = ({
onDragEnd={handleRegionDragEnd}
// Keep onClick for selection-only logic if needed by parent
onClick={handleRegionClick}
// New explicit pencil action
onOpenPianoRoll={(regionId) => {
// New explicit pencil action — disabled for audio regions
onOpenPianoRoll={audioRegion ? undefined : (regionId) => {
if (onOpenPianoRoll) {
onOpenPianoRoll(regionId);
} else if (onRegionClick) {
@@ -536,6 +549,8 @@ const TrackGridItem: React.FC<TrackGridItemProps> = ({
}
}}
midiRegion={midiRegion}
audioRegion={audioRegion}
audioBuffer={audioBuffer}
/>
);
})}