feat: add audio track support with WAV/MP3 playback and looping

Introduce KGAudioTrack and KGAudioRegion as new track/region types
alongside existing MIDI tracks. Users can import WAV, MP3, OGG, FLAC,
and AAC files into audio tracks with full playback, looping, and
solo/mute/volume support.

New core models:
- KGAudioTrack (extends KGTrack with TrackType.Wave)
- KGAudioRegion (extends KGRegion with audio file reference)
- KGAudioPlayerBus (Tone.Gain + ToneBufferSource-based playback)
- KGAudioFileStorage (OPFS media/ directory for binary audio files)

New commands:
- AddAudioTrackCommand (create audio track with player bus)
- ImportAudioCommand (import audio file, create region at playhead,
auto-expand maxBars)

Playback features:
- Audio regions scheduled via Tone.Transport alongside MIDI events
- Loop support with duration capping at loop boundaries
- Resume safety offset to prevent ToneBufferSource timing misses
- Cross-track buffer copy when moving audio regions between tracks
- Proper buffer cleanup on track deletion and project switch

UI changes:
- Separate "+ MIDI" and "+ Audio" buttons in top-left spacer
- Audio track shows upload button instead of instrument selector
- FileImportModal reused for audio file drag-and-drop upload
- Real waveform rendering on audio region canvas
- Green color scheme for audio regions (vs blue for MIDI)
- Pencil icon and resize handles hidden for audio regions
- Cross-type region moves blocked (MIDI ↔ Audio)
- Piano roll blocked for audio regions
- Audio region deletion support

Infrastructure:
- Project structure version bumped to 4 (no-op migration)
- class-transformer discriminators updated for new subtypes
- RemoveTrackCommand updated to clean up audio player buses
This commit is contained in:
Xiaohan-Tian
2026-04-09 21:54:54 -07:00
parent de2979178e
commit fc84edfc59
23 changed files with 1406 additions and 100 deletions
+15 -3
View File
@@ -5,6 +5,7 @@ import { useProjectStore } from '../stores/projectStore';
import { KGCore } from '../core/KGCore';
import { KGTrack } from '../core/track/KGTrack';
import { KGMidiRegion } from '../core/region/KGMidiRegion';
import { KGAudioRegion } from '../core/region/KGAudioRegion';
import TrackInfoPanel from './track/TrackInfoPanel';
import TrackGridPanel from './track/TrackGridPanel';
import PianoRoll from './piano-roll/PianoRoll';
@@ -35,7 +36,8 @@ const MainContent: React.FC<MainContentProps> = ({
activeRegionId,
setShowPianoRoll,
setActiveRegionId,
addTrack
addTrack,
addAudioTrack
} = useProjectStore();
// State to store regions
@@ -130,7 +132,7 @@ const MainContent: React.FC<MainContentProps> = ({
// Iterate through all regions in the track
track.getRegions().forEach(region => {
if (region instanceof KGMidiRegion) {
if (region instanceof KGMidiRegion || region instanceof KGAudioRegion) {
// Calculate bar number and length from beats
const beatsPerBar = timeSignature.numerator;
const barNumber = Math.floor(region.getStartFromBeat() / beatsPerBar) + 1;
@@ -432,6 +434,15 @@ const MainContent: React.FC<MainContentProps> = ({
// Handle explicit pencil action: select region and open piano roll
const handleOpenPianoRoll = (regionId: string) => {
// Don't open piano roll for audio regions
const project = KGCore.instance().getCurrentProject();
for (const track of project.getTracks()) {
const region = track.getRegions().find(r => r.getId() === regionId);
if (region && region.getCurrentType() === 'KGAudioRegion') {
return;
}
}
if (DEBUG_MODE.MAIN_CONTENT) {
console.log(`Open piano roll via pencil for region: ${regionId}`);
}
@@ -691,7 +702,8 @@ const MainContent: React.FC<MainContentProps> = ({
<div className="main-content-wrapper">
{/* Top-left spacer */}
<div className="top-left-spacer">
<button className="add-track-btn" onClick={() => addTrack()}>+ Add track</button>
<button className="add-track-btn" onClick={() => addTrack()}>+ MIDI</button>
<button className="add-track-btn" onClick={() => addAudioTrack()}>+ Audio</button>
</div>
{/* Bar numbers at the top */}