feat: add audio track support with WAV/MP3 playback and looping

Introduce KGAudioTrack and KGAudioRegion as new track/region types
alongside existing MIDI tracks. Users can import WAV, MP3, OGG, FLAC,
and AAC files into audio tracks with full playback, looping, and
solo/mute/volume support.

New core models:
- KGAudioTrack (extends KGTrack with TrackType.Wave)
- KGAudioRegion (extends KGRegion with audio file reference)
- KGAudioPlayerBus (Tone.Gain + ToneBufferSource-based playback)
- KGAudioFileStorage (OPFS media/ directory for binary audio files)

New commands:
- AddAudioTrackCommand (create audio track with player bus)
- ImportAudioCommand (import audio file, create region at playhead,
auto-expand maxBars)

Playback features:
- Audio regions scheduled via Tone.Transport alongside MIDI events
- Loop support with duration capping at loop boundaries
- Resume safety offset to prevent ToneBufferSource timing misses
- Cross-track buffer copy when moving audio regions between tracks
- Proper buffer cleanup on track deletion and project switch

UI changes:
- Separate "+ MIDI" and "+ Audio" buttons in top-left spacer
- Audio track shows upload button instead of instrument selector
- FileImportModal reused for audio file drag-and-drop upload
- Real waveform rendering on audio region canvas
- Green color scheme for audio regions (vs blue for MIDI)
- Pencil icon and resize handles hidden for audio regions
- Cross-type region moves blocked (MIDI ↔ Audio)
- Piano roll blocked for audio regions
- Audio region deletion support

Infrastructure:
- Project structure version bumped to 4 (no-op migration)
- class-transformer discriminators updated for new subtypes
- RemoveTrackCommand updated to clean up audio player buses
This commit is contained in:
Xiaohan-Tian
2026-04-09 21:54:54 -07:00
parent de2979178e
commit fc84edfc59
23 changed files with 1406 additions and 100 deletions
+36 -4
View File
@@ -1,5 +1,5 @@
import React, { useRef } from 'react';
import { KGTrack } from '../../core/track/KGTrack';
import { KGTrack, TrackType } from '../../core/track/KGTrack';
import { KGMidiRegion } from '../../core/region/KGMidiRegion';
import TrackGridItem from './TrackGridItem';
import { Playhead } from '../common';
@@ -9,6 +9,8 @@ import { KGMainContentState } from '../../core/state/KGMainContentState';
import { isModifierKeyPressed } from '../../util/osUtil';
import { CreateRegionCommand, ResizeRegionCommand, MoveRegionCommand } from '../../core/commands';
import { KGCore } from '../../core/KGCore';
import { KGAudioInterface } from '../../core/audio-interface/KGAudioInterface';
import { KGAudioRegion } from '../../core/region/KGAudioRegion';
import { generateNewRegionName } from '../../util/miscUtil';
interface TrackGridPanelProps {
@@ -64,7 +66,12 @@ const TrackGridPanel: React.FC<TrackGridPanelProps> = ({
// Get the track and its ID
const track = tracks[trackIndex];
const trackId = track.getId().toString();
// Don't allow manual region creation on audio tracks
if (track.getType() === TrackType.Wave) {
return;
}
if (DEBUG_MODE.TRACK_GRID_PANEL) {
console.log(`Creating region on track ${trackIndex + 1}, bar ${barNumber}`);
}
@@ -238,7 +245,19 @@ const TrackGridPanel: React.FC<TrackGridPanelProps> = ({
// Get the target track
const targetTrack = tracks[finalTrackIndex];
if (!targetTrack) return;
// Block cross-type region moves (MIDI <-> Audio)
const sourceTrack = tracks.find(t => {
return t.getRegions().some(r => r.getId() === regionId);
});
if (sourceTrack && sourceTrack.getType() !== targetTrack.getType()) {
// Snap back — don't execute the move
if (DEBUG_MODE.TRACK_GRID_PANEL) {
console.log(`Blocked cross-type move: ${sourceTrack.getType()} region cannot move to ${targetTrack.getType()} track`);
}
return;
}
// Use command pattern to move the region
try {
const command = MoveRegionCommand.fromBarCoordinates(
@@ -251,9 +270,22 @@ const TrackGridPanel: React.FC<TrackGridPanelProps> = ({
KGCore.instance().executeCommand(command);
// Copy audio buffer to target track if this is a cross-track audio region move
if (sourceTrack && targetTrack && sourceTrack.getId() !== targetTrack.getId()) {
const coreRegion = targetTrack.getRegions().find(r => r.getId() === regionId);
if (coreRegion?.getCurrentType() === 'KGAudioRegion') {
const audioRegion = coreRegion as unknown as KGAudioRegion;
KGAudioInterface.instance().copyAudioBufferBetweenTracks(
sourceTrack.getId().toString(),
targetTrack.getId().toString(),
audioRegion.getAudioFileId()
);
}
}
if (DEBUG_MODE.TRACK_GRID_PANEL) {
console.log(`Executed MoveRegionCommand: region ${regionId} moved using command pattern`);
// Verify the command worked
const movedRegion = command.getTargetRegion();
console.log(`Verified region: ${movedRegion ? 'found' : 'not found'}, startBeat=${movedRegion?.getStartFromBeat()}, trackId=${movedRegion?.getTrackId()}`);